First time visitor? Learn more.
  • No content found in .

  • Bayes Makes A Return To Statistics.

    by coldwarrior ( 79 Comments › )
    Filed under Academia, Open thread at December 4th, 2017 - 7:43 pm

    It’s also an open thread (and a SLOW night in the ER)

     

    There has been a debate in the world of Statistics on what to do now that there are such large N studies when the math is set up for much much smaller N’s.

    Please read the details here

    The most intriguing fix has been here with us for 350 years. The Reverend Thomas Bayes put forth this:

     is a conditional probability: the likelihood of event  occurring given that  is true.

     is also a conditional probability: the likelihood of event  occurring given that  is true.

     and  are the probabilities of observing  and  independently of each other; this is known as the marginal probability.

    This is the simplest example:

    Drug testing[edit]

    Tree diagram illustrating drug testing example. U, Ū, “+” and “−” are the events representing user, non-user, positive result and negative result. Percentages in parentheses are calculated.

    Suppose that a test for using a particular drug is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people are users of the drug. What is the probability that a randomly selected individual with a positive test is a user?

    33.2%

    Even if an individual tests positive, it is more likely that they do not use the drug than that they do. Why? Even though the test appears to be highly accurate, the number of non-users is large compared to the number of users. The number of false positives outweighs the number of true positives.

    To use concrete numbers, if 1000 individuals are tested, there are expected to be 995 non-users and 5 users. From the 995 non-users, 0.01 × 995 ≃ 10 false positives are expected. From the 5 users, 0.99 × 5 ≈ 5 true positives are expected. Out of 15 positive results, only 5, about 33%, are genuine.

    This illustrates the importance of base rates. Daniel Kahneman has argued that the formation of policy can be egregiously misguided if base rates are neglected when using statistics as a basis for guiding public policy.[3]

    The importance of specificity in this example can be seen by calculating that even if sensitivity is raised to 100% and specificity remains at 99% then the probability of the person being a drug user only rises from 33.2% to 33.4%, but if the sensitivity is held at 99% and the specificity is increased to 99.5% then the probability of the person being a drug user rises to about 49.9%.

    As you can see, without Bayes, and Bayesian probability interpretation, we lose how a subjective degree of belief should rationally change to account for availability of related evidence. Bayesian inference is needed for these much larger N studies and for ever more complicated statistical modeling especially if policy or important outcomes are determined by these date sets.

    Comments

    Comments and respectful debate are both welcome and encouraged.

    Comments are the sole opinion of the comment writer, just as each thread posted is the sole opinion or post idea of the administrator that posted it or of the readers that have written guest posts for the Blogmocracy.

    Obscene, abusive, or annoying remarks may be deleted or moved to spam for admin review, but the fact that particular comments remain on the site in no way constitutes an endorsement of their content by any other commenter or the admins of this Blogmocracy.

    We're not easily offended and don't want people to think they have to walk on eggshells around here (like at another place that shall remain nameless) but of course, there is a limit to everything.

    Play nice!

    79 Responses to “Bayes Makes A Return To Statistics.”
    ( jump to bottom )

    1. 1 | December 4, 2017 9:27 pm

      The importance of specificity in this example can be seen by calculating that even if sensitivity is raised to 100% and specificity remains at 99% then the probability of the person being a drug user only rises from 33.2% to 33.4%, but if the sensitivity is held at 99% and the specificity is increased to 99.5% then the probability of the person being a drug user rises to about 49.9%.

      Statistical probability good enough to track macro-trends, fortunately, also good enough to ruin 66% of the test subjects receiving a positive test results lives. Like playing Russian roulette with a fully loaded Colt Model 1911. What could possibly go wrong?


    2. rain of lead
      2 | December 5, 2017 6:46 am

      morning y’all


    3. rain of lead
      4 | December 5, 2017 8:21 am

      Branco Cartoon – Headline News
      https://legalinsurrection.com/2017/12/branco-cartoon-headline-news/


    4. eaglesoars
      5 | December 5, 2017 8:25 am

      Today on some Detroit radio show John Conyers will announce his retirement, based on the recommendation of his doctor. (Note: NOT ‘resignation’.)

      Don’t worry about his seat, tho. A current state senator, one Ian CONYERS, will be ‘running’ for it. Yeah, a relative of some kind.

      @ rain of lead:

      yep, if I were colluding with Russia, I’d definitely leave a money trail in my personal/business bank accounts.

      Like McCarthy said, this is about impeachment. John Podhoretz tried to pooh-pooh this whole idea that Strolz was the prime mover behind the change of verbiage to something anodyne in the characterization of Hillary’s emails. He tried to lay it on Comey. McCarthy has quickly corrected that. It was Obama.


    5. rain of lead
      6 | December 5, 2017 8:32 am

      @ eaglesoars:

      it’s ramping up to be an interesting christmas


    6. rain of lead
      7 | December 5, 2017 8:33 am

      @ eaglesoars:

      the girl has her 2 varsity tryout today
      then another next week


    7. eaglesoars
      9 | December 5, 2017 8:57 am

      rain of lead wrote:

      @ eaglesoars:

      the girl has her 2 varsity tryout today
      then another next week

      what sport?


    8. rain of lead
      10 | December 5, 2017 9:07 am

      @ eaglesoars:
      archery


    9. rain of lead
      11 | December 5, 2017 9:11 am

      any chance of this passing?

      http://freebeacon.com/issues/house-vote-national-gun-carry-reciprocity-background-check-fix-bill-week/

      nah, didn’t really think so


    10. eaglesoars
      13 | December 5, 2017 9:19 am

      rain of lead wrote:

      @ eaglesoars:
      archery

      THAT is incredibly demanding. yowser. give her a kiss for me, gotta go.

      CW, I have a question about false negatives but it will have to wait.


    11. coldwarrior
      14 | December 5, 2017 9:34 am

      rain of lead wrote:

      @ eaglesoars:
      archery

      mrs coldwarrior was VERY good at archery back in the soviet union.

      congrats to the girl!


    12. rain of lead
      15 | December 5, 2017 10:02 am

      @ coldwarrior:

      congrats only if she makes the team
      in the fall tryout, she missed making the team by 1 point!!!


    13. rain of lead
      17 | December 5, 2017 10:10 am

      Le gasp!

      https://hotair.com/archives/2017/12/05/new-conyers-accuser-groped-church/

      In Church!


    14. rain of lead
      18 | December 5, 2017 10:11 am

      @ rain of lead:

      they are REALLY trying to posh him out, who gets his seat if he steps down?


    15. rain of lead
      20 | December 5, 2017 12:20 pm

      rain of lead wrote:

      @ rain of lead:

      they are REALLY trying to posh him out, who gets his seat if he steps down?

      these guys

      https://twitchy.com/gregp-3534/2017/12/05/heir-and-a-spare-rep-john-conyers-jr-announces-retirement-sets-up-battle-for-seat-between-son-and-great-nephew/


    16. coldwarrior
      21 | December 5, 2017 12:27 pm

      50F in December in PA…this calls for a ritual smoking of a solstice brisket!


    17. rain of lead
      22 | December 5, 2017 12:37 pm

      @ coldwarrior:

      mmmm…..brisket
      sorry calo


    18. coldwarrior
      23 | December 5, 2017 12:46 pm

      @ rain of lead:
      And this requires a ceremonial cigar from Esteli, Nicaragua


    19. coldwarrior
      24 | December 5, 2017 12:49 pm

      And a DarkStar stout made at the world famous Fat Beagle Brewery.


    20. rain of lead
      25 | December 5, 2017 12:52 pm

      @ coldwarrior:

      I just have yuengling


    21. coldwarrior
      26 | December 5, 2017 12:52 pm

      @ coldwarrior:
      Oh snap! My brewery name got taken! Shoulda copywrote that in 91


    22. coldwarrior
      28 | December 5, 2017 12:59 pm

      rain of lead wrote:

      @ coldwarrior:

      I just have yuengling

      The pride of PA. America’s oldest brewery.

      Yinz cant go wrong with Yeungling


    23. coldwarrior
      29 | December 5, 2017 1:00 pm

      @ rain of lead:
      The CEO office is on the loading dock, he drives the forklift to relax.

      Great family business.


    24. coldwarrior
      30 | December 5, 2017 1:12 pm

      yinz know yinz is a geek when yinz use a multi-probe thermocouple to map the heat pattern inside yinz’s smoker.


    25. Aussie Infidel
      32 | December 5, 2017 2:24 pm

      The Rev knew a thing or two about getting set up by the system when the set of subjects N is large. It could be known as ’round up the usual suspects’ scenario.

      🙂


    26. coldwarrior
      33 | December 5, 2017 2:27 pm

      Turkish president Recep Tayyip Erdogan has said Donald Trump’s plan to declare Jerusalem the capital of Israel is a “red line” for Muslims and could see him breaking off diplomatic relations with the country

      .

      CALL THEIR BLUFF.


    27. eaglesoars
      34 | December 5, 2017 2:28 pm

      rain of lead wrote:

      https://twitchy.com/gregp-3534/2017/12/05/red-line-crossed-new-mueller-subpoena-of-deutsche-bank-takes-aim-at-trump-family-finances/

      guess what? That happened weeks ago. It was hauled out now to knock the Stzrok crooked agent headlines off the front page.


    28. eaglesoars
      35 | December 5, 2017 2:32 pm

      coldwarrior wrote:

      CALL THEIR BLUFF.

      he wants Gulen. This is a bargaining chip.

      While I was out I heard a report that the EU (I think it was the EU – maybe WTO?)) said we can’t change the tax code because it would violate trade agreements.

      here, watch this………….

      I figured out my false negative issue, so nevermind


    29. eaglesoars
      36 | December 5, 2017 2:35 pm

      eaglesoars wrote:

      Don’t worry about his seat, tho. A current state senator, one Ian CONYERS, will be ‘running’ for it. Yeah, a relative of some kind.

      His brother’s son. But now Conyers has thrown HIM over the side in favor of his OWN son, John Jr.

      Detroit is now officially Haiti


    30. coldwarrior
      37 | December 5, 2017 2:39 pm

      https://www.nytimes.com/2017/12/05/sports/olympics/ioc-russia-winter-olympics.html

      no. let them compete but test the snot out of each and every one of them relentlessly.

      then see how the russian do without doping.

      if the east germans never got suspended for their OBVIOUS doping, then why should the russians?


    31. coldwarrior
      38 | December 5, 2017 2:42 pm

      eaglesoars wrote:

      While I was out I heard a report that the EU (I think it was the EU – maybe WTO?)) said we can’t change the tax code because it would violate trade agreements.

      here, watch this………….

      I figured out my false negative issue, so nevermind

      😆

      well done, nice use of a real world example.


    32. eaglesoars
      39 | December 5, 2017 2:44 pm

      oh ffs

      Fury as ‘martyr Museum’ in Germany includes Paris jihadist who stormed the Bataclan, killing 90 people

      ‘Martyr Museum’ shows portraits of 20 people who ‘died for their convictions’
      Included are Bataclan bomber Omar Mostefai and 9/11 pilot Mohammed Atta
      It sparked howls of outrage in German and French media and on social networks

      I have an idea. The next time we reduce Berlin to rubble, we just leave it like that.

      http://www.dailymail.co.uk/news/article-5144931/Fury-martyr-Museum-Germany-includes-Paris-jihadist.html


    33. coldwarrior
      40 | December 5, 2017 2:47 pm

      eaglesoars wrote:

      he wants Gulen. This is a bargaining chip.

      😆

      one must be in a position of some power to negotiate.


    34. eaglesoars
      41 | December 5, 2017 2:48 pm

      coldwarrior wrote:

      then see how the russian do without doping.

      I heard a report by a journalist who does competitive cycling. He ended up doing a doc on doping in sports. It turns out that almost ALL of the anti-aging supplements people take are, in fact, the same substances used in doping, just in different doses. And yes, Russia puts a TON of money into this.

      Now see, if they were capitalists, they’d sell the stuff for its anti-aging properties but instead they use it to cheat at sports contests


    35. coldwarrior
      42 | December 5, 2017 2:54 pm

      @ eaglesoars:

      let em ALL dope, but, their names dont go into he record books, the lab’s name does.

      that goes for pro sport here too. wanna end up like lyle alzado? go ahead.


    36. eaglesoars
      43 | December 5, 2017 2:55 pm

      YES!!

      They’ve just nailed Weissman. He’s the guy on Mueller’s team who absolutely destroyed Arthur Anderson before the entire case was overturned because he absolutely played dirty (withholding exculpatory evidence, etc.) Too late for AA, tens of thousands of jobs lost. He’s a dirty dirty dirty

      Now they’ve got emails from him to Sally Yates saying how deeply proud he was of her for trying to sabotage Trump’s travel ban.


    37. eaglesoars
      44 | December 5, 2017 2:57 pm

      coldwarrior wrote:

      wanna end up like lyle alzado?

      I don’t think anabolic steroids are used for anti-aging supplements. but I get your point.


    38. coldwarrior
      45 | December 5, 2017 2:59 pm

      eaglesoars wrote:

      I don’t think anabolic steroids are used for anti-aging supplements. but I get your point.

      all medicines and supplements are poison.

      its just dose dependent.


    39. coldwarrior
      46 | December 5, 2017 3:00 pm

      eaglesoars wrote:

      He’s the guy on Mueller’s team who absolutely destroyed Arthur Anderson

      ummm…AA signed of on multiple audits of Enron and WorldCom…

      hardly fiduciary responsibility there.


    40. coldwarrior
      47 | December 5, 2017 3:17 pm

      eaglesoars wrote:

      rain of lead wrote:

      https://twitchy.com/gregp-3534/2017/12/05/red-line-crossed-new-mueller-subpoena-of-deutsche-bank-takes-aim-at-trump-family-finances/

      guess what? That happened weeks ago. It was hauled out now to knock the Stzrok crooked agent headlines off the front page.

      looks like this is fake news….a leak of false info


    41. eaglesoars
      48 | December 5, 2017 3:33 pm

      @ coldwarrior:

      I don’t think it’s false. As a matter of fact, when rain posted it I thought, “Didn’t they do that already? Maybe I’m misremembering a bank or something”

      But I rarely give my memory the benefit of the doubt, so until I saw other people confirm that it DID happen, but awhile back, did I realize it was just meant as a distraction.

      Must go fetch the intrepid beagle from day camp and fix dinner. Macker got me hungry for meatloaf. HA!


    42. eaglesoars
      49 | December 5, 2017 3:40 pm

      NO! Sarah Sanders says there has been NO subpoena of DB records.


    43. 50 | December 5, 2017 5:23 pm

      @ eaglesoars:

      Checka U E-mail! I just sent you a recipe I think you’ll like.


    44. eaglesoars
      51 | December 5, 2017 5:48 pm

      got it thanks. never made it w/turkey before


    45. eaglesoars
      52 | December 5, 2017 6:36 pm

      YOO HOO! DEPLORABLE BARBARIAN!!

      Did the prosecution tell Flynn’s lawyer that their main witness against him was removed for bias? Since Strzok led the interview and his testimony would be needed to establish untruthfulness, he is a critical witness not just a prosecutor. If not disclosed, would this not be a Giglio violation? This is the kind of misconduct that can get a case dismissed and a lawyer disbarred. It is a Constitutional violation. This has bothered me since I heard about it.

      https://pjmedia.com/instapundit/282605/


    46. rain of lead
      53 | December 5, 2017 8:26 pm

      welp, I think the girl stands a really good chance of making the varsity team

      their scores are locked in from their last tournament

      so far from two tryouts her avg is 248

      if she has another good round and her avg stays the same then she ties with three and outshoots 6 more


    47. coldwarrior
      54 | December 5, 2017 9:31 pm

      @ rain of lead:

      excellent!


    48. coldwarrior
      55 | December 5, 2017 9:32 pm

      and so it begins…most of the furniture and other non-essentials have been moved out of the bar…

      the setup for the girl’s ‘recording studio’ begins…sooo many amps!

      BWAHAHAHAAAAA!!!!!!


    49. AZfederalist
      56 | December 5, 2017 9:37 pm

      @ rain of lead:

      How many on the team?


    50. RIX
      57 | December 5, 2017 9:49 pm

      http://www.thegatewaypundit.com/2017/12/breaking-new-immigration-gun-charges-filed-illegal-alien-killed-kate-steinle/
      @ Gateway Pundit
      Feds file immigration and gun charges against the illegal white Kate Steinle.


    51. RIX
      58 | December 5, 2017 9:51 pm

      @ RIX:
      “who shot”auto correct!


    52. coldwarrior
      59 | December 5, 2017 9:55 pm

      RIX wrote:

      the illegal white Kate Steinle.

      your spell check is prescient.


    53. rain of lead
      60 | December 5, 2017 9:55 pm

      @ AZfederalist:

      28 on high school varsity team
      29 on Jr high school team

      biggest in the county


    54. coldwarrior
      61 | December 5, 2017 9:58 pm

      rain of lead wrote:

      @ AZfederalist:

      28 on high school varsity team
      29 on Jr high school team

      biggest in the county

      WOAH!

      can she get a scholarship out of this?


    55. RIX
      62 | December 5, 2017 10:01 pm

      coldwarrior wrote:

      RIX wrote:

      the illegal white Kate Steinle.

      your spell check is prescient.

      Or something.


    56. RIX
      63 | December 5, 2017 10:14 pm

      https://news.sky.com/story/terror-plot-to-assassinate-pm-may-foiled-11158388
      @ Sky News
      Two misunderstanders of the Koran busted for plotting to assasinate Theresa May.
      “It has nothing to do with Islam”!


    57. coldwarrior
      64 | December 5, 2017 10:32 pm

      @ coldwarrior:

      i shoulda hired a roadie.


    58. AZfederalist
      65 | December 5, 2017 10:32 pm

      rain of lead wrote:

      @ AZfederalist:

      28 on high school varsity team
      29 on Jr high school team

      biggest in the county

      Awesome. Sounds like she has a great chance of making the team.


    59. AZfederalist
      66 | December 5, 2017 10:33 pm

      RIX wrote:

      https://news.sky.com/story/terror-plot-to-assassinate-pm-may-foiled-11158388
      @ Sky News
      Two misunderstanders of the Koran busted for plotting to assasinate Theresa May.
      “It has nothing to do with Islam”!

      Funny, they never say that if a Christian fundamentalist is accused of a crime.


    60. Deplorable Bumr50
      67 | December 5, 2017 10:51 pm

      @ coldwarrior:

      Is it full-time and does it pay more than $13.50 an hour?


    61. coldwarrior
      68 | December 5, 2017 10:54 pm

      Deplorable Bumr50 wrote:

      @ coldwarrior:

      Is it full-time and does it pay more than $13.50 an hour?

      maybe one day…

      😆


    62. Deplorable Bumr50
      69 | December 5, 2017 11:05 pm

      @ coldwarrior:

      Keep me on the short list, LOL!

      Off to slumber…


    63. coldwarrior
      70 | December 5, 2017 11:11 pm

      Deplorable Bumr50 wrote:

      @ coldwarrior:

      Keep me on the short list, LOL!

      Off to slumber…

      we’ll have a great time om on the road!

      😆


    64. 4_Sticks
      71 | December 6, 2017 12:22 am

      “The importance of specificity in this example can be seen by calculating that even if sensitivity is raised to 100% and specificity remains at 99% then the probability of the person being a drug user only rises from 33.2% to 33.4%, but if the sensitivity is held at 99% and the specificity is increased to 99.5% then the probability of the person being a drug user rises to about 49.9%.”

      Yeah, thats what SHE said …


    65. 72 | December 6, 2017 12:49 am

      A slow night in the ER is a good night in the ER.


    66. 73 | December 6, 2017 12:55 am

      @ Deplorable Macker:
      If it’s a copy of Bunk’s Family Chili I’ll be pissed. Here it is.
      https://tackyraccoons.com/2014/05/24/herding-chili-the-strutts-family-recipe/


    67. Buckeye Abroad
      74 | December 6, 2017 4:23 am

      coldwarrior wrote:

      I used to play them on my walkman while patroling on the night shift. Glad they are still at it. I think the retro-80s sticks around because the musicians play instruments and the music industry has become a wasteland since.


    68. rain of lead
      75 | December 6, 2017 6:53 am

      morning y’all
      hump day!!!
      https://www.youtube.com/watch?v=pD-gp_QphoU


    69. coldwarrior
      76 | December 6, 2017 7:19 am

      Buckeye Abroad wrote:

      coldwarrior wrote:

      I used to play them on my walkman while patroling on the night shift. Glad they are still at it. I think the retro-80s sticks around because the musicians play instruments and the music industry has become a wasteland since.

      u-bahn, walkman, depeche mode….(some smiths thrown in for good measure…)


    70. 77 | December 6, 2017 8:06 am

      Buckeye Abroad wrote:

      I think the retro-80s sticks around because the musicians play instruments and the music industry has become a wasteland since.

      Hmmm, go figure, real musicians who wrote real music and were damned good at it.


    71. coldwarrior
      78 | December 6, 2017 9:34 am

      doriangrey wrote:

      Hmmm, go figure, real musicians who wrote real music and were damned good at it.

      crazy how that works, huh?

      NEW THREAD


    72. coldwarrior
      79 | December 6, 2017 9:35 am

      @ 4_Sticks:

      😆

      indeed!

      you have been freed from moderation, you may now post at will


    Back to the Top

    The Blogmocracy

    website design was Built By All of Us