First time visitor? Learn more.

Bayes Makes A Return To Statistics.

by coldwarrior ( 79 Comments › )
Filed under Academia, Open thread at December 4th, 2017 - 7:43 pm

It’s also an open thread (and a SLOW night in the ER)

 

There has been a debate in the world of Statistics on what to do now that there are such large N studies when the math is set up for much much smaller N’s.

Please read the details here

The most intriguing fix has been here with us for 350 years. The Reverend Thomas Bayes put forth this:

 is a conditional probability: the likelihood of event  occurring given that  is true.

 is also a conditional probability: the likelihood of event  occurring given that  is true.

 and  are the probabilities of observing  and  independently of each other; this is known as the marginal probability.

This is the simplest example:

Drug testing[edit]

Tree diagram illustrating drug testing example. U, Ū, “+” and “−” are the events representing user, non-user, positive result and negative result. Percentages in parentheses are calculated.

Suppose that a test for using a particular drug is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people are users of the drug. What is the probability that a randomly selected individual with a positive test is a user?

33.2%

Even if an individual tests positive, it is more likely that they do not use the drug than that they do. Why? Even though the test appears to be highly accurate, the number of non-users is large compared to the number of users. The number of false positives outweighs the number of true positives.

To use concrete numbers, if 1000 individuals are tested, there are expected to be 995 non-users and 5 users. From the 995 non-users, 0.01 × 995 ≃ 10 false positives are expected. From the 5 users, 0.99 × 5 ≈ 5 true positives are expected. Out of 15 positive results, only 5, about 33%, are genuine.

This illustrates the importance of base rates. Daniel Kahneman has argued that the formation of policy can be egregiously misguided if base rates are neglected when using statistics as a basis for guiding public policy.[3]

The importance of specificity in this example can be seen by calculating that even if sensitivity is raised to 100% and specificity remains at 99% then the probability of the person being a drug user only rises from 33.2% to 33.4%, but if the sensitivity is held at 99% and the specificity is increased to 99.5% then the probability of the person being a drug user rises to about 49.9%.

As you can see, without Bayes, and Bayesian probability interpretation, we lose how a subjective degree of belief should rationally change to account for availability of related evidence. Bayesian inference is needed for these much larger N studies and for ever more complicated statistical modeling especially if policy or important outcomes are determined by these date sets.

Comments

Comments and respectful debate are both welcome and encouraged.

Comments are the sole opinion of the comment writer, just as each thread posted is the sole opinion or post idea of the administrator that posted it or of the readers that have written guest posts for the Blogmocracy.

Obscene, abusive, or annoying remarks may be deleted or moved to spam for admin review, but the fact that particular comments remain on the site in no way constitutes an endorsement of their content by any other commenter or the admins of this Blogmocracy.

We're not easily offended and don't want people to think they have to walk on eggshells around here (like at another place that shall remain nameless) but of course, there is a limit to everything.

Play nice!

79 Responses to “Bayes Makes A Return To Statistics.”
( jump to bottom )

  1. 1 | December 4, 2017 9:27 pm

    The importance of specificity in this example can be seen by calculating that even if sensitivity is raised to 100% and specificity remains at 99% then the probability of the person being a drug user only rises from 33.2% to 33.4%, but if the sensitivity is held at 99% and the specificity is increased to 99.5% then the probability of the person being a drug user rises to about 49.9%.

    Statistical probability good enough to track macro-trends, fortunately, also good enough to ruin 66% of the test subjects receiving a positive test results lives. Like playing Russian roulette with a fully loaded Colt Model 1911. What could possibly go wrong?


  2. rain of lead
    2 | December 5, 2017 6:46 am

    morning y’all


  3. rain of lead
    4 | December 5, 2017 8:21 am

    Branco Cartoon – Headline News
    https://legalinsurrection.com/2017/12/branco-cartoon-headline-news/


  4. eaglesoars
    5 | December 5, 2017 8:25 am

    Today on some Detroit radio show John Conyers will announce his retirement, based on the recommendation of his doctor. (Note: NOT ‘resignation’.)

    Don’t worry about his seat, tho. A current state senator, one Ian CONYERS, will be ‘running’ for it. Yeah, a relative of some kind.

    @ rain of lead:

    yep, if I were colluding with Russia, I’d definitely leave a money trail in my personal/business bank accounts.

    Like McCarthy said, this is about impeachment. John Podhoretz tried to pooh-pooh this whole idea that Strolz was the prime mover behind the change of verbiage to something anodyne in the characterization of Hillary’s emails. He tried to lay it on Comey. McCarthy has quickly corrected that. It was Obama.


  5. rain of lead
    6 | December 5, 2017 8:32 am

    @ eaglesoars:

    it’s ramping up to be an interesting christmas


  6. rain of lead
    7 | December 5, 2017 8:33 am

    @ eaglesoars:

    the girl has her 2 varsity tryout today
    then another next week


  7. eaglesoars
    9 | December 5, 2017 8:57 am

    rain of lead wrote:

    @ eaglesoars:

    the girl has her 2 varsity tryout today
    then another next week

    what sport?


  8. rain of lead
    10 | December 5, 2017 9:07 am

    @ eaglesoars:
    archery


  9. rain of lead
    11 | December 5, 2017 9:11 am

    any chance of this passing?

    http://freebeacon.com/issues/house-vote-national-gun-carry-reciprocity-background-check-fix-bill-week/

    nah, didn’t really think so


  10. eaglesoars
    13 | December 5, 2017 9:19 am

    rain of lead wrote:

    @ eaglesoars:
    archery

    THAT is incredibly demanding. yowser. give her a kiss for me, gotta go.

    CW, I have a question about false negatives but it will have to wait.


  11. coldwarrior
    14 | December 5, 2017 9:34 am

    rain of lead wrote:

    @ eaglesoars:
    archery

    mrs coldwarrior was VERY good at archery back in the soviet union.

    congrats to the girl!


  12. rain of lead
    15 | December 5, 2017 10:02 am

    @ coldwarrior:

    congrats only if she makes the team
    in the fall tryout, she missed making the team by 1 point!!!


  13. rain of lead
    17 | December 5, 2017 10:10 am

    Le gasp!

    https://hotair.com/archives/2017/12/05/new-conyers-accuser-groped-church/

    In Church!


  14. rain of lead
    18 | December 5, 2017 10:11 am

    @ rain of lead:

    they are REALLY trying to posh him out, who gets his seat if he steps down?


  15. rain of lead
    20 | December 5, 2017 12:20 pm

    rain of lead wrote:

    @ rain of lead:

    they are REALLY trying to posh him out, who gets his seat if he steps down?

    these guys

    https://twitchy.com/gregp-3534/2017/12/05/heir-and-a-spare-rep-john-conyers-jr-announces-retirement-sets-up-battle-for-seat-between-son-and-great-nephew/


  16. coldwarrior
    21 | December 5, 2017 12:27 pm

    50F in December in PA…this calls for a ritual smoking of a solstice brisket!


  17. rain of lead
    22 | December 5, 2017 12:37 pm

    @ coldwarrior:

    mmmm…..brisket
    sorry calo


  18. coldwarrior
    23 | December 5, 2017 12:46 pm

    @ rain of lead:
    And this requires a ceremonial cigar from Esteli, Nicaragua


  19. coldwarrior
    24 | December 5, 2017 12:49 pm

    And a DarkStar stout made at the world famous Fat Beagle Brewery.


  20. rain of lead
    25 | December 5, 2017 12:52 pm

    @ coldwarrior:

    I just have yuengling


  21. coldwarrior
    26 | December 5, 2017 12:52 pm

    @ coldwarrior:
    Oh snap! My brewery name got taken! Shoulda copywrote that in 91


  22. coldwarrior
    28 | December 5, 2017 12:59 pm

    rain of lead wrote:

    @ coldwarrior:

    I just have yuengling

    The pride of PA. America’s oldest brewery.

    Yinz cant go wrong with Yeungling


  23. coldwarrior
    29 | December 5, 2017 1:00 pm

    @ rain of lead:
    The CEO office is on the loading dock, he drives the forklift to relax.

    Great family business.


  24. coldwarrior
    30 | December 5, 2017 1:12 pm

    yinz know yinz is a geek when yinz use a multi-probe thermocouple to map the heat pattern inside yinz’s smoker.


  25. Aussie Infidel
    32 | December 5, 2017 2:24 pm

    The Rev knew a thing or two about getting set up by the system when the set of subjects N is large. It could be known as ’round up the usual suspects’ scenario.

    🙂


  26. coldwarrior
    33 | December 5, 2017 2:27 pm

    Turkish president Recep Tayyip Erdogan has said Donald Trump’s plan to declare Jerusalem the capital of Israel is a “red line” for Muslims and could see him breaking off diplomatic relations with the country

    .

    CALL THEIR BLUFF.


  27. eaglesoars
    34 | December 5, 2017 2:28 pm

    rain of lead wrote:

    https://twitchy.com/gregp-3534/2017/12/05/red-line-crossed-new-mueller-subpoena-of-deutsche-bank-takes-aim-at-trump-family-finances/

    guess what? That happened weeks ago. It was hauled out now to knock the Stzrok crooked agent headlines off the front page.


  28. eaglesoars
    35 | December 5, 2017 2:32 pm

    coldwarrior wrote:

    CALL THEIR BLUFF.

    he wants Gulen. This is a bargaining chip.

    While I was out I heard a report that the EU (I think it was the EU – maybe WTO?)) said we can’t change the tax code because it would violate trade agreements.

    here, watch this………….

    I figured out my false negative issue, so nevermind


  29. eaglesoars
    36 | December 5, 2017 2:35 pm

    eaglesoars wrote:

    Don’t worry about his seat, tho. A current state senator, one Ian CONYERS, will be ‘running’ for it. Yeah, a relative of some kind.

    His brother’s son. But now Conyers has thrown HIM over the side in favor of his OWN son, John Jr.

    Detroit is now officially Haiti


  30. coldwarrior
    37 | December 5, 2017 2:39 pm

    https://www.nytimes.com/2017/12/05/sports/olympics/ioc-russia-winter-olympics.html

    no. let them compete but test the snot out of each and every one of them relentlessly.

    then see how the russian do without doping.

    if the east germans never got suspended for their OBVIOUS doping, then why should the russians?


  31. coldwarrior
    38 | December 5, 2017 2:42 pm

    eaglesoars wrote:

    While I was out I heard a report that the EU (I think it was the EU – maybe WTO?)) said we can’t change the tax code because it would violate trade agreements.

    here, watch this………….

    I figured out my false negative issue, so nevermind

    😆

    well done, nice use of a real world example.


  32. eaglesoars
    39 | December 5, 2017 2:44 pm

    oh ffs

    Fury as ‘martyr Museum’ in Germany includes Paris jihadist who stormed the Bataclan, killing 90 people

    ‘Martyr Museum’ shows portraits of 20 people who ‘died for their convictions’
    Included are Bataclan bomber Omar Mostefai and 9/11 pilot Mohammed Atta
    It sparked howls of outrage in German and French media and on social networks

    I have an idea. The next time we reduce Berlin to rubble, we just leave it like that.

    http://www.dailymail.co.uk/news/article-5144931/Fury-martyr-Museum-Germany-includes-Paris-jihadist.html


  33. coldwarrior
    40 | December 5, 2017 2:47 pm

    eaglesoars wrote:

    he wants Gulen. This is a bargaining chip.

    😆

    one must be in a position of some power to negotiate.


  34. eaglesoars
    41 | December 5, 2017 2:48 pm

    coldwarrior wrote:

    then see how the russian do without doping.

    I heard a report by a journalist who does competitive cycling. He ended up doing a doc on doping in sports. It turns out that almost ALL of the anti-aging supplements people take are, in fact, the same substances used in doping, just in different doses. And yes, Russia puts a TON of money into this.

    Now see, if they were capitalists, they’d sell the stuff for its anti-aging properties but instead they use it to cheat at sports contests


  35. coldwarrior
    42 | December 5, 2017 2:54 pm

    @ eaglesoars:

    let em ALL dope, but, their names dont go into he record books, the lab’s name does.

    that goes for pro sport here too. wanna end up like lyle alzado? go ahead.


  36. eaglesoars
    43 | December 5, 2017 2:55 pm

    YES!!

    They’ve just nailed Weissman. He’s the guy on Mueller’s team who absolutely destroyed Arthur Anderson before the entire case was overturned because he absolutely played dirty (withholding exculpatory evidence, etc.) Too late for AA, tens of thousands of jobs lost. He’s a dirty dirty dirty

    Now they’ve got emails from him to Sally Yates saying how deeply proud he was of her for trying to sabotage Trump’s travel ban.


  37. eaglesoars
    44 | December 5, 2017 2:57 pm

    coldwarrior wrote:

    wanna end up like lyle alzado?

    I don’t think anabolic steroids are used for anti-aging supplements. but I get your point.


  38. coldwarrior
    45 | December 5, 2017 2:59 pm

    eaglesoars wrote:

    I don’t think anabolic steroids are used for anti-aging supplements. but I get your point.

    all medicines and supplements are poison.

    its just dose dependent.


  39. coldwarrior
    46 | December 5, 2017 3:00 pm

    eaglesoars wrote:

    He’s the guy on Mueller’s team who absolutely destroyed Arthur Anderson

    ummm…AA signed of on multiple audits of Enron and WorldCom…

    hardly fiduciary responsibility there.


  40. coldwarrior
    47 | December 5, 2017 3:17 pm

    eaglesoars wrote:

    rain of lead wrote:

    https://twitchy.com/gregp-3534/2017/12/05/red-line-crossed-new-mueller-subpoena-of-deutsche-bank-takes-aim-at-trump-family-finances/

    guess what? That happened weeks ago. It was hauled out now to knock the Stzrok crooked agent headlines off the front page.

    looks like this is fake news….a leak of false info


  41. eaglesoars
    48 | December 5, 2017 3:33 pm

    @ coldwarrior:

    I don’t think it’s false. As a matter of fact, when rain posted it I thought, “Didn’t they do that already? Maybe I’m misremembering a bank or something”

    But I rarely give my memory the benefit of the doubt, so until I saw other people confirm that it DID happen, but awhile back, did I realize it was just meant as a distraction.

    Must go fetch the intrepid beagle from day camp and fix dinner. Macker got me hungry for meatloaf. HA!


  42. eaglesoars
    49 | December 5, 2017 3:40 pm

    NO! Sarah Sanders says there has been NO subpoena of DB records.


  43. 50 | December 5, 2017 5:23 pm

    @ eaglesoars:

    Checka U E-mail! I just sent you a recipe I think you’ll like.


  44. eaglesoars
    51 | December 5, 2017 5:48 pm

    got it thanks. never made it w/turkey before


  45. eaglesoars
    52 | December 5, 2017 6:36 pm

    YOO HOO! DEPLORABLE BARBARIAN!!

    Did the prosecution tell Flynn’s lawyer that their main witness against him was removed for bias? Since Strzok led the interview and his testimony would be needed to establish untruthfulness, he is a critical witness not just a prosecutor. If not disclosed, would this not be a Giglio violation? This is the kind of misconduct that can get a case dismissed and a lawyer disbarred. It is a Constitutional violation. This has bothered me since I heard about it.

    https://pjmedia.com/instapundit/282605/


  46. rain of lead
    53 | December 5, 2017 8:26 pm

    welp, I think the girl stands a really good chance of making the varsity team

    their scores are locked in from their last tournament

    so far from two tryouts her avg is 248

    if she has another good round and her avg stays the same then she ties with three and outshoots 6 more


  47. coldwarrior
    54 | December 5, 2017 9:31 pm

    @ rain of lead:

    excellent!


  48. coldwarrior
    55 | December 5, 2017 9:32 pm

    and so it begins…most of the furniture and other non-essentials have been moved out of the bar…

    the setup for the girl’s ‘recording studio’ begins…sooo many amps!

    BWAHAHAHAAAAA!!!!!!


  49. AZfederalist
    56 | December 5, 2017 9:37 pm

    @ rain of lead:

    How many on the team?


  50. RIX
    57 | December 5, 2017 9:49 pm

    http://www.thegatewaypundit.com/2017/12/breaking-new-immigration-gun-charges-filed-illegal-alien-killed-kate-steinle/
    @ Gateway Pundit
    Feds file immigration and gun charges against the illegal white Kate Steinle.


  51. RIX
    58 | December 5, 2017 9:51 pm

    @ RIX:
    “who shot”auto correct!


  52. coldwarrior
    59 | December 5, 2017 9:55 pm

    RIX wrote:

    the illegal white Kate Steinle.

    your spell check is prescient.


  53. rain of lead
    60 | December 5, 2017 9:55 pm

    @ AZfederalist:

    28 on high school varsity team
    29 on Jr high school team

    biggest in the county


  54. coldwarrior
    61 | December 5, 2017 9:58 pm

    rain of lead wrote:

    @ AZfederalist:

    28 on high school varsity team
    29 on Jr high school team

    biggest in the county

    WOAH!

    can she get a scholarship out of this?


  55. RIX
    62 | December 5, 2017 10:01 pm

    coldwarrior wrote:

    RIX wrote:

    the illegal white Kate Steinle.

    your spell check is prescient.

    Or something.


  56. RIX
    63 | December 5, 2017 10:14 pm

    https://news.sky.com/story/terror-plot-to-assassinate-pm-may-foiled-11158388
    @ Sky News
    Two misunderstanders of the Koran busted for plotting to assasinate Theresa May.
    “It has nothing to do with Islam”!


  57. coldwarrior
    64 | December 5, 2017 10:32 pm

    @ coldwarrior:

    i shoulda hired a roadie.


  58. AZfederalist
    65 | December 5, 2017 10:32 pm

    rain of lead wrote:

    @ AZfederalist:

    28 on high school varsity team
    29 on Jr high school team

    biggest in the county

    Awesome. Sounds like she has a great chance of making the team.


  59. AZfederalist
    66 | December 5, 2017 10:33 pm

    RIX wrote:

    https://news.sky.com/story/terror-plot-to-assassinate-pm-may-foiled-11158388
    @ Sky News
    Two misunderstanders of the Koran busted for plotting to assasinate Theresa May.
    “It has nothing to do with Islam”!

    Funny, they never say that if a Christian fundamentalist is accused of a crime.


  60. Deplorable Bumr50
    67 | December 5, 2017 10:51 pm

    @ coldwarrior:

    Is it full-time and does it pay more than $13.50 an hour?


  61. coldwarrior
    68 | December 5, 2017 10:54 pm

    Deplorable Bumr50 wrote:

    @ coldwarrior:

    Is it full-time and does it pay more than $13.50 an hour?

    maybe one day…

    😆


  62. Deplorable Bumr50
    69 | December 5, 2017 11:05 pm

    @ coldwarrior:

    Keep me on the short list, LOL!

    Off to slumber…


  63. coldwarrior
    70 | December 5, 2017 11:11 pm

    Deplorable Bumr50 wrote:

    @ coldwarrior:

    Keep me on the short list, LOL!

    Off to slumber…

    we’ll have a great time om on the road!

    😆


  64. 4_Sticks
    71 | December 6, 2017 12:22 am

    “The importance of specificity in this example can be seen by calculating that even if sensitivity is raised to 100% and specificity remains at 99% then the probability of the person being a drug user only rises from 33.2% to 33.4%, but if the sensitivity is held at 99% and the specificity is increased to 99.5% then the probability of the person being a drug user rises to about 49.9%.”

    Yeah, thats what SHE said …


  65. 72 | December 6, 2017 12:49 am

    A slow night in the ER is a good night in the ER.


  66. 73 | December 6, 2017 12:55 am

    @ Deplorable Macker:
    If it’s a copy of Bunk’s Family Chili I’ll be pissed. Here it is.
    https://tackyraccoons.com/2014/05/24/herding-chili-the-strutts-family-recipe/


  67. Buckeye Abroad
    74 | December 6, 2017 4:23 am

    coldwarrior wrote:

    I used to play them on my walkman while patroling on the night shift. Glad they are still at it. I think the retro-80s sticks around because the musicians play instruments and the music industry has become a wasteland since.


  68. rain of lead
    75 | December 6, 2017 6:53 am

    morning y’all
    hump day!!!
    https://www.youtube.com/watch?v=pD-gp_QphoU


  69. coldwarrior
    76 | December 6, 2017 7:19 am

    Buckeye Abroad wrote:

    coldwarrior wrote:

    I used to play them on my walkman while patroling on the night shift. Glad they are still at it. I think the retro-80s sticks around because the musicians play instruments and the music industry has become a wasteland since.

    u-bahn, walkman, depeche mode….(some smiths thrown in for good measure…)


  70. 77 | December 6, 2017 8:06 am

    Buckeye Abroad wrote:

    I think the retro-80s sticks around because the musicians play instruments and the music industry has become a wasteland since.

    Hmmm, go figure, real musicians who wrote real music and were damned good at it.


  71. coldwarrior
    78 | December 6, 2017 9:34 am

    doriangrey wrote:

    Hmmm, go figure, real musicians who wrote real music and were damned good at it.

    crazy how that works, huh?

    NEW THREAD


  72. coldwarrior
    79 | December 6, 2017 9:35 am

    @ 4_Sticks:

    😆

    indeed!

    you have been freed from moderation, you may now post at will


Back to the Top

The Blogmocracy

website design was Built By All of Us