Trump trap: Epistemic arrogance

From the lens of social science and economics we can easily see that Donald Trump is a poster child for mental traps and biases that get in the way of strategic thinking and rational decision making. While Trump has made millions of dollars, attained celebrity status and won the U.S. presidency either in spite of or because of his decisions and behavior, we can make the case that the mental traps and biases he continues to exhibit are leading to bad outcomes for him and for us. By examining the traps and biases that Trump seems to display, we can see where we, as fellow humans, are likewise affected, albeit most often less dramatically, by these very same mental traps and biases.

THE EVIDENCE FROM TRUMP

Donald Trump displays continuing ignorance in so many domains despite abundant evidence of the truth, as these reports show:

  • The environment. “He said the Paris climate change accord would deliver the United States to a grim future of shuttered factories, squeezed taxpayers, blackouts and brownouts, and ‘vastly diminished economic production.’ This extreme dystonic vision doesn’t correlate with the minimal regulations laid out by the agreement.”1
  • Golf and murder. “Trump not only doesn’t know the unknowns but appears to have no interest in even knowing the knowns. Fact-checkers can’t keep up. How often does Obama play golf? Who cares—let’s inflate the number by 50 percent. What’s the murder rate in a major American city? What the hell—let’s multiply it by 10.”2
  • Legislative success. This past summer Trump stated that “‘for the time in office, five months and couple of weeks, I think I’ve done more than anyone else.” And he clarifies that he means “not just executive orders’ but bills passed by Congress. By this time in his presidency, Bill Clinton had signed the Family and Medical Leave Act and the motor voter bill. George W. Bush had signed his first big round of tax cuts. Barack Obama did a major economic stimulus bill, the Lilly Ledbetter Act, and a significant SCHIP expansion. LBJ signed a big tax cut.”3
  • The Middle East. “Trump’s geopolitical obliviousness spans the globe. He irritated Israel…when he said during a news conference with the prime minister of Lebanon that Lebanon was ‘on the front lines in the fight’ against Hezbollah, which is about as wrong as wrong can be. Hezbollah—which Trump’s own State Department brands a terror group—has been part of the Lebanese government for decades, and controls that country’s most powerful military force. The Jerusalem Post, sneered: ‘Clearly, Trump has a less than satisfactory grasp of geopolitics in Lebanon. And if he does not understand who is against whom in Lebanon, he is probably not too well briefed on what is going on in Syria either.’ There were also polite snickers when Trump arrived in Israel from Saudi Arabia and said he had just come from the Middle East. Where on earth does he think Israel is, anyway?”4
  • Health care. “Health-care policy, Donald Trump has admitted, is more complex than he once assumed. ‘Nobody knew that health care could be so complicated,’ he said in February as he struggled to cobble together a plan to repeal and replace Obamacare… In an interview in May, shortly after the House passed a bill that would cause an estimated 23 million people to drop or lose their insurance coverage, Trump boasted that he had become an expert on the subject. ‘It was just something that wasn’t high on my list,’ he told Time magazine. ‘But in a short period of time I understood everything there was to know about health care.’”5
  • Intelligence. “Even before he took office, news broke that Trump was refusing the intelligence briefings meant to get him up to speed. When challenged, he explained he didn’t need them: ‘I’m, like, a really smart person.’”6
  • Auto imports. “President Trump called on Japan to build more cars in the U.S. during his stop in the country as part of his first official tour of Asia as president. ‘Try building your cars in the United States instead of shipping them over,’ Trump said at an event with Japanese business executives, according to a pool report… In reality, the percentage of Japanese cars sold in the United States that are built in this country has, in the last three decades, gone from 12% to 75%. The average of all American-sold cars is that only 65% of them were assembled here. In fact, Japan exports over 400,000 cars built here to other countries. And the top five most American cars — they must contain 75% American-made parts and be assembled in America — are all made by either Toyota or Honda. So Japan doesn’t have to ‘try’ to build their cars here that they sell to Americans; they already do that. But Trump doesn’t know that because Trump knows almost nothing about virtually any subject. His claims are almost never informed by the facts, which are completely irrelevant to him.”7
  • And so much more… Trump also makes random factual errors: In one interview “he says Japanese Prime Minister Shinzo Abe’s wife doesn’t speak any English, but she seems to speak English fine. He says Deputy Attorney General Rod Rosenstein is from Baltimore, when he grew up in the Philadelphia suburbs and has lived for years in Bethesda, just outside of DC. He says the FBI director ‘reports directly to the president of the United States,’ which also isn’t true. He says the Russia investigation is ‘not an investigation’ (whatever that means) and also that ‘it’s not on me’ (it is). He says James Comey wrote a letter to him, when he actually wrote a letter to his former colleagues at the FBI. He says 51 Republican senators came to his health care meeting at the White House, when in fact Susan Collins and Rand Paul didn’t attend and John McCain was sick, so the number was 49.”8

Boris_karloff_as_the_monster_in_bride_of_frankenstein_film_trailer
THE TRAP

Epistemic arrogance (from the Greek “episteme,” knowledge)

“It takes extraordinary wisdom and self-control to accept that many things have a logic we do not understand that is smarter than our own.” – Nassim Nicholas Taleb, The Bed of Procrustes: Philosophical and Practical Aphorisms9

Definition: We think we know more than we really do know. As our learning increases, we grow even more confident in our knowledge and tend to ignore our ignorance and what we still do not know. Nassim Nicholas Taleb labels epistemic arrogance as “our hubris concerning the limits of our knowledge.” He explains, “We overestimate what we know and underestimate uncertainty, by compressing the range of possible uncertain states (i.e., by reducing the space of the unknown).”10

Other researchers label the trap “delusional self-assurance” and “excessive certainty, the tendency we have to believe that our knowledge is more certain than it really is.”11,12

How it works: Humans want to be in control. Uncertainty fuels anxiety and fear and a sense of being out of control. We fool ourselves by wanting to believe that our knowledge is all the knowledge worth having. Through our epistemic arrogance, “to support an illusion of control, we diminish the possibilities and consequences of ‘randomness.'”  We gain greater confidence that we are in control because we think we can better explain past events and better predict  future events. But research shows that while what we know may comfort us, it does not to the same degree increase our capability to control future  uncertainties.13

Epistemic arrogance manifests itself in overconfidence, resulting in “overprecision—excessive confidence that one knows the truth.” Research, conducted by Albert Mannes of The Wharton School of the University of Pennsylvania and Don Moore of the Haas School of Business at the University of California, Berkeley, revealed that the more confident participants were about their estimates of an uncertain quantity, the less they adjusted their estimates in response to feedback about their accuracy and to the costs of being wrong. “The findings suggest that people are too confident in what they know and underestimate what they don’t know,” says Mannes.14,15

Other researchers observe, “One might expect confidence to affect subsequent information search, independent of actual knowledge. What drives information search in this case is one’s feelings of confidence (i.e., perceived knowledge); indeed, one’s actual knowledge level is unknown to the decision maker.”16

Why it’s a problem: The prevalence of overconfidence appears to pose a serious obstacle to effective learning, problem solving and decision making.

“Overprecision — excessive confidence in the accuracy of our beliefs — can have profound consequences, inflating investors’ valuation of their investments, leading physicians to gravitate too quickly to a diagnosis, even making people intolerant of dissenting views,” write researcher Mannes and his co-author. “People also cling too fervently to beliefs that are poorly supported by evidence, adjusting their beliefs too little in light of the evidence or the consequences of being wrong.”17 

Michael Patrick Lynch, Professor of Philosophy at the University of Connecticut, sees epistemic arrogance as “the defining trait of the age,” calling it “the arrogance of thinking that you know it all and that you don’t need to improve because you are just so great already.” He adds, “We blur the line between what’s inside our heads and what’s not. Some philosophers have argued that this blurring is actually justified because knowing itself is often an extended process, distributed in space. If that’s right, then living in a knowledge economy literally increases my knowledge because knowing is not just an individual phenomenon.” However, he laments that “the personalized internet, with its carefully curated social-media feeds and individualized search results” leads to many different knowledge economies, “each bounded by different assumptions of which sources you can trust and what counts as evidence and what doesn’t.” This creates “not only an explosion of overconfidence in what you individually understand but an active encouragement of epistemic arrogance. The Internet of Us becomes one big reinforcement mechanism, getting us all the information we are already biased to believe, and encouraging us to regard those in other bubbles as misinformed miscreants.”18 

“This freedom to doubt is an important matter…. It was born of a struggle. It was a struggle to be permitted to doubt, to be unsure. And I do not want us to forget the importance of the struggle and, by default, to let the thing fall away. I feel a responsibility as a scientist who knows the great value of a satisfactory philosophy of ignorance, and the progress made possible by such a philosophy, progress which is the fruit of freedom of thought. I feel a responsibility to proclaim the value of this freedom and to teach that doubt is not to be feared, but that it is, to be welcomed as the possibility of a new potential for human beings. If you know that you are not sure, you have a chance to improve the situation.” – Richard P. Feynman,The Meaning of It All: Thoughts of a Citizen Scientist 19 

Examples:

US_1stCavDiv_Fallujah _Nov_12 _2004
Finding validation that was not there.
 Analysts now generally agree that the U.S.invasion of Iraq was predicated on faulty intelligence, mistruths or lies, ideology and jingoism. The epistemic arrogance of U.S. decision makers led them to not see or discount that both the seeming evidence and the construct in which it was considered were faulty.

Max Fisher has written, “The US primarily invaded Iraq…because of an ideology. A movement of high-minded ideologues had, throughout the 1990s, become obsessed with deposing Saddam Hussein. When they assumed positions of power under Bush in 2001, they did not seek to trick America into that war, but rather tricked themselves. In 9/11, and in fragments of intelligence that more objective minds would have rejected, they could see only validation for their abstract and untested theories about the world — theories whose inevitable and obvious conclusion was an American invasion of Iraq.”20

Matt Taibbi has offered a similar analysis revealing epistemic arrogance: “The Iraq invasion was always an insane exercise in brainless jingoism that could only be intellectually justified after accepting a series of ludicrous suppositions. First you had to accept a fictional implied connection between Saddam Hussein and 9/11. Then you had to buy that this heavily-sanctioned secular dictator (and confirmed enemy of Islamic radicals) would be a likely sponsor of radical Islamic terror. Then after that you had to accept that Saddam even had the capability of supplying terrorists with weapons that could hurt us… And then, after all that, you still had to buy that all of these factors together added up to a threat so imminent that it justified the immediate mass sacrifice of American and Iraqi lives. It was absurd, a whole bunch of maybes piled on top of a perhaps and a theoretically possible or two.”21

PSM_V76_D622_Lord_kelvin_with_his_compass
Nothing left to know.
 Some physicists around the turn of the 20th century predicted that nothing of great value was left to be found in physics. It seems that those who knew the most thought the least was unknown.

In 1900, eminent British physicist Lord Kelvin purportedly declared: “There is nothing new to discover in physics. All that remains is to more accurately measure its quantities.” Lord Kelvin (William Thomson) was among the leading physicists in the world at that time: He had devised the absolute temperature scale. “the Kelvin scale,” formulated the second law of thermodynamics and had been instrumental in installing telegraph cables under the Atlantic Ocean. He made his declaration “in the same year quantum physics was born and three decades later it, and Einstein’s theory of relativity, had completely revolutionized and transformed physics.”22,23

American_Cash
I’m smart about my money.
 Research by global asset manager Schroders shows that Millennials are overly confident about their investment knowledge. Although 83% of American investors between 18 and 35 surveyed said they knew more about investments than the average investor, only 28% could correctly identify what an investment management company does.Globally, the figures for millennials were 61% and 32%, respectively.

In fact, overconfidence in investment expertise seems prevalent across many age groups: 51% of all investors surveyed said their investment knowledge was above average. but only 37% could accurately identify what an asset management firm does.24,25

Baby computer are-we-there-yet
The internet makes us…dumber?
  That’s the implication of a recent set of experiments conducted by Yale University researchers who studied the effect of finding information by searching the Internet. They concluded that searching the internet for knowledge “creates an illusion whereby people mistake access to information for their own personal understanding of the information.”  They explained, “After using Google to retrieve answers to questions, people seem to believe they came up with these answers on their own; they show an increase in ‘cognitive self-esteem,’ a measure of confidence in one’s own ability to think about and remember information, and predict higher performance on a subsequent trivia quiz to be taken without access to the Internet.”26

The experiments showed that people who acquire information through internet search tend to become irrationally more confident in knowledge unrelated to what they found through search. The researchers found that “searching the Internet may cause a systematic failure to recognize the extent to which we rely on outsourced knowledge. Searching for explanations on the Internet inflates self-assessed knowledge in unrelated domains.” Justin Barrett observed in his article about the research in Slate that “after using the Internet to check how glass is made or why there are leap years, a national sample of American adults suddenly regarded themselves as having a better understanding of completely unrelated topics such as the causes of the U.S. Civil War and how tornadoes form.”27

Even more bothersome is the finding that “the illusion of knowledge from Internet use appears to be driven by the act of searching. The effect does not depend on previous success on a specific search engine [or] when the queries posed to the search engine are not answered and remains even…where the search query fails to provide relevant answers or even any results at all.”28 Barrett wrote, “Just the impression of having access to so much (mis)information seems to swell one’s intellectual self-assessment. The worrisome implication of this study is that those of us who regularly use the Internet to get information may be giving ourselves a steady dose of arrogance enhancement at the same time.”29

So when we rely on the internet but believe “we already knew that,” we seem to believe that we are smarter than we really are. “Erroneously situating external knowledge within their own heads, people may unwittingly exaggerate how much intellectual work they can do in situations where they are truly on their own.”30

6829245ea05969844e359ac03ff244e3--health-psychology-wikimedia-commons
I can diagnose the easy cases, so why not the difficult ones? 
Given their success in diagnosing easy cases, physicians appear to be overly confident that they can come up with the correct diagnosis in more difficult cases, according to multiple studies. The result is a higher likelihood of diagnostic error.

One study concludes that because of their epistemic arrogance “physicians might not request the additional resources to facilitate diagnosis when they most need it.”31 Another study finds, “Other effects of overconfidence that may lead to diagnostic and other errors include widespread non-compliance with clinical guidelines and ‘the general tendency on the part of physicians to disregard, or fail to use, decision-support resources.'”32

TAC-activists-2
330,000 lives lost.
 Starting in the 1980s, AIDS killed millions of people in the prime of their lives (and at present AIDS-related illnesses have killed 35 million people worldwide since the start of the epidemic).33 However, the epidemic was slowed when “virtually the entire medical establishment” agreed with research findings that the human immunodeficiency virus (HIV) caused AIDS. By 1996, a combination of drugs was discovered that could control the virus, allowing infected people to live long and productive lives. Also, “doctors and public health officials saved countless lives through measures aimed at preventing its transmission. ” Today, thanks to antiretroviral treatment for HIV and AIDS, the disease is manageable and 37 million people are living with HIV.”34

But at the same time that a consensus developed on the cause, treatment and prevention of AIDS, a few “AIDS denalists” continued to challenge the understanding that HIV causes AIDS. One of these denialists is University of California Professor Peter Duesberg, who asserts that HIV is harmless and AIDS results from long-time use of recreational or antiretroviral drugs and malnutrition.35

The story of fringe dissent would just be a footnote in the fight against AIDS, except that it led to deadly epistemic ignorance on the part of a leader of a major nation. A press release on a study by researchers from the Harvard Study of Public Health states what then happened: “In 2000, Duesberg sat on a panel which advised then-President of South Africa Thabo Mbeki about the cause of the AIDS virus. Mbeki later denied that AIDS was caused by a virus and limited the treatments in the country.”36 Despite the concerted efforts of scientists and world leaders to dissuade over-confident Mbeki of his false belief, according to the study report “more than 330,000 lives…were lost because a feasible and timely ARV treatment program was not implemented in South Africa [and] thirty-five thousand babies were born with HIV.”37

Rawpixel-com-267082
The overconfident get “phished.”
 Researchers from the University of Texas San Antonio and Columbia College had 600 subjects try to identify the emails they thought were legitimate among 25 actual business emails sent by banks or financial institutions in the U.S. and 25 “phishing” emails that were targeted at customers of banks or financial institutions seeking personal information such as bank account, credit card number, and online banking login information. Then the subjects explained why they made their choices.38

What the researchers found was that overconfidence affected the subjects’ decision-making processes, leading many to misidentify phishing emails as legitimate. Eighty percent of the participants displayed overprecision: They overestimated the probability that the email was not a phishing email. The results showed that constrained decision time or diverted attention (variability in attention allocation), level of optimism and familiarity with the business entities increased the participants’ overconfidence, and only greater time to reach a judgment (cognitive effort) reduced overconfidence.

Researcher Raghav Rao explained in USA Today that people tend to believe they’re smarter than the “phishers,” which is why so many fall for their schemes. “A big advantage for phishers is self efficacy,” Rao said. “Many times, people think they know more than they actually do, and are smarter than someone trying to pull off a scam via an e-mail.”39


 

Big decisions book cover 1 560 high
This series relies on some of the research conducted for my upcoming book, Big Decisions: Why we make decisions that matter so poorly. How we can make them better.. It will be an antidote for bad decision individual and organizational decision making. You can help me get it published and in the hands of decision makers whose decisions not only affect their lives but all of ours.

Learn more about Big Decisions: Why we make decisions that matter so poorly. How we can make them better and my special half-price pre-publication offer. Thank you!


References

  1. Katy Waldman,”We the Victims,” Slate,  June 1, 2017.  http://www.slate.com/articles/news_and_politics/politics/2017/06/trump_is_projecting_his_own_psychological_issues_all_over_america.html
  2. Graydon Carter, “Donald Trump: A Pillar of Ignorance and Certtude,” Vanity Fair, January 2017. https://www.vanityfair.com/news/2017/01/donald-trump-a-pillar-of-ignorance-and-certitude-graydon-carter
  3. Matthew Yglesias, “A new interview reveals Trump’s ignorance to be surprisingly wide-ranging,” Vox, July 20, 2017. https://www.vox.com/2017/7/20/16003218/trump-nyt-interview-ignorance
  4. Paul Brandus, “Opinion: Trump’s global ignorance is hurting America,” MarketWatch, Oct 12, 2017. https://www.marketwatch.com/story/trumps-global-ignorance-is-hurting-america-2017-10-12
  5. Abigail Tracy, “Donald Trump’s Ignorance is Becoming a National Crisis,” Vanity Fair, June 28, 2017. https://www.vanityfair.com/news/2017/06/donald-trump-ignorance-health-care-mitch-mcconnell
  6. Mollie Wilson O’Reilly, “The Man Who Knew Too Little,” Commonweal Magazine, October 9, 2017. https://www.commonwealmagazine.org/man-who-knew-too-little
  7. Ed Brayton, “Trump Displays His Ignorance to Japanese Leaders,” Patheos, November 7, 2017.
    http://www.patheos.com/blogs/dispatches/2017/11/07/trump-displays-ignorance-japanese-leaders/#yGiF2xUPAuSfZMuM.99
  8. Yglesias, “A new interview reveals Trump’s ignorance to be surprisingly wide-ranging,” Vox.
  9. Nassim Nicholas Taleb, The Bed of Procrustes: Philosophical and Practical Aphorisms, Random House, 2010. https://www.amazon.com/Bed-Procrustes-Philosophical-Practical-Aphorisms/dp/1400069971
  10. Nassim Nicholas Taleb, The Black Swan: Second Edition: The Impact of the Highly Improbable: With a new section: On Robustness and Fragility. (Random House Trade Paperbacks, 2010)https://www.amazon.com/Black-Swan-Second-Improbable-Incerto-ebook/dp/B00139XTG4/ref=sr_1_1?ie=UTF8&qid=1511877885&sr=8-1&keywords=the+black+swan+taleb
  11. Pedro Figueiredo, “Metacognition, Epistemic Arrogance and the Dunning–Kruger Effect ,” pedronfigueiredo.com, February 26, 2017. http://pedronfigueiredo.com/dunning-kruger-effect/
  12. Malcom Galloway, “Challenging diagnostic overconfidence.” Excellence in Medical Education; 2015, 6, 16–19.
  13. “Epistemic arrogance,” Prof on Call, July 15, 2012. http://profoncall.blogspot.com/2012/07/epistemic-arrogance.html
  14. “People are overly confident in their own knowledge, despite errors,” Science Daily, June 10, 2013. https://www.sciencedaily.com/releases/2013/06/130610113012.htm
  15. A. E. Mannes and D. A. Moore.” A Behavioral Demonstration of Overconfidence in Judgment,” Psychological Science, 2013; DOI: 10.1177/0956797612470700 http://learnmoore.org/papers/Mannes%20Moore%202013.pdf
  16. Andrew M. Parker and Eric R. Stone, “Identifying the Effects of Unjustified Confidence versus Overconfidence: Lessons Learned from Two Analytic Methods.” Journal of Behavioral Decision Making, 27.2 (2014): 134–145. PMC. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4191861/
  17. Mannes and Moore, A Behavioral Demonstration of Overconfidence in Judgment,” Psychological Science.
  18. Michael Patrick Lynch, “Teaching Humility in an Age of Arrogance,”The Chronicle of Higher Education, June 5, 2017. https://www.chronicle.com/article/Teaching-Humility-in-an-Age-of/240266
  19. Richard P. Feynman, The Meaning of It All: Thoughts of a Citizen Scientist, Addison-Wesley, 1998. https://www.amazon.com/Meaning-All-Thoughts-Citizen-Scientist/dp/0465023940
  20. Max Fisher, “America’s unlearned lesson: the forgotten truth about why we invaded Iraq,” Vox, February 16, 2016. https://www.vox.com/2016/2/16/11022104/iraq-war-neoconservatives
  21. Matt Taibbi, “Forget What We Know Now: We Knew Then the Iraq War Was a Joke,” Rolling Stone, May 18, 2015.  http://www.rollingstone.com/politics/news/forget-what-we-know-now-we-knew-then-the-iraq-war-was-a-joke-20150518
  22. Johan Hansson. (2015). The 10 Biggest Unsolved Problems in Physics,”  International Journal of Modern Physics and Applications, Vol.1, No.1, Mar. 2015. https://www.researchgate.net/publication/272795905_The_10_Biggest_Unsolved_Problems_in_Physics
  23. “Lord Kelvin (1824-1907),” Scottish Science Hall of Fame, National Library of Scotland, 2009. http://digital.nls.uk/scientists/biographies/lord-kelvin/
  24. “Study Finds Millennials Overconfident About Investment Knowledge,” planadviser, September 14, 2016. https://www.planadviser.com/study-finds-millennials-overconfident-about-investment-knowledge/
  25. Javier Simon, “Global Investor Study,” Schroders, 2016. http://www.schroders.com/en/insights/global-investor-study/2016findings/
  26. Matthew Fisher, Mariel K. Goddu and Frank C. Keil, “Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge,” Journal of Experimental Psychology, 2015, Vol. 144, No. 3, 674 – 687. http://gradpsych.apags.org/pubs/journals/releases/xge-0000070.pdf
  27. Justin Barrett, “You Aren’t As Smart As You Think You Are,” Slate. http://www.slate.com/bigideas/what-do-we-know/essays-and-opinions/justin-barrett-opinion
  28. Fisher, Goddu and Keil, “Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge,” Journal of Experimental Psychology.
  29. Barrett, “You Aren’t As Smart As You Think You Are,” Slate.
  30. Fisher, Goddu and Keil, “Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge,” Journal of Experimental Psychology.
  31. A. Meyer, V. Payne, D. Meeks, R. Rao and H. Singh (2013), “Physicians’ diagnostic accuracy, confidence, and resource requests: A vignette study,” JAMA Internal Medicine, 173 (21): 1952–1958.
  32. E. Berner and M. Graber (2008), “Overconfidence as a cause of diagnostic error in medicine,” The American Journal of Medicine, 121 (Suppl. 5): S2–S23
  33. “The Global HIV/AIDS Epidemic,” HIV.gov, November 20, 2017. https://www.hiv.gov/hiv-basics/overview/data-and-trends/global-statistics
  34. Tom Nichols, “America’s Cult of Ignorance,” The Daily Beast, April 1, 2017.  https://www.thedailybeast.com/americas-cult-of-ignorance
  35. Nicoli Nattrass, “AIDS Denialism vs. Science,” Skeptical Inquirer, Volume 31.5, September / October 2007.  https://www.csicop.org/si/show/aids_denialism_vs._science
  36. “Researchers estimate lives lost due to delay in antiretroviral drug use for HIV/AIDS in South Africa,” Harvard School of Public Health, October 20, 2008. https://www.hsph.harvard.edu/news/press-releases/researchers-estimate-lives-lost-delay-arv-drug-use-hivaids-south-africa/
  37. Pride Chigwedere, George R. Seage III, Sofia Gruskin, Tun-Hou Lee and M. Essex, “Estimating the lost benefits of antiretroviral drug use in South Africa.” JAIDS Journal of Acquired Immune Deficiency Syndromes, December 1, 2008, Volume 49, Issue 4,  p 410-415, doi: 10.1097/QAI.0b013e31818a6cd5.  https://www.ncbi.nlm.nih.gov/pubmed/19186354
  38. Jingguo Wang, Yuan Li and H. Raghav Rao, (2016) “Overconfidence in Phishing Email Detection,” Journal of the Association for Information Systems, Vol. 17, Iss. 11 , Article 1. http://aisel.aisnet.org/cgi/viewcontent.cgi?article=1757&context=jais
  39. Joanna Carver, “UTSA study shows how phishing scams thrive on overconfidence,” USA Today, November 28, 2017. https://www.utsa.edu/today/2017/01/phishing.html