When Do Anecdotes Become Useful?
There's been a lot of discussion recently over on science Twitter about just how useful anecdotes - that is, people's individual experience - can be when it comes to making evidence based conclusions. For a very long time, anecdotes have been viewed critically by anyone examining evidence, because the experience of one does not automatically constitute the experience of the many.
I think most people understand this, whether they're aware of it or not. After all, if I tell you that my throat closes up when I eat Nutella, you'd be an idiot to assume it would do that to you too unless you'd tried it. (Especially because I've got a nut allergy so to be honest I'm not exactly impartial about Nutella). There's also the fact that most people tend to be fairly critical about things a stranger tells them unless they've experienced it as well or understand enough about the subject to think they're telling the truth.
But the thing is, we all rely on anecdotes to make decisions that effect our everyday lives. If somebody tells you they had a horrible experience with a company (*cough* TUI *cough*), you're probably pretty likely to let that affect your impression of them, even if you think you're one of those rare people that can't be influenced by others at all. Similarly, if you spot some one saying something online, or you read about an experience somebody had in a magazine, even if you're the most critical person in the world there's still going to be a part of your mind that remembers it.
So that then begs the question, if anecdotes are so bad, why are our lives not dumpster-fire messes caused by bad decisions made on the experience of one other person?
This, I think, is what makes 'anecdata' so important. If you're not a contributor to Science Twitter, or Med Twitter, or any other vaguely nerdy blog, then chances are you've never heard the term. 'Anecdata' has been, for many years, a slightly sneery term used by the old hats of science to look down their noses at anyone claiming their single experience represents the whole truth. In the most case, these old timers are right - the single bloke claiming that homeopathy 'changed his life' can't exactly be reliable evidence for everybody to judge their decision. But recently a lot of the community have started pushing back against the term, and I think they've got a point.
This article by Ben Goldacre (I'm a bit of a fan so I'll take any chance I have to reference his work) is just one of the many interesting blog posts written on the subject. (Just in case you thought I was blowing preconceptions out of the water here). It talks about the power of anecdotes, and how they really only become interesting when they're a unqiue event happening in rare circumstances. So there's our first point on how anecdotes can actually be quite useful - if they're the first data on a brand new experience, then yes you should absolutely take note. However, if 20,000 people have all eaten my mum's raw trout and promptly got food poisoning, you probably shouldn't take her claims that she's never gotten sick as gospel.
Secondly, many anecdotes are part of a much wider picture. To illustrate this point, we're going to focus on drugs and their side effects, and how they can all be reported to the Yellow Card Scheme online (I really just like linking to things otherwise you could probably just have Googled that yourself.)
If you're on a new medication and when your doctor asks you if you've had any side effects you say 'weight gain' (thanks, birth control), then there's no evidence to suggest that your experience is nothing more than a fluke or a weird side effect of something else. You might just be pregnant and have no idea, or have eaten a few too many chocolate digestives in a week. However, when all of the data from every patient on that new drug is collected (under something like an Open Data policy or the Yellow Card Scheme) then you can pick out trends and patterns. If lots of people are experiencing weight gain, then actually your personal experience could now be both anecdote and truth. At this point, I think it becomes 'anecdata', useful data drawn from individual's anecdotal experiences that can be used to come to a conclusion. That, ladies and gentleman, is a pretty important part of science, and part of the reason why i think we should be less sneery about anecdotes.
There's also the important point that anecdotes can actually be useful for the general public understanding of complicated topics. I don't know about you, but when I do my nightly reading on rocket science I always find it pretty helpful to hear from people who are working in the field. (Honestly, I don't think I've read a single article on rocket science in my life, I just know there's a really hot end that makes fire and a slightly pointier end with people in it.) For example, let's take the idea of a new drug: I've got no clue, even with my stunningly unhelpful Chemistry A-Level, what certain pharmaceuticals do to the body and how they work. What's definitely going to help me, as a lay person, is someone who's had experience with the drug coming forward and talking about their experience.
But this is where things get dicey.
These anecdotes must be representative. If it's a hail Mary cure that only works in one in ten million people (so, chance), then it's probably a bad idea to hear from the one person it helped. I want to know about the experience of the many not the one. It's also probably not a great idea to have these anecdotes come from people with bias - i.e. I don't want to hear from from CEO Brian about the life changing effects of Brian's Heart Drug. So again, provided this anecdata is representative of the wider experience, then there's no harm in using them to provide information to non specialist people.
Finally, anecdotes can still be useful even if they're the only story saying something different to 10,000 other stories. Whistleblowing cases are built on the principle that only one person has to say something in order to trigger a wider investigation into a company or management system. In fact, if these 'anecdotes' weren't considered useful, some of the biggest scandals that have been revealed around the world would never have been revealed. I'm looking at you, Cambridge Analytica.
But the thing is with this type of anecdote, is there's usually a tonne of evidence to support the claim - it's just not immediately visible. You could probably therefore argue that it's not really an anecdote, just information from somebody brave enough to speak out about something shady. If you do view it in that light, however, then it's obvious why it's useful. It doesn't need an eighteen year old girl to explain it to you.
Â
Thanks for reading this week's offering! I see a few new readers which is always promising ;) (quite possibly my nan has been sending email chains again, so i won't get too excited)
