Bit of a shorter post today. I made a hierarchy of evidence, which as the name implies, is a ranking for which types of evidence we should take more seriously than others.
People have made various hierarchies for various fields of science. I was looking for an image of a more ‘general’ hierarchy that could easily be dropped into any online conversation to quickly improve the debate. I found none that had all the features I was looking for. So I took an old hierarchy: expanded it, made it more aesthetically pleasing, and made it into a jpeg, pdf and pages-file so people can easily share and modify it (e.g. translate it or convert it to different files).
Here it is:
Keep in mind that this is only a heuristic and not an ironclad rule. E.g. a good case-control study is still preferable over a bad cohort study. However, when presented with studies of similar quality, or if you can’t assess which study is high quality, go for the one higher up the chain.
Also keep in mind that this is not a hierarchy of ‘usefulness’. I generally prefer listening to experts because they can explain vast amounts of things in “human” terms, inform me how different things interact, and subsequently answer my specific questions. It's just that for any single piece of information you'd rather have a meta-analysis backing you up than an expert opinion.
This is also not a hierarchy of ‘prowess as a scientist’ or ‘what we should fund’. Just because someone has worked on one of the top studies doesn’t mean they’re a better scientist than those who did studies near the bottom. The top also tends to be more expensive and labor intensive. Sometimes, especially with exploratory research, it’s a better use of time and money to go for a lower one.