This was prompted by Mad Scientist’s recently updated Meta post, a Meta answer by Anongoodnurse that has received disturbingly little attention, and my own recent response to a question on Main that ended up revealing as much about my interests in site-wide education on critical data analysis as it did about the question at hand.
This site has the potential to be really useful. There are tons of people with great questions, and there are many people who are excited to help them find answers. There are also quite a few people with good background knowledge that will allow them to create good answers. The main problem at this point seems to be that the community has a hard time distinguishing between high quality and low quality answers. Requiring references doesn’t seem to be enough.
The concepts required to evaluate data critically are not, in my opinion, beyond the grasp of most people here. However, if you’ve never heard about the problem of multiple comparisons, it’s just not likely to occur to you to think about it when you read those shocking results about how daily tooth flossing actually delays the onset of gray hair by 3 years!
What can we do to help more people here gain this background knowledge? Would it be appropriate to have a succinct explanation on meta of concepts that frequently complicate understanding of health-related data so that we could link to them? (I’m reminded of Judaism.SE’s glossary.)
I’m thinking here of things like:
- measures of central tendency
- random and systematic errors
- statistical significance
- problem of multiple comparisons
- false positives and false negatives
- standard deviation and standard error
- receiver operating characteristics
- types of bias