13

This was prompted by Mad Scientist’s recently updated Meta post, a Meta answer by Anongoodnurse that has received disturbingly little attention, and my own recent response to a question on Main that ended up revealing as much about my interests in site-wide education on critical data analysis as it did about the question at hand.

This site has the potential to be really useful. There are tons of people with great questions, and there are many people who are excited to help them find answers. There are also quite a few people with good background knowledge that will allow them to create good answers. The main problem at this point seems to be that the community has a hard time distinguishing between high quality and low quality answers. Requiring references doesn’t seem to be enough.

The concepts required to evaluate data critically are not, in my opinion, beyond the grasp of most people here. However, if you’ve never heard about the problem of multiple comparisons, it’s just not likely to occur to you to think about it when you read those shocking results about how daily tooth flossing actually delays the onset of gray hair by 3 years!

What can we do to help more people here gain this background knowledge? Would it be appropriate to have a succinct explanation on meta of concepts that frequently complicate understanding of health-related data so that we could link to them? (I’m reminded of Judaism.SE’s glossary.)

I’m thinking here of things like:

  • measures of central tendency
  • random and systematic errors
  • confounding
  • statistical significance
  • problem of multiple comparisons
  • false positives and false negatives
  • standard deviation and standard error
  • receiver operating characteristics
  • types of bias
  • etc.
3
  • Everything you mention is mathematics albeit statistics. How do you get people to understand mathematical concepts? The answer is for people to solve mathematical problems dealing with the concepts at hand. Mathematicians dislike statistics (but not measure theory) and most normal people do as well so it may be a difficult task to get every reader to brush up on statistics.
    – user139
    Commented Apr 21, 2015 at 14:29
  • @dustin Math-related, definitely. Most of them can be understood conceptually without any math, though. I dare say that most could be explained in a paragraph that most critically thinking people, even without a math background, could understand. Some of them - confounding, bias, false positive/negatives - can be quantified but aren't fundamentally mathematical issues to start with. Standard deviation and measures of central tendency are indeed math-y, but many people learned the difference between mean and median in 6th grade and just need to be reminded. IMO.
    – Susan
    Commented Apr 21, 2015 at 14:45
  • If that was necessarily true, I would think less people would complain and avoid mathematics. Everything in mathematics has a clear definition, theorem, or lemma, and if reading it and being able to understand it without putting it to practice was the case, I would be a lot better at mathematics than I am. Unfortunately, that isn't the case. The issue is that everyone views math concepts differently. Definitions I find trivial someone else may struggle with whereas I may struggle with other definitions others think are no brainers.
    – user139
    Commented Apr 21, 2015 at 16:26

1 Answer 1

Reset to default
4

So, as an academic epidemiologist, my I really want to say yes to this, because I think an understanding of these kinds of concepts are important. I certainly don't think the existence of such a key would hurt.

But I also don't think it will help.

This is a site inherently driven by popular voting, and is open to answers of all types. While requiring citations might help (but also has some drawbacks), I think the solution to this is inherently a community-level problem. We, as a community, need to push up answers that try not to just find "A Paper that Says What I Want" but tries to genuinely tackle the problem in a methodologically rigorous way.

Your idea, however, could be an interesting collaboration with the good folks over at CrossValidated?

2
  • Thank you for your thoughts. I was wondering if we might be able to use such a meta post as something to link to in comments or answers, to encourage people to bring up these things when answering or giving constructive feedback about methodological issues. That depends primarily on a user base that is willing to implement it, though. I agree - just having a glossary of such concepts sitting around isn't going to do anything. Regardless, I hope you'll consider sticking around to help us figure it out!
    – Susan
    Commented Apr 13, 2015 at 18:42
  • 1
    As for collaboration with CrossValidated.....those do seem like people who could help us. If we can just make it out of private beta....
    – Susan
    Commented Apr 13, 2015 at 18:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .