28

The first meta question asked Should we require references to back up all answer?. The general consensus was yes (most of the time). This is a quote from the top voted answer by @Jez

I think the approach should be that we "strongly encourage" references to reliable web sources in answers

Notice the word reliable. Obviously we can't have our sources coming from our aunt's blog, or Yahoo answers. But what is a reliable source? Published studies are good sources. How about WebMD or Mayo Clinic? Are independent articles written by doctors good?

What do you think some reliable sources are? Should we downvote answers that don't have reliable sources? If you have any good reliable sites, make sure to add to the community wiki :)

5
  • 1
    This discussion came up in the comments of this answer re: chiropractor sources
    – Zaralynda
    Commented Apr 4, 2015 at 15:32
  • 1
    One thing to remember with reliable sources is if the supporters/detractors have anything to gain/lose from the side they are taking.
    – Joe W
    Commented Apr 4, 2015 at 21:01
  • 1
    This question should be made more clear that this is not a whitelist, it is for recommended sources
    – Zach Saucier
    Commented Apr 6, 2015 at 18:54
  • @ZachSaucier Made it a bit more clear
    – michaelpri
    Commented Apr 6, 2015 at 18:55
  • @michaelpri, Also, what if the answer is "common knowledge"? E.g. this answer. Does it need to be backed up?
    – Pacerier
    Commented Jun 5, 2015 at 4:55

9 Answers 9

Reset to default
19

We should downvote if there are no sources, unless it is widely known (for example, the statement "If you get too cold you die" is a little unnecessary to backup. The actual temperature (35°C/95°F and less) should be linked.

What sites?

These sites are reputable and I would use as a reference. I have made this CW, please edit to include more. Note that this is not a whitelist. This is just suggestions if you need a site and want to know that at least one other person approves. Please use citation not inline link style.

US:

UK:

EU:

Australia:

  • Therapeutic Goods Administration (TGA), Australian regulatory body.

    As part of the Department of Health, TGA safeguards and enhances the health of the Australian community through effective and timely regulation of therapeutic goods.

Other:

Scientific journals:

13
  • 2
    +1 for Center for Disease Control
    – JorgeArtware
    Commented Apr 5, 2015 at 1:11
  • 1
    I appreciate the addition (@anon) of PubMed, although the quality of journals indexed there varies tremendously. Honestly, I only trust peer -reviewed literature that either is itself primary data or properly cites primary data. The trouble with using other websites that seem reliable but don’t follow normal rules about references for non-obvious points is exemplified here. I cited primary data suggesting that low-oxalate diet is not helpful, and she cited a website that says that it is helpful. How is the reader to interpret?
    – Susan
    Commented Apr 5, 2015 at 17:15
  • 3
    I have no problem removing PubMed. The problem about references to me is that not everybody (including me when at home) has access to NEJM, Nature, Science, Cell, etc. It's a problem. In general, though, I think the Mayo Clinic does a good job; I'm inclined to keep that as a reference.
    – anongoodnurse
    Commented Apr 5, 2015 at 18:29
  • 3
    @anongoodnurse I didn’t mean to imply that Pubmed shouldn’t be there - most peer-reviewed stuff is indexed there, the links are organized and easy to create from a PMID, and usually there is at least an abstract there. I see your point about the access issue, and I’m not really advocating taking any of those other (publicly accessible) sources out of here. I was just voicing a concern without offering any particular solution. Helpful, isn’t it? ;-)
    – Susan
    Commented Apr 5, 2015 at 18:49
  • @Kenorb is the Clinical trials one atually a nih one?
    – Tim
    Commented Apr 6, 2015 at 13:23
  • @Tim All these databases are linked on the main NIH site. It's also saying on that site ('A service of the U.S. National Institutes of Health').
    – kenorb
    Commented Apr 6, 2015 at 13:25
  • Okay, that's fine :) Thanks for the edits!
    – Tim
    Commented Apr 6, 2015 at 13:45
  • @Mad why do you not think it is (just asking, not arguing)? :)
    – Tim
    Commented Apr 9, 2015 at 14:07
  • @Sue - I like your two sources, but the Harvard site is mostly behind a paywall. :-(
    – anongoodnurse Mod
    Commented May 20, 2015 at 22:21
  • @Sue - I should have added, though (please forgive me) that if you have access to it, by all means, use it! As I said, I agree that it's a very good site.
    – anongoodnurse Mod
    Commented May 21, 2015 at 1:00
  • Strictly speaking, arXiv is not a scientific journal. While they ensure their publications have the form of scientific papers, the contents are not peer reviewed on technical grounds. It is just a preprint repository, and any article there may or may not have been accepted in a peer reviewed publication.
    – lvella
    Commented Mar 7, 2017 at 21:14
  • Thank you for this excellent list! I would add two more: PsycINFO (subscription required but available through many public and university libraries); and Google Scholar - quite comprehensive (which means some junk, but all search engines produce questionable articles), and links to free PDF downloads if available (more than any other site of this type).
    – Mark D Worthen PsyD
    Commented Jul 26, 2017 at 4:42
  • @MarkDWorthenPsyD feel free to edit. This is a CW post so you can.
    – Tim
    Commented Jul 26, 2017 at 8:50
10

I don't thik we should start building a list of trusted sources (this is what we are doing by filling the What Sites list): will an approval process be required when later on someone will want to add an entry to this list?

Users are able to evaluate the reliability of the sources, case by case, as they do with the answer itself. At least statistically: bad source will just mean bad score.

3
  • 5
    No, the list is to help people if they are struggling to find a good site. I would accept almost any link.
    – Tim
    Commented Apr 4, 2015 at 21:09
  • 2
    @Tim Oh, I see! Sorry I didn't get it initially, I thought it was a whitelist.
    – Shlublu
    Commented Apr 4, 2015 at 22:14
  • I agree, white-listing is dangerous, but a wikilist of suggested sources is useful.
    – JorgeArtware
    Commented Apr 9, 2015 at 11:41
10

The medical research literature is rife with badly designed studies (and has a strong publication bias). This leads to, e.g. up to 70% of scientific studies not be reproduced. So even published studies are often not so reliable. One way to sort this mess would be to establish some basic criteria for assessing the quality of evidence in a medical research article.

7

I would caution that it is important to state the (mal)intentions of the "research" or "study", especially so for times when there is a conflict of interest or probable suspicion of intellectual dishonesty.

For example, if "a study to investigate the safety of bar soap was done by a major company that makes bar soap", the reader would probably decide to delve furthur, to read up on the methodology of the "research" to see if there are any possible inaccuries or biasness involved.

4

Reliable source should show a documented evidence. Nothing to say against Mayo Clinic or WebMD, but their articles usually do not have any reference links...so how do you know if what they say is evidence or not. Evidence comes from published studies, not from doctors' or institutions' opinions.

For "Is it possible?" questions, quote a case study from an online medical journal (abstracts are often available in PubMed). So, if it is documented that it has happened, it probably is possible...

For "Does it work...is it effective" questions, quote one or more systematic reviews. Often, one study will say that something works and another one that it doesn't. A systematic review at least tries to narrow down all the randomness and remove low quality studies...Most systematic reviews are available on PubMed. In a search engine type: site:gov keywords "systematic review" and it will usually lead you to PubMed.

PubMed is a library of studies, not a "website" so it should not be considered good or bad by itself.

10
  • 1
    The Mayo Clinic does indeed have references; you just have to click through for them.
    – anongoodnurse
    Commented Feb 26, 2016 at 6:48
  • 1
    @anongoodnurse Some articles on Mayo Clinic do not have any have references, e.g. mayoclinic.org/healthy-lifestyle/adult-health/expert-answers/… (mirror).
    – Franck Dernoncourt
    Commented Nov 11, 2018 at 22:14
  • @FranckDernoncourt - Not going to look. Usually barely visible, but have never seen one without. If that's true, good for you, you've won a million dollars (from them, not me. I don't really care.)
    – anongoodnurse
    Commented Nov 11, 2018 at 23:57
  • 2
    @FranckDernoncourt - There I went and fell for it. There are references. Click on "references" button on the bottom. Duh. No million dollars for you. I'll settle for all being right with my world.
    – anongoodnurse
    Commented Nov 11, 2018 at 23:59
  • @anongoodnurse Thanks, good catch, I'm glad there are some references! (would be slightly more convenient to indicate the references within the text to more quickly see what claim each reference is supposed to support)
    – Franck Dernoncourt
    Commented Nov 12, 2018 at 0:04
  • 1
    @FranckDernoncourt - That there are no references and that they are hard to find are two very different things. Just look for the word references or citations.
    – anongoodnurse
    Commented Nov 12, 2018 at 0:07
  • @anongoodnurse sure.
    – Franck Dernoncourt
    Commented Nov 12, 2018 at 0:09
  • @anongoodnurse From the 5 references, only 1 is a proper research journal article. Which makes no recommendation concerning fluoride. Also a serious article puts citations in-text next to where they are relevant, not lists them silently at the bottom hidden by default and easy to miss.
    – Dmitri Zaitsev
    Commented Jun 3, 2019 at 3:33
  • 1
    @DmitriZaitsev - Serious article? Are you familiar with the Mayo Clinic site? Please.
    – anongoodnurse
    Commented Jun 3, 2019 at 4:42
  • @anongoodnurse This is what every research article does, nothing specific to this site, no I am not familiar with it, but from what I see it does not impress...
    – Dmitri Zaitsev
    Commented Jun 3, 2019 at 7:18
0

Only secondary or tertiary review articles that are widely cited in the literature are reliable sources. Peer reviewed articles are not necessarily reliable sources to back up statements; the whole point of publishing research results is to make a relevant point in the scientific discourse about the subject under discussion. It's then not necessarily the case that what is written in the article is actually correct. In many cases it can make a point that is not widely supported in the scientific community. This is why it is better not to cite primary research articles to back up statements in an authoritative way and instead only use secondary or tertiary review articles. for that purpose.

Primary research articles can be used as examples about e.g. recent research or to make a point other than to support the reliability of your answer. This is why I'm not in favor of the requirement of backing up answers here by citing references. In many cases good answers cannot be backed up adequately. By not citing references one makes clear that the given answer may lack a rigorous scientific basis, which would then distinguishes that answer form another answer that does have such references. But that other answer isn't necessarily better, often what can be proven rigorously are weaker statements.

1
  • If you doubt about references, why not being honest to your readers and stating your reservations?
    – Dmitri Zaitsev
    Commented Jun 3, 2019 at 3:47
0

If I can add my $0.02 worth: I think we need to recognize what type of information is being cited. Certainly the listed sources have reliable medical information, but often we have a need to cite sources for pharmacology, nutrition, biology, chemistry, and maybe even physics (and probably a few others I haven't thought of). The listed sources may or may not be reliable repositories for that information, and often mundane sources like Wikipedia may.

0

The TRIP database is a very good resource to use in this case. It arranges results by hierarchy of evidence including systematic reviews, clinically-appraised topics & articles, randomized controlled trials. TRIP also cross-references various trusted medical journals and databases talked about here in other answers, including NIH's PubMed and MEDLINE indices. All of this is available via a text box and search button, so it's also very easy for users to look up keywords.

-5

If an answer by a recognized MD, maybe even specialized in the field the question is asked in, would not be accepted as an answer, then one should ask himself: Why would I go do a doctor when he doesn't cite the sources the communitywiki described for every sentence he says?

We should downvote if there are no sources, unless it is widely known (for example, the statement "If you get too cold you die" is a little unnecessary to backup. The actual temperature (35 C and less) should be linked.

To such a specialized MD, the statement "widely known" will have another meaning than to a normal user. And also note that it's this group of people who writes the articles you'd like to have as citations.

So my answer to your question would be: If there is a possibility to identify a user as a specialist in the field the question is asked in, an answer can be accepted without citations. If not, citations will be required for backing up the statements.

Please also note that decision-making in clinical environments is also not only not entirely based on evidence (as many mechanisms of treatments that evidentially help are simply unkown), but also rely on experience which is only in latter stages eventually transformed into citable articles. This is always a matter for discussion amongst professionals, but should it therefore be completely left out of this site? In that case, chances are high that not the best answer to the question, but the most citable one will end up as most upvoted (or least downvoted).

7
  • 1
    I disagree, this has been discussed and decided against, and for very good reasons: meta.health.stackexchange.com/questions/5/…. If you want to distinguish between types of user, the SE network is not the place for it.
    – rumtscho
    Commented May 18, 2015 at 9:00
  • 1
    Well, then obviously SE will not become the place to find the best answers in Healthcare, due to the reasons I mentioned above. This is not comparable to computer science, where a good answer becomes deductible to others simply by re-thinking its logic. If SE sticks to this policy, the most upvoted answers will just be the lowest common denominator in openly available information, and not the best from common, everyday practice, which itself will then remain uncomprehensible to many. In healthcare, there is still a large gap between these two.
    – cirko
    Commented May 18, 2015 at 9:20
  • 2
    Yes, the "lowest common denominator" problem is known and already appears on many SE sites. I'd love to see a better solution for it, but I don't actually think that having "recognized MD" status will help. I can't tell you the number of known misconceptions I've heard repeated by real MDs in real life. Popular =/= correct is indeed something which plagues us, but I am not aware of any way this can be resolved in a crowdsourced resource. So of course it won't be the place for the "best answers", non-crowdsourced places are better suited by design.
    – rumtscho
    Commented May 18, 2015 at 9:26
  • Well I agree with you on misconceptions by real MDs, but if discussions amongst those are not to be recognizably held here on SE, with arguments for everyone to see, then the amount of valuable output on SE Health will be very little. I would see no point in using it then, anyway (or recommending it to others), if SE doesn't at least admit the usage of books as citations for answers.
    – cirko
    Commented May 18, 2015 at 9:54
  • 1
    there must be a misunderstanding. Books certainly can be used as citations for answers. See also in the upvoted answer: this is not intended as a whitelist, using resources outside of these is OK. What we don't want to have is to give some users the status "real doctor" and from there on say that they are exempt from providing citations, or give their votes more weight, or other privileges. Apart from that, thank you for openly discussing your position: it is important for this community to know how its potential users react to its rules.
    – rumtscho
    Commented May 18, 2015 at 9:59
  • Well if that is the case, I'm fine with that. I think that any healthcare professional should be at least able to cite a book his information is based on. As for the discussion, provoking a dispute is the least I can do to help. ;)
    – cirko
    Commented May 18, 2015 at 21:25
  • Honestly, the idea that I could cite my own paper and it would pass muster but if not I have to go track down a source (often spending awhile trying to find evidence of a negative) while essentially doing someone a favor is one of the reasons I've essentially walked away from this site.
    – Fomite
    Commented May 3, 2016 at 7:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .