Site specific reason for deleting answers: Misinterpreted references

Question

Misinterpretation is a major problem

The Internet is swamped with misinformation on health matters, and we want to do better. So maybe we can pack one of the major sources at the root: laypeople hearing of legitimate studies and then jumping to conclusions. They do so in good faith, but the result is dangerous.

A fictional example: A scientist publishes a study in which he fed rats with mountains of lettuce and nothing else. The rats got digestive problems. The next day, the tabloids all cry: "Lettuce is bad for you!" A year later, nobody can dispel the notion in the public mind that lettuce is responsible for colon cancer, coeliac disease and haemorrhoids.

New policy proposal

To prevent this, my proposal: A discrepancy between what an answer claims and what its references prove should be a sufficient reason for taking action, up to and including deletion.

Implementation details

Answers deviate from references to different levels, so there is no one-size-fits-all approach. The detailed suggestion:

• If an answer is based on a reference which is clearly not applicable to the circumstances, it should be deleted. Example: an answer suggests that taking a vitamin A supplement will improve the vision of a myopic person, and uses a reference which says that lack of vitamin A causes eyesight deterioration. This is an obvious misunderstanding and deletion is appropriate.

• If an answer is based on a study which uncovers signs for an interesting interaction between some substance X and health (or intervention X, or whatever) but is not sufficient to warrant behavioral change in the public, we should expect the answer to be clear about that, and edit it if it is not. Example: answer claims that people should start fasting one day per week, citing a study in which fruit flies' life span is extended under calorie restriction. Here, the recommendation for fasting can be edited out and replaced with a text describing the situation more clearly, such as "In fruit flies, calorie restriction leads to longer life span. It is unclear if this effect is applicable to humans." The users who find out that the evidence presented is lacking should edit themselves (and in this case, the "too intrusive edits are not allowed" policy has to be superseded by the "no misinformation" policy), or flag the question with a custom reason.

• An answer can represent a minority opinion. This is the case which is hardest to deal with. We cannot know if the referenced study was written by the next Andrew Wakefield or the next Hermann Joseph Muller¹. So what do we do? Simply leaving the existing answer as it is and writing an opposing answer won't be enough, as voting won't sort the truth out in this case.

  • the non-invasive option is to leave the answer as it is. A user who is aware that the theory represented is controversial can leave a comment challenging the answer, and in the best case, write an answer explaining the controversy and link to it from a comment under the first answer (so that users won't miss the explanation).
  • the more radical option is to suspend the "no invasive editing" policy in this case too, and allow other users with more evidence to change the text such that the controversial suggestion is still present, but it is clearly called "controversial". "More evidence" means that the editing user should include a literature review finding that the first opinion is a minority opinion, or a current textbook passage which shows that the opposing opinion is accepted as true. If we don't require that, we will have fringe science fans festoon every usual statement with a "this is a disputed opinion" claim.

I cannot say which option would be better for the site, I hope to see input on it in the answers.

^{1 Andrew Wakefield was the scientist who published a study linking vaccines to autism. The study was never replicated and later shown to have been fraud, but the anti vaccination movement is still a fact. Hermann Joseph Muller is the first scientist to discover that ionizing radiation is linked to cancer - he published in an era when drugstores sold radioactive toothpaste advertised as extra teeth whitening. It took decades to change the popular opinion that radiation is safe. Both theories were controversial when they were published.}

Moderator burden

This will place burden on the moderators, but I believe it is doable. The most difficult part will be the balance act. Bad answers should be appropriately changed, without allowing pedants to police decent answers whose references don't reach the lofty standard of ultimate proof. It is a judgment call, and it requires the site to have moderators knowledgeable in both medicine and research (insofar as judging whether a reference supports a given statement is a typical researcher's task), which is more than what is needed from moderators on other SE sites. But I think this community will not have trouble attracting moderator candidates who are qualified for that - the pro tem moderators are already up to it.

One good thing about such a policy is that it does not require the moderator to be perfectly versed in the area of the answer, or know the absolute truth. Comparing answer claims with its references is not an easy task, but it is much easier than knowing which answers are right and which ones are wrong.

What this policy is not about

The intention is to stop simple misunderstanding of references, not any of the other problems which can lead to dangerously wrong answers. These other problems should be addressed in separate discussions.

• if an answer claims exactly what its references claim, then this policy should not be applicable, even if the answer is quackery to the best of the moderators' knowledge, or the answer comes from a disreputable source. For example, if an answer claims that wearing a magnet bracelet can heal rheumatoid arthritis and links to a site which explains that magnet bracelets heal arthritis. Unreliable sources are a problem separate from misunderstood sources, and should be addressed in a separate discussion.

• if an answer has no references whatsoever, this policy is not applicable. Answers without references are a problem separate from misunderstood references, and should be addressed in a separate discussion. Of course, this creates the "loophole" where a quack submits an answer without references so it can't be edited under this policy. But this is a problem we can solve in a different way. Also, I hope that people will place less faith in an answer without references when they would have been impressed by one with a reference list even if they did not check these references. So the side effect of quacks purposefully not adding bad references is a good thing.

• This policy proposal deliberately addresses answers only. Questions based on misunderstandings, premature translation of tentative findings to actionable suggestions, or controversial theories, should not be deleted or edited. If a question exhibits this problem, the goal should be to explain the problem in the answers, not change the question.

• We should not expect moderators to preemptively scan answers, read all references in detail, and make the judgment. The policy should be enforced by the users, through edits and flags, as always. This does not meet the high "first, do no harm" standard of the medical profession, but in this case, I believe this is acceptable. The existence of this policy does not mean that users can trust all answers on the site to have been fact checked for them.

---

As usually, the above is a proposal, open for voting and new suggestions. I wrote it up in an overly formal way, because I know that such Meta questions can end up being the "law" of the community, and both moderators and users appreciate having clear, actionable guidance then. But of course, it is as open to discussion as any other Meta question - don't think that my style means I'm trying to impose a directive on everybody.

Answer 1

What I agree with:

A discrepancy between what an answer claims and what its references prove should be a sufficient reason for taking action, up to and including deletion.

This would be ideal (and has been done), but is it realistic?

Many people don't read the references. Of those that do, I think a comment under the post challenging the reference would be ideal. I can see a lengthy comment war where the OP has a difficult time accepting an interpretation that's different from his. But it attracts the attention of others who might then read the reference and agree with the OP or with the commentor, and vote. Either way, that's good.

Too intrusive an edit changes the author's answer; some people can accept this graciously, others can't. An edit here would lead to a rollback war.

It has always been the policy that the person who posts has the most control over their post. I don't think editing it is the way to go; I think commenting and giving the OP an opportunity to change the answer or expand on it to reinforce his point are options. If a significant number of people down vote, the community can elect to delete. Or a moderator can, in lieu of the fact that too few 4K users are on the site.

As on other sites, a final option would be to post a better answer.

Finally, this:

The existence of this policy does not mean that users can trust all answers on the site to have been fact checked for them.

This cannot even be implied, and such a policy would imply to the reader that answers which are unchallenged are correct.

I still think that comments and downvotes are more appropriate than radical edits.

I answer here as a user who has experienced the problems even comments alone cause, and as one of the "professionals" who does not want to be held in any way responsible - directly or even indirectly - for a bad answer.