5

We think site introspection is a good idea once in a while and there's already been a lot of it here on Health. But now that there are a bunch of real questions from real users floating around, I think it's about time to look at a random sample to really dig into specific problems, concerns and (perhaps) successes.

Below, I've listed 5 questions chosen at random from the last quarter that have received at least some anonymous feedback. I also included two scores*:

  1. How helpful the page seemed to regular users.
  2. How helpful the page seemed to anonymous visitors.

The range is from 0 to 100% where bigger numbers mean more helpful than small numbers. The final number is the number of total views the question had received at the time of sampling.

If you'd like to participate, please copy the question list into an answer for evaluation. Exactly how you evaluate the questions and answers is up to you, but it would be useful to:

  • Find better answers on the internet (if they exist).

    Exactly what "better" means is up to you, but the criteria might include:

    • correctness,
    • understandability,
    • authoritative (e.g. citing official sources),
    • clean formatting, or
    • easier to find with a search.
  • Figure out why regular users and anonymous visitors have different opinions of the usefulness of the page.

    The most common reason for low visitor scores is low views, which limits the number of anonymous users who provide feedback. It's not a problem if some questions are so niche that nobody outside of the community reads them. But chance visitors who find answers via search are the best source of new contributors.

    More interesting: sometimes visitors disagree with the site's community about the usefulness of a question and its answers. Obviously we can't know why people who can't leave comments might differ from current users, but we can make an educated guess.

  • Fix any problems you notice and describe your changes.

    After doing the above analysis, you will be in good shape to edit titles and tags to help Google direct searchers to the question, clean up formatting and grammar problems, and link to authoritative sources. There's no need to wait; go edit.

The goal is to use the sample to reflect on how the site is progressing in terms of "making the internet a better place for people to get answers to their health questions". If this site already does a good job, that's wonderful. If there are some things that need fixing, please talk about that. In either case, this evaluation mostly stands to improve this community's understanding of itself.


Key: link (registered voters; anonymous voters; views)
  1. Does fasting inhibit healing? (61%; 0%; 33)
  2. I have painful/weak knees - which practitioner is best to see? (74%; 51%; 1,055)
  3. Should all the stuff be thrown away when moving out from house with mold? (61%; 21%; 50)
  4. Can I continue to eat a cereal my cat ate out of? (70%; 21%; 75)
  5. What exactly is a diuretic and its relationship to dehydration (51%; 34%; 29)

Footnote:

* The scores are actually the lower bound of Wilson score confidence interval for a Bernoulli parameter, which is a measure of how likely a given set of upvotes and downvotes is overall positive. For the first score, I used the sum of all votes on the question and all answers. For the second score, I used anonymous feedback. Please see the query I used for implementation details.

  • Can the views be differentiated between logged in users of the site, SE users and (one-time) unknown visitors? – LаngLаngС Sep 16 '17 at 11:47
  • @LangLangC Not really. However, only the second has meaningful views. That suggests it was seen by outside users and the others mostly weren't. I had to lower the anonymous feedback threshold to just one vote in order to get 5 questions. – Jon Ericson Sep 16 '17 at 18:17
  • Somehow, this question + answer of mine (which is only 2 hours old) has received >400 views. That is somewhere near 40 times the views other answer accumulate over multiple days. Would you mind including this in your self evaluation as well? I'd be interested where all that attention comes from, and it could teach us a lot about what anonymous users find interesting on health – Narusan Sep 20 '17 at 19:42
  • 1
    @Narusan-in-coma they come from HNQ (Hot Network Questions) on the right side of every site. – Andrew T. Sep 20 '17 at 19:46
  • @AndrewT. oh wow, that's the first time I have ever seen a question of Health.SE on the HNQ list. That explains a lot. Didn't expect that to happen.... – Narusan Sep 20 '17 at 19:47
  • You can also see the current set here. Indeed, your question is on the list. – Jon Ericson Sep 20 '17 at 19:47
3

I'm going to go ahead and answer this as an outsider. As a preface, I'd like to ask you to read FAQ: Welcome to New Users on the Skeptics site. As some of you might know, that site has very successfully navigated around the difficulties hosting a site that deals in a wide variety of topics that have dubious claims. In fact, their topic tag is [medical-claims] which could be ontopic here too. It should be noted that the site does include users with specialized knowledge, but is structured in a way that does not require expertise. I think this is smart for a site which covers all sorts of topic, but might not be ideal for a site, like this one, with a somewhat narrower focus.


1. Does fasting inhibit healing? (61%; 0%; 33)

As is often the case, the problems with this question start from the question itself. The asker mentions "in all my research", but doesn't actually list any research or indicate where the idea came from in the first place. As a result, the extant answer is simultaneously broad and has to make guesses about the specific situation. There's a "citation needed" post notice under that answer, which is resonable. But since there are no other answers, this isn't a great page for people to land on. The one anonymous feedback vote was negative, which reflects the unhelpfulness of the content.

This is unfortunate because a Google search on the topic shows there's a lot of content available from sorta sketchy-looking websites. This would be a great opportunity for someone who is interested in reading the literature to find some research on the topic. (Google Scholar turns up many results, but you'd probably want to narrow it down by using the correct nedical jargon.) The silver lining is almost nobody has seen this content.

If I were to attempt to fix the problem, I'd probably close the question and follow through on deleting the answer. Alternatively, I could see someone writing a comprehensive answer on the interactions of fasting and healing that would rescue the question. Again, this probably is a matter of finding, quoting, summarizing and referencing existing research. Probably this can be done by anyone with a familiarity with the technical language.

2. I have painful/weak knees - which practitioner is best to see? (74%; 51%; 1,055)

This is the only page seen by a significant number of people. The feedback has been mostly positive other than an antidotal answer that's been downvoted. Looking around the internet, there are a number of general articles about who someone with runner's knee might visit. So it's handy to have something more specific.

I'd say the question is maybe a bit on the vague side, which is why the top answer needs to disclaim the second half of the answer. That portion also reads like it was copied from somewhere ("normally the information below isn’t given to patients"). That's fine as long as it's properly referenced.

The answers also haves some comments that aren't relevant anymore. I'd probably want to clean those up. But otherwise, this seems like a good question with useful answers.

3. Should all the stuff be thrown away when moving out from a house with mold? (61%; 21%; 50)

It's not hard to find definitive information about mold cleanup and this is a fairly general question. Both of the answers are useful. The top-voted one is a digest of a CDC page, which is probably the best sort of answer. The other answer is anecdotal, which can be unhelpful. In this case, I think it's useful prospective. Personally I find a lot news coverage of mold problems (typically after flooding events) to be overly dramatic. The two answers together make a great pairing of expert advice and practical experience. (But as always, the well-referenced answer is prefered.)

4. Can I continue to eat a cereal my cat ate out of? (70%; 21%; 75)

I'm typing this at my computer with my cereal behind me on a tall stool.

Um. Ok. This doesn't seem like a very helpful question at all. I downvoted it and I'd probably vote to close if I were a regular here. The answer is commonsensical enough. But the question is just specific in all the wrong ways. (It's also migrated from Pets, for some reason. That doesn't do it any favors.)

5. What exactly is a diuretic and its relationship to dehydration (51%; 34%; 29)

I like the question (except it forgot to actually link to the BBC article) and the answer. The one quibble I have is that the comment about whether drinking coffee, tea or alcohol increases dehydration somehow hasn't been addressed in the answer.

Unfortunately, this question is already well covered on the internet a Google search turns up:

Diuretic dehydration

Conclusion

I was on the lookout for dangerous answers or questions with the potential to attract them. And I didn't really find much. Granted, I didn't find a lot to recommend about 4 of the 5 questions and I had some concerns about the 5th. All of my concerns are the fairly typical problems you might find in a site that could use more community moderation.

Still plenty of work to be done to make this an excellent site, but I agree with Robert's sentiment:

On a site like this, folks are allowed to talk about this strange and mysterious world of "medical health." Folks will be discussing how we can live longer, stay healthy, fix problems, and how all this stuff works. Our hope in creating a Stack Exchange site like this is that we can separate the good information from the mis-information.

3

OK, here goes. But please keep in mind I'm prickly when it comes to this site. And I've never done anything like this before, so I don't know if I'm doing it correctly.

  1. Does fasting inhibit healing? (61%; 0%; 33)

I like this question, and don't think the OP's lack of cited research is a problem (it would be on Skeptics.) I like it because of the popularity of fasting these days and the importance of nutrition in healing. I would do a tag edit to include diet, healing, and nutrition.

The answer in comment should be removed because, well, the policy is no answers in comments, and there's no sources cited.

The answer should have been removed not only because of the lack of sources, but it's also pseudo-y, meaning it really doesn't address the issue but throws out a lot of impressive sounding factoids.. like "excessive protein consumption can drive cancerous processes in the body." Huh? How does this relate?

The top Google hit sounds good, but it's a small study, wound healing was carried out while refeeding after four cycles of feeding. So, very, very 'meh'. No wonder it was cited all of 6 times since 2011.

The second sciency-looking hit, Factors Affecting Wound Healing, is a really nice paper, and even answers the question. It's strange to find in a dental journal, but hey, they inflict a lot of wounds. But there's a lot of reading to do if you don't scroll down to the right section. The fact that it was cited 1423 times says something, but bad papers have been cited often, too.

The rest are bloggy, wrong, or don't support with the literature.

The "fix" (?) is to tag correctly, remove comment and answer, and hope someone who can interpret the literature feels strongly enough to answer. I like this topic, so I would have when I was active.

The best source on googling (looked for an easy, quick, etc. answer; no go.) is the paper I liked above.

  1. I have painful/weak knees - which practitioner is best to see? (74%; 51%; 1,055)

I like the question, even though knees are tricky for me. It's a good question.

The upvoted answer is full of information but lacking in good answers (imo). The user assumes the most likely dx is Chondromalacia Patella, but, though it's an extremely common source of knee pain, it is not usually accompanied by weakness. It's really lacking in that it does not recommend anyone but a Physical therapist, playing down the orthopedist's role. In my state, a PT can't order an MRI, and with weakness/knee instability (people with instability often describe it as weakness because they're not sure the knee will hold in certain movements), this patient needs an MRI.

The answer isn't copypasta, so needs no blockquote/does not need to be excised.

Disclaimer/ethical disclosure: I'm a doctor, therefore rely a lot on tests to help confirm a diagnosis. However I'm not an internist (they order "serum porcelain levels", in other words, everything including the kitchen sink); in my specialty, we rely less on tests. If I order a test, it's because I think the proper diagnosis can't be made without one.

Can a better answer be found on Googling? I liked my first pick among the first hits better. Disclaimer: my training was in primary care, and they agree an MRI is necessary. Heh.

Also, a PCP will know the better orthopods in the area, so will avoid sending you to a money-hungry one.

Dr.Whom's answer is much better than the fancy accepted answer.

Why is the DV'ed answer still there even?

The "fix": - ask for clarification in comments (where does it hurt? Do you feel weak/unstable? What's it like first thing in the am? etc. But that would make it more personal.
-DV and comment on the top-voted answer
-UV and comment on the far better answer
-remove the DV'ed answer.
-Have more experts on the site who can evaluate the answers for quality. and finally,
-Have more experts on the site who can evaluate the answers for quality.

  1. Should all the stuff be thrown away when moving out from house with mold? (61%; 21%; 50)

Dr.Whom's answer excellent. Googling first hit that wasn't an Ad was CDC site. So great!

Did anyone notice the tag: stachybotrys-black-mold? It matters, but only a little bit. Some people actually die from this particular mold. So remove the tag or address the problem.

Fix: remove the answer without citations.

  1. Can I continue to eat a cereal my cat ate out of? (70%; 21%; 75)

Umm... I don't know what to say here. Downvote and delete?

  1. What exactly is a diuretic and its relationship to dehydration (51%; 34%; 29)

The question makes two faulty assumptions (the last two box quotes) and should be edited to simply, what does a diuretic do? (And experts should know to do this.)

It's a common question, and very commonly misunderstood. While @CareyGregory's answer was correct, it did not address the common misperception that alcohol, caffeine, chocolate, and other diuretics found in food really do not dehydrate you to any significant extent. (The body's regulation of water is one of the tightest and most efficient because of the dangers of fluctuations in serum sodium. If drinking five cups of coffee made you urinate more water than you drank/ate, there would be less of a population problem.)

Googling? When I google off scholar and the Mayo Clinic comes up, that's my first choice. See @JonEricson's answer.

The fix: Remove the two erroneous block quotes, and address not only what a diuretic does, but also what it doesn't do.


Are there any answers that can actively hurt someone on this site?

YES

This answer got six upvotes before I felt compelled to answer it myself and accept my own answer to undo the harm. How embarrassing.

Even worse is that I brought it up to the mods. It only got 1 DV.

Does that say something about mod participation here?

That's just one off the top of my head.

I hate to say this for many reasons, not the least of which is because I was asked to step down as a moderator from this site (which kinda questions the reliability of my observations), but:

If this site doesn't get different moderators, nothing will change.

I'm not saying I want the role. I couldn't distance myself then, and I don't want to try now.

  • I agree with everything you said. However, a mod change might not be necessary. I think a change in the ideal of moderation would suffice. Mods are taught to be as little involved as possible, and Health right now needs rigorous moderation. Besides that, who could become a mod? DoctorWhom would be a perfect pick, but she'll be quite busy the next few months so I'm not sure whether she wants the extra work, I'm not experienced enough with SE, not knowledgeable enough and ultimately, just don't want the job. Carey might be a good fit, if he wants the extra responsibility. Mike and LangLangC are – Narusan Sep 20 '17 at 19:54
  • [cont'd] too new and haven't has access to the review queues until very recently, so I doubt they'd want the job. We're stuck with the mods we have. And I do appreciate the support I've received in the past few weeks. They could be a lot more aggressive though. Whenever you have time and want to talk about that, you could tell me in chat why you resigned. That might also give me a better understanding of the "old" Health.SE – Narusan Sep 20 '17 at 19:56
  • In relation to the answer you called out, why isn't there a comment explaining the danger? I've read both answers and it seems like yours is a bit more strident that people should be vaccinated after a bat encounter. Did the answerer misinterpret or misrepresent the CDC link's information? Or did they not dig deep enough? In any case, it's hard to see why the moderators should have acted when there are no flags and no comments. – Jon Ericson Sep 20 '17 at 20:04
  • Thank you for your analysis of the 5 example questions. It's helpful to know your thoughts and I largely agree with them. (I don't know much about the actual medical information, but your explanations make a lot of sense.) – Jon Ericson Sep 20 '17 at 20:11
  • @JonEricson - I communicated the problem to the mods in the mod chat room. Only one mod took the problem seriously. The answerer did not dig deeply enough. If my answer was strident, I apologize. I wanted it to be very plain that any bat encounter needs - absolutely, not "it can't hurt" - to be prophyllaxed. – anongoodnurse Sep 20 '17 at 20:12
  • @Narusan-in-coma - The present mods are burnt out (?) and can't or won't moderate differently. – anongoodnurse Sep 21 '17 at 3:34
  • @anongoodnurse My question remains: Who should be a new mod – Narusan Sep 21 '17 at 5:35
  • @Narusan-in-coma - I have not been active on the site for over a year; I can't make any recommendations. I only know so much. – anongoodnurse Sep 21 '17 at 14:12
  • @anongoodnurse It was rather a rhetorical question. If DoctorWhom isn‘t willing to do it, we basically have no one that would be suited. – Narusan Sep 21 '17 at 14:13
  • 1
    @Narusan-in-coma - And I'm saying, you can't improve the site with the present moderators. You are excited about rebooting the site. I'm just saying, rethink this; nothing will change if the moderation does not. Moderation is critical when there aren't enough hi-rep users to do the work. And here, there are not. Even if you were to get CMs to take over (which most likely won't happen), their lack of knowledge of the field would hurt the site, as it would be run on the SE model, with which Health is not a good fit; is Medical Sciences? You need to ask yourselves that. – anongoodnurse Sep 21 '17 at 14:21
  • @anongoodnurse I see your point. I don’t know whether MedicalSciences would be a better fit. I am certain it’s not a worse fit, this is why I want to give it a shot. But I‘ll think about what you have said. – Narusan Sep 21 '17 at 14:25
  • @Narusan-in-coma I would be more than happy to moderate, actually - I am just uncertain what my availability will be. Sometimes I can do a lot, sometimes a little, sometimes I will be gone for a week or more. I want to be conscientious in my commitments, and cannot commit to being here daily, for example. But I can some days. Therefore if we have a LIMIT on the # of mods, I'm not sure if I'm the ideal candidate, since maybe another candidate will be online more. BUT if there is not a limit, I would be HAPPY to mod when I can be here. – DoctorWhom Sep 24 '17 at 1:05
  • Also, you and @anongoodnurse make a very good point - there aren't other physicians (or other medical professionals for that matter) actively modding this Health site - so I have to agree that my intermittent participation would be better than not adding any new mods. – DoctorWhom Sep 24 '17 at 1:19
  • 3
    I commented in chat that I "can't conscientiously add another responsibility by agreeing to regularly participate." and that I "can't take on a responsibility to REGULARLY monitor the site," and with my uncertain schedule I need to come/go when I need to, or it would become a burden rather than a labor of love. But I also said that I'd happily serve as a mod when I am able to be on. I just don't know the rules of being a mod. Would that be okay, a mod that comes/goes based on schedule availability? – DoctorWhom Sep 24 '17 at 1:23
  • 2
    @DoctorWhom - Mods are volunteers. Yes, you can come and go as you please/are able. But your availability should (as you would, I'm sure) be explained up front. They might add an extra mod then. – anongoodnurse Sep 24 '17 at 1:47
2

While the site evaluation is a good starting point, it omits a few problems because of the selection of posts.

A data query returned that 25% of the posts posted in August were either deleted, closed or had a net score < 0. That's like a quarter.

Why is this the case? From my experience, roughly 60% of those bad posts are questions we can't answer. I think the site name plays a role there, one would expect questions about one's health to be on-topic on a site called "Health", but those questions are not on topic. There is a misconception about the site's purpose. I believe that simply renaming the site into something like "MedicalSciences" would solve this issue.

The other 40% of posts are answers that are either wrong, based on experience, not facts, or without sources. Again, this is largely a misconception due to the site's name. Everybody has health, and everybody has experience regarding health. But we humans differ so much that experience isn't helpful. Furthermore, everybody believes to know something about health (which often turns out as fairytale). Again, a site name would help here as well.

This core issue leads to other symptoms: Veteran users burn out (25% of bad posts is quite a task to moderate, both for the Community and the Moderation Team), and good posts suffocate in the mess of mediocre and bad posts.

From the 5 example questions, I found the "diuretic and dehydration" question the only question I'd upvote, the "which practitioner to see" question valid, but with a bit of research by the OP unnecessary, and I have downvoted all other questions.

Here again, we have a manifestation of a very few good questions (albeit being already covered), and a few questions tangential to the scope.

And this list doesn't even include the 25% I was talking about above.

  • I leave out the clearly bad questions because they aren't very interesting when it comes to self-evaluation. Every site has some percentage of unanswerable and unhelpful questions. This site has more than most and that's certainly something important to consider. But the purpose of this exercise to identify what works and what doesn't in a sample of questions the community has deemed good enough to upvote. – Jon Ericson Sep 20 '17 at 21:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .