7

So this is a subject that's come up before, twice, but I'm not so sure we got to anywhere actionable. As mentioned in a comment, the fact that this community is struggling with professional involvement is a bit of an elephant in the room.

And I like to point at elephants.

I think that a first step to addressing this problem should be requiring a minimum level of research, and an attempt to answer your own question.

Explain how you encountered the problem you're trying to solve, and any difficulties that have prevented you from solving it yourself.

I honestly think that this is a bare minimum to make the site work, really any SE site, and that we are not currently enforcing it.

Thus I have 3 questions that are sequential:

  1. Should we require a basic level of research for a question to be posted?
  2. Can we agree on some guidelines for a basic level of research?
  3. Can this be a reason to close a question if a basic level of research is not reached.

For (1) I would argue that the way someone shows that they can repeat the problem and solve it themselves in health is to do a basic search.

For (2) I think this should probably be at least a look at Wikipedia and the first page of Google hits. Even if the source is some crazy wrong blog article, it would at least substantiate the belief, and often articulate the point of view more clearly than the question. I think that if there isn’t a single citation or link in the question, it is very likely to fail this requirement.

For (3) I think without the full teeth of a VTC, the above attempt is not enforceable in a way that will get a point across to new and problem users.

  • How to show some research was done when nothing was found? e.g. for Recommended size of diclofenac patch, I haven't found anything after searching a few minutes on Google and Gscholar. – Franck Dernoncourt Jan 15 '16 at 22:35
  • 1
    @FranckDernoncourt I also wanted to point out, that I assume that you are at least somewhat familiar with them given the questions you've asked here. But rather a more basic question of dosing seems to be missing. I'm always pro working to sculpt a better question, but if it was truly left that way without any edits, I would indeed want to vote to close it – Atl LED Jan 16 '16 at 3:34
  • @FranckDernoncourt I don't think that searching for a few minutes qualifies as research. If you are going to make claims that you where not able to find anything it would me that you did a decent amount of searching in multiple areas before giving up not just a quick search. – Joe W Jan 16 '16 at 16:45
  • @JoeW My previous comments got deleted, I don't feel like rewriting them. – Franck Dernoncourt Jan 16 '16 at 16:59
  • @FranckDernoncourt That may be the case but a little time searching on google for information (even if you are knowledgeable about the subject) does not count as research. – Joe W Jan 16 '16 at 17:01
  • @JoeW My previous comments were answering this question. – Franck Dernoncourt Jan 16 '16 at 17:01
  • Related from PsychologySE. Although downvoting/locking 0 research questions would drive away a large portion of users, it would increase quality. Maybe in the long run it would benefit the site. Seeing how many questions are problematic, I don't think anymore that driving away non professions would be that bad (therefor i deleted my answer). – Fermi paradox Feb 20 '16 at 9:50
4

I couldn't decide if I wanted to put this in as an edit or an answer, but after going through several questions tonight, I thought I'd throw it up here.

We only have 1 of our 3 canned off topic options occupied right now. I propose the following off topic VTC as a middle ground:

"This question is off topic because it concerns an unsourced or unsubstantiated position about health. To be considered for reopening, please provide a reference that clearly states your position."

I think that this will only solve half the problem, but at least it's the more annoying half. You will still find people unable to find the 1st Google hit that covers their questions (like the dietary recommendations of the Kidney Foundation for dialysis pts).

Edits:

In response to @JohnP mentioning this meta question, and the logic tree displayed in the answer, I thought I would propose a different logic tree for us:

Flow Diagram for closing based on lack of research

Again I think this is different from the close for general reference answers. I think our problem is users not actually trying to answer the question themselves.

For examples regarding "Are the search terms unclear," I think our current bounty question is a great example. That question would require knowledge of words I would not expect the public to know (stance instead of standing, biomechanics, and pathomechanics). Using the terms from within the question you can tell that it would be very difficult to get a good answer to the question.

However, there are plenty of questions, the one @JohnP pointed to for example, where even just DDG-ing (or Google I tried both) the question exactly as written provides an answer on the first hit or page of results.

  • I was kind of confused about the flowchart when I saw this - it seemed to put a lot of burden on the person asking the question. I totally agree that questions with clear, obvious answers are not great (though I'd like to avoid getting into the debate about downvote vs close), but if the answer is not obvious, asking the OP to summarize their research seems like an unusually high bar. For example, on health.stackexchange.com/q/4645/631 do you expect me to summarize several search results in the question? – Cascabel Feb 3 '16 at 17:05
  • I'm guess I'm wondering if at some point the flowchart should ask "is the question (completely) answered by a simple search?" It seems like that's the standard we usually unconsciously apply on most sites: if we can already tell you're not going to be able to find a reasonably complete answer by searching, why bother making you go do that search and then prove to us that you still have a question? – Cascabel Feb 3 '16 at 17:09
  • 1
    @Jefromi 1) That is EXACTLY the kind of question I want to have research or be closed. Yes, I expect you to point to any reference that goes into "different drinks can cause varying severity of hangovers." Look into phenol's if you want a hint. I really mean quite exactly what I have in the flow chart, as bare minimum to show that you attempted to answer the question your self, that it's "reproducible" [other people care], and to explain your position as exactly as possible. Same rules as SO, but research is the only way I think that can be shown on this site. – Atl LED Feb 3 '16 at 17:50
  • Okay, I told you what's in the top Google results, which talk about congeners, not phenols. Is that an improvement? I feel like you're taking the research effort thing to a bit of an extreme here, not just asking me to make an effort to see if the question is easily answered, but to answer the question myself. One of the really great things about StackExchange is that for questions with somewhat more complex answers that aren't necessarily all in one place (even if they're out there) we consolidate things into a useful single page - seems a shame to lose that. – Cascabel Feb 3 '16 at 17:55
  • And... I'm confused. I don't have a position. I'm asking what's known. I don't care whether the answer is that there's barely any difference (but X affects it) or there's a huge difference (due to X, Y, Z, and probably other things). Do I have to have a position in order to ask a question? Why do I have to demonstrate that other people care about the question? Why do I have to prove to you that I can't just find an answer with Google, when I already know I can't? Or are you trying to limit yourself only to academically interesting questions? – Cascabel Feb 3 '16 at 17:58
  • I don't think it is off, anywhere in StackExchange, to want the OP to "answer the question myself." Yes that is exactly what I am asking you to do. Show me that you tried to answer the question your self, ran into trouble somewhere along the way, and now would like help. I think that's pretty standard in SE sites like SO or .Bio. I don't know that even tried, let alone aren't able, to find the answer with out a single reference. It seems like a really low bar. – Atl LED Feb 3 '16 at 18:00
  • @Jefromi with your edit, show me a mass-media article and link to it. Let me know you read something from say CNN or BBC. Then follow up with maybe the wiki article on congeners to show me you know what they are. Again, it seems like you've done the research, so why is it an issue to include it in the question. – Atl LED Feb 3 '16 at 18:04
  • @Jefromi as further proof on my idea, if actually look at the Wiki article on relevant congeners, you will see that references 2 and 3 are quite on point to cover your question. – Atl LED Feb 3 '16 at 18:06
  • Let us continue this discussion in chat. – Cascabel Feb 3 '16 at 18:07
3

Researching a question before you post it is a very good idea. However trying to measure the amount of research that was put into a question from reading the post can be impossible depending on the amount of information found. If it is a question about something where there is not a lot of information available, either outside of certain areas or at all, a person can spend a significant amount of time and find no information.

If you are breaking down what you are asking I would have some concerns about what you are looking for.

  1. If the person doing the research spent some time ( at least a few hours ) trying to find the information but was unable to find any what do they post? Do they just describe how they tried to find the information?
  2. Guidelines for research would most likely need to be different depending on the question. A question about the common cold should have more information from research than a question about a rare or new disease.
  3. A close reason for lack of research can be hard to do as determining what is enough research is primarily opinion based. This close reason has been removed from other sites because it was being abused to close questions so I don't see it being added again.
  • 2
    1) Yes, if someone searches in earnest, then they need to clearly describe what and how they searched for the information. Often including what they found instead could be helpful. 2) I think some bare minimums can be reached. If the question doesn't have a single citation, it will need a very compelling reason to stay open in my mind. 3) Which sites was it removed from? – Atl LED Jan 18 '16 at 16:09
  • @AtlLED I know it was removed from Stack Overflow because it was a very abused close reason. – Joe W Jan 19 '16 at 0:25
  • 1
    Currently it seems like questions here abuse the lack of that requirement. I think requiring 1 reference or a compelling narrative on not being able to find information is a pretty objective line that can be drawn. I think you comment to warn the OP about the policy then move to close. – Atl LED Jan 19 '16 at 2:50
  • @AtlLED To be fair if you are asking a question does a reference really help improve the question other than to provide some hint that you tried to look it up on your own? – Joe W Jan 19 '16 at 2:52
  • 1
    Assuming you read it and it covered the same (or even similar) topic? Yes, I think it would be a drastic improvement – Atl LED Jan 19 '16 at 3:15
  • StackOverflow definitely has "primarily opinion-based" still. (I have close privileges there.) – Cascabel Feb 3 '16 at 20:49
  • @Jefromi This isn't about the primarily opinion based close reason but a close reason for lack of research. – Joe W Feb 3 '16 at 23:11
  • @JoeW My bad, misread - I didn't know there ever had been a lack of research reason on SO. (I thought that they declined to ever add it.) – Cascabel Feb 3 '16 at 23:17
3

I don't think that strict rules and close voting are a good way to enforce research effort. My big suggestions would be:

  • Close conspiracy theory questions. Probably add a custom reason for this; it's common enough.
  • Close general reference questions. Probably add a custom reason for this too.
  • Close everything else that's known bad. "Too broad" and "unclear what you're asking" cover an awful lot of ground.
  • Don't close otherwise good questions, regardless of research effort. Downvoting and asking for clarification and/or research is fine though!

I think the most important question to ask yourself when evaluating a question with respect to research effort is:

Would the OP still be asking the question if they'd done a little research?

If no, then it is indeed a bad question. I think the best thing to do in that case is downvote (after all, the site does explicitly provide this as a reason to downvote questions). If there are serious problems remaining (after dealing with conspiracy theory and general reference questions), that's where you can focus additional efforts.

If yes, then the question should be regarded as fine in terms of research effort, whether or not the OP actually has explicitly proved that they did the research. (Note that it could still be too broad, too unclear, and so on - all that is separate.) If the question can't easily be answered by non-expert research, then including a proof of that in the question doesn't actually make the question better.

The point here is to consider the merits of the actual question being asked, not just the degree to which the OP has attempted to answer it. If the question is "good", the site should be happy to have it, and if it's "bad", the site should be happy to get rid of it, all regardless of the amount of research demonstrated in the question.

I get the impression that you're trying to use research effort as a proxy for making those decisions about more specific bad categories, i.e. if the question goes away after asking for research, it must have been bad. But I think it's counterproductive to impose additional burden on good questions in order to weed out the bad ones.


To take a step back, I see that you're worried about attracting experts. That's a legitimate concern! And I agree, a good way to do that is to have really interesting questions!

However, I think there are a few disconnects between that and a "close without research effort" policy:

  • Lack of research effort isn't exactly the same as not interesting. Making strict rules here is a good way to throw away both bad and good questions, not just bad ones.

  • Making it take more work to ask questions, in general, discourages people from asking questions - the opposite of your goal.

  • Getting rid of bad questions increases the fraction of good questions, not the number of good questions. Even if you could come up with a rule that only got rid of bad questions, you still wouldn't really be making progress. (If you were taking an unmanageable 100 questions/day and pruning it to 50 good questions/day, sure. If the site gets there, revisit this! But right now the site has 7 questions/day.)

  • I think you make some very good points. There are some problems specific to this site, though, to add to the equation that changes it a bit. One is that there are too few active (and/or) high-rep users to downvote +/- delete bad questions; another is that there is an unwillingness (among most of the medical professionals here at least) to put time and effort into good answers when the OP won't put any effort into it at all. But, as I said, these are good points. – anongoodnurse Feb 3 '16 at 19:40
  • @anongoodnurse I would be interested to see what "bad" questions there are which don't fall afoul of any bad question categories besides lack of research. My aim here is to avoid using that blunt hammer, and instead look for the things that actually make those questions bad. – Cascabel Feb 3 '16 at 19:41
  • Your edit added some classes that expand the close reasons. :) Granted, if the community were willing to close (and delete, but close would be good) the kinds of questions you've highlighted, I think the rest would be interesting. I have answered questions without research that were good/interesting, and I know others have as well. We're saying we are willing to answer poorer questions (poor questions don't prevent good information from being provided in the answer) if the OP puts in a minimum of effort. – anongoodnurse Feb 3 '16 at 19:47
  • @anongoodnurse They were actually there before, I just made them more prominent - sorry for posting in a less well-organized form first. (I originally wrote the answer trying to focus on "research effort is a bad metric" rather than "here are some better metrics".) – Cascabel Feb 3 '16 at 19:49
  • 1
    My reading of the "general reference" meta posts was that it was discouraged because it was highly abused. That's one of the reasons I proposed the outline I did, as a more objective metric. I would love to be able to close for the first two reasons you outline, but there's just so much debate on every single question that it becomes tedious. This would be a nice, more objective bar to place. I agree with many of the issues/points brought up here though. – Atl LED Feb 3 '16 at 20:39
  • 1
    @AtlLED Yeah, it certainly is fuzzier. But I think this might be a case where the benefits of having it outweigh that cost? It would also be easier to avoid it being abused if it's part of the site from much earlier on, perhaps with some initial guidelines, e.g. "can it be answered in one or two sentences from one of the top Google results for an obvious query, especially a Wikipedia article?" – Cascabel Feb 3 '16 at 20:44
3

TL;DR: Don't close questions because they don't show research effort; that's the wrong tool.

I come from far far away on the Stack Exchange network, and I've only just come across this. However, there's something going on here which I need to point out.

Scroll back up to the question. Hover over the downvote button. Read the tooltip. Go on, see for yourself. It says this:

This question does not show any research effort; it is unclear or not useful

Emphasis mine, and very deliberate.

Lack of research, by itself, is not a reason to close a question. It is okay to close a question if the answer could be found as the top result of an obvious Google search - that's just lazy and shouldn't be around here. But research is somewhat subjective: someone could have done their own research, have nothing to show for it, and not be easily able to demonstrate it.

...a minimum level of research, and an attempt to answer your own question.

Why is this community here, if not to answer questions? OK, it's fair to require minimal effort to solve your own problem - a Google search or two is not a difficult task. But if someone's tried that, it's unfair to close their question because you think they haven't. Downvote it, if you must, but don't close it.

On the other hand, you have the downvote button. It's there for exactly cases like this: questions that you don't think are very high-quality, are unlikely to help anyone else, etc etc. That's the correct tool for this job.

See also: A Close Vote is not a Super-Downvote. Please don't use it as one.

2

I like the logic tree in @AtlLED’s answer and I think that we all need to follow some sort of method to enforce prior research when asking questions.

I am also an active member of Psychology.SE where, being also a science-based site, they actively insist on prior research. Now we have rebranded to Medical Sciences I think we need to follow suit.

In my answer in meta to standard comments I spoke of an off-topic close reason for this purpose which Psychology.SE use, which has subsequently been modified within the answer.

This question is not framed in the sciences of health. It is based on assumptions which are not made explicit, are not well-motivated (e.g., referenced), or are not held to be true within any of the research fields on-topic here. Please edit your question to provide more information on… (your research, why you are asking this question, what problems are you having understanding your research…)

Motivation and effort needs to be shown that the OP has tried to research their question. Answers can take a lot of time and effort in themselves, especially with complex subjects requiring a lot of referencing (which is also required here). Why should an anyone answer a question posed which shows no prior effort from the OP?

If no prior research was attempted, it can often be evident when a quick Google search immediately provides an answer. If you attempted to research but can't find anything, then provide an idea on what you searched for. We are all helpful here and will provide assistance where needed.

We are not looking for references for literally everything. Anything commonly known such as "a common cold is primarily evident with a runny nose" doesn't need referencing, but anything not as well known does. Especially when someone says "I read that...". What did you read? How can we also read it? (What are the details?)

Adding at least one link to an article or book on the subject not only provides evidence of prior research, but also gives any potential answerer an idea on what the OP has already read in order for them to not repeat that and concentrate on what is not known or understood.

I would suggest we create the close reason I mentioned for us to use and maybe in the meantime we use the "other" close reason and put the reason in there.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .