8

I have developed a query to determine a SE site's "Welcoming Index", which is a measure of how well we treat new users compared to regulars.

http://data.stackexchange.com/health/query/328147/welcoming-index

Running the query shows that new users receive twice as many down-votes than regular users, relative to up-votes received. (an evil index of 1.8).

Could this trend be potentially driving away the new users who we need away from this site?

What can we do to increase our "Welcoming Index"?

14
  • 1
    Of possible interest: what are the most effective ways to guide new users?
    – Monica Cellio
    Commented Jun 22, 2015 at 3:40
  • @MonicaCellio - great resource!
    – anongoodnurse
    Commented Jun 22, 2015 at 5:14
  • 1
    How do these numbers compare to other SE sites?
    – Garrett
    Commented Jun 27, 2015 at 2:59
  • @Garrett, not sure but you can run the query for the other sites and see. Personally I'm not convinced other sites are friendly to new users either so I'm not sure if they set a good bench mark to compare to.
    – Kenshin
    Commented Jun 27, 2015 at 7:25
  • can you give the definitions of new user here?
    – Ooker
    Commented Aug 6, 2015 at 4:20
  • @Ooker, No I will not. I have no strict criteria for new.
    – Kenshin
    Commented Aug 6, 2015 at 7:55
  • @NinjaDoc why not?
    – Ooker
    Commented Aug 6, 2015 at 7:55
  • @Ooker, I don't want to arbitrarily restrict the scope of my question because any definition of new user will be arbitrary. However in the SQL code referenced, new is defined as an account less than 30 days old.
    – Kenshin
    Commented Aug 6, 2015 at 7:56
  • 1
    @Ooker, the ideal answer should have a low sensitivity to various definitions of new.
    – Kenshin
    Commented Aug 6, 2015 at 7:57
  • @NinjaDoc nice say
    – Ooker
    Commented Aug 6, 2015 at 7:58
  • If you don't mind doing the leg work, I'd love to know how things compare on other SE sites. Even the big 3, not just where we have a lot of cross over (like Bio). If this is going to be a meaningful metric at all, it needs to be understood in context.
    – Atl LED
    Commented Apr 13, 2016 at 13:39
  • @AtlLED, the query can be run on any site. I've just run it on stack overflow and they have an evil index of 6+!
    – Kenshin
    Commented Apr 14, 2016 at 8:38
  • So by this metric, they would be more harsh than we are? That seems like a good reason to think that we are not , or that this is not a good metric.
    – Atl LED
    Commented Apr 14, 2016 at 13:10
  • @AtlLED, 8% of all votes on overflow are downvotes, while 10% of all votes on health SE are downvotes. 50% of all votes for new users (joined in last 30 days) on overflow are down, while 20% for health. So we are harsher than overflow, but overflow is harsher on new users than us.
    – Kenshin
    Commented Apr 15, 2016 at 2:53

3 Answers 3

Reset to default
2

I think it's a cultural issue. A clique forms with the initial users and the subsequent users follow the culture developed by the early users. It becomes self propagating, and when someone new posts outside those cultural norms, they get down voted. Many new users will leave. Those rules may be also unwritten so that a new user will not know what they are doing incorrectly. So, your post maybe entirely within topic for the group in question, but it may have violated some unwritten rule. It's entirely human, natural and predictable because users/moderators are human. Perhaps it could be circumvented a little if votes were cast like those for closing a question where a reason has to be selected from a formal list.

I saw one user who got down voted or criticized for answering correctly a basic biochemistry answer. He was told he needed to provide a reference ( must be a rule specific to health). Was he supposed to link to page number and edition of a basic biochemistry text which is unlikely to be online? I see he hasn't come back.

So, at least from this example I see health as inimical to knowledgeable users who don't wish to play the game, and moderated by other users with far less knowledge then some new posters.

6
  • 1
    Quite a few users, including some medical professionals, have left due to this.
    – Franck Dernoncourt
    Commented Apr 13, 2016 at 0:56
  • 3
    @FranckDernoncourt FWIW, as a medical professional, the voting had nothing to do with my discouragement, nor the 10+ colleagues I showed this to. Rather it was the allowable low question quality, and having to sift through to find good questions. I accept that the community doesn't want a high bar on questions (gathered from meta post on the topic), but that is what will contribute to me answering less. I have a hard time believing that real health professionals will have difficulty citing sources, the one basic requirement we have.
    – Atl LED
    Commented Apr 13, 2016 at 13:57
  • 2
    There are some assertions in this answer that are not entirely true. Firstly, the rule on references is most certainly not unwritten, since there is extensive discussion on meta about it. Secondly, the SE model is such that the users who participate in forming the site, participate in forming the rules before the site graduates. This can hardly be called a clique dynamic, since it is the same for all 153 SE websites. Thirdly, the question is about down votes and anyone with rep of 125 can do this - one can be a new user with 125 rep. [cont]
    – Lucky
    Commented Apr 18, 2016 at 0:45
  • 1
    Ultimately, I fail to see the logic of your claim that someone who wants to help enforce the rules which are created for a reason, has far less knowledge than the poster who doesn't obey them. Sometimes, when users prompt the poster to add references, they don't do it for their own sake, but for the sake of users who may not be from the field. And frankly, joining a community and choosing to ignore the rules because one doesn't wish to play the game is rude and somewhat disdainful.
    – Lucky
    Commented Apr 18, 2016 at 0:57
  • The clique observation applies to many stackexchange sites. It's not a new observation. There is always going to be someone more knowledgeable then yourself on a particular subject. Someone might spend some considerable time on an answer to get it downvoted without a reason. Now, that is rude.
    – Graham Chiu
    Commented Apr 18, 2016 at 1:33
  • @Lucky Some of us didn't "Join a community then choose to ignore the rules". We started the community then left when the rules developed in a different direction. There's a substantial difference between those two. My own departure is a mix of Franck and Atl's reasoning. But that does mean that there's not a subset of people available to be welcoming to that particular genre of new user.
    – Fomite
    Commented Jul 7, 2016 at 6:04
10

I will give the same answer (with differences dependant on site) that I gave to this exact same question posted at the same time (using the exact same terminology) on Biology.SE:

There is an inherent bias in your query. Of course some new users are going to get down votes. Some get many downvotes (does your query differentiate that?) while others will get only one (or none).

You're comparing questions asked by new users, which are sometimes very poor (or even trollish), and as such, they receive multiple down votes.

Seasoned users (though this is a much newer site than Health.SE) know what fits on this site, and as such, will not ask as many inappropriate questions.

I don't think down votes are given superfluously, and I don't think they are excessive here. They are part of the SE model, which many people find intrinsically unfriendly. * They are an integral part of teaching all users what is considered appropriate aor of poor quality on every site.

To answer your question, if you base "friendliness of a site" on down votes on new users, then no, Health.SE is not too harsh on new users. You need to look at the whole picture. Were helpful comments left? Are edits done to new users' questions to try to salvage them where possible? Are migrations (to better sites) suggested? This counts as much - if not more - towards the friendliness of the site.

*The fact that this same question was asked on two SE sites serves to emphasize that the SE model is one that is seen as intrinsically unfriendly by many. The fact that this was asked on such such similar sites reinforces this further.

14
  • 2
    I think that it is also important to note that our "evil index" is much lower than that of other more established SE sites. I don't believe that "evil index" is really a stat that can be accurately measured.
    – michaelpri
    Commented Jun 21, 2015 at 19:50
  • Many companies are presently being investigated by a number of government agencies, law enforcement and watchdog groups for "online endorsements", writing fake reviews, creating fake accounts, writing fake comments on blogs, etc. How vulnerable is SE to this? It's hardly beyond the realm of imagination that "consultants" from the food industry would defend fast food, write answers critical of nutrients, defend chemical additives, downplay cholesterol and trans-fats, etc. How can anyone assure me that it's not taking place here? Frankly I'm appalled by users who defend fast food here, for one.
    – Shon
    Commented Jun 22, 2015 at 19:51
  • 1
    @sss4r - I'm not sure I understand this comment. Just like the poor, we will always have the dishonest among us, but we do the best we can under the circumstances, which is pretty much how most everyone else gets on. I wouldn't worry overly much about it. We endorse solid endorse conflict-free sites as sources, not blogs, and especially not for-profit blogs. Bad answers get down voted. Good answers rise, eventually.
    – anongoodnurse
    Commented Jun 22, 2015 at 23:44
  • Then please let me try to explain further, I think it's important for moderators to consider this. If it's a real problem that corporations pay people to endorse their products online, such as writing fake reviews, advertising products, creating fake accounts, drowning out complaints by posting fake rebukes, etc.--what's the stop a pharmaceutical company, for example, from paying a dozen people to create a dozen fake accounts each and polluting questions with biased answers? 10 answers to a health question by the same person using fake accounts to say, Ask your doc about drug X!
    – Shon
    Commented Jun 23, 2015 at 0:11
  • But you're probably right, it's probably not a problem, it's simply something many companies are actually doing -- US and UK authorities are joining efforts to combat this -- so it's something that moderators should at least be aware of and on the lookout for and aim to recognize such patterns.
    – Shon
    Commented Jun 23, 2015 at 0:13
  • I've seen a number of answers/comments on Health.SE that defend fast food, ridicule alternative medicine, take an antagonistic position towards nutrition, etc. and I just have to wonder what motivates an individual to do that on a health site, it's the kind of views I'd expect someone from the drug and food industries to advocate. In truth we have no idea who contributes content to this site. I'm not trying to be alarmist, I'm simply asking about a minor concern.
    – Shon
    Commented Jun 23, 2015 at 0:19
  • @sss4r - Ah. That is why you have the option to vote, comment, flag, make meta posts, and also, to post in chat if you feel a post is really off the mark. Though it's not done much on this site, I've seen it occasionally (and done it once or twice myself) on other sites. Use these options freely.
    – anongoodnurse
    Commented Jun 23, 2015 at 0:24
  • @anongoodnurse Very good, thank you, I'm just looking out for the users :-)
    – Shon
    Commented Jun 23, 2015 at 0:34
  • @sss4r I understand your concern, but I think @ anongoodnurse is right, you can counter any answer or comment that you disagree with by posting your own (or flagging it if it's harmful). Your concerns cast a wide net; it would be impossible to make a universal rule to cover it all. E.g. someone who disagrees with alternative medicine doesn't have to be on a certain company's payroll - they just might be a health professional who learnt and did things differently throughout their entire studies and career - it doesn't have to involve hidden/malicious intent. It has to be accessed case by case
    – Lucky
    Commented Jun 26, 2015 at 9:57
  • 3
    I totally agree with this answer. I'll add that I'd prefer a site that's a bit harsh on new users than a site like Yahoo Answers which has very low quality content, probably due to its low barriers to entry for newcomers.
    – Garrett
    Commented Jun 27, 2015 at 3:03
  • 2
    new user (to health) commenting here. Down votes to Qs and As by new users will be seen as hostile because new users don't know any better. If there isn't an accompanying comment on why the user got downvoted, it is hostile. Please don't "drive by downvote". While we all would love for new users to read the help center, they don't. So a blurb explaining what a downvote is for new users would also be nice. I've seen this policy enforced on other sites quite successfully.
    – anon
    Commented Jun 30, 2015 at 14:39
  • @ton.yeung - We quite often leave comments. It mostly doesn't matter. The SE model isn't for everybody.
    – anongoodnurse
    Commented Jun 30, 2015 at 16:42
  • 1
    not saying you guys don't. Just suggesting to anyone who doesn't already do so to keep it in mind.
    – anon
    Commented Jun 30, 2015 at 16:45
  • The SE model has its good spots and it's flaws. However, it's good to note that a lot of people I've seen on the site recently new and old have been ignoring it or have shown upset about it. Not that that makes it a problematic model, it's just a note and caution.
    – Pobrecita
    Commented Apr 12, 2016 at 17:43
0

As a new user of less than 2 days, here's my honest opinion.

I gave 4 answers, with 2 accepted (50 points), combined with negative ratings of -13 at one point. I quickly learned that answering horrible worded questions will lead to negative votes. The two questions that I got me a combined score of -13, were down voted to oblivion as well.

That being said, I deleted my "bad answers" to "bad questions" and eventually recovered to 45 points. The other -5 points were for not providing "references" and for something I posted? Not really sure because the person who down voted never explained nor responded, just a down vote. Now, here's the problem, new users who do not understand the SE model will be bombarded with negative points. It's bad enough to not have any "privileges" such as editing a question (1000 points) to make it sound better, new users are not able to comment (50 points) and ask the OP to clarify question.

I can understand the importance of using references as to prevent psuedoscience and broscience from infiltrating this place, yet, sometimes this place feels more like the Ivory Tower where bureaucracy and hierarchy is important. Those with more "privileges" have more "rights" and are more "authorized" than others. Now think about this. How can someone recover if they post a bad question and receive negative votes as their first interaction with the site? They won't even be able to answer a question. The point I want to make with references is this. Do we just want references or can this place be a place for some sort of discussion, in the comments section of course (not as an answer).

Is this SE limited to only Medical Doctors, or can researchers, phDs, paramedics, also participate? Can we give personal experience if we are health care professionals? What about personal experience for patients? I think this site can be much better if there is an active participation of both patients, regular folks, as well as health care professionals. Yet, I feel as if the rules of this site is rigid and exclusive. Anyone other than healthcare professionals are unqualified to answer, even those who are healthcare professionals are being down voted.

Here is an example of over zealous down voting, Which medical specialist should be consulted for anus polyp? I got two down votes for my answer, do we need to provide references for which doctors to consult? I later clarified the answer, as I feel this question can be improved on (obviously as a new user, Im not qualified to improve answers)

4
  • 1
    The reason one needs a certain amount of points (i.e. reputation) across SE for certain actions (editing, commenting etc.) is so that those actions are not misused. Reputation is gained by participation - both asking and answering good questions, through which one "learns the ropes". It is the experience with the SE network that is valued and necessary to access some actions. It is always a good idea when someone is new to check if there are any rules - on SE that would be taking the tour or visiting the help center. On health SE we have great contributors who are not healthcare professionals.
    – Lucky
    Commented Sep 28, 2016 at 13:15
  • [cont.] However, a healthcare professional or not, references are necessary and you yourself have recognised this. Personal experience is not acceptable without the references. You do make a good point on recommendations which type of specialist to see for certain conditions - I am not sure that it requires references. It would be a great separate question, and I encourage you to ask it here on meta. (Another hint on SE policies: if you have a question you should post it as a new question, not include it in an answer to another question). I hope this helps and that you won't feel unwelcome.
    – Lucky
    Commented Sep 28, 2016 at 13:18
  • @Lucky Thanks for the response. It's definitely an eye opener to the world of the Internet. I work as an Intern in the hospital and even though we have an MD degree and are licensed, I know that the "public" does not view us as full self-practicing doctors. Yet, this SE is definitely different from what I or what other doctors experience in the real world, where people take our advice as authoritative. While on this SE, there are users (I looked up their profile and they claimed to be IT managers) questioning my answers.
    – 辛祐賦 hsinyofu
    Commented Sep 28, 2016 at 19:49
  • It's definitely eye rolling if you understand what I mean, but come to think of it, no wonder why they say doctors have huge egos. This sites style is also quite different from what we are used to in the hospital, that is, medical discussion. Ill admit this is my own fault for being too optimistic and thinking that "Health Stack Exchange is for educational purposes", educational purposes as a good medical discussion. I just realized that the site still adheres to its SE roots, which are concise answers to save people's time. A stark contrast from expectations to reality.
    – 辛祐賦 hsinyofu
    Commented Sep 28, 2016 at 19:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .