Arguello-et-al-2006
Arguello, Jaime, & Butler, Brian, & Joyce, Elisabeth, & Kraut, Robert, & Ling, Kimberly S., & Rose, Carolyn, & Wang, Xiaoqing (2006). Talk to Me: Foundations for Successful Individual-Group Interactions in Online Communities. In Proceedings of Human-Computer Interactions.
We and Me: A Test in Language Usage
Hmm, first to post again. I guess I'm more anxious than others to get to "Spring" Break ;-)
This was an incredibly interesting article detailing the study of over 6000+ messages from a year on various UseNet groups. The "Abstract" on the first page does an excellent job summarizing the authors' findings so I won't repeat that here, instead I'd just like to highlight a few of the points that I thought were most interesting and applicable to our site(s). I thought it was interesting that this article was housed in this week's topic of "Getting People to Stay," because it just as easily could have been beneficial in next week's "Socializing Newcomers." In fact, one of the main conclusions and suggestions comes on the last page when they advise site owners to be cognizant of newcomers' postings so that they'll be sure to receive a response and stay active in the community. Indeed, this seems to back up legitmate concerns of newcomers' continued usage since within this study only 27.8% of newcomers post again after an initial posting, whereas old-timers are 72.2% likely to post again. This is compounded by their finding that old-timers get more responses to their posts. The authors here speculate that this may be because of familarity with previous posts.
However, the authors' findings about message-level content that may inform this causal relationship indicate that it may be beyond simple familarity amongst users. Linguistically, the researchers found that "sentences containing more first person singular pronouns and third person pronouns got more replies than those that didn't [and] words expressing either positive or negative emotion were also more likely to get a reply" (964). This find, above all the others, fascinates me because of the implications of language use. Whereas using "I" or "me" may signal that the poster is emotionally invested in the community enough to disclose personal feelings, the "we" or "us" indicates a more cohesive understanding of community that may build trust. The researchers here are saying both together (or apart) lead to postive outcomes, but I think we should take notice of the subtle rhetorical effect of both. As evidenced in sentences like:
As a class, we should really consider the effect our language has in building upon each others' posts so that we're building a consistent conversation and not just a string of individual posts.
OR
I wish people would respond directly to my posts instead of just writing a new summary/response that posits a new idea.
It's interesting to see if either sentence ellicits more of a response or makes others want to stay engaged in the community longer.
Some thoughts on Arguello et al. reading...
Nice summary and observation/example about language, Chris. I agree, there is definitely a big difference between how people might read posts (especially from new users) that contain a lot of "I" and "me" as opposed to alternative tones and words that express "solidarity" within groups. I guess one related problem, is that new users often are not in the position (or may not want to, even if its acceptable) to speak out in a more "group-cohesive" manner. This leads to a few questions/concerns that were on my mind while reading Arguello et al.'s reading.
I was a little concerned while reading the design of the author's research. While the authors do mention some of the caveats and possible loopholes in their study, I felt a few issues were actually quite important. For instance, individual status seems rather vague because the authors defined new users to be users who have never posted. While I do not necessarily have a better idea as how to best distinguish users, I guess it is still important to keep in mind that in this study, all "non-new users", ranging from "celebrity users" to users with 2 posts, are all considered as "old users". I would assume the response rate a regular member would receive would be much higher than the other "non-new users" who in the community, are not much different from the "new users"
I was also concerned about the communities included in the study. The authors explain that they have covered 3 out of 5 major groups, but with the exception of maybe the sports-related groups, politics or health-related support groups are really a lot different than some other groups. First of all, the basic target audience may be really different. For example, I would assume that the average age of users in politics or heath-related support forums would be much higher than a forum, on say, a gaming forum. While I have no statistics to back up this claim, I think it is important to consider how the implications would be different if the authors included some vastly different online community. Similarly, the interaction on health support forums or any technical support forum might be vastly different than say, the "off topic" section of some music forum. In most tech support forums, users (often "new users") will come just to post an answer. Depending on the answer, other users in the community may not find it necessary to comment if a single sufficient response has already been given. However, given that this is a tech support forum, most questions (assuming they are not overly obnoxious) get answered. But the same can't be said in other forums. In one of the small online communities that I am familiar with, several posts from regulars that everyone knows, get zero responses. Interestingly, there appears to be nothing awkward about this situation. Basically, some communities are more like a "questions/answers" site, and others are more like "Here's a picture. Discuss". Response rates obviously differ a lot.
Another thing I was concerned about was how the authors are testing to see if the original message poster returns within 3 days to either post a response to a response (if any), or create a new related thread. I don't feel that this is a really accurate way to test whether new users would return. As previously mentioned, if a new user is asking a technical question, he or she may not necessarily post a follow-up question or even a "thank you". Similarly, if the context is within a "quesitons/answers" type community or thread, users may not find it necessary to see what other people have to say. In a heath support forum, where a community might in general be more "close" to each other or feel more passionate about a certain condition, more discussion and back-and-forth conversation might be expected. Also, users may simply come back at a later date to comment on other threads, or create totally un-related threads. I guess these are just some things to keep in mind.
Nice
I have many similar thoughts as your post,
nice one!
good points
I have similar reservations particularly on two of the points
1. the generalization of 'old' members: There will be a big difference in most cases between people who are regulars to the site and well known as compared to people who have posted three times on the forums, each time to get some quick answers to their questions
2 . Whether a post was replied to or not would for a large part depend on the community. I am a part of a web community where people post music by independent artists and leaving a note saying thank you bumps the post up so people generally don't post thank yous, although they might acknowledge that in other places. So i'm not sure whether the replies tells tells us much without the any information about the complex nuances of the community interactions
Good points Geoff. But I'm
Good points Geoff. But I'm more interested in how to fix those problems if we have a chance to redesign the study. I don't have an answer yet.
some of your critiques aren't quite right
I think some ofyour critiques are not right.
I think you've got a good point about the need to see whether results differ across different types of communities; more comments on that in response to other students later.
I go into this detail because I laud your effort to engage these papers in sufficient detail to generate a critique of this kind, but I also think it's important that we hold ourselves accountable to reading accurately and making careful inferences. The people who replied in support of your critiques should ask themselves why they were so easily taken in.
Arguello et al & Grice
Chris, I feel compelled to respond to your post. And having said that, I wonder if there are any implications to using a lot of 2nd person pronouns.
Frequently using first person singular pronouns ("I/me") might motivate responses for different reasons in different communitites. In a support type community, it might signal a willingness to open up and build trust. In other contexts, like a political community, it might be interpreted as egocentric or antagonizing and elicit more defensive replies. Also interesting was the fact that using a lot of 3rd person pronouns ("he/she/they") increased the likelihood of a reply. The authors mention that using 3rd person pronouns distinguishes between in-group and out-group, although I'm not sure why that would have an effect on getting a reply.
What struck me was how the authors' findings resemble some of the Gricean maxims for effective conversation. Being direct, staying relevant, and being brief all increased the chance of getting a reply, so it seems that online conversational norms may not be all that different from those in the real world. The challenge is making that obvious to the community, especially newcomers.
Nice point
This is a really good point Matt. With all of the talk about talk, I’m surprised that we haven’t spent more time looking at how interpersonal communication conventions compare to online communication conventions.
I have always imagined that Paradise will be a kind of library.
-Jorge Luis Borges
Good point
I agree with your point, Matt, about I/me not always being the 'right' way to communicate in all communities. I think this could relate to identity-based vs. bond-based attachments that we learned about in the other article - at identity-based communities like political discussion groups, talking about yourself (using I/me) would be more off-topic because people are there to discuss the topic, not get to know each other.
But in a health support group that is bond-based, using I/me is welcomed, because people are there to support each other and create bonds. So, in the context of this study, assuming that there is a difference in using I/me vs. we/us does not consider the whole picture.
would like to see interaction effects
I agree that it would be nice to see whether the effects were different in different communities. Statistically, the way to do that would be by adding a bunch of "interaction" terms to the probit regression reported in Table 3.
With only a couple newsgroups of each type, however, there may not be enough statistical power to make inferences about this from this study.
good question; answer is in Table 3, model 4
Thanks for the connection to Grice, Matt.
In Table 3, model 4, it shows no significant effect of using the word "you" a lot.
I would have thought would be the biggest factor. 2nd person, especially 2nd person plural, is distancing (it announces that that the writer is not part of the group). But that's not what they found-- there was only a small, and not statistically significant, negative effect of using "you".
Similarly, I was surprised that "we" statements did not have a significant positive effect, but "I" statements did. I agree with your explanation of why "I" statements might be effective.
That ctools site...
I am compelled to respond to your second statement, Chris. Well done.
Trust is the key to getting response for the newcomers, and a clever play of words by the newcomer may just do it. But it requires skill.
I remember the Ctools site Judy Lawson and Marta Ray set up for us newcomers to SI. The forum where people introduce themselves contain nothing much other than introductions. I dont recall getting a welcome response from the owners nor from fellow students. There really was no trust. Then again, I guess there was no statement like "Talk to me!!" at the end that will compel anyone to respond.
Hmmm
I actually recall coming back to the ctools site a few times and looking for responses and, in some cases, getting them! It was kind of nice but eventually I removed my membership since I actually met people in RL.
language structure probability and tool design implications
I think there are some interesting implications for tool design in this analysis of language -- if you can determine certain message attributes are likely to be undesirable, you could potentially offer users different strategies for composition of comments.
While it may seem a bit obtuse, this sort of automated "help" for newcomers could have its uses. Encoding certain rhetorical conventions into an interface could be one way to help people post more coherently. Of course, having software that gives reminders against talking about yourself all the time seems a bit too close to having a nosy old aunt.
--------------------------------------------------------------
oostendo@umich.edu
--------------------------------------------------------------
good idea
I like that idea. In my opinion people are often times looking for guidance on what and how to write in the setting of an unfamiliar community. So if I would be given some unobtrusive pointers on what is expected in this community and how the best way to get a reply is, I would really like that.
It could maybe be like a short checklist at the side of the input field I'm typing in. It would say in very short not nosy-old-aunt-like style the top 5 things to do to be heard. And you could check them of one by one when you think that you comply with them. This could even be helpful if this is not my first post at all.
What do other people think? Helpful? Annoying?
I like this idea
Yes, I think this is a great idea! But I think it’s also important to keep the originality that newcomers may bring. The standard checklist might frame how people post things in some ways.
maybe, but i'm skeptical
I thought about this, too - when new users post you can include a checklist of thing they can do to increase the probability that their post will get a response.
But I have 2 concerns that make me skeptical about this actually working.
First, this article doesn't have any causal conclusion, only correlational. We don't know that framing a post in terms of "we" instead of "I" actually causes an increase in responses. Maybe someone who has spent more effort reading other posts and getting a lot of background information tends to write with "we" pronouns instead of "I". But the real cause for increased responses is that the person is more informed about what's going on in the community so their posts are more relevant or more tailored to the community concerns. Telling a newcomer to use "we" pronouns may not actually help, because it's the wrong prescription.
Second, if we give users a list of things to do to increase response probability, and lots of newcomers start using those tactics, will they lose their effectiveness? Think about the person who always writes "URGENT: RESPONSE NEEDED" in their email subject headings. The effectiveness of the message gets worn out with overuse.
Another Skeptic
I also think it's dicey to try to automate suggestions for users' posts. I agree with Erin's second point that overuse of signals of quality will degrade the strength of those signals. But I think there's also a larger issue.
What do personal pronouns, self-disclosure, and being an old-timer have in common? ...familiarity, personality, and a sense that you're making a real connection! Posts with these qualities yield the most results because the poster seems more real, more engaging. In a community, it's the bonds that matter the most to members, and these posting styles exploit that.
Without a lot of care, I think that automated writing assistance that claims to yield higher response rates will be seen more as manipulation, which runs contrary to the idea of establishing relationships.
agreed
I'm in agreement with you that these kinds of tools would only help to manipulate your posts, to change your actual tone into one of more friendly/familiar style. Maybe they just get better responses because they're a friendlier person who directly addresses other users and asks their opinion more often.
I think that a tool to help you mimic the linguistic statistics of "good" posts that generate lots of replies would only perhaps jump one hurdle - and that Erin's points about correlation not being the same as causation.
a follow-up experiment
To address the correlation/causation concern, they did a controlled experiment.
Burke, M., Joyce, E., Kim, T., Anand, V., and Kraut, R. (2007).
Introductions and Requests: Rhetorical Strategies That Elicit Response
in Online Communities. Third International Conference on Communities
& Technologies 2007. PDF
Thoughts on User Manipulation
I love your example here about subject lines (urgent: response needed). I'm on a wide range of human rights action list servs, and am constantly bombarded with messages bearing close resemblance to the guilt-inducing "Children's International" marketing strategy. This strategy backfires with me, leaving me pissy with the agency and less likely to donate.
As others have mentioned, manipulation comes into play the more you "guide" your users to post appropriately. I recently joined a "women in computing" list serv that has strict, but tacit norms. I was scared to post, and when I finally did, it bounced back from the sys admin, shot down for being too long.
Lesson learned: Cogent and brief posts are best.
I was not surprised this was one of the findings of the study....
Lisa McLaughlin
not their finding
You are skeptical, Erin, that framing in terms of "we" instead of "I" causes an increase in responses. In fact, the paper did not find that! It found that "I" language increased responses; "we" language did not (at least not statistically significantly).
*blush*
Hrm - so much for being a critical reader. Thanks for pointing that out - I'm not sure where I got mixed up.
Stupid filter
I like your idea a lot. One related idea is the stupid filter. it’s more fun to check it out than to have me explain it.
http://stupidfilter.org/main/
I wonder if any of our posts would be filtered by stupidfilter..
that looks cool, thanks for sharing!
an alternative; exemplars
Another possibility might be to use statistical analysis of what features of first posts lead to responses in a particular group to select a few exemplars-- examples of actual posts from this group in the past that were most and least likely to get responses. Hopefully, the most likely ones will actually have gotten responses!
Getting the right people to stay
John Blair
I wonder if too much effort is being directed at getting people to stay as opposed to taking care of those that got the community where it is. Everyone was once a newbie to the community (with the possible exceptions of the founders and his/her colleagues) and certainly efforts should be made to attract the desired people so that the community grows (if that’s the intent).
Designs targeted toward potential members should include making entry barriers appropriate to the mission of the community, providing easily found community information (its purpose, norms, policies etc.). The design of these features should also consider the type of member candidates being sought. For example, it would not serve the purpose of the alt.hacker community to lower its’ entry barriers because they would not accept most of the people who joined – and that’s ok, it’s their world.
I think it would be more beneficial to the community to weight their efforts more heavily on the side of maintaining a happy membership. After all, the other side of the membership growth equation is losing existing members. Not only do the member numbers go down when members depart, but they take with them the knowledge and attachment they dedicated to the community in the first place (this isn’t considering the viral campaign they may engage in regarding why they left). It is far more difficult to replace a departing member than to add a new one; and simply adding a new member isn’t going to balance the scales in that scenario.
If you want people to stay, you have to realize there is only so much of that you can control. As the authors here suggest, if you to increase the odds of engaging them, do so in a way that provides them “beneficial responses and builds commitment.” People are going to come and go, and a bad design will make them go faster than a good design will attract them. I believe nurturing the core members while focusing on the activities (design, features, information, etc.) to attract the right people will provide a much more sustainable community model as the existing membership will continue to perceive value and new members will recognize that and what to attach to it.
Not usually either/or
Sometimes there's a tension between welcoming newcomers and retaining oldtimers (e.g., forcing conversation to stay on topic, which is better for newcomers). But most of the design suggestions for getting newcomers to stay will not have any detrimental effect on old-timers.
Not perfect one, but can be a good exmaple to look at
In general, this paper is pretty interesting to provide substantial
empirical data analysis to reflect on the theories in online community.
But may be just because it attempts to grasp too many things embedded
in the complex context, it is hard to void concerns in validity, as
many have mentioned above. However, I thought the way they derive from
theories and conduct analysis using semantic approaches, is very
valuable for us to look at.
In addition, I am a bit concerned about the significance of those
predictors in the models. The improvement of the model by adding new
predictor has been changing relatively small, for example, the
complexity factors improved 26/6756, which is rather small. In general
all these factors don't count significantly for the variance in terms
of how people would or would not reply. So it could be that there are
some other more critical factors counting more on the result.
How to win friends and influence communities
Hoping to add to the insightful discussions about language choice and causation with a compilation of my design-related takeaway points from this reading:
Newcomers:
-are more likely to return if someone responds to their post (about 6%)
-are more likely to respond faster if someone responds to their post, but are just as likely to return at some point
-are less likely to receive a response than established members (about 4%)
-can increase the likelihood of a first-post response by addressing members with 1) questions (about 6%), 2) personal histories that establish legitimacy, 3) an appeal to member interests by staying on-topic (about 10% improvement), and 4)simple language when possible.
-generally post again about 28% of the time, compared to 72% for established members.
Inter-Group Posting
-reduces the incentive for members to respond, however posts to multiple groups were 9% more likely to receive a reply
Large Groups
-have generally less committed members
-will repel members if too many messages appear (cognitive demands)
-will repel members if too few messages appear (lack of useful content)
Messages
-weren't more or less likely to receive a reply based on the total length.
-were more likely to receive a reply when sentences and words were short.
-were more likely to receive a reply when the poster used 1st/3rd person pronouns
-reflecting mental processes had a higher rate of response.
-with both positive and negative emotions had a higher rate of response.
If Usenet interactions represent the state of conversation across the Internet, community developers should take steps to ensure that newcomers have some guidance in their first post. There should be incentives to stay on topic, ask relevant questions, keep it simple, and write something about their personal background. In the interest of attracting new members, established members should also have additional incentives to reply to posts by newcomers. In general, posts that incite antipathy are less likely to receive a response than posts that encourage sympathy. (63.6% of posts in liberalism discussion group received responses, vs. 81.6% of posts in the depression discussion group)
a couple nitpicks
Mostly good, but a couple nitpicks.
Useful suggestion to improve welcome kit
I identified one goal for online communities: adapting general knowledge to suit individual's specific needs. When a user a posts a questions to Y! answers, the expert will use his/her expertise and information from other publicly available sources to provide a solution that is specific to the user.
A design alternative I picked was providing a welcome kit with community metrics to a new user when they join a community. As mentioned in the paper the metrics can based on context, rhetoric, linguistic complexity and word choices. Users may not find the actual suggestions surprising, everyone knows that it is better to ask 'simple questions relevant to the domain of the community'. But when the suggestions are backed with scientific evidence and chances of increasing the response (in percentage) it is more persuasive.
It is also worth noting that the research was conducted on Usenet data. Hence the results may not be statistically significant when extended to other types of communities like Wikipedia, Youtube, etc that don't have a similar request/response type of interaction.
Social status and responding rate
This paper discusses main factors affecting community responsiveness and individual commitment at the group and individual level including context, content and source of a message.
One limitation of this study which the authors also point out is that different online community operates differently. For instance, in a typical online fans-club, although idols don’t always visit their site, or even he or she is the new user in their site, their posters are always the message with the highest responding rate. So, it is also interesting for us to investigate individuals’ social status on online communities which is also a factor to affect responsiveness.
In addition, the importance of content/topic for online communities initiates me to think about the future of Facebook. Obviously, it is not a community based on content/ topic but an online social network, how far can it reach?
Damn Statistics
While it may be statistically more likely that using certain words will garner a response, it just seems to me that the actual CONTENT matters more. The semantics, not the syntax, is really what should be the subject of research here. It's just plain lazy to count the number of "I"s and be done with it.
classifying affect might help
I have heard that some linguistic classifiers (namely one that one of the authors, Carolyn Rose worked on called taghelper tools) can do a decent job at classifying affect. What about that as a contribution towards accurate classification? I think you might mean something more along the affect than just content - more of the social meaning.
Agreed
I agree with you on this, Hung - it does seem that the software they used "Minorthird" wasn't even especially accurate at classifying testimonial and questions.
I see this as a major drawback to their research, since much of their analysis depended on these results. This seems like a case where a human would be much better than a computer at classifying the content, even if it takes a lot more time.
si fueris Romae, Romano vivito more; si fueris alibi, vivito sic
I agree with syntax vs. content argument just as well.
Furthermore, i would add that there is something to be said about the "culture" that permeates a site. And when surfing such a site you are either are part of it or not. I and We go only so far in the english language. It is limiting. however, in most romance languages the difference btwn, as an axample, "tu" versus "usted" are enormous and grave. used incorrectly at the wrong time with the wrong person and you are persona non grata. I recently learned that in Ukrainian it is not much different with vys and vu if I recall correctly.
There are online communities where individuals refer to themselves in the formal context while others prefer the informal. and one must abide by their rules, culture, even the vernacular.
My point: If you are in Rome, live in the Roman way; if you are elsewhere, live as they do there. It will elicit a reaction as Arguello has markedly pointed out through empirical evidence.
Enhance responsiveness
The main design claim of this reading is to enhance responsiveness to initiate conversations to increase user’s commitments to the community. Some design alternatives I found are:
1. Enhance responsiveness by establishing group identity, maintaining a desirable group size and volume, and allowing cross-posting (?).
2. Enhance responsiveness by encouraging self-disclosure to increase other group members' willingness to reply and can increase the poster's commitment to the group.
Do responses to cross-posts have different welcoming effect?
I think some kind of separate analysis might be needed to understand the dynamics of cross-posts.
To post or not to post ?
At the end of the day whether a person posts or not probably depends on two important factors:
1. The value he thinks his post will add to the discussion.
2. How comfortable he is posting to the group
For example a large group size implies that the message will be read by a lot of people, a lot of whose reactions it's hard for the author to predict. This is particularly true for someone who has not posted much to the community. Thus his comfort level in posting to the group is reduced. The person can be made more comfortable posting if he can read posts from othernewcomers and enthusiastic responses to them, or can be hindered if he hears some rude remarks to the postings of a newcomer.
The most important thing a designer can do to get people to participate is to inculcate the belief that that their posts will be welcome. It is particularly important to acknowledge newcomers. Not only does the encouragement help that newcomer transition to a possible regular poster the visibility of this fact to others also acts to indicate that newcomers posts are appreciated.
Overcoming newcomer status and problems with the analysis
I was struck by one of the theories that Arguello et al posit about newcomer contribution. Members with more history will have an easier time getting a response. So what is a newcomer to do?
It seems like a typical reputation problem for newbies. Because they have no history and no tie to the community (especially important in bond-based communities), they are less likely to garner a response... but does this apply generally as the authors suggest?
This leads me to a problem I have with their study. First, I think it was important for them to get different types of communities – sports, health, political. Their explanation on different behaviors within those types of communities seems congruent to what we might expect, e.g. political groups are generally topic-based, identifying with the community as whole (as opposed to bond-based where member's identification to the community is based on their relationships with each other).
So now to the problem: they split out each of the communities in the dataset with dummy variables, but they don't report any of the statistical results for the specific groups. I would have like to have seen them group the types of communities and then run the analysis on each community type. I would like to see statistical differences between community types. For example, I would expect bond-based communities (medical communities) would actually have a positive newcomer coefficient or at least a statistically insignificant coefficient. My intuition leads me to believe that when someone posts a new message to a breast cancer forum seeking support, they will be quickly shepherded into the community. It would have been nice to see some sort of statistical support across community types.
yes, or interaction terms
Generally, it's better to include interaction terms in a single model, rather than run the regression separately with subsets of the data.
But, as I noted in an earlier comment, there may not be enough data in this study (only a few groups of each type) to conduct this analysis.
I totally agree that type of
I totally agree that type of attachment should be goal driven and tradeoffs always exist. The implications are worth taking a look.
1. Newcomers. The author mentioned that newcomers can always be a problem but being able to recruit the retain newcomers on a regular basis is vital for all online communities. This is definitely true. There are numbers of newcomers registered in the community I am studying. These newcomers keep asking similar common questions all the time before they even start to search the old postings a little bit. This could be a problem as it annoys old members frequently. In order to deal with this, site managers will gradually pull out the most frequently asked questions and try to group all related answers together in a new post and then put it on top of all the postings in its subgroup and marked as recommended. In addition, notices will be sent to newcomers as a remind that quite some questions could be found there.
2. As for the off-topic discussion, it seems that most forum communities dislike it and tend to impose policies to control such discussion. And my community is just one example.
3. In the first reading, author argued that larger groups may have less-committed members. Here, the author’s argument seems more convincing to me. First, since community growth often leads to diversification in community purposes and member preferences, for communities in which both types of attachment are important, one option might be to group participants into clusters with similar backgrounds or needs. I think the community I am studying is a good example of this. Also, having large numbers of members and high turnover is less a problem for identity-based communities than for bond based communities because for the identity based groups the large membership provides their core resource and rich new content and this is what I have seen in my study.
good point, but for other reading?
Did you mean to post this with the Ren et al reading?
I enjoyed this paper, and I
I enjoyed this paper, and I think it helps set the stage nicely for the research that appeared at CCT 2007 on using machine learning to evaluate whether or not a post was likely to receive a response.
One of the challenges with this type of research is that it suggests user behaviors that may be optimal, rather than design choices that may be optimal, so what route should designers take? On the one hand, designers can try to encourage user behavior that is likely to receive an answer (with affordances for that behavior, or the more direct approach of saying "hey, if you rephrase this question you are about to answer, you'll be more likely to get an answer..."), or they might try to instead use design cues to change the question answerers' behavior so that they answer questions that do not currently receive answers.
I'm skeptical of either of these approaches. The way someone phrases a question signals what sort of answer would be useful, and how much effort they've already put into the problem (e.g., did they search the archives to see if it was already answer?). It probably isn't a good idea to pursue design choices that weaken this signal. Solutions that prompt user to search the archives are already used in some situations (particularly companies' technical support interfaces).
I wonder if there would be another possible approach that would help better match questions with potential answerers, based not only on skills (as is already done in many spaces) but also based on what sorts of questions people like to answer (maybe there are users that like to answer vague, indirect questions, but have trouble finding them?).
The choice of newsgroups is also interesting to me. There's the benefit that the designed interface across groups has very little variability (though will vary based on users' selected newsgroup reader), so the authors have basically eliminated the issue of group-specific design cues (apart from policy design cues) affecting user behavior.
Foundations
The authors begin with a a hypothesis that "community responsiveness to attempts to initiate conversations is an essential element of community success," noting that this is "consistent with research showing that individuals who post for the first time to an online group are more likely to return when others respond to them [11]."
Their literature review leads them to the following factors:
Context
--Group-Level Factors
----Group identity
----Cross-posting
----Group size and volume
--Individual-Level Factors
----Newcomer status
Message Characteristics
--Rhetorical strategies
----Topical coherence
--Linguistic Complexity
--Word choice
Then they state that "The study described in this paper examines the impact of these classes of factors on individual-group interaction in a sample of Usenet newsgroups." They show their findings on the effects of the factors on further posting, noting 1) that the statistical relationships are only correlations, and 2) that they may be caused by an unexamined z-variable. They conclude with some interesting design implications of their results.
This study is so straightforward that its results are easily interpretable, portable, and the whole study could be repeated in multiple communities over time to generate even more results. In other words, it is groundwork for an entire series of studies. It could be improved by taking into account potential confounding variables and looking for (or generating) groups that vary by those factors (effectively control groups).
--------------------------------------------------------
PHartzog@umich.edu
--------------------------------------------------------
The Universe is made up of stories, not atoms.
--Muriel Rukeyser
Support for newcomers: [Revised posting from blog]
I believe that if a newcomer does proper research (and I think that
most people do) before posting the question or answering one, there is
bigger probability of getting a response. For example: If he/she is
asking a question, then it would be good to know 1) Which thread should
he/she post the question 2) Has this topic been discussed in the past
3) What is the current context of the discussion and how does my
question/response fit in that context.
If the newcomer takes these simple steps, then the probability of getting a response might go up.
Based on my personal experience about being a newcomer on few
communities, I understand how it feels when no one replies to the post.
We may have an analogy of someone new walking into a bar where he/she
does not know anyone. Maybe not as strong as the real world
communities, but similar logic might apply to the online communities
also. Getting an answer (for the first posted question) is analogous to
(mayb e) an acceptance by the other members in the bar.
Another observation: Many a times, I do not look at the user ID of
the user who posted some comments/article. Instead, I am more
interested in what the person is saying. (Does anyone else do the same
thing?). What it means is that the history (or the newness) of an
individual might not be that big a factor as his/her content and how it
relates to the discussion context. Newcomers might get instant approval
of the community if their content provides additional value to the
ongoing discussion.