At the request of Paul Resnick, what follows is a brief description/summary of:
Kraut, R. E., Olson, J., Manaji, M., Bruckman, A., Cohen, J. & Couper, M. (2003). Psychological Research Online: Opportunities and Challenges. American Psychologist, 59(2), 105-117.
-----------------------------------------------------------------
-----------------------------------------------------------------
According to the abstract, the Internet offers new opportunities and difficulties to psychology research. "This paper describes some benefits and challenges" (p1).
The Internet provides new opportunities for psychology research. First, it makes empirical research easier in four key ways: 1) by providing access to subjects, 2) by providing effective mechanisms for observing social behavior, 3) by providing access to archival data, and 4) by providing automation and experimental control. Second, beyond improved efficiencies in performing traditional kinds of research, the Internet also is "an important phenomenon in its own right," (p6) and a virtual laboratory for examining new social phenomena.
Nonetheless, the Internet presents two important challenges to researchers: 1) data quality, and 2) the protection of human subjects.
The concern over data quality is twofold. First, the sample of human subjects accessible to online researchers is "by no means representative of the nation as a whole" (p7). Second, the Internet possesses a relative lack of control over the data-collection setting as compared to other environments.
Regarding the protection of human subjects, the authors refer to the basic ethical principles described in the "Belmont Report, prepared by the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research in 1979" which includes: 1) Respect for Persons, 2) Beneficence, and 3) Justice (p9). A helpful flowchart (Figure 1) displays "some factors relevant to Internet research influencing whether informed consent is required and must be documented" (p10/p23). The Internet is (at present) an ambiguously defined participatory space that makes it difficult to determine what information can appropriately be considered public vs. private, as well as whether an individual participant is anonymous or identifiable. These factors present risks to subjects in the form of potential harms as a result of participating in online research. Consequently, researchers must be reflective of these potential harms and utilize debriefing and confidentiality wherever and whenever possible, always mindful of the necessity of informed consent on the part of research participants. Online, meeting these ethical obligations can be more difficult than in other settings.
Some key lessons emerge from the above considerations. Researchers should acquire facility with online research practices by starting with small simple studies. Tracking numerous data points (IP addresses, etc.) from all participants allows researchers to investigate data problems later (such as sampling biases). Researchers should practice matching risks with commensurate protections for all participants involved. Finally, researchers should not simply acquire a set of skills and re-use them, but should be continually reflective of how the speed of change online demands continual revision of research methods and techniques.
In the authors' summary, they suggest that the new risks of online research "require researchers and Institutional Review Boards to keep abreast of changes in online behavior, community standards, and available technology. They also require a degree of reflection about the research process that may not be necessary in more established domains" (p22). (The authors provide a bibliography, figures, tables, and sidebars at the end of the article).
-----------------------------------------------------------------
-----------------------------------------------------------------
-------------------------------------------------------- PHartzog@umich.edu
--------------------------------------------------------
The Universe is made up of stories, not atoms.
--Muriel Rukeyser
Submitted by Rozaidi Rashid on Thu, 01/10/2008 - 23:49.
0
points
I have never done psychological research, what more on the Internet. (My experience in Internet research is analyzing my website's log files.. originally scouring the text, then with awstats, and now with Google Analytics.)
But the article allows me to appreciate the challenges presented by the Internet, in terms of data quality and human subjects. I like the flowchart to check whether informed consent is required when researching human subjects.
Great summary! Phsycology researches always seem exciting but hard to me and I have little experience as well. In spite of this, the reading still seems interesting to me and the above summary is just great. The lessons drawn from the research are useful and I totally agree that the new risks of online research "require researchers and Institutional Review Boards to keep abreast of changes in online behavior, community standards, and available technology.
Kraut brings up the idea that the internet allows researchers a lot of control over environment (or at least environment of the computer screen interface). Because of this ready control, it is attractive to conduct a lot of research based on changes in interfaces. For example, a researcher could partner with an existing online community administrator to conduct a study on the effect of photos paired with user names on community activities. This type of research poses little harm to subjects. IRB processes often obstruct research - sometimes for the better. However, in situations with so little risk, IRB seems unnecessary.
The more important point that this discussion on risks and IRB brings up is the normative mechanisms that are in place in academics. Academic researchers should be brought up in an environment that teaches them the boundaries of ethical
behavior, empowering them to make correct decisions without the parenting of
the IRB. This relates to the discussion thread about Wikipedia editing. Moving away from overzealous administrators to a more self-administrated system. Good research enviroments should foster inherent ethical behavior.
This chapter comprehensively explores the many issues involved in the possibilities of Psychological studies on people’s online behaviors. In addition to many we had long noticed such as the lack of control over data collecting process, it is particularly valuable in mentioning the ambiguities on all the notions we have clarified to some degree in the real life but still very vague in online sittings, as a new type of public space for people’s social interactions.
This place requires a considerable period of time for people to figure it out. In particular, a person uses her knowledge to make assumption on the privacy issue when participating in online activities. This knowledge determines how she would consider the issues. For example in the paper, the college student posts her family pictures on her personal website, assuming that only her friends and family members would be interested in browsing it. However technically, the reality is that everyone on the Internet can access those photos. And this fact actually can lead to the way harming people’s privacy or themselves. For example, exploring one’s friends’ profiles on the Facebook can infer something about the person that he does not want to show. When one releases some of her privacy on a particular place online, she had the assumption that it can be exclusive within the group who access here. Another example is that a guy’s on some social website’ activities behind his wife was found by a three degree friend of his wife.
The richness of information online brings the possibilities of inferring underlying relations; some time even beyond what a researcher intends to do.
The above summaries are very comprehensive and definitely helped me to get to the major issues of this article. Given that this reading comes under the ethics portion of the class, though, I think it's important to focus in particular on the public vs. private debate which begins on approximately p.12. What I think is most interesting is that even though the authors are highlighting the benefits and problems with conducting research online, it is not until this point in the article that they begin citing legal statutes and cases. Before this the discussion was about economics and ethics, concrete and abstract respecitvely, but the legal discussion is where things begin to blur into grey. The above cited examples begin a great discussion of public/private disclosure and the ethics of that. But the ethics and legality of a virtual space will probably be the number one thing researchers and academics battle in the coming century. Since the Internet is "public" and sites may be "private" nearly every site is a literal site of conflict. This article does a nice job of admitting this, but I find it troubling/sad that legal matters (and to some extent the ever-expanding role of IRB) is where these discussions will ultimately end up. This is especially poignant when viewed side by side with the Kim and Powazek pieces, which discuss a kind of utopia of online communities with users freely and gladly giving content.
Submitted by John Blair on Thu, 01/10/2008 - 21:02.
0
points
John Blair
To expand on Chris's points a bit further regarding economics, ethics and privacy. If we think back a mere 10-15 years ago, the Internet was a much more "friendly" place. For the most part, people contributed because they wanted to share the information with others. Then, as the general public started to gain more access through better tools, the economics of that many potential buyers began to take over. In the U.S. there is essentially no privacy protections guaranteed to users. The marketing departments have ensured that the bills don't become laws because then accountability and consideration of the users information would become one of their requirements, which of course would cost them. Look carefully at the breach laws we do have and you'll see that there are a multitude of conditions as to when a company actually has to inform consumers that their information may be at risk. The U.S. does a dispicable job of protecting our privacy because it's too profitable to not protect it. Privacy in the 21st century is highly unlikely given the amount of information that already exists in databases everywhere about all of us.
Regarding this particular reading, I found it much like Powazek's in that they both are providing the basic foundation considerations for the respective topicss of online communties and online research. Both articles compare some of the hurdles while providing some useful guidance if one chooses to proceed.
The privacy issue raised by this reading stood out as the most dubious and unresolved challenge with which researchers must contend when conducting research online. I don't claim to have a solution, but I'm more than pleased to point out the inadequacy of the current laws.
The reading states that "research involves human subjects only if data is collected through interaction with a subject or if it collects 'identifiable private information'." and further defines "private information" as information about which there is a "reasonable expectation" of privacy. Of course, a "reasonable expectation" is highly subjective. To make matters worse, many Internet users have naive and uninformed ideas about their privacy online.
As one example, informed Internet users understand that email is not private. However, as specified in the reading, there are laws on the books that protect the privacy of email messages between two individuals. If people didn't already have the expectation of privacy in email (they did anyway, by and large), a law purporting to protect email privacy certainly lends credibility to that notion. (Personally, I'd prefer not to pass laws that can't conceivably be enforced, but that's just one of the many reasons I would never make it as a legislator.) For the "reasonable expectation" principle to have any weight, I think it's important to have a better educated population. Until better legislation or better education comes along, it looks like online researchers will have to tread lightly.
Submitted by Tracy Liu on Thu, 01/10/2008 - 12:53.
0
points
This paper gives us a blueprint that which topic we can explore on the Internet and what we need to pay attention to as psychosis, especially it gives helpful information on the aspect of legality, which we(at least myself) knows little before.
As a beginning researcher, I would like to ask what would be a good research topic on the Internet from the perspective of (social) psychologists. Or what is an exciting research question when we explore users’ behaviors online? As Internet provides a broader platform to study human behavior, there are so many new phenomena generated by this new community, however, what would be the most valuable one instead of transitory events? From my opinion, those activities which also take place offline, such as E-commerce are one of the most important topics, however, how about those activities, such as Blog, which seldom emerges offline(we could not read others’ diary in daily life!!), how do we value this kind of activities or how long can it sustain active?
We have done a bunch of studies on the Internet, what’s the next hot topic? What else can we explore from the old topic? What is indeed wrong/trivial in current research results?
The Internet with all of its digitized behavior is too tempting of a source of content to be resisted but the social scientist, but how to gather the data ethically? This article explores a number of interesting aspects of gathering data online, but one that I found most interesting was how a tragedy of commons scenario can emerge by overly zealous researcher practices.
There is evidence that Second Life is already suffering for research fatigue. If the users get sick of being pestered it will undoubtedly lead the administrators scrutinizing research more and more. I think it will be interesting to see how researchers conduct them selves in order to ensure that they are seen as beneficial to the online communities rather than just another online nuisance
I have always imagined that Paradise will be a kind of library.
I agree with others who have posted so far that this paper seems to bring up more issues than it purports to solve, especially concerning the ambiguities of public versus private online. Yet there are a lot of practical lessons to be gained here for researchers in the field, things like: start with smaller online studies, don't overdo technology and informed consent for minimal risk studies, separate log data from identifying information, etc.
As for the issues brought up by the study, some of them are large enough that they should persuade researchers to not use the Internet for certain studies. The authors seem to make this claim - the Internet is great because it enables studies to be cheaper, easier to administer and analyze, and reach more people. But the risks include not being able to manage the user's physical environment, and the anonymous nature of the Internet which may encourage some to participate with the express purpose of damaging data. Ethical issues imposed on the researcher include whether to reveal pseudoymns, and whether it is ethical to be a 'lurker' to collect data from websites where users have no idea their communication is being collected.
Overall, this article brought up many good issues for us to have in mind as researchers, so that we are aware of the risks and benefits involved.
Submitted by LizBlankenship on Thu, 01/10/2008 - 20:22.
2
points
I thought this was a good introduction to the issues involved in online research, discussing such issues as privacy, ethics, and legality.
I think some of the privacy issues are interesting from a psychological point of view. Just as the college girl posted photos online thinking noone would look, I myself have in the past left a link to a rather personal blog on a public website that identified me, only to discover my blog being featured by a professor on the first day of my undergraduate Human Factors of Technlogy class. Bridging the gap between a researcher and an average person on the internet that doesn't think their actions through in terms of the rather permanent, ubiquitous, accessible nature of the internet is one of many challenges for internet research. Sheltered restricted-access environments, and the ever-growing apathy and mindless automatic click-through of User Agreements of all forms should make us ever more cautious.
I think Kraut's suggestions are a good beginning for tackling some of these issues, but we still have a way to go in uncomplicating internet research.
I had a similar blog exposure incident, wherein the head of a nonprofit who I knew professionally only very casually, asked if I was doing ok. Unabashedly, he admitting to reading my blog on myspace and concerns for my emotional health.
I deleted my myspace account less than an hour later, not because I had something to hide, but because my angsty poetry was less than exemplary on second glance. I also saw very little resemblance in how I presented myself in writing then and who I think I am now. It didn't authentically represent me.
This is my issue with internet research, how to measure the validity of interactions and discussions that occur spontaneously and in so many different contexts that its difficult to tell how representive it really is of a population.
Another case in point: When I was doing HR for a UM department I knew we had a student coming in for an interview and looked them up on facebook so I could visually identify them. Not sure how ethical this was, but I stumbled upon a less than savory visual depiction of my candidate's zeal for Jack Daniels and "ho's in different area codes."
I was then uncomfortable interviewing this person, and wasn't sure if I had crossed some kind of ethical boundary by looking them up. My perspective on the candidate was undoubtably changed, utterly changed. There are certainly many nuances to cover in this issue, both as a researcher and professional of any sort.
Submitted by LizBlankenship on Tue, 01/22/2008 - 23:00.
0
points
You make an interesting point when you say how the things you once posted were no longer representative of you. I have often felt the same about my old internet selves - whether it be the middle school MUD addict part of me, or my very first webpage (on Angelfire with javascript flashing colors) or even as I've progressed through yet another two website designs to finally end up with my current site. When will I decide this site isn't "me"? (I've decided to keep it simple and grow it when I know what I want.) And why is it that people do this? I think it's a unique duality of behavior that we had yet to observe before the advent of technology to allow us to do so. Sherry Turkle's book, Life on the Screen, does a pretty good job analyzing things from the MUD days but I'd be really interested to read more modern assessments of our new psychological phenomena.
Given the horrible privacy practices that users display, I'd say that it is a real issue using data from the internet for research. I agree that IRB approval might be a bit of a hurdle for researchers to cross. I think if users and researchers are careful with the way they handle online information, there's really no need to explicitly use information that is already freely available to anyone with an internet connection.
Kraut et al talk about the opportunities and challenges the Internet has brought up to psychology research. It may reduce cost to conduct research and make empirical research easier, but it also may decrease the quality of data and infringe personal rights. I believe not only psychology research encounters these issues, but also other disciplines may face similar problems.
I found this reading is insightful and useful for me to think about issues I have never thought about before. However, I doubted that some points that are stressed by this reading may be out of date. For example, it said when given choice of Internet or paper questionnaires, respondents still overwhelmingly choose paper (p8). From my point of view, if I was given the choice, I will defiantly choose the Internet version, since I don’t want to write. Therefore, I guess it would be important to examine the same standard or rule in different settings and times while doing research, since the environment evolve.
Kraut et al were ahead of their time in 2003, and while they were framing arguments for ethics in psych experiments, foresaw privacy breaches like the 2006 AOL search log release:
"Probaby the greater risk of harm in online research comes not from the experience of participating, but from possible disclosure of personal information at a later time." (p16)
AOL gave a dataset of 20,000,000 search queries from 650,000 unique members to the academic community for psychological research. While the logs don't explicitly store personally identifiable information like a screen name, searches can of course be used to infer someone's identity. (Who doesn't search for information about themselves from time to time?) Screen Names were replaced with "anonymous" pseudonyms like "091283", which aren't aren't so anonymous...
A search query is in some ways like thinking out loud. Expectations for privacy with a search engine are likely to be a measure of how much you trust yourself to keep a secret about yourself.
Submitted by Daniel Zhou on Thu, 01/10/2008 - 22:38.
1
point
Paul Hartzog has posted a very nice summary of the paper in the beginning, which I will not repeat here. The value of the paper is that it proposed the ethical issues related to online research, and it gives some suggestions/guidelines on how to conduct online research ethically.
I also learned the pros and cons of doing online research, which helps me better understand this tool.
This paper describes the intricacies of performing Psychology research online. Like other fields Internet has disrupted the research methods as well. It has presented new opportunities to collect data, process data, collaborate with other researchers and publishing the research.
With the Internet it is very easy to collect data, the marginal cost of one subject is usually close to zero. As people spend more time online they leave more social footprints, this provides a rich data for the researches to study how people are forming connections. There is already plenty of archived data that can be used for research. Internet in itself has given rise to new phenomenons like new forms of interpersonal communication.
But all these advantages come with a host of challenges like sample bias, data collection setting leading to deviant behavior, protecting identity of subjects, risks to internet subjects.
Users are putting private data in public domain. This is blurring the line between traditional public and private data domains. I think initiative like dataportability.org are in the right direction. It will encourage users to think about their as a closed entity they can control.
The description of 'publically accepted behavior' is being redefined online. I think we are developing new ethos for Internet, hence a period of trial and error will exist. Hopefully we will reach consensus quickly.
In the beginning portion of this reading, the authors discuss in detail the reasons and benefits of conducting academic research and experiments on the internet. Obvious benefits of this method could range from saving costs to discovering new areas of interests, perhaps otherwise unrepresented in existing studies on offline communities. The authors then continue with an in-depth analysis on the potential risks and caveats of conducting research on the internet. The list of things researchers must keep in mind seems endless at first glance. However, this simply reiterates the point that with the improved access to information (made available through the internet in this case), researchers need to become even more careful and responsible for their own work and research. The most practical solution, as represented by the fact that the authors chose to close their paper with a list of recommendations and suggestions, is that best practices and improved standards of research and data collection might need to be explored.
Submitted by Sean Munson on Fri, 01/11/2008 - 00:41.
1
point
The authors raise many important concerns for researchers working online. One addition concern I've heard from some community members (both directly and from other researchers) is the concern that being studied will send more people to their community, when they don't necessarily want to grow from that sort of exposure. This may be something researchers don't worry about for traditional research publications (how many people read them anyway?), but as researchers also start semi-publishing online (blogs, etc), the chances of research publication having an impact on a community's membership seems to increase.
Another top concern that I've heard, and that is covered in the reading (and not unique to online communities) is that the act of being studied or the study's results (when members find and read any publication) will change how the community members interact long after the study is done. This is something that I've had to think very hard about in at least one past work, where a few interviews provided data that would have challenged how members perceived their community, and spent a lot of effort trying to change enough details about the individuals so they could not be identified with their group (fortunately we were studying a number of groups) while still preserving what I and the other researchers felt were the important characteristics and not causing validity problems with the work. I don't know what I would have done if the study only included people from one group.
Submitted by Satyendra on Fri, 01/11/2008 - 01:57.
0
points
The article reminds me of John Battelle's reference to the web and its usage creating a database of intentions of mankind. He defines a database of intentions as "The aggregate results of
every search ever entered, every result list ever tendered, and every
path taken as a result. It lives in many places, but three or four
places in particular hold a massive amount of this data (ie MSN,
Google, and Yahoo). This information represents, in aggregate form, a
place holder for the intentions of humankind - a massive database of
desires, needs, wants, and likes that can be discovered, supoenaed,
archived, tracked, and exploited to all sorts of ends."
This in a way summarizes one of the primary issues of the article. Once you have such a massive dtabase of people's needs, desires and wants it becomes a rich dataset from a research perspective and ethical ways of researching the data can make it a source of great value. However, this also makes it vulnerable to being exploited for all sorts of reasons leading to it results that can be genuine mistakes or malicious intents.
It is clear that there should be some way of managing the privacy of people while at the same time allowing genuine research to proceed as smoothly as possbile. What kind of access controls we can have to allow these diverging goals to converge is of course a much harder question.
I think it is good to understand whether we are observing people's behaviors online, or we just want to use Internet as a channel to collect data. For the former, it is a relatively new field with great research value, and more experiences are called for to help us better understand the phenomena online. Additionally, all the constraints and risks are probably a must in this case. However, for the latter, we need to know the differences of using Internet and traditional ways to collect data. The differences can be huge yet subtle, as pointed out in this paper.
I think it is important to have the distinction in mind while conducting research online, especially when we are using the Internet only as a way to collect data. Sometimes it may be better, in some cases, to collect data with traditional ways when we are not that clear about the Internet effect on the data (and subjects).
While this article focuses on the IP address as a meaningful way to identify agents, there are a number of other pieces of information internet consumers are often not aware of.
- User Agent strings -- every time you hit a web page your web browser sends information about itself, ie what OS you are running and the type and version of a browser it is. Spyware and "search toolbars" also will sometimes add ID strings to this, so that even when your IP is in a dynamic pool your activities can be tracked.
- Cookies -- many people are unaware that all the pages they view with Google adsense are being tracked continually, no matter what IP they are coming from. This is because google sets "cookies" and will record what pages you view and what ads you click on, for targeting purposes. Considering this thouroughly, not only is the publishing of material becoming part of a public record, but a large publically owned company is getting a record of what a large share of internet consumers are reading.
- Custom URLs and referrer logs -- often a URL will contain identifying information such as a session code, which can be used to identify users. My Yahoo was infamous for including users names on it's "my" portal page, and web administrators could see from pages linked from it exactly who was viewing their pages.
This article raises an important issue with the perception of privacy. It would seem that one of the biggest issues with conducting research online, is determining what is 'public' data.
As Nate mentioned, very little that we do online is not logged somewhere, and we broadcast cookies, Referrer-URLs (the page that you clicked on to get to this page try typing "javascript:alert(document.referrer)" in your browser's status bar as you browse around the internet. ), IP addresses (can be traced to ISPs -- and often mapped to particular regions of the country), and User-Agents (browser name) to every site we visit.
In addition, advertising companies like DoubleClick (now owned by google), can track individual users as they navigate across any website in their advertising network. (And double click is huge -- Google is pretty big too)
Even individual webmasters maintain history logs of how their visitors browse their website.
Does the existence of these records count as public record?
In short, we broadcast a lot of information everytime we open a browser window, even when we aren't explicitly placing content online.
However, I am confident that very few internet users are aware of this... (or the fact that anything they place online can be found on google)
It seems that if researchers could educate the average internet user, they could ethically collect more data, as users would be aware of their lack of privacy.
Will the internet begin to water down empirical data or the
process by which empirical data is captured, processed and interpreted? Will the sciences become like the arts where
the world is now everyone’s oyster? Is
the auteur of tomorrow on equal footing with the great scientist(thinker) that
may one day break the genome code or gift the world with analysis on how the
internet has rewired humanity to act, feel and think? Certainly, the net has sped up the capacity to amass knowledge in
a relatively short amount of time but qualitative research needs to be held to
very strict guidelines.
Maybe one day it will come to regarding these ”young punks”
(i.e. nick yee, pp.4) doing research as a second class style of executing
research? However shabby their methods
might be, you can’t erase the fact that this is inspiring many young future
scientists to have a go.
Overall, I would agree with Mouly’s statement above, “hence
a period of trial and error will exist”.
Has this period of trials even begun though?
One additional point, just as we at SI at some point
in our scholarly careers learn about Ethics, does anyone with an advanced
degree in Psychology know if there are required classes for outbound students
on using technology to conduct research and the different methods used to not
compromise the data and/or subject?
Does the researcher of today know the difference between cryptography,
cookies, HTML and how these can facilitate research or compromise a study?
Psychological Research Online: Opportunities and Challenges
At the request of Paul Resnick, what follows is a brief description/summary of:
Kraut, R. E., Olson, J., Manaji, M., Bruckman, A., Cohen, J. & Couper, M. (2003). Psychological Research Online: Opportunities and Challenges. American Psychologist, 59(2), 105-117.
http://www.cs.cmu.edu/~kraut/RKraut.site.files/articles/kraut04-Psycholo...
-----------------------------------------------------------------
-----------------------------------------------------------------
According to the abstract, the Internet offers new opportunities and difficulties to psychology research. "This paper describes some benefits and challenges" (p1).
The Internet provides new opportunities for psychology research. First, it makes empirical research easier in four key ways: 1) by providing access to subjects, 2) by providing effective mechanisms for observing social behavior, 3) by providing access to archival data, and 4) by providing automation and experimental control. Second, beyond improved efficiencies in performing traditional kinds of research, the Internet also is "an important phenomenon in its own right," (p6) and a virtual laboratory for examining new social phenomena.
Nonetheless, the Internet presents two important challenges to researchers: 1) data quality, and 2) the protection of human subjects.
The concern over data quality is twofold. First, the sample of human subjects accessible to online researchers is "by no means representative of the nation as a whole" (p7). Second, the Internet possesses a relative lack of control over the data-collection setting as compared to other environments.
Regarding the protection of human subjects, the authors refer to the basic ethical principles described in the "Belmont Report, prepared by the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research in 1979" which includes: 1) Respect for Persons, 2) Beneficence, and 3) Justice (p9). A helpful flowchart (Figure 1) displays "some factors relevant to Internet research influencing whether informed consent is required and must be documented" (p10/p23). The Internet is (at present) an ambiguously defined participatory space that makes it difficult to determine what information can appropriately be considered public vs. private, as well as whether an individual participant is anonymous or identifiable. These factors present risks to subjects in the form of potential harms as a result of participating in online research. Consequently, researchers must be reflective of these potential harms and utilize debriefing and confidentiality wherever and whenever possible, always mindful of the necessity of informed consent on the part of research participants. Online, meeting these ethical obligations can be more difficult than in other settings.
Some key lessons emerge from the above considerations. Researchers should acquire facility with online research practices by starting with small simple studies. Tracking numerous data points (IP addresses, etc.) from all participants allows researchers to investigate data problems later (such as sampling biases). Researchers should practice matching risks with commensurate protections for all participants involved. Finally, researchers should not simply acquire a set of skills and re-use them, but should be continually reflective of how the speed of change online demands continual revision of research methods and techniques.
In the authors' summary, they suggest that the new risks of online research "require researchers and Institutional Review Boards to keep abreast of changes in online behavior, community standards, and available technology. They also require a degree of reflection about the research process that may not be necessary in more established domains" (p22). (The authors provide a bibliography, figures, tables, and sidebars at the end of the article).
-----------------------------------------------------------------
-----------------------------------------------------------------
--------------------------------------------------------
PHartzog@umich.edu
--------------------------------------------------------
The Universe is made up of stories, not atoms.
--Muriel Rukeyser
A rookie comments
I have never done psychological research, what more on the Internet. (My experience in Internet research is analyzing my website's log files.. originally scouring the text, then with awstats, and now with Google Analytics.)
But the article allows me to appreciate the challenges presented by the Internet, in terms of data quality and human subjects. I like the flowchart to check whether informed consent is required when researching human subjects.
Great summary! Phsycology
Great summary! Phsycology researches always seem exciting but hard to me and I have little experience as well. In spite of this, the reading still seems interesting to me and the above summary is just great. The lessons drawn from the research are useful and I totally agree that the new risks of online research "require researchers and Institutional Review Boards to keep abreast of changes in online behavior, community standards, and available technology.
When IRB Approval is Needed
Kraut brings up the idea that the internet allows researchers a lot of control over environment (or at least environment of the computer screen interface). Because of this ready control, it is attractive to conduct a lot of research based on changes in interfaces. For example, a researcher could partner with an existing online community administrator to conduct a study on the effect of photos paired with user names on community activities. This type of research poses little harm to subjects. IRB processes often obstruct research - sometimes for the better. However, in situations with so little risk, IRB seems unnecessary.
The more important point that this discussion on risks and IRB brings up is the normative mechanisms that are in place in academics. Academic researchers should be brought up in an environment that teaches them the boundaries of ethical
behavior, empowering them to make correct decisions without the parenting of
the IRB. This relates to the discussion thread about Wikipedia editing. Moving away from overzealous administrators to a more self-administrated system. Good research enviroments should foster inherent ethical behavior.
The Assumption and Reality of Online Privacy
This chapter comprehensively explores the many issues involved in the possibilities of Psychological studies on people’s online behaviors. In addition to many we had long noticed such as the lack of control over data collecting process, it is particularly valuable in mentioning the ambiguities on all the notions we have clarified to some degree in the real life but still very vague in online sittings, as a new type of public space for people’s social interactions.
This place requires a considerable period of time for people to figure it out. In particular, a person uses her knowledge to make assumption on the privacy issue when participating in online activities. This knowledge determines how she would consider the issues. For example in the paper, the college student posts her family pictures on her personal website, assuming that only her friends and family members would be interested in browsing it. However technically, the reality is that everyone on the Internet can access those photos. And this fact actually can lead to the way harming people’s privacy or themselves. For example, exploring one’s friends’ profiles on the Facebook can infer something about the person that he does not want to show. When one releases some of her privacy on a particular place online, she had the assumption that it can be exclusive within the group who access here. Another example is that a guy’s on some social website’ activities behind his wife was found by a three degree friend of his wife.
The richness of information online brings the possibilities of inferring underlying relations; some time even beyond what a researcher intends to do.
Is privacy possible in 21st century?
The above summaries are very comprehensive and definitely helped me to get to the major issues of this article. Given that this reading comes under the ethics portion of the class, though, I think it's important to focus in particular on the public vs. private debate which begins on approximately p.12. What I think is most interesting is that even though the authors are highlighting the benefits and problems with conducting research online, it is not until this point in the article that they begin citing legal statutes and cases. Before this the discussion was about economics and ethics, concrete and abstract respecitvely, but the legal discussion is where things begin to blur into grey. The above cited examples begin a great discussion of public/private disclosure and the ethics of that. But the ethics and legality of a virtual space will probably be the number one thing researchers and academics battle in the coming century. Since the Internet is "public" and sites may be "private" nearly every site is a literal site of conflict. This article does a nice job of admitting this, but I find it troubling/sad that legal matters (and to some extent the ever-expanding role of IRB) is where these discussions will ultimately end up. This is especially poignant when viewed side by side with the Kim and Powazek pieces, which discuss a kind of utopia of online communities with users freely and gladly giving content.
Privacy vs. marketing
John Blair
To expand on Chris's points a bit further regarding economics, ethics and privacy. If we think back a mere 10-15 years ago, the Internet was a much more "friendly" place. For the most part, people contributed because they wanted to share the information with others. Then, as the general public started to gain more access through better tools, the economics of that many potential buyers began to take over. In the U.S. there is essentially no privacy protections guaranteed to users. The marketing departments have ensured that the bills don't become laws because then accountability and consideration of the users information would become one of their requirements, which of course would cost them. Look carefully at the breach laws we do have and you'll see that there are a multitude of conditions as to when a company actually has to inform consumers that their information may be at risk. The U.S. does a dispicable job of protecting our privacy because it's too profitable to not protect it. Privacy in the 21st century is highly unlikely given the amount of information that already exists in databases everywhere about all of us.
Regarding this particular reading, I found it much like Powazek's in that they both are providing the basic foundation considerations for the respective topicss of online communties and online research. Both articles compare some of the hurdles while providing some useful guidance if one chooses to proceed.
Reasonable Expectations
The privacy issue raised by this reading stood out as the most dubious and unresolved challenge with which researchers must contend when conducting research online. I don't claim to have a solution, but I'm more than pleased to point out the inadequacy of the current laws.
The reading states that "research involves human subjects only if data is collected through interaction with a subject or if it collects 'identifiable private information'." and further defines "private information" as information about which there is a "reasonable expectation" of privacy. Of course, a "reasonable expectation" is highly subjective. To make matters worse, many Internet users have naive and uninformed ideas about their privacy online.
As one example, informed Internet users understand that email is not private. However, as specified in the reading, there are laws on the books that protect the privacy of email messages between two individuals. If people didn't already have the expectation of privacy in email (they did anyway, by and large), a law purporting to protect email privacy certainly lends credibility to that notion. (Personally, I'd prefer not to pass laws that can't conceivably be enforced, but that's just one of the many reasons I would never make it as a legislator.) For the "reasonable expectation" principle to have any weight, I think it's important to have a better educated population. Until better legislation or better education comes along, it looks like online researchers will have to tread lightly.
This paper gives us a
This paper gives us a blueprint that which topic we can explore on the Internet and what we need to pay attention to as psychosis, especially it gives helpful information on the aspect of legality, which we(at least myself) knows little before.
As a beginning researcher, I would like to ask what would be a good research topic on the Internet from the perspective of (social) psychologists. Or what is an exciting research question when we explore users’ behaviors online? As Internet provides a broader platform to study human behavior, there are so many new phenomena generated by this new community, however, what would be the most valuable one instead of transitory events? From my opinion, those activities which also take place offline, such as E-commerce are one of the most important topics, however, how about those activities, such as Blog, which seldom emerges offline(we could not read others’ diary in daily life!!), how do we value this kind of activities or how long can it sustain active?
We have done a bunch of studies on the Internet, what’s the next hot topic? What else can we explore from the old topic? What is indeed wrong/trivial in current research results?
Psychological research online
The Internet with all of its digitized behavior is too tempting of a source of content to be resisted but the social scientist, but how to gather the data ethically? This article explores a number of interesting aspects of gathering data online, but one that I found most interesting was how a tragedy of commons scenario can emerge by overly zealous researcher practices.
There is evidence that Second Life is already suffering for research fatigue. If the users get sick of being pestered it will undoubtedly lead the administrators scrutinizing research more and more. I think it will be interesting to see how researchers conduct them selves in order to ensure that they are seen as beneficial to the online communities rather than just another online nuisance
I have always imagined that Paradise will be a kind of library.
-Jorge Luis Borges
Ethics of Online Studies
I agree with others who have posted so far that this paper seems to bring up more issues than it purports to solve, especially concerning the ambiguities of public versus private online. Yet there are a lot of practical lessons to be gained here for researchers in the field, things like: start with smaller online studies, don't overdo technology and informed consent for minimal risk studies, separate log data from identifying information, etc.
As for the issues brought up by the study, some of them are large enough that they should persuade researchers to not use the Internet for certain studies. The authors seem to make this claim - the Internet is great because it enables studies to be cheaper, easier to administer and analyze, and reach more people. But the risks include not being able to manage the user's physical environment, and the anonymous nature of the Internet which may encourage some to participate with the express purpose of damaging data. Ethical issues imposed on the researcher include whether to reveal pseudoymns, and whether it is ethical to be a 'lurker' to collect data from websites where users have no idea their communication is being collected.
Overall, this article brought up many good issues for us to have in mind as researchers, so that we are aware of the risks and benefits involved.
I thought this was a good
I thought this was a good introduction to the issues involved in online research, discussing such issues as privacy, ethics, and legality.
I think some of the privacy issues are interesting from a psychological point of view. Just as the college girl posted photos online thinking noone would look, I myself have in the past left a link to a rather personal blog on a public website that identified me, only to discover my blog being featured by a professor on the first day of my undergraduate Human Factors of Technlogy class. Bridging the gap between a researcher and an average person on the internet that doesn't think their actions through in terms of the rather permanent, ubiquitous, accessible nature of the internet is one of many challenges for internet research. Sheltered restricted-access environments, and the ever-growing apathy and mindless automatic click-through of User Agreements of all forms should make us ever more cautious.
I think Kraut's suggestions are a good beginning for tackling some of these issues, but we still have a way to go in uncomplicating internet research.
the blurring of private and public identities
I had a similar blog exposure incident, wherein the head of a nonprofit who I knew professionally only very casually, asked if I was doing ok. Unabashedly, he admitting to reading my blog on myspace and concerns for my emotional health.
I deleted my myspace account less than an hour later, not because I had something to hide, but because my angsty poetry was less than exemplary on second glance. I also saw very little resemblance in how I presented myself in writing then and who I think I am now. It didn't authentically represent me.
This is my issue with internet research, how to measure the validity of interactions and discussions that occur spontaneously and in so many different contexts that its difficult to tell how representive it really is of a population.
Another case in point: When I was doing HR for a UM department I knew we had a student coming in for an interview and looked them up on facebook so I could visually identify them. Not sure how ethical this was, but I stumbled upon a less than savory visual depiction of my candidate's zeal for Jack Daniels and "ho's in different area codes."
I was then uncomfortable interviewing this person, and wasn't sure if I had crossed some kind of ethical boundary by looking them up. My perspective on the candidate was undoubtably changed, utterly changed. There are certainly many nuances to cover in this issue, both as a researcher and professional of any sort.
Lisa McLaughlin
You make an interesting
You make an interesting point when you say how the things you once posted were no longer representative of you. I have often felt the same about my old internet selves - whether it be the middle school MUD addict part of me, or my very first webpage (on Angelfire with javascript flashing colors) or even as I've progressed through yet another two website designs to finally end up with my current site. When will I decide this site isn't "me"? (I've decided to keep it simple and grow it when I know what I want.) And why is it that people do this? I think it's a unique duality of behavior that we had yet to observe before the advent of technology to allow us to do so. Sherry Turkle's book, Life on the Screen, does a pretty good job analyzing things from the MUD days but I'd be really interested to read more modern assessments of our new psychological phenomena.
Protecting Users From Themselves
Given the horrible privacy practices that users display, I'd say that it is a real issue using data from the internet for research. I agree that IRB approval might be a bit of a hurdle for researchers to cross. I think if users and researchers are careful with the way they handle online information, there's really no need to explicitly use information that is already freely available to anyone with an internet connection.
use or ask permission?
Are you saying no need to use information that is freely available, or no need to ask permission?
Nice Reading
Kraut et al talk about the opportunities and challenges the Internet has brought up to psychology research. It may reduce cost to conduct research and make empirical research easier, but it also may decrease the quality of data and infringe personal rights. I believe not only psychology research encounters these issues, but also other disciplines may face similar problems.
I found this reading is insightful and useful for me to think about issues I have never thought about before. However, I doubted that some points that are stressed by this reading may be out of date. For example, it said when given choice of Internet or paper questionnaires, respondents still overwhelmingly choose paper (p8). From my point of view, if I was given the choice, I will defiantly choose the Internet version, since I don’t want to write. Therefore, I guess it would be important to examine the same standard or rule in different settings and times while doing research, since the environment evolve.
AOL can't keep a secret
Kraut et al were ahead of their time in 2003, and while they were framing arguments for ethics in psych experiments, foresaw privacy breaches like the 2006 AOL search log release:
"Probaby the greater risk of harm in online research comes not from the experience of participating, but from possible disclosure of personal information at a later time." (p16)
AOL gave a dataset of 20,000,000 search queries from 650,000 unique members to the academic community for psychological research. While the logs don't explicitly store personally identifiable information like a screen name, searches can of course be used to infer someone's identity. (Who doesn't search for information about themselves from time to time?) Screen Names were replaced with "anonymous" pseudonyms like "091283", which aren't aren't so anonymous...
A search query is in some ways like thinking out loud. Expectations for privacy with a search engine are likely to be a measure of how much you trust yourself to keep a secret about yourself.
Link: http://www.techcrunch.com/2006/08/06/aol-proudly-releases-massive-amount...
summary
Paul Hartzog has posted a very nice summary of the paper in the beginning, which I will not repeat here. The value of the paper is that it proposed the ethical issues related to online research, and it gives some suggestions/guidelines on how to conduct online research ethically.
I also learned the pros and cons of doing online research, which helps me better understand this tool.
Tricky world of Internet
This paper describes the intricacies of performing Psychology research online. Like other fields Internet has disrupted the research methods as well. It has presented new opportunities to collect data, process data, collaborate with other researchers and publishing the research.
With the Internet it is very easy to collect data, the marginal cost of one subject is usually close to zero. As people spend more time online they leave more social footprints, this provides a rich data for the researches to study how people are forming connections. There is already plenty of archived data that can be used for research. Internet in itself has given rise to new phenomenons like new forms of interpersonal communication.
But all these advantages come with a host of challenges like sample bias, data collection setting leading to deviant behavior, protecting identity of subjects, risks to internet subjects.
Users are putting private data in public domain. This is blurring the line between traditional public and private data domains. I think initiative like dataportability.org are in the right direction. It will encourage users to think about their as a closed entity they can control.
The description of 'publically accepted behavior' is being redefined online. I think we are developing new ethos for Internet, hence a period of trial and error will exist. Hopefully we will reach consensus quickly.
Some thoughts on Kraut et al. reading
In the beginning portion of this reading, the authors discuss in detail the reasons and benefits of conducting academic research and experiments on the internet. Obvious benefits of this method could range from saving costs to discovering new areas of interests, perhaps otherwise unrepresented in existing studies on offline communities. The authors then continue with an in-depth analysis on the potential risks and caveats of conducting research on the internet. The list of things researchers must keep in mind seems endless at first glance. However, this simply reiterates the point that with the improved access to information (made available through the internet in this case), researchers need to become even more careful and responsible for their own work and research. The most practical solution, as represented by the fact that the authors chose to close their paper with a list of recommendations and suggestions, is that best practices and improved standards of research and data collection might need to be explored.
beyond individual anonymity: anonymity of / spotlight on groups
The authors raise many important concerns for researchers working online. One addition concern I've heard from some community members (both directly and from other researchers) is the concern that being studied will send more people to their community, when they don't necessarily want to grow from that sort of exposure. This may be something researchers don't worry about for traditional research publications (how many people read them anyway?), but as researchers also start semi-publishing online (blogs, etc), the chances of research publication having an impact on a community's membership seems to increase.
Another top concern that I've heard, and that is covered in the reading (and not unique to online communities) is that the act of being studied or the study's results (when members find and read any publication) will change how the community members interact long after the study is done. This is something that I've had to think very hard about in at least one past work, where a few interviews provided data that would have challenged how members perceived their community, and spent a lot of effort trying to change enough details about the individuals so they could not be identified with their group (fortunately we were studying a number of groups) while still preserving what I and the other researchers felt were the important characteristics and not causing validity problems with the work. I don't know what I would have done if the study only included people from one group.
Also from my personal experiences, this is a thread of some users discussing a course project that studied their community and others.
The database of intentions
The article reminds me of John Battelle's reference to the web and its usage creating a database of intentions of mankind. He defines a database of intentions as "The aggregate results of
every search ever entered, every result list ever tendered, and every
path taken as a result. It lives in many places, but three or four
places in particular hold a massive amount of this data (ie MSN,
Google, and Yahoo). This information represents, in aggregate form, a
place holder for the intentions of humankind - a massive database of
desires, needs, wants, and likes that can be discovered, supoenaed,
archived, tracked, and exploited to all sorts of ends."
This in a way summarizes one of the primary issues of the article. Once you have such a massive dtabase of people's needs, desires and wants it becomes a rich dataset from a research perspective and ethical ways of researching the data can make it a source of great value. However, this also makes it vulnerable to being exploited for all sorts of reasons leading to it results that can be genuine mistakes or malicious intents.
It is clear that there should be some way of managing the privacy of people while at the same time allowing genuine research to proceed as smoothly as possbile. What kind of access controls we can have to allow these diverging goals to converge is of course a much harder question.
A target or a channel?
I think it is good to understand whether we are observing people's behaviors online, or we just want to use Internet as a channel to collect data. For the former, it is a relatively new field with great research value, and more experiences are called for to help us better understand the phenomena online. Additionally, all the constraints and risks are probably a must in this case. However, for the latter, we need to know the differences of using Internet and traditional ways to collect data. The differences can be huge yet subtle, as pointed out in this paper.
I think it is important to have the distinction in mind while conducting research online, especially when we are using the Internet only as a way to collect data. Sometimes it may be better, in some cases, to collect data with traditional ways when we are not that clear about the Internet effect on the data (and subjects).
identification on the internet
While this article focuses on the IP address as a meaningful way to identify agents, there are a number of other pieces of information internet consumers are often not aware of.
- User Agent strings -- every time you hit a web page your web browser sends information about itself, ie what OS you are running and the type and version of a browser it is. Spyware and "search toolbars" also will sometimes add ID strings to this, so that even when your IP is in a dynamic pool your activities can be tracked.
- Cookies -- many people are unaware that all the pages they view with Google adsense are being tracked continually, no matter what IP they are coming from. This is because google sets "cookies" and will record what pages you view and what ads you click on, for targeting purposes. Considering this thouroughly, not only is the publishing of material becoming part of a public record, but a large publically owned company is getting a record of what a large share of internet consumers are reading.
- Custom URLs and referrer logs -- often a URL will contain identifying information such as a session code, which can be used to identify users. My Yahoo was infamous for including users names on it's "my" portal page, and web administrators could see from pages linked from it exactly who was viewing their pages.
--------------------------------------------------------------
oostendo@umich.edu
--------------------------------------------------------------
Perception of Privacy
This article raises an important issue with the perception of privacy. It would seem that one of the biggest issues with conducting research online, is determining what is 'public' data.
As Nate mentioned, very little that we do online is not logged somewhere, and we broadcast cookies, Referrer-URLs (the page that you clicked on to get to this page try typing "javascript:alert(document.referrer)" in your browser's status bar as you browse around the internet. ), IP addresses (can be traced to ISPs -- and often mapped to particular regions of the country), and User-Agents (browser name) to every site we visit.
In addition, advertising companies like DoubleClick (now owned by google), can track individual users as they navigate across any website in their advertising network. (And double click is huge -- Google is pretty big too)
Even individual webmasters maintain history logs of how their visitors browse their website.
Does the existence of these records count as public record?
In short, we broadcast a lot of information everytime we open a browser window, even when we aren't explicitly placing content online.
However, I am confident that very few internet users are aware of this... (or the fact that anything they place online can be found on google)
It seems that if researchers could educate the average internet user, they could ethically collect more data, as users would be aware of their lack of privacy.
Will the internet begin to
Will the internet begin to water down empirical data or the
process by which empirical data is captured, processed and interpreted? Will the sciences become like the arts where
the world is now everyone’s oyster? Is
the auteur of tomorrow on equal footing with the great scientist(thinker) that
may one day break the genome code or gift the world with analysis on how the
internet has rewired humanity to act, feel and think? Certainly, the net has sped up the capacity to amass knowledge in
a relatively short amount of time but qualitative research needs to be held to
very strict guidelines.
Maybe one day it will come to regarding these ”young punks”
(i.e. nick yee, pp.4) doing research as a second class style of executing
research? However shabby their methods
might be, you can’t erase the fact that this is inspiring many young future
scientists to have a go.
Overall, I would agree with Mouly’s statement above, “hence
a period of trial and error will exist”.
Has this period of trials even begun though?
One additional point, just as we at SI at some point
in our scholarly careers learn about Ethics, does anyone with an advanced
degree in Psychology know if there are required classes for outbound students
on using technology to conduct research and the different methods used to not
compromise the data and/or subject?
Does the researcher of today know the difference between cryptography,
cookies, HTML and how these can facilitate research or compromise a study?