From: Benjamin Mako Hill Date: Fri, 7 Sep 2012 14:21:29 +0000 (-0400) Subject: added draft of research statement X-Git-Url: https://projects.mako.cc/source/bmh-research_statement/commitdiff_plain/3ad5d7d91cefffdb63b6fcf83d47dd4b285cc82d added draft of research statement --- diff --git a/refs.bib b/refs.bib index 90822b9..52c1387 100644 --- a/refs.bib +++ b/refs.bib @@ -41,7 +41,7 @@ title = {Computers can't give credit: how automatic attribution falls short in an online remixing community}, booktitle = {Proceedings of the 2011 annual conference on Human factors in computing systems}, publisher = {{ACM}}, - author = {Monroy-Hernandez, Andres and Hill, Benjamin Mako and Gonzalez-Rivero, Jazmin and boyd, danah}, + author = {Monroy-Hern\'{a}ndez, Andr\'{e}s and Hill, Benjamin Mako and Gonzalez-Rivero, Jazmin and boyd, danah}, year = {2011}, pages = {3421--3430} }, diff --git a/research_statement.tex b/research_statement.tex index f9b94b5..72688c8 100644 --- a/research_statement.tex +++ b/research_statement.tex @@ -57,7 +57,7 @@ particularly interested in how the design of communication and information technologies shape social outcomes like the decision to join a community or contribute to a public good. My research is deeply interdisciplinary and lies at the intersection of sociology, -communication, and human-computer interaction. I combine data from +communication, and human-computer interaction. I analyze data from online communities that make failures of collective action newly visible with ``big data'' research methods from software engineering to answer fundamental social scientific questions. @@ -66,22 +66,23 @@ Seeking to understand the determinants of collective action, my research has been driven by three overlapping themes: (1) population-level observational studies comparing failures to build communities to the rare successful attempts; (2) attention to the role -of reputation and status in collective action; and (3) analyses of -design changes treated as ``natural experiments'' to build a deeper, -and often causal, understanding of social processes from observational -data. Almost all of my work incorporates at least two of these themes. +of reputation and status in the mobilization of volunteers; and (3) +analyses of design changes as ``natural experiments'' to build a +deeper, and often causal, understanding of social processes from +observational data. Nearly all of my work incorporates at least two of +these themes. \section{Population-Level Observational Studies} Although there have been many thousands of studies of online collective action and peer production, the vast majority of these studies have only considered successful projects like Wikipedia and -GNU/Linux. The majority of research on collective action -- both -online and off -- has only considered projects that have successfully -mobilized contributors. In this sense, most previous analyses of +GNU/Linux. The majority of research on collective action -- online +and off -- has only considered projects that have successfully +mobilized. In this sense, most previous analyses of collection action have systematically selected on their dependent variable. Most of my research treats projects as the unit of analysis -and mobilization as the dependent variable, comparing successful +and mobilization as the dependent variable to compare successful examples of collective actions to failures. % \begin{wrapfigure}{r}{0.4\textwidth} @@ -96,7 +97,7 @@ examples of collective actions to failures. \begin{wrapfigure}{r}{2.6in} \begin{centering} \includegraphics[width=2.6in]{figures/scratch_screenshot_default.png} - \caption{A screenshot of the Scratch desktop programming environment + \caption{A screenshot of the Scratch programming environment where users create animations and interactive games.} \label{fig:scratchapp} \end{centering} @@ -107,22 +108,22 @@ examples of collective actions to failures. For example, in a working paper that is part of my dissertation, I compare Wikipedia to seven attempts to create online collaborative encyclopedia projects that were launched before Wikipedia -\cite{hill_almost_2012}. Using a inductive, grounded-theory based +\cite{hill_almost_2012}. Using an inductive, grounded-theory based analysis of founder interviews and archival data, I propose four -hypothesis for why Wikipedia built a large community while extremely -similar projects struggled to do so. Although the paper's methods -diverge from the quantitative, ``big data'' approach typical of most -of my work, the research question and strategy is representative. +hypothesis to explain why Wikipedia attracted many more contributors +than similar projects. Although the paper's methods diverge from the +quantitative, ``big data'' approach typical of most of my work, the +research question and strategy is representative. -I have also addressed this question in a series of quantitative +I have also followed this strategy in a series of quantitative studies of the Scratch online community: a public website with a large community of users who create, share, and remix interactive media. The community is built around the Scratch programming environment: a freely downloadable desktop application that allows amateur creators to combine media with programming code (see Figure -\ref{fig:scratchapp}. Despite the fact that Scratch is a community +\ref{fig:scratchapp}). Despite the fact that Scratch is a community designed to promote collaboration through content remixing, only about -ten percent of Scratch projects will attract even a second +ten percent of Scratch projects attract a second contributor. \begin{wrapfigure}{l}{2.6in} @@ -136,39 +137,36 @@ contributor. \end{wrapfigure} In one study, forthcoming in American Behavioral Scientist, I test -three of the most commonly cited theories associated with +several of the most commonly cited theories associated with ``generativity'' (i.e., qualities of technology or content that make -some works more fertile group for collaboration). I find some support +some works more fertile ground for collaboration). I find some support for previous theories but also find that, across the board, factors -associated with increased collaboration are associated with less -original and less transformative modes of joint-work -\cite{hill_remixing_2012}. In another published study using data from -Scratch, I show that more superficial collaboration leads to negative -reactions and displeasure in the community -\cite{hill_responses_2010}. - -I am conducting a similar population based analysis in a new dataset I -have created with a population of 80,000 public attempts at ``wikis'' -(i.e., public websites using the same infrastructure as Wikipedia). In -a working paper using this dataset, I consider inter-organizational -effects of competition for volunteer labor and find little support for -a widely cited ecological model of collective action from sociology -that treats volunteer labor as fixed and finite resource. Using data -on wikis hosted by both Wikia and the Wikimedia Foundation, I show -that contributions across peer production projects are driven by -environment-level changes in interest and that projects can even -benefit from complimentarities between similar projects -\cite{hill_is_2012}. +associated with increased collaboration are also associated with less +original and transformative modes of joint-work +\cite{hill_remixing_2012}. In another study of Scratch, I show that +more superficial collaboration leads to negative reactions and +community displeasure \cite{hill_responses_2010}. + +I am conducting a similar population-level analysis in a new dataset I +have created for my dissertation that includes 80,000 public attempts +at wikis (i.e., public, editable, websites similar to Wikipedia). In +my first working paper using this dataset, I consider +inter-organizational effects of competition for volunteer labor and +find little support for a widely cited ecological model of collective +action from sociology that treats volunteer labor as fixed and finite +resource. Instead, I show that contributions to different wikis on the +same topic or theme are driven primarily by environment-level changes +in interest and that projects can even benefit from complimentarities +and synergies \cite{hill_is_2012}. \section{Reputation and Status} Although empirical research comparing successful and unsuccessful peer -projects has been rare, theories offering to explain the success of -collective action online have been widespread. No theory has been more -influential or widespread than the suggestion that, in the absence of -pecuniary rewards, engagement in collective action in online -communities is driven by the possibility of increased reputation and -status conferred upon contributors to an online public good. +projects has been rare, theories have been widespread. No theory has +been more influential than the suggestion that, in the absence of +pecuniary rewards, contributions to online public good are driven by +the possibility of increased reputation and status conferred upon +contributors. \begin{wrapfigure}{r}{0.3\textwidth} \vspace{-1em} @@ -186,20 +184,20 @@ In a study of status-based awards in Wikipedia called ``barnstars'' sociology journal by the end of this year, I provide an empirical test of an influential status-based theory of collective action from sociology. Although the study finds support for the widely -hypothesized virtuous cycle of status rewarding active contributors, -it also finds that these effects are limited to a sub-population of -Wikipedia contributors -- i.e., those who show their awards off to -others in virtual trophy cases \cite{hill_status_2012}. This result +hypothesized ``virtuous cycle'' of status rewards both causing and +being caused by contributions, it also finds that this effect is +limited to a sub-population of contributors to Wikipedia -- i.e., +those who show off their awards \cite{hill_status_2012}. This result has broad implications for both status-based theories of collective -action as well the design of reputation-based reward systems. +action as well the design of reputation-based rewards. -In another study of Scratch, nominated for best paper at the CHI 2011 -conference \cite{monroy-hernandez_computers_2011}, I paired a -quantitative analysis of a design change with in-depth interviews of -users to demonstrate how credit-giving is ineffective when it stems -from an automated system because the system fails to reinforce -status-ordering through credible human expressions of social deference -and gratitude. +In a mixed methods study of Scratch, nominated for best paper at the +CHI 2011 conference \cite{monroy-hernandez_computers_2011}, I +presented both a quantitative analysis of a design change and in-depth +interviews of users to demonstrate how credit-giving is ineffective +when it stems from an automated system because systems fail to +reinforce status-ordering with credible human expressions of social +deference and gratitude. \section{Design-Driven Natural Experiments} @@ -213,32 +211,31 @@ and gratitude. Although nearly all of my work has important implications for the design of socio-technical systems, I have structured much of my work -around the evaluation of design interventions. In several papers, I -treat design changes in technologies as ``natural experiments'' that +around the evaluation of technological design changes. In several papers, I +treat design changes as ``natural experiments'' that exogenously change the ways that social structure is enacted in order -to both build causal understanding from field data and to tighten the -distance between theories and and design. +to both build causal understanding of social phenomena from field data and to tighten the +distance between theory and and design. For example, to evaluate the impact of status-based incentives and collaboration in Scratch, I use a regression discontinuity framework -to measure the causal effect of increased status for collaboration in -Scratch \cite{hill_causal_2012}. In that study, which I am preparing -for submission to a communication journal this fall, I show that the -highlighting the work of the authors of collaborative projects on the -Scratch web page (see the bottom of Figure \ref{fig:scratchfrontpage}) -resulted in more collaboration but also caused a decrease in the -amount of total effort exerted by contributors. Speaking to -fundamental sociological work in the literature on collective action, -I present evidence that this decrease is driven by both an the influx -of new contributors and a decrease in the effort and contributions of -established contributors. - -In other papers, I have helped analyze sales records of hobbyists +to measure the causal effect of increased status for collaboration +\cite{hill_causal_2012}. In that study, which I am preparing for +submission to a communication journal this fall, I show that +highlighting collaborative projects on the Scratch web page (see the +bottom of Figure \ref{fig:scratchfrontpage}) resulted in more +collaboration but also caused a decrease in the amount of total effort +exerted by contributors. Speaking to fundamental sociological work in +the literature on collective action, I present evidence that this +decrease is driven by both an the influx of new contributors and a +decrease in the effort and contributions of established participants. + +In other papers, I have helped analyze sales records of hobbyist microcontrollers to suggest that relatively simple design changes in the \emph{LilyPad Arduino} -- a electronics toolkit minimally re-designed for women and girls (see Figure \ref{fig:lilypad}) -- can -lead to large increases in the proportion of women contributors -through important shifts in the type of projects created +lead to large increases in the proportion of women contributors and +drastic shifts in the type of projects created \cite{buechley_lilypad_2010}. In other work, I have explored how technical errors may be able to provide similar opportunities for analysis \cite{hill_revealing_2010}. @@ -249,38 +246,36 @@ analysis \cite{hill_revealing_2010}. My research agenda involves further exploration of the determinants of collection action online -- especially using a series of large new -datasets I have built in my dissertation research. I plan to both -continue this research trajectory and to create new social and -technical infrastructure that will allow others researchers to join me -in the type ``big data'' observational research in active communities -that my research involves. This section outlines some future -directions I plan to explore. +datasets I have assembled for my dissertation. I plan to both continue +on this research trajectory and to create new social and technical +infrastructure that will allow others researchers to join me in ``big +data'' observational research in active communities. This section +outlines some future directions I plan to explore. \emph{Toolkits for Experimental Social Design} -- My research has been -possible through relationships I have with a series of organizations -with large active online communities (e.g., the MIT Media Lab and the -Wikimedia Foundation). These organizations, like many others, make -design changes to the software that supports their communities to -encourage contributions and improve aspects of their users' -experiences. Most of the time, these organizations have very little -idea if these changes work. I plan to build on my own experience -evaluating natural experiments in online communities to create a -technical framework, and a network of academic and practitioner -collaborators, to facilitate well-designed experiments by the hosts of -large online communities and a system for data sharing that allows for -academic evaluation of these experiments. +possible through personal relationships I have with a series of +organizations with large active online communities (e.g., the MIT +Media Lab and the Wikimedia Foundation). These organizations, like +many others, make design changes to the software that supports their +communities to encourage contributions and improve aspects of their +users' experiences. Most of the time, these organizations have very +little idea if these changes are effective. I plan to build on my own +experience to create a technical framework, and a network of academic +and practitioner collaborators, to facilitate well-designed natural +experiments by the hosts of large online communities and a system for +data sharing that allows for academic evaluation of these experiments. \emph{Understanding the Relationship Between Collective Action and Performance} -- My work has treated collective action and production as ends in themselves and has largely avoided the consideration of -issues of performance, efficiency, or quality. Using my existing -datasets, I will compare the performance of collaborative production -to individually produced works to understand when successful -collection action leads to higher performance or quality. In a -manuscript currently under review using data from Scratch, I show -important limitations of collaboration in remixing quality, -particularly in regards to more artistic or media-intensive projects -\cite{hill_cost_2012}. I will further explore this direction in future +issues of performance, efficiency, and quality. Using my existing +datasets, I plan to compare the performance of collaborative +production to individually produced works to understand when +successful collection action leads to higher performance and +quality. In a manuscript currently under review using data from +Scratch, I show important limitations of collaboration in remixing +quality, particularly in regards to more artistic or media-intensive +works \cite{hill_cost_2012}. I will explore this direction in future work. \emph{Integrated Theory of Design for Collective Action} -- @@ -289,14 +284,15 @@ of collection action in relation to one important independent variable. In future work, I plan to evaluate the effect of governance and different systems of authority, framing, modularity and project complexity. In the long term, I hope to work toward a broad set of -principles of design for collection action and community. +principles of design for online collection action and community. In graduate school, I have been fortunate to collaborate with many -co-authors in many academic departments and I intend to continue this -tradition going forward. In sum, my research uses design to contribute -to social scientific theories of collective action, and using theories -of collective action to influence design and offers implications and -opportunities for a broad range of disciplines and fields. +co-authors in many academic departments and I intend to continue going +forward. In sum, my research uses design to contribute to social +scientific theories of collective action, and uses theories of +collective action to influence design. I believe my work offers +implications and opportunities for a broad range of disciplines and +fields. % bibliography here \renewcommand{\bibsection}{\section{\bibname}\prebibhook}