X-Git-Url: https://projects.mako.cc/source/bmh-research_statement/blobdiff_plain/9a949f29fc3ce175d3e6f5e04baaf537ab948b50..2d8640cdc15752dec4daa249664baf86884a4085:/research_statement.tex diff --git a/research_statement.tex b/research_statement.tex index 78cb500..ede334a 100644 --- a/research_statement.tex +++ b/research_statement.tex @@ -49,41 +49,40 @@ \maketitle -My research is focused on collective action in online communities and -seeks to understand why some attempts at collaborative production -- -like Wikipedia and Linux -- build large volunteer communities while -the vast majority never attract even a second contributor. I am -particularly interested in how the design of communication and -information technologies shape social outcomes like the decision to -join a community or contribute to a public good. My research is deeply -interdisciplinary and lies at the intersection of sociology, -communication, and human-computer interaction. I analyze data from -online communities that make failures of collective action newly -visible with ``big data'' research methods from software engineering -to answer fundamental social scientific questions. - -Seeking to understand the determinants of collective action, my -research has been driven by three overlapping themes: (1) -population-level observational studies comparing failures to build -communities to the rare successful attempts; (2) attention to the role -of reputation and status in the mobilization of volunteers; and (3) -analyses of design changes as ``natural experiments'' to build a -deeper, and often causal, understanding of social processes from +I study collective action in online communities and seek to understand +why some attempts at collaborative production -- like Wikipedia and +Linux -- build large volunteer communities while the vast majority +never attract even a second contributor. I am particularly interested +in how the design of communication and information technologies shape +fundemental social outcomes with broad theoretical and practical +implications -- like the decision to join a community or contribute to +a public good. My research is deeply interdisciplinary, consists +primarily of ``big data'' quantitative analyses, and lies at the +intersection of sociology, communication, and human-computer +interaction. + +Using Internet-based peer production projects as my research settings, +my work seeks to understand the conditions for collective action using +observational data from real communities. This work has been shaped +by three complentary approaches: (1) the comparison of failures to +build communities to rare successful attempts through the use of +projects as the unit of analysis; (2) attention to the role of +reputation and status in the mobilization of volunteers; and (3) +analysis of design changes as ``natural experiments'' building a +deeper, and often causal, understanding of social processes using observational data. Nearly all of my work incorporates at least two of -these themes. +these approaches. -\section{Population-Level Observational Studies} +\section{Projects As Unit of Analysis} Although there have been many thousands of studies of online -collective action and peer production, the vast majority of these -studies have only considered successful projects like Wikipedia and -GNU/Linux. The majority of research on collective action -- online -and off -- has only considered projects that have successfully -mobilized. In this sense, most previous analyses of -collection action have systematically selected on their dependent -variable. Most of my research treats projects as the unit of analysis -and mobilization as the dependent variable to compare successful -examples of collective actions to failures. +collective action the vast majority have only considered successful +projects like Wikipedia and Linux. The majority of research on +collective action -- on and offline -- has only considered projects +that have successfully mobilized. In this sense, most previous +analyses of collection action have systematically selected on their +dependent variable. Most of my research treats projects as the unit of +analysis and collective action as the outcome of interest. % \begin{wrapfigure}{r}{0.4\textwidth} % \begin{centering} @@ -104,20 +103,18 @@ examples of collective actions to failures. \vspace{-2em} \end{wrapfigure} - -For example, in a working paper that is part of my dissertation, I -compare Wikipedia to seven attempts to create online collaborative -encyclopedia projects that were launched before Wikipedia +In one study, I compare Wikipedia to seven attempts to create online +collaborative encyclopedia projects that were launched previously \cite{hill_almost_2012}. Using an inductive, grounded-theory based analysis of founder interviews and archival data, I propose four -hypothesis to explain why Wikipedia attracted many more contributors -than similar projects. Although the paper's methods diverge from the +hypothesis to explain why Wikipedia attracted many more +contributors. Although the paper's methods diverge from the quantitative, ``big data'' approach typical of most of my work, the research question and strategy is representative. I have also followed this strategy in a series of quantitative -studies of the Scratch online community: a public website with a large -community of users who create, share, and remix interactive media. The +studies of the Scratch online community: a public website where a large +community of users create, share, and remix interactive media. The community is built around the Scratch programming environment: a freely downloadable desktop application that allows amateur creators to combine media with programming code (see Figure @@ -126,6 +123,17 @@ designed to promote collaboration through content remixing, only about ten percent of Scratch projects attract a second contributor. +In one study, forthcoming in American Behavioral Scientist, I test +several of the most widely cited theories associated with +``generativity'' (i.e., qualities of technology or content that make +some works more fertile ground for collaboration). I find some support +for existing theory but also find that, across the board, factors +associated with more collaboration are also associated with less +original and transformative types of joint-work +\cite{hill_remixing_2012}. In another study of Scratch, I show that +this type of superficial collaboration leads to negative reactions and +community displeasure \cite{hill_responses_2010}. + \begin{wrapfigure}{l}{2.6in} \begin{centering} \includegraphics[width=2.6in]{figures/frontpage_modified-topremix.png} @@ -136,36 +144,25 @@ contributor. \vspace{-2em} \end{wrapfigure} -In one study, forthcoming in American Behavioral Scientist, I test -several of the most commonly cited theories associated with -``generativity'' (i.e., qualities of technology or content that make -some works more fertile ground for collaboration). I find some support -for previous theories but also find that, across the board, factors -associated with increased collaboration are also associated with less -original and transformative modes of joint-work -\cite{hill_remixing_2012}. In another study of Scratch, I show that -more superficial collaboration leads to negative reactions and -community displeasure \cite{hill_responses_2010}. - -I am conducting a similar population-level analysis in a new dataset I -have created for my dissertation that includes 80,000 public attempts -at wikis (i.e., public, editable, websites similar to Wikipedia). In -my first working paper using this dataset, I consider -inter-organizational effects of competition for volunteer labor and -find little support for a widely cited ecological model of collective -action from sociology that treats volunteer labor as fixed and finite -resource. Instead, I show that contributions to different wikis on the -same topic or theme are driven primarily by environment-level changes -in interest and that projects can even benefit from complimentarities -and synergies \cite{hill_is_2012}. +This year, I am conducting a population-level analysis in a new +dataset I have created that includes 80,000 attempts at wikis (i.e., +public, editable, websites similar to Wikipedia). In my first working +paper using this dataset, I consider inter-organizational effects of +competition for volunteer labor and find little support for a widely +cited ecological model of collective action from sociology that treats +volunteer labor as a fixed and finite resource. Instead, I show that +contributions to different wikis on the same topic or theme are driven +primarily by environment-level changes in interest and that projects +may even benefit from complimentarities and synergies +\cite{hill_is_2012}. \section{Reputation and Status} Although empirical research comparing successful and unsuccessful peer -projects has been rare, theories have been widespread. No theory has -been more influential than the suggestion that, in the absence of -pecuniary rewards, contributions to online public good are driven by -the possibility of increased reputation and status conferred upon +production projects has been rare, theories have been widespread. No +theory has been more influential than the suggestion that, in the +absence of pecuniary rewards, contributions to online public goods are +driven by the possibility of increased reputation and status for contributors. \begin{wrapfigure}{r}{0.3\textwidth} @@ -180,65 +177,65 @@ contributors. \end{wrapfigure} In a study of status-based awards in Wikipedia called ``barnstars'' -(see Figure \ref{fig:barnstar}) that I will be submitting to a major -sociology journal by the end of this year, I provide an empirical test -of an influential status-based theory of collective action from +(see Figure \ref{fig:barnstar}) I provide an empirical test of an +influential status-based theory of collective action from sociology. Although the study finds support for the widely hypothesized ``virtuous cycle'' of status rewards both causing and being caused by contributions, it also finds that this effect is -limited to a sub-population of contributors to Wikipedia -- i.e., -those who show off their awards \cite{hill_status_2012}. This result +limited to a sub-population of Wikipedia contributors -- ``signalers'' +who show off their awards \cite{hill_status_2012}. This result has broad implications for both status-based theories of collective action as well the design of reputation-based rewards. In a mixed methods study of Scratch, nominated for best paper at the CHI 2011 conference \cite{monroy-hernandez_computers_2011}, I -presented both a quantitative analysis of a design change and in-depth +present both a quantitative analysis of a design change and in-depth interviews of users to demonstrate how credit-giving is ineffective when it stems from an automated system because systems fail to reinforce status-ordering with credible human expressions of social deference and gratitude. +%\newpage \section{Design-Driven Natural Experiments} +Although nearly all of my work has important implications for the +design of socio-technical systems, I have structured much of my work +around the evaluation of technological design changes. In several +papers, I treat design changes as ``natural experiments'' that +exogenously change the ways that social structure is enacted. By doing +so, I can both build causal understandings of social phenomena from +field data, and can tighten the distance between theory and design. + \begin{wrapfigure}{r}{0.25\textwidth} + \vspace{-1em} \begin{centering} \includegraphics[width=1.5in]{figures/lilypad.png} \caption{A image of the LilyPad Arduino microcontroller.} \label{fig:lilypad} \end{centering} + \vspace{-1em} \end{wrapfigure} -Although nearly all of my work has important implications for the -design of socio-technical systems, I have structured much of my work -around the evaluation of technological design changes. In several -papers, I treat design changes as ``natural experiments'' that -exogenously change the ways that social structure is enacted in order -to both build causal understanding of social phenomena from field data -and to tighten the distance between theory and and design. - For example, to evaluate the impact of status-based incentives and collaboration in Scratch, I use a regression discontinuity framework to measure the causal effect of increased status for collaboration -\cite{hill_causal_2012}. In that study, which I am preparing for -submission to a communication journal this fall, I show that -highlighting collaborative projects on the Scratch web page (see the -bottom of Figure \ref{fig:scratchfrontpage}) resulted in more -collaboration but also caused a decrease in the amount of total effort -exerted by contributors. Speaking to fundamental sociological work in -the literature on collective action, I present evidence that this -decrease is driven by both an the influx of new contributors and a -decrease in the effort and contributions of established participants. - -In other papers, I have helped analyze sales records of hobbyist -microcontrollers to suggest that relatively simple design changes in -the \emph{LilyPad Arduino} -- a electronics toolkit minimally -re-designed for women and girls (see Figure \ref{fig:lilypad}) -- can -lead to large increases in the proportion of women contributors and -drastic shifts in the type of projects created -\cite{buechley_lilypad_2010}. In other work, I have explored how -technical errors may be able to provide similar opportunities for -analysis \cite{hill_revealing_2010}. +\cite{hill_causal_2012}. I show that highlighting collaborative +projects on the Scratch web page (see the bottom of Figure +\ref{fig:scratchfrontpage}) resulted in more collaboration but also +caused a decrease in the amount of total effort exerted by +contributors. Speaking to fundamental sociological work in the +literature on collective action, I present evidence that this decrease +is driven by both an the influx of new contributors and a decrease in +the effort and contributions of established participants. + +In other work, I have analyzed sales records of hobbyist +microcontrollers to argue that relatively simple design changes in the +\emph{LilyPad Arduino} -- a electronics toolkit minimally re-designed +for women and girls (see Figure \ref{fig:lilypad}) -- lead to large +increases in the proportion of women contributors and drastic shifts +in the type of projects created \cite{buechley_lilypad_2010}. I have +also explored how technical errors may be able to provide similar +opportunities for analysis \cite{hill_revealing_2010}. % or changes in socio-technical systems describing responsibility for a piece of software can lead to an important impact in the type and structure of contributions in peer production \cite{michlmayr_quality_2003} @@ -246,52 +243,52 @@ analysis \cite{hill_revealing_2010}. My research agenda involves further exploration of the determinants of collection action online -- especially using a series of large new -datasets I have assembled for my dissertation. I plan to both continue -on this research trajectory and to create new social and technical +datasets I have recently assembled. I plan to both continue on this +research trajectory and to create new social and technical infrastructure that will allow others researchers to join me in ``big -data'' observational research in active communities. This section +data'' observational research with active communities. This section outlines some future directions I plan to explore. -\emph{Toolkits for Experimental Social Design} -- My research has been -possible through personal relationships I have with a series of -organizations with large active online communities (e.g., the MIT -Media Lab and the Wikimedia Foundation). These organizations, like -many others, make design changes to the software that supports their -communities to encourage contributions and improve aspects of their -users' experiences. Most of the time, these organizations have very -little idea if these changes are effective. I plan to build on my own -experience to create a technical framework, and a network of academic -and practitioner collaborators, to facilitate well-designed natural -experiments by the hosts of large online communities and a system for -data sharing that allows for academic evaluation of these experiments. - \emph{Understanding the Relationship Between Collective Action and Performance} -- My work has treated collective action and production as ends in themselves and has largely avoided the consideration of issues of performance, efficiency, and quality. Using my existing datasets, I plan to compare the performance of collaborative production to individually produced works to understand when -successful collection action leads to higher performance and -quality. In a manuscript currently under review using data from -Scratch, I show important limitations of collaboration in remixing -quality, particularly in regards to more artistic or media-intensive -works \cite{hill_cost_2012}. I will explore this direction in future -work. - -\emph{Integrated Theory of Design for Collective Action} -- -My studies of status provide a detailed understanding of the dynamics -of collection action in relation to one important independent -variable. In future work, I plan to evaluate the effect of governance -and different systems of authority, framing, modularity and project -complexity. In the long term, I hope to work toward a broad set of -principles of design for online collection action and community. - -In graduate school, I have been fortunate to collaborate with many -co-authors in many academic departments and I intend to continue going -forward. In sum, my research uses design to contribute to social -scientific theories of collective action, and uses theories of -collective action to influence design. I believe my work offers -implications and opportunities for a broad range of disciplines and +successful collection action leads to increased performance. For +example, in a manuscript currently under review using data from +Scratch, I show important limitations of collaboration through +remixing in regards to project quality, particularly for more artistic +or media-intensive works \cite{hill_cost_2012}. + +\emph{Integrated Theory of Design for Collective Action} -- My studies +of status and reputation provide a detailed understanding of the dynamics of +collection action in relation to one set of important predictors. In future +work, I plan to evaluate the effect of governance and different +systems of authority, framing, modularity and project complexity. In +the long term, I hope to offer a broad set of principles of +design for online collection action and community. + +\emph{Toolkits for Experimental Social Design} -- My research has been +possible through personal relationships I have with a series of +organizations with large, active, online communities (e.g., the MIT +Media Lab and the Wikimedia Foundation). These organizations, like +many others, make design changes to the software that supports their +communities to encourage contributions and improve users' +experiences. Most of the time, these organizations have very little +idea if these changes are effective. I plan to seek funding for, and +to create, a technical framework and a network of academic and +practitioner collaborators, to facilitate well-designed natural +experiments by the hosts of large online communities and to share data +that allows for academic evaluation of these experiments. + +Although I study cooperation, I also practice it. In graduate school, +I have collaborated with a large and engaged group of co-authors in +many academic departments. I intend to continue doing so. In sum, my +research uses design to contribute to social scientific theories of +collective action, and uses theories of collective action to influence +design. Although my research settings are online communities, I +believe my work has implications for a broad range of disciplines and fields. % bibliography here