1 \documentclass[xcolor=dvipsnames]{beamer}
3 % set up the file to create notes in the output PDFs
7 \renewcommand{\rmdefault}{ugm}
8 \usepackage[garamond]{mathdesign}
10 \renewcommand{\sfdefault}{phv}
14 \usepackage[utf8x]{inputenc}
15 \usepackage[T1]{fontenc}
18 % add tikz and a bunch of tikz foo
20 \usetikzlibrary{shapes,shapes.misc,backgrounds,fit,positioning}
21 \tikzstyle{every picture}+=[overlay,remember picture]
23 % add functions to circle parts of slides (e.g., in tables)
24 \newcommand\marktopleft[1]{%
25 \tikz[overlay,remember picture]
26 \node (marker-#1-a) at (0,1.5ex) {};%
28 \newcommand\markbottomright[1]{%
29 \tikz[overlay,remember picture]
30 \node (marker-#1-b) at (0,0) {};%
31 \tikz[overlay,remember picture,dashed,inner sep=3pt]
32 \node[violet!75,ultra thick,draw,rounded rectangle,fit=(marker-#1-a.center) (marker-#1-b.center)] {};%
35 % DEPRECATED function to build a huge centered dropshadow
36 \newcommand\dropshadow[3]{%
37 \node[black!30!white] at (#1+0.1,#2-0.1) {
38 \scalebox{2}{\Huge \textbf{#3}}
41 \scalebox{2}{\Huge \e{#3}}
45 % more flexible non-tikz alternative with no dropshadow
46 \newlength{\centertxtlen}
48 \newcommand\centertext[2]{%
49 \setlength{\centertxtlen}{#1}%
50 \setlength{\centertxtlen}{0.35\centertxtlen}%
52 \fontsize{#1}{2\centertxtlen}\selectfont
58 % create an empty quotetxt so we can reuse it
59 \newcommand{\quotetxt}{}
61 % add function to stop numbering appendix slides
62 \newcommand{\backupbegin}{
63 \newcounter{framenumberappendix}
64 \setcounter{framenumberappendix}{\value{framenumber}}
66 \newcommand{\backupend}{
67 \addtocounter{framenumberappendix}{-\value{framenumber}}
68 \addtocounter{framenumber}{\value{framenumberappendix}}
71 % packages i use in essentially every document
74 % \usepackage{dcolumn}
75 % \usepackage{booktabs}
77 % replace footnotes with symbols instead of numbers
78 \renewcommand*{\thefootnote}{\fnsymbol{footnote}}
80 \MakePerPage{footnote}
83 \usetheme[pageofpages=/,% String used between the current page and the
85 bullet=default,% Use circles instead of squares for bullets.
86 titleline=false,% Show a line below the frame title.
87 alternativetitlepage=true,% Use the fancy title page.
88 titlepagelogo=figures/logo.pdf,% Logo for the first page.
89 %watermark=watermark-polito,% Watermark used in every page.
90 watermarkheight=100px,% Height of the watermark.
91 watermarkheightmult=4,% The watermark image is 4 times bigger
92 % than watermarkheight.
96 \useinnertheme{rectangles}
97 %\setbeamertemplate{blocks}[rounded][]
98 \setbeamercolor{block title}{bg=makopurple3, fg=White}
100 \setbeamertemplate{items}[default]
101 \setbeamertemplate{blocks}[shadow=true]
103 \usepackage{tcolorbox}
104 % These options will be applied to all `tcolorboxes`
107 colback=makopurple5, %background color of the box
108 colframe=makopurple1, %color of frame and title background
109 coltext=black, %color of body text
110 coltitle=white, %color of title text
115 alerted/.style={coltitle=red,
117 example/.style={coltitle=black,
123 %\useoutertheme{infolines}
124 %\usepackage[breaklinks]{hyperref}
126 \hypersetup{colorlinks=true, linkcolor=Black, citecolor=Black, filecolor=makopurple1,
127 urlcolor=Plum, unicode=true}
129 % create a boldface version of the header
130 \setbeamerfont{frametitle}{series=\bfseries}
131 \setbeamerfont{title}{series=\bfseries}
133 % tweak the beamer font to make it a bit lists a bit smaller
134 \setbeamerfont*{itemize/enumerate body}{size=\small}
135 \setbeamerfont*{itemize/enumerate subbody}{size=\footnotesize}
136 \setbeamerfont*{itemize/enumerate subsubbody}{size=\footnotesize}
138 % indent the margins of the itemize lists a little bit
139 \setlength{\leftmargin}{0pt}
140 \setlength{\leftmargini}{0.7cm}
141 \setlength{\leftmarginii}{0.7cm}
143 % create a new \e{} command to make things purple and bold
144 \newcommand{\e}[1]{\textcolor{makopurple1}{\textbf{#1}}}
146 % remove the nagivation symbols
147 \setbeamertemplate{navigation symbols}{}
149 \title{Presentation Title}
150 % \subtitle{Presentation Subtitle}
151 \author[Benj. Mako Hill]{\textbf{Benjamin Mako Hill}\\ mako@mit.edu}
153 \institute[MIT/Harvard]{\textbf{Massachusetts Institute of Technology}\\
154 Sloan School of Management\\
157 \textbf{Harvard University}\\
158 Berkman Center for Internet and Society}
160 \date{December 2, 1980}
164 % remove some of the space in the itemize to make it quite compact
165 \let\olditemize\itemize
166 \renewcommand\itemize{\olditemize\itemsep-1pt}
168 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
169 \section{Introduction}
170 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
172 %% SLIDE: Title Slide
173 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
177 \node at (current page.center) [xshift=-3.5cm, yshift=0.5cm, opacity=0.4]
178 {\includegraphics[height=\paperheight]{figures/wikimedia_projects.png}};
181 \node at (current page.south east)
182 [anchor=south east,text width=1.8\paperwidth,align=right,color=black]
185 \fontsize{2.5em}{2.5em}
186 \selectfont {\bf \color{makopurple4} The State of Wikimedia\\
187 Research: 2012-2013} \par}
192 \fontsize{2.0em}{2.1em}
193 \selectfont {\bf \color{black} Benjamin Mako Hill\\
194 Wikimania 2013, Hong Kong\\
195 August 6, 2013} \par}
201 \tikz[overlay,shift=(current page.south west)]{\node [xshift=5.6em,yshift=0.5em]{\colorbox{makopurple1}{\color{white} \tt \smaller \smaller \smaller revision:\ \VCRevision\ (\VCDateTEX)}};}
203 \note{I've been doing this for many years. I started in 2008 and
204 skipped one year, I think.
206 This began as an excuse for me to make sure I was up to date on
211 %% SLIDE: Anecdote from Wikimania 2008
212 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
213 \renewcommand{\quotetxt}{``This talk will try to [provide] a quick
214 tour – a literature review in the scholarly parlance – of the last
215 year's academic landscape around Wikimedia and its projects geared
216 at non-academic editors and readers. It will try to categorize,
217 distill, and describe, from a birds eye view, the academic landscape
218 as it is shaping up around
220 \hfill – \e{From my Wikimania 2008 Submission}}
226 \includegraphics[width=\textwidth]{figures/google_scholar_result.png}
229 \tikz{\draw (current page.center) [xshift=-2.1cm, yshift=0.9cm, color=red]
230 ellipse (1.5cm and 0.5cm);}
232 \note<1>{Back in Wikimania 2008, I set out to run a session at
233 Wikimania that would provide a comprehensive literature review of
234 articles in Wikipedia published in the last year.
240 Then, about two weeks before Wikimania, I did the scholar search
241 so I could build the literature.}
243 \note<2->{I tried to import the whole list into Zotero and managed
244 to get banned for abusing the Google Scholar because they thought
245 that no human being could realistically consume the amount of
246 material published on Wikipedia that year.
248 So anyway, I had a 45 minute talk so it worked out to 3.45 seconds
251 And believe it or not, this year is even bigger.
253 And my talk is even shorter.}
257 %% SLIDE: Citations Per Year
258 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
261 \includegraphics[width=\textwidth]{figures/citations_by_year.pdf}
265 {\smaller \emph{Number of citation, per year, with the term
266 “wikipedia” in the title.\\
267 (Source: Google scholar results. Accessed: 2013-08-06)}}
269 \note{Academics have written \e{a lot} of papers about
270 Wikipedia. There are more than 500 papers published about
271 Wikipedia each year and although we've reached a peak, it's not
274 We're on track this year to meet or surpass that.}
278 % %% SLIDE: breakdown by time?
279 % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
282 % \includegraphics[width=\textwidth]{figures/wikipeda_citations_bytime.png}
286 %% SLIDE: My Scope Conditions
287 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
290 \includegraphics[width=\textwidth]{figures/multiple_issues.png}
293 In selecting papers for this session, the goal is always to choose
294 examples of work that:
298 \item Represent \e{important themes} from Wikipedia in the last year.
299 \item Research that is likely to be of \e{interest} to Wikimedians.
300 \item Research by people who are \e{not at Wikimania}.
303 \note{This is my disclaimer slide...
305 Within these goals, the selections are \e{incomplete}, and \e{wrong}.}
308 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
309 \section{Paper Summaries}
310 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
312 \subsection{Wikipedia in Context}
313 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
315 %% SLIDE: Reagle and Loveland Citation
316 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
317 \begin{frame}{Wikipedia in Historical Context}
319 \larger \larger Loveland, Jeff, and Joseph Reagle. “Wikipedia and
320 Encyclopedic Production.” \emph{New Media \& Society}
321 (2013). DOI:10.1177/1461444812470428.
323 \note{Jeff Loveland is a historian of encyclopedias. Joseph Reagle
324 is a media studies scholar who wrote the first book length
325 academic treatment of Wikipedia.
327 Loveland heard about Reagle's book through an article in the
328 Signpost but felt it was weak on history. So, they got together
329 and put together a great piece of work that places Wikipedia into
333 %% SLIDE: Reagle and Loveland Overview
334 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
335 \begin{frame}{Wikipedia in Historical Context}
337 \larger \larger \larger Loveland and Reagle cite three modes
338 of encyclopedia production:
341 \larger \larger \larger
342 \item Compulsive collection
343 \item Stigmergic accumluation
344 \item Corporate production
347 In each case, they see a connection between Wikipedia and methods of
350 \note<1>{The authors identify three historical methods through which
351 encyclopedias were written and they suggest that, in different
352 ways, each plays a role in Wikipedia:
355 \item \e{Compulsive collection} were people who were individually
356 driven to collect information. Think Pliny the Elder. And then
357 think Wikipediaholics and WikiBreak enforcing software.
358 \item \e{Stigmergic accumulation} references the `stigmergy' is a
359 word form Zoology that describes how wasps build nests and
360 references accumulation. In the past, this meant piracy and
361 building off of others. In Wikipedia, it means revision,
362 incorporation of other sources, and more.
363 \item \e{Corporate production} means working together with many
364 other people. Diderot took advantage of at least 140 different
365 authors. Think of the OED collecting information from
366 others. Wikipedia of course uses a similar model.
369 In each case, they think that Wikipedia's model is not a total
370 break from the past in the way many people talk about it.}
372 \note<2>{\e{Takeaway:} But I think is a great example how much of the more humanities
373 focused work on Wikipedia can do a wonderful job of providing us
374 context and a better way to think about and talk about what we're
380 \subsection{Wikipedia as Data Source}
381 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
384 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
385 \begin{frame}{Wikipedia as a Data Source}
389 Sérasset, Giles. “Dbnary: Wiktionary as a LMF Based Multilingual RDF
390 Network.” In \emph{Proceedings of the Eight International Conference on
391 Language Resources and Evaluation}, 2012.
394 \visible<2>{\url{http://dbnary.forge.imag.fr/}}
397 \note<1>{There's a whole genre of paper that is about Wikipedia only
398 in that is uses WP as its dataset. This might even be a
399 \e{majority} of all papers published on Wikipedia.
401 This paper up here, on a project called ``Dbnary'', is attempt to
402 build a \e{lexical network} out of Wiktionary data. Essentially,
403 they are using Wiktionary as a network of words and their
404 relationships -- including definitions, translations, synonyms,
405 antonyms, etc. -- in different languages, often connected through
408 Lexical networks are are essential to a whole family of
409 computerized natural language processing and a variety of
412 What I like about what Sérraseset did was that he created not only
413 use it as a dataset but really did a bunch of work to make
414 Wiktionary more useful to other resources.}
416 \note<2>{The researcher has created an open source tool – available
419 And anybody can use this tool, along with the dumps as published
420 by WMF, to produce their own, on their computers, is about 5
423 The paper also contains a list of challenges that Wiktionary
424 contributors might be able to use to extract data more effectively
427 \e{Takeaway:} I think that this paper suggests, like a lot of
428 simliar work, how Wikipedia's effect is broader than just what
429 comes through viewership on the web. And that there are important
430 ways we might be able to work with researchers like this to become
435 \subsection{Wikipedia and Quality}
436 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
438 %% SLIDE: Wikipedia and Quality Citation
439 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
440 \begin{frame}{Wikipedia and Quality}
444 Volsky, Peter G., Cristina M. Baldassari, Sirisha Mushti, and Craig
445 S. Derkay. ``Quality of Internet Information in Pediatric
446 Otolaryngology: A Comparison of Three Most Referenced Websites.''
447 \emph{International Journal of Pediatric Otorhinolaryngology} 76,
448 no. 9 (September 2012): 1312–1316. DOI:10.1016/j.ijporl.2012.05.026.
450 \note{There is little industry of articles designed to evaluate
451 Wikipedia's quality. There are literally dozens of these each
454 This is an example of one from pediatric otorhinolayrnology. That is,
455 the study of dieases of the ear, nose, and throat -- in children.}
460 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
461 \begin{frame}{Wikipedia and Quality: Evaluation of Otolaryngology Articles}
464 \column{0.53\textwidth}
467 \includegraphics[width=0.6\textwidth]{figures/oto-content_accuracy.png}
469 Accuracy as scored for content against a rubric\\
470 developed from otolaryngology textbooks.
474 \includegraphics[width=0.6\textwidth]{figures/oto-errors_omissions.png}
476 Mean numbers of errors and omissions.
478 \column{0.47\textwidth}
481 \includegraphics[width=0.6\textwidth]{figures/oto_reading_level.png}
483 Composite score for user interface.
487 \includegraphics[width=0.6\textwidth]{figures/oto-user_interface.png}
489 Flesch–Kinkaid Reading Level.
496 {\larger WK=Wikipedia; ML=MedLinePlus; EM=eMedicine.}
500 \note{Like many of these studies, this study cmpares Wikipedia to
501 other sites. In this case, eMedicine, and Medicine Plus. They used
502 a series of textbooks and experts to evaluate the the content
503 errors and they used some standard systems to evaluate usability
506 They find that Wikipedia has the most errors, the least accuracy,
507 and a medium reading level. But in most cases it is similar to MedLinePlus.
509 And Wikipedia had a rather good user interface compared to the
512 I'm not sure what that says about the others user interface.
514 \e{Takeaway:} We need to be better about getting these datsets and
515 helping integrate these into improving the encyclopedia. One thing that frustrates me is that it's very rare
516 that the people doing these studies coordinate with Wikipedia or that
517 Wikipedians systematically reach out. At a minimum, we could try to collect a list of errors papers like this discover and correct them.}
520 \subsection{Perception of Quality}
521 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
524 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
525 \begin{frame}{Viewership}
527 \larger \larger Towne, W. Ben, Aniket Kittur, Peter Kinnaird, and
528 James Herbsleb. “Your Process Is Showing: Controversy Management and
529 Perceived Quality in Wikipedia.” In \emph{Proceedings of the 2013
530 Conference on Computer Supported Cooperative Work}, 1059–1068. CSCW
531 ’13. New York, NY, USA: ACM, 2013. DI:10.1145/2441776.2441896.
533 \note{A group at Carnegie Mellon put together a really nice piece
534 that tried to surface Wikipedia's talk pages. Now, as many of you
535 will know intuitive, a majority of Wikipededia's work happens on
536 talk pages are invisible to many users. What would happen if we
537 made this more visible?}
540 \begin{frame}{Perception of Quality: Towne et al.}
543 ``Laws, like sausages, cease to inspire respect in proportion
544 as we know how they are made.''\\
545 \hfill -- John G. Saxe,
549 \item<2-> Discussion $\Rightarrow$ Lower Ratings
550 \item<3-> Unresolved conflict $\Rightarrow$ Even lower ratings
551 \item<4-> Discussion $\Rightarrow$ Higher reported preception of
552 Wikipedia and article!
555 \note{The goal was to test this theory in Wikipedia.
557 An experiment, on Mechanical Turk, to show people Wikipedia
558 articles and also to show them the talk pages. Then then asked
559 people to rate the articles, and their perception of the article
563 \item When discussion is shown, quality rating were significantly lower.
564 \item When discussion involving conflict was displayed, article
565 quality ratings were even lower yet.
566 \item If the editors involved in the conflict resolved it
567 through a positive collaboration approach, the negative
568 effects of conflict disappeared.
569 \item Participants reported that reading the discussion raised
570 their perceptions of both the article’s quality and Wikipedia
571 in general. (i.e., they were not aware of the rating-lowering
572 effects of the discussion.)
575 \e{Takeaway:} There's a deep and interesting tradeoff that cuts to
576 the core of Wikimedia's two missions to empower folks by getting
577 involved in the process to display material. This kind of work
578 explores big important questions at the heart of the foundation's
583 \subsection{Tool Building for Wikipedians}
584 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
586 %% SLIDE: Tool Building for Wikipedians
587 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
588 \begin{frame}{Tool Building for Wikipedians}
590 \larger \larger Solorio, Thamar, Ragib Hasan, and Mainul Mizan. ``A
591 Case Study of Sockpuppet Detection in Wikipedia.'' In
592 \emph{Proceedings of the Workshop on Language in Social Media},
593 59–68. Atlanta, Georgia, USA: Association for Computational
596 \note<1>{This is paper from a computational linguistics conference. And
597 they set out to create a method to identify sockpuppets in
600 There's a little academic industry designed to detect authorship
601 across texts and alias. But one problem that literature has is
602 that they almost no data of people \e{trying} to hide their
603 identity where the identity was later confirmed.
605 Wikipedia has no such problem. There were more than 2,700 cases of
606 suspected sock-puppeting in Wikipedia in 2012 alone.}
608 \note<2>{They use a database of confirmed (with checkuser) and rejected
609 cases of sockpuppeting to train a machine learning based approach
612 The system achieved an accuracy of 68.83\% in the tested cases.
614 This is not very good because simply always confirming the
615 suspected sockpuppet abuse would have achieved 53.24\% accuracy.
617 After adding features based on the user's edit frequency by time
618 of day and day of the week, it achieved 84.04\% confidence.
620 The authors have ideas of creating a system that could run in the
621 background and detect sockpuppets. But even if that never happens,
622 community members have done similar work in the past. And this
623 represents a set of tools and techniques from which the community
624 could directly benefit.
626 \e{Takeaway:} Again, we need to get better about working with all the
627 people, like this, building tools for our communities.}
633 \subsection{Effects of Feedback}
634 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
636 %% SLIDE: Effects of Feedback Citation
637 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
638 \begin{frame}{Effects of Feedback}
640 \larger \larger Zhu, Haiyi, Amy Zhang, Jiping He, Robert Kraut, and Aniket
641 Kittur. ``Effects of Peer Feedback on Contribution: A Field
642 Experiment in Wikipedia.'' In \emph{Proceedings of the SIGCHI
643 Conference on Human Factors in Computing Systems}. Paris, France:
646 \note{There have been a whole bunch of studies which have looked at
647 the effects of feedback on contribution to Wikipedia. Reverts,
648 welcome messages, et. And they have shown a series of effects.
650 But one concern with this work is that it is not causal. People
651 who receive negative messages are often behaving differently than
654 This reflects a real experiment, done in Wikipedia, where
655 different types of feedback were randomly assigned.
657 In August-November 2011, they left feedback for 703 creators of
658 new articles in Wikipedia after at least two days and making sure
659 the article had a certain amount of content and had not been
660 tagged for speedy deletion.}
664 %% SLIDE: Effects of Feedback Figures
665 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
666 \begin{frame}{Effects of Feedback: Zhu et al.}
669 \includegraphics[height=0.85\textheight]{figures/shared_leadership-figures.pdf}
671 \note{They left four kinds of feedback: positive, negative,
672 directive, and social.
674 And they were interested in both the effect on editing in the new
675 article they mention and on general editing on Wikipedia.
677 Feedback had no effect at all on experienced contributors. At
678 all. This was surpising to the folks running the study but maybe
679 not to the folks in this room.
681 In newbies, they found that negative feedback and directive
682 feedback had a positive effect on editing in the focal article and
683 positive feedback had a effect on general editing (but not the
684 article in question). And they found no other effects.
686 \e{Takeaway:} We should learn from and improve our processes based
687 on studies like these. We should work with researchers to do more
688 experiments. There are important ethical implications. There was a
689 long section of the paper about talking to the research ctte and
695 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
697 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
699 %% SLIDE: Other Resources
700 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
701 \begin{frame}{More Resources}
704 \larger \larger \larger
705 \item \e{Wikimedia Research Newsletter} [[:meta:Research:Newsletter]]
706 \item \e{WikiSym} (Last week in Hong Kong!)
707 \item \e{WikiPapers Repository} [http://wikipapers.referata.com]
711 \note{Those are my six exemplary studies from the past year.
713 There has been just tons and tons of work in this area. Trying to
714 talk about this in 20 minutes strikes me as increasingly crazy
715 every year I try to do it.
717 The most important source, now going for a couple years, is the
718 Wikimedia Research Newsletter which is published monthly in the (English)
721 But there are other resources as well. And I encourage you to get