1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
4 <meta name="generator" content=
5 "HTML Tidy for Linux (vers 25 March 2009), see www.w3.org">
6 <meta http-equiv="Content-Type" content=
7 "text/html; charset=us-ascii">
8 <title>To Fork or Not To Fork</title>
9 <meta name="generator" content="DocBook XSL Stylesheets V1.78.1">
11 <body bgcolor="white" text="black" link="#0000FF" vlink="#840084"
14 <div class="titlepage">
17 <h2 class="title"><a name="paper-11194" id="paper-11194"></a>To
18 Fork or Not To Fork</h2>
21 <h3 class="subtitle"><i>Lessons From Ubuntu and Debian</i></h3>
25 <h3 class="author"><span class="firstname">Benjamin</span>
26 <span class="othername">Mako</span> <span class=
27 "surname">Hill</span></h3>
28 <div class="affiliation"><span class="orgname">Canonical
29 Limited<br></span></div>
30 <div class="affiliation"><span class="orgname">The Debian GNU/Linux
31 Project<br></span></div>
32 <div class="affiliation"><span class="orgname">Software in the
33 Public Interest, Inc.<br></span></div>
37 <p class="copyright">Copyright © 2005 Benjamin Mako
41 <div class="legalnotice"><a name="idp27154528" id=
43 <p>This material is licensed under the <a class="ulink" href=
44 "http://creativecommons.org/licenses/by-sa/2.0/" target=
45 "_top">Creative Commons Attribution-Sharealike 2.0 License</a>.</p>
46 <p>The canonical location for the most recent version of this
47 document is <a class="ulink" href="http://mako.cc/" target=
48 "_top">at the author's website</a>.</p>
52 <div class="revhistory">
53 <table style="border-style:solid; width:100%;" summary=
56 <th align="left" valign="top" colspan="2"><b>Revision
60 <td align="left">Revision 0.2</td>
61 <td align="left">August 7, 2005</td>
64 <td align="left" colspan="2">Correction and improvements.</td>
67 <td align="left">Revision 0.1</td>
68 <td align="left">May 15, 2005</td>
71 <td align="left" colspan="2">
72 <p>The first version of this paper was written to an accepted talk
73 given at Linuxtag 2005 given in Karlsruhe, Germany.</p>
82 <p><b>Table of Contents</b></p>
84 <dt><span class="section"><a href=
85 "#idp27161152">Introduction</a></span></dt>
86 <dt><span class="section"><a href="#idp27077936">"Fork" Is A Four
87 Letter Word</a></span></dt>
88 <dt><span class="section"><a href="#idp27089792">Case
92 <dt><span class="section"><a href="#idp26985776">The Debian
93 Project</a></span></dt>
94 <dt><span class="section"><a href=
95 "#idp26996864">Ubuntu</a></span></dt>
96 <dt><span class="section"><a href=
97 "#idp32030512">Applicability</a></span></dt>
100 <dt><span class="section"><a href="#idp26879744">Balancing Forking
101 With Collaboration</a></span></dt>
104 <dt><span class="section"><a href="#idp26880432">Derivation and
105 Problem Analysis</a></span></dt>
106 <dt><span class="section"><a href="#idp32061728">Distributed Source
107 Control</a></span></dt>
108 <dt><span class="section"><a href="#idp32075664">Problem Specific
109 Tools</a></span></dt>
110 <dt><span class="section"><a href="#idp32079568">Social
111 Solutions</a></span></dt>
114 <dt><span class="section"><a href=
115 "#idp32092080">Conclusions</a></span></dt>
118 <div class="section">
119 <div class="titlepage">
122 <h2 class="title" style="clear: both"><a name="idp27161152" id=
123 "idp27161152"></a>Introduction</h2>
127 <p>The explosive growth of free and open source software over the
128 last decade has been mirrored by an equally explosive growth in the
129 ambitiousness of free software projects in choosing and tackling
130 problems. The free software movement approaches these large
131 problems with more code and with more expansive communities than
132 was thinkable a decade ago. Example of these massive projects
133 include desktop environments — like GNOME and KDE — and
134 distributions like Debian, RedHat, and Gentoo.</p>
135 <p>These projects are leveraging the work of thousands of
136 programmers — both volunteer and paid — and are
137 producing millions of lines of code. Their software is being used
138 by millions of users with diverse sets of needs. This paper focuses
139 on two major effects of this situation:</p>
140 <div class="itemizedlist">
141 <ul class="itemizedlist" style="list-style-type: disc;">
142 <li class="listitem">
143 <p>The communities that free software projects — and in
144 particular large projects — serve are increasingly diverse.
145 It is becoming increasingly difficult for a single large project to
146 release any single product that can cater to all of its potential
149 <li class="listitem">
150 <p>It's becoming increasingly difficult to reproduce these large
151 projects. While reproducing entire project is impossible for small
152 groups of hackers, it is often not even possible for small groups
153 to even track and maintain a fork of a large project over time.</p>
157 <p>Taken together, these facts imply an increasingly realized free
158 software community in which programmers frequently derive but where
159 traditional forking is often untenable. "Forks," as they are
160 traditionally defined, must be improved upon. Communities around
161 large free software projects must be smarter about the process of
162 derivation than they have been in the past.</p>
163 <p>We are already seeing this with GNU/Linux distributions. New
164 distributions are rarely built from scratch today. Instead, they
165 adapted from and built on top of the work of existing projects. As
166 projects and user-bases grow, these derived distributions are
167 increasingly common. Most of what I describe in this essay are
168 tools and experiences of derived distributions.</p>
169 <p>Software makers must pursue the idea of an <span class=
170 "emphasis"><em>ecosystem</em></span> of free software projects and
171 products that have forked but that maintain a close relationship as
172 they develop parallelly and symbiotically. To do this, developers
174 <div class="itemizedlist">
175 <ul class="itemizedlist" style="list-style-type: disc;">
176 <li class="listitem">
177 <p>Break down the process of derivation into a set of different
178 types of customization and derivation and prioritize methods of
181 <li class="listitem">
182 <p>Create and foster social solutions to the social aspects of the
183 derivation problem.</p>
185 <li class="listitem">
186 <p>Build and use new tools specifically designed to coordinate
187 development of software in the context of an ecosystem of
190 <li class="listitem">
191 <p>Distribute and utilize distributed version control tools with an
192 emphasis on maintaining differences over time.</p>
196 <p>This paper is an early analysis of this set of problems. As
197 such, it is highly focused on the experience of the Ubuntu project
198 and its existence as a derived Debian distribution. It also pulls
199 from my experience with Debian-NP and the Custom Debian
200 Distribution (CDD) community. Since I participate in both the
201 Ubuntu and CDD projects, these are areas that I can discuss with
202 some degree of knowledge and experience.</p>
204 <div class="section">
205 <div class="titlepage">
208 <h2 class="title" style="clear: both"><a name="idp27077936" id=
209 "idp27077936"></a>"Fork" Is A Four Letter Word</h2>
213 <p>The act of taking the code for a free software project and
214 bifurcating it to create a new project is called "forking." There
215 have been a number of famous forks in free software history. One of
216 the most famous was the schism that led to the parallel development
217 of two versions of the Emacs text editor: GNU Emacs and XEmacs.
218 This schism persists to this day.</p>
219 <p>Some forks, like Emacs and XEmacs, are permanent. Others are
220 relatively short lived. An example of this is the GCC project which
221 saw two forks — EGCS and PGCC — that both eventually
222 merged back into GCC. Forking can happen for any number of reasons.
223 Often developers on a project develop political or personal
224 differences that keep them from continuing to work together. In
225 some cases, maintainers become unresponsive and other developers
226 fork to keep the software alive.</p>
227 <p>Ultimately though, most forks occur because people do not agree
228 on the features, the mechanisms, or the technology at the core of a
229 project. People have different goals, different problems, and want
230 different tools. Often, these goals, problems and tools are similar
231 up until a certain point before the need to part ways becomes
233 <p>A fork occurs on the level of code but a fork is not merely
234 — or even primarily — technical. Many projects create
235 "branches." Branches are alternative versions of a piece of
236 software used to experiment with intrusive or unstable features and
237 fixes. Forks are distinguished from branches both in that they are
238 often more significant departures from a technical perspective
239 (i.e., more lines of code have been changed and/or the changes are
240 more invasive or represent a more fundamental rethinking of the
241 problem) and in that they are bifurcations defined in social and
242 political terms. Branches involve a <span class=
243 "emphasis"><em>single</em></span> developer or community of
244 developers — even if it does boil down to distinct subgroups
245 within a community — whereas forks are separate projects.</p>
246 <p>Forking has historically been viewed as a bad thing in free
247 software communities: they are seen to stem from people's inability
248 to work together and have ended in reproduction of work. When I
249 published the first version of the <a class="ulink" href=
250 "http://mako.cc/projects/howto/" target="_top">Free Software
251 Project Management HOWTO</a> more than four years ago, I included a
252 small subsection on forking which described the concept to future
253 free software project leaders with this text:</p>
254 <div class="blockquote">
255 <blockquote class="blockquote">
256 <p>The short version of the fork section is, don't do them. Forks
257 force developers to choose one project to work with, cause nasty
258 political divisions, and redundancy of work.</p>
261 <p>In the <span class="emphasis"><em>best</em></span> situations, a
262 fork means that two groups of people need to go on developing
263 features and doing work they would ordinarily do <span class=
264 "emphasis"><em>in addition to</em></span> tracking the forked
265 project and having to hand-select and apply features and fixes to
266 their own code-base. This level of monitoring and constant
267 comparison can be extremely difficult and time-consuming. The
268 situation is not helped substantially by traditional source control
269 tools like diff, patch, CVS and Subversion which are not optimized
270 for this task. The worse (and much more common) situation occurs
271 when two groups go about their work ignorant or partially ignorant
272 of the code being cut on the other side of the fork. Important
273 features and fixes are implemented twice — differently and
275 <p>The most substantial bright side to these drawbacks is that the
276 problems associated with forking are so severe and notorious that,
277 in most cases, the threat of a fork is enough to force maintainers
278 to work out solutions that keep the fork from happening in the
280 <p>Finally, it is worth pointing out that fork is something of a
281 contested term. Because definitions of forks involve, to one degree
282 or another, statements about the political, organization, and
283 technical distinctions between projects, bifurcations that many
284 people call branches or parallel trees are described by others as
285 forks. Recently, fueled by the advent of distributed version
286 control systems, the definition of what is and is not a fork has
287 become increasingly unclear. In part due to the same systems, the
288 benefits and drawbacks of what is increasingly problematically
289 called forking is equally debatable.</p>
291 <div class="section">
292 <div class="titlepage">
295 <h2 class="title" style="clear: both"><a name="idp27089792" id=
296 "idp27089792"></a>Case Study</h2>
300 <p>In my introduction, I described how the growing scope of free
301 software projects and the rapidly increasingly size and diversity
302 of user communities is spearheading the need for new type of
303 derivation that avoids, as best as possible, the drawbacks of
304 forking. Nowhere is this more evident than in the largest projects
305 with the broadest scope: a small group of projects that includes
306 operating system distributions.</p>
307 <div class="section">
308 <div class="titlepage">
311 <h3 class="title"><a name="idp26985776" id="idp26985776"></a>The
316 <p>The Debian project is by many counts the largest free software
317 distribution in terms of code. It is the also, arguably, the
318 largest free software project in terms of the number of volunteers.
319 Debian includes more than 15,000 packages and the work of well over
320 1,000 official volunteers and many more contributors without
321 official membership. Projects without Debian's massive volunteer
322 base cannot replicate what Debian has accomplished; they can rarely
323 hope to even maintain what Debian has produced.</p>
324 <p>At the time that this paper was written, Distrowatch lists 129
325 distributions based on Debian<a href="#ftn.idp26988128" class=
326 "footnote" name="idp26988128" id="idp26988128"><sup class=
327 "footnote">[1]</sup></a> — most of them are currently active
328 to varying degrees. Each distribution represents at least one
329 person — and in most cases a community of people — who
330 disagreed with Debian's vision or direction strongly enough to want
331 to create a new distribution <span class=
332 "emphasis"><em>and</em></span> who had the technical capacity to
333 follow through with this goal. Despite Debian's long-standing
334 slogan — "the universal operating system" — the fact
335 that the Debian project has become the fastest growing operating
336 system while spawning so many derivatives is testament to the fact
337 that, as far as software is concerned, one size <span class=
338 "emphasis"><em>can not</em></span> fit all.<a href=
339 "#ftn.idp26990736" class="footnote" name="idp26990736" id=
340 "idp26990736"><sup class="footnote">[2]</sup></a></p>
341 <p>Organizationally, Debian derivers are located both inside and
342 outside of the Debian project. A group of derivers working within
343 the Debian project has labeled themselves "Custom Debian
344 Distributions" and has created nearly a dozen projects customizing
345 and deriving from Debian for specific groups of users including
346 non-profit organization, the medical community, lawyers, children
347 and many others.<a href="#ftn.idp26993168" class="footnote" name=
348 "idp26993168" id="idp26993168"><sup class="footnote">[3]</sup></a>
349 These projects build on the core Debian distribution and the
350 canonical archive from <span class=
351 "emphasis"><em>within</em></span> the organizational and political
352 limits of the Debian project and constantly seek to minimize the
353 delta by focusing on less invasive changes and by advancing
354 creative ways of building the <span class=
355 "emphasis"><em>ability</em></span> to alter the core Debian code
356 base through established and policy compliant procedures.</p>
357 <p>A second group of Debian customizers includes those working
358 outside of the Debian project organizationally. Notable among this
359 list are (in alphabetical order) Knoppix, Libranet, Linspire
360 (formerly Lindows), Progeny, MEPIS, Ubuntu, Userlinux, and Xandros.
361 With its strong technological base, excellent package management,
362 wide selection of packages to choose from, and strong commitment to
363 software freedom which ensures derivability, Debian provides an
364 ideal point from which to create a GNU/Linux distribution.</p>
366 <div class="section">
367 <div class="titlepage">
370 <h3 class="title"><a name="idp26996864" id=
371 "idp26996864"></a>Ubuntu</h3>
375 <p>The Ubuntu project was started by Mark Shuttleworth in April
376 2004 and the first version was built almost entirely by a small
377 group of a Debian developers employed by Shuttleworth's company
378 Canonical Limited.<a href="#ftn.idp26998016" class="footnote" name=
379 "idp26998016" id="idp26998016"><sup class="footnote">[4]</sup></a>
380 It was released to the world in late 2004. The second version was
381 released six months later in April 2005. The goals of Ubuntu are to
382 provide a distribution based on a subset of Debian with:</p>
383 <div class="itemizedlist">
384 <ul class="itemizedlist" style="list-style-type: disc;">
385 <li class="listitem">
386 <p>Regular and predictable releases — every six months with
387 support for eighteen months.</p>
389 <li class="listitem">
390 <p>An emphasis on free software that will maintain the derivability
391 of the distribution.</p>
393 <li class="listitem">
394 <p>An emphasis on usability and a consistent desktop vision. As an
395 example, this has translated into less questions in the installer
396 and a default selection and configuration of packages that is
397 usable for most desktop users "out of the box."</p>
401 <p>The Ubuntu project provides an interesting example of a project
402 that aims to derive from Debian to an extensive degree. Ubuntu made
403 code-level changes to nearly 1300 packages in Debian at the time
404 that this paper was written and the speed of changes will not
405 decelerate with time; the total number of changes and the total
406 size of the delta will grow.<a href="#ftn.idp27004416" class=
407 "footnote" name="idp27004416" id="idp27004416"><sup class=
408 "footnote">[5]</sup></a> The changes that Ubuntu makes are
409 primarily of the most intrusive kind — changes to the code
411 <p>That said, the Ubuntu project is explicit about the fact that it
412 could not exist without the work done by the Debian
413 project.<a href="#ftn.idp27006336" class="footnote" name=
414 "idp27006336" id="idp27006336"><sup class="footnote">[6]</sup></a>
415 More importantly, Ubuntu explains that it cannot continue to
416 provide the complete set of packages that its users depend on
417 without the ongoing work by the Debian project. Even though Ubuntu
418 has made changes to the nearly 1300 packages, this is less than ten
419 percent of the total packages shipped in Ubuntu and pulled from
421 <p>Scott James Remnant, a prominent Debian developer and a hacker
422 on Ubuntu who works for Canonical Ltd., described the situation
423 this way on his web log to introduce the Ubuntu development
424 methodology in the week after the first public announcement of
425 Canonical and Ubuntu:<a href="#ftn.idp27008624" class="footnote"
426 name="idp27008624" id="idp27008624"><sup class=
427 "footnote">[7]</sup></a></p>
428 <div class="blockquote">
429 <blockquote class="blockquote">
430 <p>I don't think Ubuntu is a "fork" of Debian, at least not in the
431 traditional sense. A fork suggests that at some point we go our
432 separate way from Debian and then occasionally merge in changes as
433 we carry on down our own path.</p>
434 <p>Our model is quite different; every six months we take a
435 snapshot of Debian's unstable distribution, apply any outstanding
436 patches from our last release to it and spend a couple of months
437 testing and bug-fixing it.</p>
438 <p><span class="inlinemediaobject"><img src=
439 "tfontf-picture-01.png"></span></p>
440 <p>One thing that should be obvious from this is that our job is a
441 lot easier if Debian takes all of our changes. The model actually
442 encourages us to give back to Debian.</p>
443 <p>That's why from the very first day we started fixing bugs we
444 began sending <a class="ulink" href=
445 "http://www.no-name-yet.com/patches/" target="_top">the patches</a>
446 back to Debian through the BTS. Not only will it make our job so
447 much easier when we come to freeze for "hoary", our next release,
448 but it's exactly what every derivative should do in the first
452 <p>There is some debate on the degree to which Ubuntu developers
453 have succeeded in accomplishing the goals laid out by Remnant.
454 Ubuntu has filed hundreds of patches in the bug tracking system but
455 it has also run into problems in deciding <span class=
456 "emphasis"><em>what</em></span> constitutes something that should
457 be fed back to Debian. Many changes are simply not relevant to
458 Debian developers. For example, they may include changes to a
459 package in response to another change made in another package in
460 Ubuntu that will not or has not been taken by Debian. In many other
461 cases, the best action in regards to a particular change, a
462 particular package, and a particular upstream Debian developer is
464 <p>The Ubuntu project's track record in working constructively with
465 Debian is, at the moment, a mixed one. While an increasingly large
466 number of Debian developers are maintaining their packages actively
467 within both projects, many in both Debian and Ubuntu feel that
468 Ubuntu has work left to do in living up to its own goal of a
469 completely smooth productive relationship with Debian.</p>
470 <p>That said, the importance of the goals described by Remnant in
471 the context of of the Ubuntu development model cannot be
472 overstated. Every line of delta between Debian and Ubuntu has a
473 cost for Ubuntu developers. Technology, social practices, and wise
474 choices may reduce that cost but it cannot eliminate it. The
475 resources that Ubuntu can bring to bear upon the problem of
476 building a distribution are limited — far more limited than
477 Debian's. As a result, there is a limit to how far Ubuntu can
478 diverge; it is always in Ubuntu's advantage to minimize the delta
481 <div class="section">
482 <div class="titlepage">
485 <h3 class="title"><a name="idp32030512" id=
486 "idp32030512"></a>Applicability</h3>
490 <p>Ubuntu and Debian are distributions and — as such —
491 operate on a different scale than the vast majority of free
492 software projects. They include more code and more people. As a
493 result, there are questions as to whether the experiences and
494 lessons learned from these projects are particularly applicable to
495 the experience of smaller free software projects.</p>
496 <p>Clearly, because of the difficulties associated with forking
497 massive amount of code and the problems associated with duplicating
498 the work of large volunteer bases, distributions are forced into
499 finding a way to balance the benefits and drawbacks of forking.
500 However, while the need is stronger and more immediate in larger
501 projects, the benefits of their solutions will often be fully
503 <p>Clearly, modifiability of free software to better fit the needs
504 of its users lies at the heart of the free software movement's
505 success. However, while modification usually comes in the form of
506 collaboration on a single code-base, this is a function of
507 limitations in software development methodologies and tools rather
508 than the best response to the needs or desires of users or
510 <p>I believe that the fundamental advantage of free software in the
511 next decade will be in the growing ability of any single free
512 software project to be multiple things to multiple users
513 simultaneously. This will translate into the fact that, in the next
514 ten years, technology and social processes will evolve, so that
515 forking is increasingly less of a bad thing. Free software
516 development methodology will become less dependent on a single
517 project and begin to emphasize parallel development within an
518 ecosystem of related projects. The result is that free software
519 projects will gain a competitive advantage over propriety software
520 projects through their ability to better serve the increasingly
521 diverse needs of increasingly large and increasingly diverse
522 user-bases. Although it sounds paradoxical today, more projects
523 will derive and less redundant code will be written.</p>
524 <p>Projects more limited in code and scope may use the tools and
525 methods described in the remainder of this paper in different
526 combinations, in different ways, and to different degrees than the
527 examples around distributions introduced here. Different projects
528 with different needs will find that certain solutions work better
529 than others. Because communities of the size of Debian are
530 difficult to fork in a way that is beneficial to any party, it is
531 in these communities that the technology and development
532 methodologies are first emerging. With time, these strategies and
533 tools will find themselves employed productively in a wide variety
534 of projects with a broad spectrum of sizes, needs, scopes and
538 <div class="section">
539 <div class="titlepage">
542 <h2 class="title" style="clear: both"><a name="idp26879744" id=
543 "idp26879744"></a>Balancing Forking With Collaboration</h2>
547 <div class="section">
548 <div class="titlepage">
551 <h3 class="title"><a name="idp26880432" id=
552 "idp26880432"></a>Derivation and Problem Analysis</h3>
556 <p>The easiest step in creating a productive derivative software
557 project is to break down the problems of derivations into a series
558 of different classes of modification. Certain types of modification
559 are more easily done and are intrinsically more maintainable.</p>
560 <p>In the context of distributions, the problem of derivation can
561 be broken down into the following types of changes (sorted roughly
562 according to the intrusiveness inherent in solving the problem and
563 the severity of the long-term maintainability problems that they
565 <div class="orderedlist">
566 <ol class="orderedlist" type="1">
567 <li class="listitem">
568 <p>Selection of individual pieces of software;</p>
570 <li class="listitem">
571 <p>Changes to the way that packages are installed or run (e.g., in
572 a Live CD type environment or using a different installer);</p>
574 <li class="listitem">
575 <p>Configuration of different pieces of software;</p>
577 <li class="listitem">
578 <p>Changes made to the actual software package (made on the level
579 of changes to the packages code);</p>
583 <p>By breaking down the problem in this way, Debian derivers have
584 been able to approach derivation in ways that focus energy on the
585 less intrusive problems first.</p>
586 <p>The first area that Ubuntu focused on was selecting a subset of
587 packages that Ubuntu would support. Ubuntu selected and supports
588 approximate 2,000 packages. These became the <span class=
589 "command"><strong>main</strong></span> component in Ubuntu. Other
590 packages in Debian were included in a separate section of the
591 Ubuntu archive called <span class=
592 "command"><strong>universe</strong></span> but were not guaranteed
593 to be supported with bug or security fixes. By focusing on a small
594 subset of packages, the Ubuntu team was able to select a
595 maintainable subsection of the Debian archive that they could
596 maintain over time.</p>
597 <p>The most simple derived distributions — often working
598 within the Debian project as CDDs but also including projects like
599 Userlinux — are merely lists of packages and do nothing
600 outside of package selection. The installation of lists of packages
601 and the maintenance of those lists over time can be aided through
602 the creation of what are called <span class=
603 "emphasis"><em>metapackages</em></span>: empty packages with long
604 lists of "dependencies."</p>
605 <p>The second item, configuration changes, is also relatively
606 low-impact. Focusing on moving as many changes as possible into the
607 realm of configuration changes is a sustainable strategy that
608 derivers working within the Debian project intent on a single
609 code-base have pursued actively. Their idea is that rather than
610 forking a piece of code due to disagreement in how the program
611 should work, they can leave the code intact but add the
612 <span class="emphasis"><em>ability</em></span> to work in a
613 different way to the software. This alternate functionality is made
614 toggleable through a configuration change in the same manner that
615 applications are configured through questions asked at install
616 time. Since the Debian project has a unified package configuration
617 framework called Debconf, derivers are able to configure an entire
618 system in a highly centralized manner.<a href="#ftn.idp32057520"
619 class="footnote" name="idp32057520" id="idp32057520"><sup class=
620 "footnote">[8]</sup></a> This is not unlike RedHat's Kickstart
621 although the emphasis is on maintenance of those configuration
622 changes over the life and evolution of the package; Kickstart is
623 focused merely on installation of the package.</p>
624 <p>A third type of configuration is limited to changes in the
625 environment through which a system is run or installed. One is
626 example is Progeny's Anaconda-based Debian installer which provides
627 an alternate installer but results in an identical system. Another
628 example is the Knoppix project which is famous for its "Live CD"
629 environments. While, Knoppix makes a wide range of invasive changes
630 that span all items in my list above, other Live CD projects,
631 including Ubuntu's "Casper" project, are much closer to an
632 alternate shell through which the same code is run.</p>
633 <p>Because these three methods are relatively non-invasive, they
634 are reasonable strategies for small teams and individuals working
635 on creating a derived distribution. However, many desirable changes
636 — and in the case of some derived distributions, <span class=
637 "emphasis"><em>most</em></span> desirable changes — require
638 more invasive techniques. The final and most invasive type of
639 change — changes to code — is the most difficult but
640 also the most promising and powerful if it can be done sustainably.
641 Changes of this type involve bifurcations of the code-base and will
642 be the topic of the remainder of this paper.</p>
644 <div class="section">
645 <div class="titlepage">
648 <h3 class="title"><a name="idp32061728" id=
649 "idp32061728"></a>Distributed Source Control</h3>
653 <p>One promising method of maintaining deltas in forked or branched
654 projects lies in distributed version control systems (VCS).
655 Traditional VCS systems work in a highly centralized fashion. CVS,
656 the archetypal free software VCS and the basis for many others, is
657 based around the model of a single centralized server. Anyone who
658 wishes to commit to a project must commit to the centralized
659 repository. While CVS allows users to create branches, anyone with
660 commit rights has access to the entire repository. The tools for
661 branching and merging over time are not particularly good.</p>
662 <p>The branching model is primarily geared toward a system where
663 development is bifurcated and then the branch is merged completely
664 back into the main tree. Normal use of a branch might include
665 creating a development branch, making a series of development
666 releases while maintaining and fixing important bugs in the stable
667 primary branch, and then ultimately replacing the stable release
668 with the development release. The CVS model is <span class=
669 "emphasis"><em>not</em></span> geared toward a system where an
670 arbitrary delta, or sets of deltas, are maintained over time.</p>
671 <p>Distributed version control aims to solve a number of problems
672 introduced by CVS and alluded to above by:</p>
673 <div class="itemizedlist">
674 <ul class="itemizedlist" style="list-style-type: disc;">
675 <li class="listitem">
676 <p>Allowing people to work disconnected from each other and to sync
677 with each other, in whole or in part, in an arbitrary and ad-hoc
680 <li class="listitem">
681 <p>Allowing deltas to be maintained over time.</p>
685 <p>Ultimately, this requires tools that are better at merging
686 changes and in <span class="emphasis"><em>not</em></span> merging
687 certain changes when that is the desired behavior. It also leads to
688 tools capable of history-sensitive merging.</p>
689 <p>The most famous switch to a distributed VCS model from a
690 centralized VCS model was the move by the Linux kernel development
691 community to the proprietary distributed version control system
692 BitKeeper. In his recent announcement of the decision to part ways
693 with BitKeeper, Linus Torvalds said:</p>
694 <div class="blockquote">
695 <blockquote class="blockquote">
696 <p>In fact, one impact BK has had is to very fundamentally make us
697 (and me in particular) change how we do things. That ranges from
698 the fine-grained changeset tracking to just how I ended up trusting
699 sub-maintainers with much bigger things, and not having to work on
700 a patch-by-patch basis any more.<a href="#ftn.idp32069728" class=
701 "footnote" name="idp32069728" id="idp32069728"><sup class=
702 "footnote">[9]</sup></a></p>
705 <p>At the time of the switch, free distributed version control
706 tools were less advanced than they are today. At the moment, an
707 incomplete list of free software VCS tools includes GNU Arch,
708 Bazaar, Bazaar-NG, Darcs, Monotone, SVK (based on Subversion), GIT
709 (a system developed by Linus Torvalds as a replacement for
710 BitKeeper) and others.</p>
711 <p>Each of these tools, at least after they reach a certain level
712 of maturity, allow or will allow users to develop software in a
713 distributed fashion and to, over time, compare their software and
714 pull changes from others significantly more easily than they could
715 otherwise. The idea of parallel development lies at the heart of
716 the model. The tools for merging and resolving conflicts over time,
717 and the ability to "cherry pick" certain patches or changes from a
718 parallel developer each make this type of development significantly
719 more useful than it has been in the past.</p>
720 <p>VCSs work entirely on the level of code. Due to the nature of
721 the types of changes that Ubuntu project is making to Debian's
722 code, Ubuntu has focused primarily on this model and Canonical
723 currently funds two major distributed control products — the
724 Bazaar and Bazaar-NG projects.</p>
725 <p>In many ways, employing distributed version control effectively
726 is a much easier problem to solve for small, more traditional, free
727 software development projects than it is for GNU/Linux
728 distributions. Because the problems associated with maintaining
729 parallel development of a single piece of software in a set of
730 related distributed repositories is the primary use case for
731 distributed version control systems, distributed VCS alone can be a
732 technical solution for certain types of parallel development. As
733 the tools and social processes for distributed VCS evolve, they
734 will become increasingly important tools in the way that free
735 software is developed.</p>
736 <p>Because the problems of scale associated with building an entire
737 derivative distribution are more complicated than those associated
738 with working with a single "upstream" project, distributed version
739 control is only now being actively deployed in the Ubuntu project.
740 In doing so, the project is focusing on integrating these into
741 problem specific tools built on top of distributed version
744 <div class="section">
745 <div class="titlepage">
748 <h3 class="title"><a name="idp32075664" id=
749 "idp32075664"></a>Problem Specific Tools</h3>
753 <p>Another technique that Canonical Ltd. is experimenting with is
754 the creation of high level tools built on top of distributed
755 version control tools specifically designed for maintaining
756 difference between packages. Because packages are usually
757 distributed as a source file with a collection of one or more
758 patches, this introduces the unique possibility of creating a
759 high-level VCS system based around this fact.</p>
760 <p>In the case of Ubuntu and Debian, the ideal tool creates one
761 branch per patch or feature and uses heuristics to analyze patch
762 files and create these branches intelligently. The package build
763 system section of the total patch can also be kept as a separate
764 branch. Canonical's tool, called the Hypothetical Changeset Tool
765 (HCT) (although no longer hypothetical), is one experimental way of
766 creating a very simple, very streamlined interface for dealing with
767 a particular type of source that is created and distributed in a
768 particular type of way with a particular type of change.</p>
769 <p>While HCT promises to be very useful for people making derived
770 distributions based on Debian, its application outside distribution
771 makers will, in all likelihood, be limited. That said, it provides
772 an example of the way that problem and context specific tools may
773 play an essential role in the maintenance of derived code more
776 <div class="section">
777 <div class="titlepage">
780 <h3 class="title"><a name="idp32079568" id="idp32079568"></a>Social
785 <p>It has been said that it is a common folly of a technophile to
786 attempt to employ technical solutions toward solving social
787 problems. The problem of deriving software is both a technical
788 <span class="emphasis"><em>and</em></span> social problem and
789 adequately addressing the larger problems requires approaches that
790 take into consideration both types of solution.</p>
791 <p>Scott James Remnant compares the relationship between
792 distributions and derived distributions as similar to the
793 relationship between distributions and upstream maintainers:</p>
794 <div class="blockquote">
795 <blockquote class="blockquote">
796 <p>I don't think this is much different from how Debian maintainers
797 interact with their upstreams. As Debian maintainers we take and
798 package upstream software and then act as a gateway for bugs and
799 problems. Quite often we fix bugs ourselves and apply the patch to
800 the package and send it upstream. Sometimes the upstream don't
801 incorporate that patch and we have to make sure we don't
802 accidentally drop it each subsequent release, we much prefer it if
803 they take them, but we don't get angry if they don't.</p>
804 <p>This is how I see the relationship between Ubuntu and Debian,
805 we're no more a fork of Debian than a Debian package is a fork of
809 <p>Scott alludes the fact that, at least in the world of
810 distributions, parallel development is already one way to view the
811 <span class="emphasis"><em>modus operandi</em></span> of existing
812 GNU/Linux distributions. The relationship between a deriver and
813 derivee on the distribution level mirrors the relationship between
814 the distribution and the "upstream" authors of the packages that
815 make up the distribution. These relationships are rarely based
816 around technological tools but are entirely in the realm of social
818 <p>Ubuntu has pursued a number of different initiatives along these
819 lines. The first of these has been to regularly file bugs in the
820 Debian bug tracking system when bugs that exist in Debian are fixed
821 in Ubuntu. While this can be partially automated, the choice to
822 automate this and the manner in which it it is set up is a purely
824 <p>However, as I alluded to above, Ubuntu is still left with
825 questions in regards to changes that are made to packages that do
826 not necessarily fix bugs or that fix bugs that do not exist in
827 Debian but may in the future. Some Debian developers want to hear
828 about the full extent of changes made to their software in Ubuntu
829 while others do not want to be bothered. Ubuntu should continue to
830 work with Debian to find ways to allow developers to stay in
832 <p>There are also several initiatives by developers in Debian,
833 Ubuntu, and in other derivations to create a stronger relationship
834 between the Debian project and its ecosystem of derivers and
835 between Ubuntu and Debian in particular. While the form that this
836 will ultimately take is unclear, projects existing within an
837 ecosystem should explore the realm of appropriate social
838 relationships that will ensure that they can work together and be
839 informed of each others' work without resorting to "spamming" each
840 other with irrelevant or unnecessary information.</p>
841 <p>Another issue that has recently played an important role in the
842 Debian/Ubuntu relationship is the importance of both giving
843 adequate credit to the authors or upstream maintainers of software
844 without implying a closer relationship than is the case. Derivers
845 must walk a file line where they credit others' work on a project
846 without implying that the others work for, support, or are
847 connected to the derivers project to which, for any number of
848 reasons, the "upstream" author might not want to be associated.</p>
849 <p>In the case of Debian and Ubuntu, this has resulted in an
850 emphasis on keeping or importing changelog entries when changes are
851 imported and in noting the pedigree of changes more generally. It
852 has recently also been discussed in terms of the "maintainer" field
853 in each package in Ubuntu. Ubuntu wants to avoid making changes to
854 every unmodified source package (and introducing an unnecessary
855 delta) but does not want to give the impression that the maintainer
856 of the package is someone unassociated with Ubuntu. While no
857 solution has been decided at the time of writing, one idea involved
858 marking the maintainer of the package explicitly as a Debian
859 maintainer at the time that the binary packages are built on the
860 Ubuntu build machines.</p>
861 <p>The emphasis on social solutions is also essential when using
862 distributed VCS technology. As Linus Torvalds alluded to in the
863 quote above, the importance of technological changes to distributed
864 VCS technology is only felt when people begin to work in a
865 different way — when they begin to employ different social
866 models of developer interaction.</p>
867 <p>While Ubuntu's experience can provide a good model for tackling
868 some of these source control issues, it can only serve as a model
869 and not as a fixed answer. Social solutions must be appropriate for
870 a given social relationship. Even in situations where a package is
871 branched because of social disagreements, a certain level of
872 collaboration on a social level will be essential to the long term
873 viability of the derivative.</p>
876 <div class="section">
877 <div class="titlepage">
880 <h2 class="title" style="clear: both"><a name="idp32092080" id=
881 "idp32092080"></a>Conclusions</h2>
885 <p>As the techniques described in this paper evolve, the role that
886 they play in free software development becomes increasingly
887 prominent and increasingly important. Joining them will be other
888 techniques and models that I have not described and cannot predict.
889 Because of the size and usefulness of their code and the size of
890 their development communities, large projects like Debian and
891 Ubuntu have been forced into confronting and attempting to mediate
892 the problems inherent in forking and deriving. However, as these
893 problems are negotiated and tools and processes are advanced toward
894 solutions, free software projects of all sizes will be able to
895 offer users exactly what they want with minimal redundancy and
896 little duplication of work. In doing this, free software will
897 harness a power that proprietary models cannot compete with. They
898 will increase their capacity to produce better products and better
899 processes. Ultimately, it will help free software capture more
900 users, bring in more developers, and produce more free software of
901 a higher quality.</p>
903 <div class="footnotes"><br>
904 <hr style="width:100; text-align:left;margin-left: 0">
905 <div id="ftn.idp26988128" class="footnote">
906 <p><a href="#idp26988128" class="para"><sup class=
907 "para">[1]</sup></a> Information is listed on the distrowatch
908 homepage here: <a class="ulink" href=
909 "http://distrowatch.com/dwres.php?resource=independence" target=
910 "_top">http://distrowatch.com/dwres.php?resource=independence</a></p>
912 <div id="ftn.idp26990736" class="footnote">
913 <p><a href="#idp26990736" class="para"><sup class=
914 "para">[2]</sup></a> Netcraft posts yearly updates on the speed at
915 which Linux distributions are growing. The one in question can be
916 found at: <a class="ulink" href=
917 "http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html"
919 "_top">http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html</a></p>
921 <div id="ftn.idp26993168" class="footnote">
922 <p><a href="#idp26993168" class="para"><sup class=
923 "para">[3]</sup></a> I spearheaded and help build a now mostly
924 defunct derivation of Debian called Debian-Nonprofit (Debian-NP)
925 geared for non-profit organizations by working within the Debian
928 <div id="ftn.idp26998016" class="footnote">
929 <p><a href="#idp26998016" class="para"><sup class=
930 "para">[4]</sup></a> Information Ubuntu can be found on the
931 <a class="ulink" href="http://www.ubuntu.com" target="_top">Ubuntu
932 homepage.</a> Information Canonical Limited can be found at
933 <a class="ulink" href="http://www.canonical.com" target=
934 "_top">Canonical's homepage</a>.</p>
936 <div id="ftn.idp27004416" class="footnote">
937 <p><a href="#idp27004416" class="para"><sup class=
938 "para">[5]</sup></a> Scott James Remnant maintains a list of these
939 patches online here: <a class="ulink" href=
940 "http://people.ubuntu.com/~scott/patches/" target=
941 "_top">http://people.ubuntu.com/~scott/patches/</a></p>
943 <div id="ftn.idp27006336" class="footnote">
944 <p><a href="#idp27006336" class="para"><sup class=
945 "para">[6]</sup></a> You can see that explicit statement on
946 Ubuntu's website here: <a class="ulink" href=
947 "http://www.ubuntulinux.org/ubuntu/relationship/" target=
948 "_top">http://www.ubuntulinux.org/ubuntu/relationship/</a></p>
950 <div id="ftn.idp27008624" class="footnote">
951 <p><a href="#idp27008624" class="para"><sup class=
952 "para">[7]</sup></a> The entire post can be read here: <a class=
954 "http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html"
956 "_top">http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html</a></p>
958 <div id="ftn.idp32057520" class="footnote">
959 <p><a href="#idp32057520" class="para"><sup class=
960 "para">[8]</sup></a> More information on Debconf can be found
961 online at: <a class="ulink" href=
962 "http://www.kitenet.net/programs/debconf/" target=
963 "_top">http://www.kitenet.net/programs/debconf/</a></p>
965 <div id="ftn.idp32069728" class="footnote">
966 <p><a href="#idp32069728" class="para"><sup class=
967 "para">[9]</sup></a> The full message can be read online at:
968 <a class="ulink" href=
969 "http://kerneltrap.org/mailarchive/1/message/48393/thread" target=
970 "_top">http://kerneltrap.org/mailarchive/1/message/48393/thread</a></p>
projects / to_fork_or_not / blob