+++ /dev/null
-<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>To Fork or Not To Fork</title><meta name="generator" content="DocBook XSL Stylesheets V1.78.1"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="article"><div class="titlepage"><div><div><h2 class="title"><a name="paper-11194"></a>To Fork or Not To Fork</h2></div><div><h3 class="subtitle"><i>Lessons From Ubuntu and Debian</i></h3></div><div><div class="author"><h3 class="author"><span class="firstname">Benjamin</span> <span class="othername">Mako</span> <span class="surname">Hill</span></h3><div class="affiliation"><span class="orgname">Canonical Limited<br></span></div><div class="affiliation"><span class="orgname">The Debian GNU/Linux Project<br></span></div><div class="affiliation"><span class="orgname">Software in the Public Interest, Inc.<br></span></div></div></div><div><p class="copyright">Copyright © 2005 Benjamin Mako Hill</p></div><div><div class="legalnotice"><a name="idp27154528"></a><p>This material is licensed under the <a class="ulink" href="http://creativecommons.org/licenses/by-sa/2.0/" target="_top">Creative
- Commons Attribution-Sharealike 2.0 License</a>.</p><p>The canonical location for the most recent version of this
- document is <a class="ulink" href="http://mako.cc/" target="_top">at the author's
- website</a>.</p></div></div><div><div class="revhistory"><table style="border-style:solid; width:100%;" summary="Revision History"><tr><th align="left" valign="top" colspan="2"><b>Revision History</b></th></tr><tr><td align="left">Revision 0.2</td><td align="left">August 7, 2005</td></tr><tr><td align="left" colspan="2">Correction and improvements.</td></tr><tr><td align="left">Revision 0.1</td><td align="left">May 15, 2005</td></tr><tr><td align="left" colspan="2">
- <p>The first version of this paper was written to an
- accepted talk given at Linuxtag 2005 given in Karlsruhe,
- Germany.</p>
- </td></tr></table></div></div></div><hr></div><div class="toc"><p><b>Table of Contents</b></p><dl class="toc"><dt><span class="section"><a href="#idp27161152">Introduction</a></span></dt><dt><span class="section"><a href="#idp27077936">"Fork" Is A Four Letter Word</a></span></dt><dt><span class="section"><a href="#idp27089792">Case Study</a></span></dt><dd><dl><dt><span class="section"><a href="#idp26985776">The Debian Project</a></span></dt><dt><span class="section"><a href="#idp26996864">Ubuntu</a></span></dt><dt><span class="section"><a href="#idp32030512">Applicability</a></span></dt></dl></dd><dt><span class="section"><a href="#idp26879744">Balancing Forking With Collaboration</a></span></dt><dd><dl><dt><span class="section"><a href="#idp26880432">Derivation and Problem Analysis</a></span></dt><dt><span class="section"><a href="#idp32061728">Distributed Source Control</a></span></dt><dt><span class="section"><a href="#idp32075664">Problem Specific Tools</a></span></dt><dt><span class="section"><a href="#idp32079568">Social Solutions</a></span></dt></dl></dd><dt><span class="section"><a href="#idp32092080">Conclusions</a></span></dt></dl></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="idp27161152"></a>Introduction</h2></div></div></div><p>The explosive growth of free and open source software over
- the last decade has been mirrored by an equally explosive growth
- in the ambitiousness of free software projects in choosing and
- tackling problems. The free software movement approaches these
- large problems with more code and with more expansive
- communities than was thinkable a decade ago. Example of these
- massive projects include desktop environments — like GNOME
- and KDE — and distributions like Debian, RedHat, and
- Gentoo.</p><p>These projects are leveraging the work of thousands of
- programmers — both volunteer and paid — and are
- producing millions of lines of code. Their software is being
- used by millions of users with diverse sets of needs. This
- paper focuses on two major effects of this situation:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>The communities that free software projects — and
- in particular large projects — serve are increasingly
- diverse. It is becoming increasingly difficult for a single
- large project to release any single product that can cater
- to all of its potential users.</p></li><li class="listitem"><p>It's becoming increasingly difficult to reproduce these
- large projects. While reproducing entire project is
- impossible for small groups of hackers, it is often not even
- possible for small groups to even track and maintain a fork
- of a large project over time.</p></li></ul></div><p>Taken together, these facts imply an increasingly realized
- free software community in which programmers frequently derive
- but where traditional forking is often untenable. "Forks," as
- they are traditionally defined, must be improved upon.
- Communities around large free software projects must be smarter
- about the process of derivation than they have been in the
- past.</p><p>We are already seeing this with GNU/Linux distributions. New
- distributions are rarely built from scratch today. Instead, they
- adapted from and built on top of the work of existing projects.
- As projects and user-bases grow, these derived distributions are
- increasingly common. Most of what I describe in this essay are
- tools and experiences of derived distributions.</p><p>Software makers must pursue the idea of an
- <span class="emphasis"><em>ecosystem</em></span> of free software projects and
- products that have forked but that maintain a close relationship
- as they develop parallelly and symbiotically. To do this,
- developers should:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>Break down the process of derivation into a set of
- different types of customization and derivation and
- prioritize methods of derivation.</p></li><li class="listitem"><p>Create and foster social solutions to the social aspects
- of the derivation problem.</p></li><li class="listitem"><p>Build and use new tools specifically designed to
- coordinate development of software in the context of an
- ecosystem of projects.</p></li><li class="listitem"><p>Distribute and utilize distributed version control tools
- with an emphasis on maintaining differences over
- time.</p></li></ul></div><p>This paper is an early analysis of this set of problems. As
- such, it is highly focused on the experience of the Ubuntu
- project and its existence as a derived Debian distribution. It
- also pulls from my experience with Debian-NP and the Custom
- Debian Distribution (CDD) community. Since I participate in both
- the Ubuntu and CDD projects, these are areas that I can discuss
- with some degree of knowledge and experience.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="idp27077936"></a>"Fork" Is A Four Letter Word</h2></div></div></div><p>The act of taking the code for a free software project and
- bifurcating it to create a new project is called "forking."
- There have been a number of famous forks in free software
- history. One of the most famous was the schism that led to the
- parallel development of two versions of the Emacs text editor:
- GNU Emacs and XEmacs. This schism persists to this day.</p><p>Some forks, like Emacs and XEmacs, are permanent. Others are
- relatively short lived. An example of this is the GCC project
- which saw two forks — EGCS and PGCC — that both
- eventually merged back into GCC. Forking can happen for any
- number of reasons. Often developers on a project develop
- political or personal differences that keep them from continuing
- to work together. In some cases, maintainers become unresponsive
- and other developers fork to keep the software alive.</p><p>Ultimately though, most forks occur because people do not
- agree on the features, the mechanisms, or the technology at the
- core of a project. People have different goals, different
- problems, and want different tools. Often, these goals, problems
- and tools are similar up until a certain point before the need
- to part ways becomes essential.</p><p>A fork occurs on the level of code but a fork is not merely
- — or even primarily — technical. Many projects create
- "branches." Branches are alternative versions of a piece of
- software used to experiment with intrusive or unstable features
- and fixes. Forks are distinguished from branches both in
- that they are often more significant departures from a technical
- perspective (i.e., more lines of code have been changed and/or
- the changes are more invasive or represent a more fundamental
- rethinking of the problem) and in that they are bifurcations
- defined in social and political terms. Branches involve a
- <span class="emphasis"><em>single</em></span> developer or community of developers
- — even if it does boil down to distinct subgroups within a
- community — whereas forks are separate projects.</p><p>Forking has historically been viewed as a bad thing in free
- software communities: they are seen to stem from people's
- inability to work together and have ended in reproduction of
- work. When I published the first version of the <a class="ulink" href="http://mako.cc/projects/howto/" target="_top">Free Software Project
- Management HOWTO</a> more than four years ago, I included
- a small subsection on forking which described the concept to
- future free software project leaders with this text:</p><div class="blockquote"><blockquote class="blockquote"><p>The short version of the fork section is, don't do them.
- Forks force developers to choose one project to work with,
- cause nasty political divisions, and redundancy of
- work.</p></blockquote></div><p>In the <span class="emphasis"><em>best</em></span> situations, a fork means
- that two groups of people need to go on developing features and
- doing work they would ordinarily do <span class="emphasis"><em>in addition
- to</em></span> tracking the forked project and having to
- hand-select and apply features and fixes to their own code-base.
- This level of monitoring and constant comparison can be
- extremely difficult and time-consuming. The situation is not
- helped substantially by traditional source control tools like
- diff, patch, CVS and Subversion which are not optimized for this
- task. The worse (and much more common) situation occurs when two
- groups go about their work ignorant or partially ignorant of the
- code being cut on the other side of the fork. Important features
- and fixes are implemented twice — differently and
- incompatibly.</p><p>The most substantial bright side to these drawbacks is that
- the problems associated with forking are so severe and notorious
- that, in most cases, the threat of a fork is enough to force
- maintainers to work out solutions that keep the fork from
- happening in the first place.</p><p>Finally, it is worth pointing out that fork is something of
- a contested term. Because definitions of forks involve, to one
- degree or another, statements about the political, organization,
- and technical distinctions between projects, bifurcations that
- many people call branches or parallel trees are described by
- others as forks. Recently, fueled by the advent of distributed
- version control systems, the definition of what is and is not a
- fork has become increasingly unclear. In part due to the same
- systems, the benefits and drawbacks of what is increasingly
- problematically called forking is equally debatable.</p></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="idp27089792"></a>Case Study</h2></div></div></div><p>In my introduction, I described how the growing scope of
- free software projects and the rapidly increasingly size and
- diversity of user communities is spearheading the need for new
- type of derivation that avoids, as best as possible, the
- drawbacks of forking. Nowhere is this more evident than in the
- largest projects with the broadest scope: a small group of
- projects that includes operating system distributions.</p><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp26985776"></a>The Debian Project</h3></div></div></div><p>The Debian project is by many counts the largest free
- software distribution in terms of code. It is the also,
- arguably, the largest free software project in terms of the
- number of volunteers. Debian includes more than 15,000
- packages and the work of well over 1,000 official volunteers
- and many more contributors without official membership.
- Projects without Debian's massive volunteer base cannot
- replicate what Debian has accomplished; they can rarely hope
- to even maintain what Debian has produced.</p><p>At the time that this paper was written, Distrowatch lists
- 129 distributions based on Debian<a href="#ftn.idp26988128" class="footnote" name="idp26988128"><sup class="footnote">[1]</sup></a> — most of them
- are currently active to varying degrees. Each distribution
- represents at least one person — and in most cases a
- community of people — who disagreed with Debian's vision
- or direction strongly enough to want to create a new
- distribution <span class="emphasis"><em>and</em></span> who had the technical
- capacity to follow through with this goal. Despite Debian's
- long-standing slogan — "the universal operating system"
- — the fact
- that the Debian project has become the fastest growing
- operating system while spawning so many derivatives is
- testament to the fact that, as far as software is concerned,
- one size <span class="emphasis"><em>can not</em></span> fit all.<a href="#ftn.idp26990736" class="footnote" name="idp26990736"><sup class="footnote">[2]</sup></a>
- </p><p>Organizationally, Debian derivers are located both inside
- and outside of the Debian project. A group of derivers working
- within the Debian project has labeled themselves "Custom
- Debian Distributions" and has created nearly a dozen projects
- customizing and deriving from Debian for specific groups of
- users including non-profit organization, the medical
- community, lawyers, children and many others.<a href="#ftn.idp26993168" class="footnote" name="idp26993168"><sup class="footnote">[3]</sup></a> These projects build on the core Debian distribution and
- the canonical archive from <span class="emphasis"><em>within</em></span> the
- organizational and political limits of the Debian project and
- constantly seek to minimize the delta by focusing on less
- invasive changes and by advancing creative ways of building
- the <span class="emphasis"><em>ability</em></span> to alter the core
- Debian code base through established and policy compliant
- procedures.</p><p>A second group of Debian customizers includes those
- working outside of the Debian project organizationally.
- Notable among this list are (in alphabetical order) Knoppix,
- Libranet, Linspire (formerly Lindows), Progeny, MEPIS, Ubuntu,
- Userlinux, and Xandros. With its strong technological base,
- excellent package management, wide selection of packages to
- choose from, and strong commitment to software freedom which
- ensures derivability, Debian provides an ideal point from
- which to create a GNU/Linux distribution.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp26996864"></a>Ubuntu</h3></div></div></div><p>The Ubuntu project was started by Mark Shuttleworth in
- April 2004 and the first version was built almost entirely
- by a small group of a Debian developers employed by Shuttleworth's
- company Canonical Limited.<a href="#ftn.idp26998016" class="footnote" name="idp26998016"><sup class="footnote">[4]</sup></a> It was released to the world in late 2004.
- The second version was released six months later in April
- 2005. The goals of Ubuntu are to provide a distribution based
- on a subset of Debian with:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>Regular and predictable releases — every six months
- with support for eighteen months.</p></li><li class="listitem"><p>An emphasis on free software that will maintain the
- derivability of the distribution.</p></li><li class="listitem"><p>An emphasis on usability and a consistent desktop
- vision. As an example, this has translated into less
- questions in the installer and a default selection and
- configuration of packages that is usable for most desktop
- users "out of the box."</p></li></ul></div><p>The Ubuntu project provides an interesting example of a
- project that aims to derive from Debian to an extensive
- degree. Ubuntu made code-level changes to nearly 1300 packages
- in Debian at the time that this paper was written and the
- speed of changes will not decelerate with time; the total
- number of changes and the total size of the delta will
- grow.<a href="#ftn.idp27004416" class="footnote" name="idp27004416"><sup class="footnote">[5]</sup></a> The changes that Ubuntu makes are primarily of the
- most intrusive kind — changes to the code itself.</p><p>That said, the Ubuntu project is explicit about the fact
- that it could not exist without the work done by the Debian
- project.<a href="#ftn.idp27006336" class="footnote" name="idp27006336"><sup class="footnote">[6]</sup></a> More importantly, Ubuntu explains that it cannot
- continue to provide the complete set of packages that its
- users depend on without the ongoing work by the Debian
- project. Even though Ubuntu has made changes to the nearly
- 1300 packages, this is less than ten percent of the total
- packages shipped in Ubuntu and pulled from Debian.</p><p>Scott James Remnant, a prominent Debian developer and a
- hacker on Ubuntu who works for Canonical Ltd., described the
- situation this way on his web log to introduce the Ubuntu
- development methodology in the week after the first public
- announcement of Canonical and Ubuntu:<a href="#ftn.idp27008624" class="footnote" name="idp27008624"><sup class="footnote">[7]</sup></a>
- </p><div class="blockquote"><blockquote class="blockquote"><p>I don't think Ubuntu is a "fork" of Debian, at least not
- in the traditional sense. A fork suggests that at some
- point we go our separate way from Debian and then
- occasionally merge in changes as we carry on down our own
- path.</p><p>Our model is quite different; every six months we take a
- snapshot of Debian's unstable distribution, apply any
- outstanding patches from our last release to it and spend a
- couple of months testing and bug-fixing it.</p><p>
- <span class="inlinemediaobject"><img src="tfontf-picture-01.png"></span>
- </p><p>One thing that should be obvious from this is that our
- job is a lot easier if Debian takes all of our changes. The
- model actually encourages us to give back to
- Debian.</p><p>That's why from the very first day we started fixing
- bugs we began sending <a class="ulink" href="http://www.no-name-yet.com/patches/" target="_top">the
- patches</a> back to Debian through the BTS. Not only
- will it make our job so much easier when we come to freeze
- for "hoary", our next release, but it's exactly what every
- derivative should do in the first place.</p></blockquote></div><p>There is some debate on the degree to which Ubuntu
- developers have succeeded in accomplishing the goals laid out
- by Remnant. Ubuntu has filed hundreds of patches in the bug
- tracking system but it has also run into problems in deciding
- <span class="emphasis"><em>what</em></span> constitutes something that should be
- fed back to Debian. Many changes are simply not relevant to
- Debian developers. For example, they may include changes to a
- package in response to another change made in another package
- in Ubuntu that will not or has not been taken by Debian. In
- many other cases, the best action in regards to a particular
- change, a particular package, and a particular upstream Debian
- developer is simply unclear.</p><p>The Ubuntu project's track record in working
- constructively with Debian is, at the moment, a mixed one.
- While an increasingly large number of Debian developers are
- maintaining their packages actively within both projects, many
- in both Debian and Ubuntu feel that Ubuntu has work left to do
- in living up to its own goal of a completely smooth productive
- relationship with Debian.</p><p>That said, the importance of the goals described by
- Remnant in the context of of the Ubuntu development model
- cannot be overstated. Every line of delta between Debian and
- Ubuntu has a cost for Ubuntu developers. Technology, social
- practices, and wise choices may reduce that cost but it cannot
- eliminate it. The resources that Ubuntu can bring to bear upon
- the problem of building a distribution are limited — far
- more limited than Debian's. As a result, there is a limit to
- how far Ubuntu can diverge; it is always in Ubuntu's advantage
- to minimize the delta where possible.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp32030512"></a>Applicability</h3></div></div></div><p>Ubuntu and Debian are distributions and — as such
- — operate on a different scale than the vast majority of
- free software projects. They include more code and more
- people. As a result, there are questions as to whether the
- experiences and lessons learned from these projects are
- particularly applicable to the experience of smaller free
- software projects.</p><p>Clearly, because of the difficulties associated with
- forking massive amount of code and the problems associated
- with duplicating the work of large volunteer bases,
- distributions are forced into finding a way to balance the
- benefits and drawbacks of forking. However, while the need is
- stronger and more immediate in larger projects, the benefits
- of their solutions will often be fully transferable.</p><p>Clearly, modifiability of free software to better fit the
- needs of its users lies at the heart of the free software
- movement's success. However, while modification usually comes
- in the form of collaboration on a single code-base, this is
- a function of limitations in software development methodologies
- and tools rather than the best response to the needs or
- desires of users or developers.</p><p>I believe that the fundamental advantage of free software
- in the next decade will be in the growing ability of any
- single free software project to be multiple things to multiple
- users simultaneously. This will translate into the fact that,
- in the next ten years, technology and social processes will
- evolve, so that forking is increasingly less of a bad thing.
- Free software development methodology will become less
- dependent on a single project and begin to emphasize parallel
- development within an ecosystem of related projects. The
- result is that free software projects will gain a competitive
- advantage over propriety software projects through their
- ability to better serve the increasingly diverse needs of
- increasingly large and increasingly diverse user-bases.
- Although it sounds paradoxical today, more projects will
- derive and less redundant code will be written.</p><p>Projects more limited in code and scope may use the tools
- and methods described in the remainder of this paper in
- different combinations, in different ways, and to different
- degrees than the examples around distributions introduced
- here. Different projects with different needs will find that
- certain solutions work better than others. Because communities
- of the size of Debian are difficult to fork in a way that is
- beneficial to any party, it is in these communities that the
- technology and development methodologies are first
- emerging. With time, these strategies and tools will find
- themselves employed productively in a wide variety of projects
- with a broad spectrum of sizes, needs, scopes and
- descriptions.</p></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="idp26879744"></a>Balancing Forking With Collaboration</h2></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp26880432"></a>Derivation and Problem Analysis</h3></div></div></div><p>The easiest step in creating a productive derivative
- software project is to break down the problems of derivations
- into a series of different classes of modification. Certain
- types of modification are more easily done and are
- intrinsically more maintainable.</p><p>In the context of distributions, the problem of derivation
- can be broken down into the following types of changes (sorted
- roughly according to the intrusiveness inherent in solving the
- problem and the severity of the long-term maintainability
- problems that they introduce):</p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><p>Selection of individual pieces of software;</p></li><li class="listitem"><p>Changes to the way that packages are installed or run
- (e.g., in a Live CD type environment or using a different
- installer);</p></li><li class="listitem"><p>Configuration of different pieces of software;</p></li><li class="listitem"><p>Changes made to the actual software package (made on
- the level of changes to the packages code);</p></li></ol></div><p>By breaking down the problem in this way, Debian derivers
- have been able to approach derivation in ways that focus
- energy on the less intrusive problems first.</p><p>The first area that Ubuntu focused on was selecting a
- subset of packages that Ubuntu would support. Ubuntu selected
- and supports approximate 2,000 packages. These became the
- <span class="command"><strong>main</strong></span> component in Ubuntu. Other packages in
- Debian were included in a separate section of the Ubuntu
- archive called <span class="command"><strong>universe</strong></span> but were not
- guaranteed to be supported with bug or security fixes. By
- focusing on a small subset of packages, the Ubuntu team was
- able to select a maintainable subsection of the Debian archive
- that they could maintain over time.</p><p>The most simple derived distributions — often
- working within the Debian project as CDDs but also including
- projects like Userlinux — are merely lists of packages
- and do nothing outside of package selection. The installation
- of lists of packages and the maintenance of those lists over
- time can be aided through the creation of what are called
- <span class="emphasis"><em>metapackages</em></span>: empty packages with long
- lists of "dependencies."</p><p>The second item, configuration changes, is also
- relatively low-impact. Focusing on moving as many changes as
- possible into the realm of configuration changes is a
- sustainable strategy that derivers working within the Debian
- project intent on a single code-base have pursued actively.
- Their idea is that rather than forking a piece of code due to
- disagreement in how the program should work, they can leave
- the code intact but add the <span class="emphasis"><em>ability</em></span> to
- work in a different way to the software. This alternate
- functionality is made toggleable through a configuration
- change in the same manner that applications are configured
- through questions asked at install time. Since the Debian
- project has a unified package configuration framework called
- Debconf, derivers are able to configure an entire system in a
- highly centralized manner.<a href="#ftn.idp32057520" class="footnote" name="idp32057520"><sup class="footnote">[8]</sup></a> This is not unlike RedHat's Kickstart although the
- emphasis is on maintenance of those configuration changes over
- the life and evolution of the package; Kickstart is focused
- merely on installation of the package.</p><p>A third type of configuration is limited to changes in the
- environment through which a system is run or installed. One is
- example is Progeny's Anaconda-based Debian installer which
- provides an alternate installer but results in an identical
- system. Another example is the Knoppix project which is famous
- for its "Live CD" environments. While, Knoppix makes a wide
- range of invasive changes that span all items in my list
- above, other Live CD projects, including Ubuntu's "Casper"
- project, are much closer to an alternate shell through which
- the same code is run.</p><p>Because these three methods are relatively non-invasive,
- they are reasonable strategies for small teams and individuals
- working on creating a derived distribution. However, many
- desirable changes — and in the case of some derived
- distributions, <span class="emphasis"><em>most</em></span> desirable changes
- — require more invasive techniques. The final and most
- invasive type of change — changes to code — is the
- most difficult but also the most promising and powerful if it
- can be done sustainably. Changes of this type involve
- bifurcations of the code-base and will be the topic of the
- remainder of this paper.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp32061728"></a>Distributed Source Control</h3></div></div></div><p>One promising method of maintaining deltas in forked or
- branched projects lies in distributed version control systems
- (VCS). Traditional VCS systems work in a highly centralized
- fashion. CVS, the archetypal free software VCS and the basis
- for many others, is based around the model of a single
- centralized server. Anyone who wishes to commit to a project
- must commit to the centralized repository. While CVS allows
- users to create branches, anyone with commit rights has access
- to the entire repository. The tools for branching and merging
- over time are not particularly good.</p><p>The branching model is primarily geared toward a system
- where development is bifurcated and then the branch is merged
- completely back into the main tree. Normal use of a branch
- might include creating a development branch, making a series
- of development releases while maintaining and fixing important
- bugs in the stable primary branch, and then ultimately
- replacing the stable release with the development release. The
- CVS model is <span class="emphasis"><em>not</em></span> geared toward a system
- where an arbitrary delta, or sets of deltas, are maintained
- over time.</p><p>Distributed version control aims to solve a number of
- problems introduced by CVS and alluded to above by:</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"><p>Allowing people to work disconnected from each other
- and to sync with each other, in whole or in part, in an
- arbitrary and ad-hoc fashion.</p></li><li class="listitem"><p>Allowing deltas to be maintained over time.</p></li></ul></div><p>Ultimately, this requires tools that are better at merging
- changes and in <span class="emphasis"><em>not</em></span> merging certain
- changes when that is the desired behavior. It also leads to tools capable
- of history-sensitive merging.</p><p>The most famous switch to a distributed VCS model from a
- centralized VCS model was the move by the Linux kernel
- development community to the proprietary distributed version
- control system BitKeeper. In his recent announcement of the
- decision to part ways with BitKeeper, Linus Torvalds
- said:</p><div class="blockquote"><blockquote class="blockquote"><p>In fact, one impact BK has had is to very fundamentally
- make us (and me in particular) change how we do things. That
- ranges from the fine-grained changeset tracking to just how
- I ended up trusting sub-maintainers with much bigger things,
- and not having to work on a patch-by-patch basis any
- more.<a href="#ftn.idp32069728" class="footnote" name="idp32069728"><sup class="footnote">[9]</sup></a>
- </p></blockquote></div><p>At the time of the switch, free distributed version
- control tools were less advanced than they are today. At the
- moment, an incomplete list of free software VCS tools includes
- GNU Arch, Bazaar, Bazaar-NG, Darcs, Monotone, SVK (based on
- Subversion), GIT (a system developed by Linus Torvalds as a
- replacement for BitKeeper) and others.</p><p>Each of these tools, at least after they reach a certain
- level of maturity, allow or will allow users to develop
- software in a distributed fashion and to, over time, compare
- their software and pull changes from others significantly more
- easily than they could otherwise. The idea of parallel
- development lies at the heart of the model. The tools for
- merging and resolving conflicts over time, and the ability to
- "cherry pick" certain patches or changes from a parallel
- developer each make this type of development significantly
- more useful than it has been in the past.</p><p>VCSs work entirely on the level of code. Due to the nature
- of the types of changes that Ubuntu project is making to
- Debian's code, Ubuntu has focused primarily on this model and
- Canonical currently funds two major distributed control
- products — the Bazaar and Bazaar-NG projects.</p><p>In many ways, employing distributed version control
- effectively is a much easier problem to solve for small, more
- traditional, free software development projects than it is for
- GNU/Linux distributions. Because the problems associated with
- maintaining parallel development of a single piece of software
- in a set of related distributed repositories is the primary
- use case for distributed version control systems, distributed
- VCS alone can be a technical solution for certain types of
- parallel development. As the tools and social processes for
- distributed VCS evolve, they will become increasingly
- important tools in the way that free software is
- developed.</p><p>Because the problems of scale associated with building an
- entire derivative distribution are more complicated than those
- associated with working with a single "upstream" project,
- distributed version control is only now being actively
- deployed in the Ubuntu project. In doing so, the project is
- focusing on integrating these into problem specific tools
- built on top of distributed version control.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp32075664"></a>Problem Specific Tools</h3></div></div></div><p>Another technique that Canonical Ltd. is experimenting
- with is the creation of high level tools built on top of
- distributed version control tools specifically designed for
- maintaining difference between packages. Because packages are
- usually distributed as a source file with a collection of one
- or more patches, this introduces the unique possibility of
- creating a high-level VCS system based around this fact.</p><p>In the case of Ubuntu and Debian, the ideal tool creates
- one branch per patch or feature and uses heuristics to
- analyze patch files and create these branches
- intelligently. The package build system section of the total
- patch can also be kept as a separate branch. Canonical's tool,
- called the Hypothetical Changeset Tool (HCT) (although no
- longer hypothetical), is one experimental way of creating a
- very simple, very streamlined interface for dealing with a
- particular type of source that is created and distributed in a
- particular type of way with a particular type of
- change.</p><p>While HCT promises to be very useful for people making
- derived distributions based on Debian, its application outside
- distribution makers will, in all likelihood, be limited. That
- said, it provides an example of the way that problem and
- context specific tools may play an essential role in the
- maintenance of derived code more generally.</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a name="idp32079568"></a>Social Solutions</h3></div></div></div><p>It has been said that it is a common folly of a
- technophile to attempt to employ technical solutions toward
- solving social problems. The problem of deriving software is
- both a technical <span class="emphasis"><em>and</em></span> social problem and
- adequately addressing the larger problems requires approaches that
- take into consideration both types of solution.</p><p>Scott James Remnant compares the relationship between
- distributions and derived distributions as similar to the
- relationship between distributions and upstream
- maintainers:</p><div class="blockquote"><blockquote class="blockquote"><p>I don't think this is much different from how Debian
- maintainers interact with their upstreams. As Debian
- maintainers we take and package upstream software and then
- act as a gateway for bugs and problems. Quite often we fix
- bugs ourselves and apply the patch to the package and send
- it upstream. Sometimes the upstream don't incorporate that
- patch and we have to make sure we don't accidentally drop it
- each subsequent release, we much prefer it if they take
- them, but we don't get angry if they don't.</p><p>This is how I see the relationship between Ubuntu and
- Debian, we're no more a fork of Debian than a Debian package
- is a fork of its upstream.</p></blockquote></div><p>Scott alludes the fact that, at least in the world of
- distributions, parallel development is already one way to view
- the <span class="emphasis"><em>modus operandi</em></span> of existing GNU/Linux
- distributions. The relationship between a deriver and derivee
- on the distribution level mirrors the relationship between the
- distribution and the "upstream" authors of the packages that
- make up the distribution. These relationships are rarely based
- around technological tools but are entirely in the realm of
- social solutions.</p><p>Ubuntu has pursued a number of different initiatives along
- these lines. The first of these has been to regularly file
- bugs in the Debian bug tracking system when bugs that exist in
- Debian are fixed in Ubuntu. While this can be partially
- automated, the choice to automate this and the manner in which
- it it is set up is a purely social one.</p><p>However, as I alluded to above, Ubuntu is still left with
- questions in regards to changes that are made to packages that
- do not necessarily fix bugs or that fix bugs that do not exist
- in Debian but may in the future. Some Debian developers want
- to hear about the full extent of changes made to their
- software in Ubuntu while others do not want to be
- bothered. Ubuntu should continue to work with Debian to find
- ways to allow developers to stay in sync.</p><p>There are also several initiatives by developers in
- Debian, Ubuntu, and in other derivations to create a
- stronger relationship between the Debian project and its
- ecosystem of derivers and between Ubuntu and Debian in
- particular. While the form that this will ultimately take is
- unclear, projects existing within an ecosystem should explore
- the realm of appropriate social relationships that will ensure
- that they can work together and be informed of each others'
- work without resorting to "spamming" each other with
- irrelevant or unnecessary information.</p><p>Another issue that has recently played an important role
- in the Debian/Ubuntu relationship is the importance of both
- giving adequate credit to the authors or upstream maintainers
- of software without implying a closer relationship than is the
- case. Derivers must walk a file line where they credit others'
- work on a project without implying that the others work for,
- support, or are connected to the derivers project to which, for
- any number of reasons, the "upstream" author might not want to
- be associated.</p><p>In the case of Debian and Ubuntu, this has resulted in an
- emphasis on keeping or importing changelog entries when
- changes are imported and in noting the pedigree of changes
- more generally. It has recently also been discussed in terms
- of the "maintainer" field in each package in Ubuntu. Ubuntu
- wants to avoid making changes to every unmodified source
- package (and introducing an unnecessary delta) but does not
- want to give the impression that the maintainer of the package
- is someone unassociated with Ubuntu. While no solution has
- been decided at the time of writing, one idea involved marking
- the maintainer of the package explicitly as a Debian
- maintainer at the time that the binary packages are built on
- the Ubuntu build machines.</p><p>The emphasis on social solutions is also essential when
- using distributed VCS technology. As Linus Torvalds alluded to
- in the quote above, the importance of technological changes to
- distributed VCS technology is only felt when people begin to
- work in a different way — when they begin to employ
- different social models of developer interaction.</p><p>While Ubuntu's experience can provide a good model for
- tackling some of these source control issues, it can only
- serve as a model and not as a fixed answer. Social solutions
- must be appropriate for a given social relationship. Even in
- situations where a package is branched because of social
- disagreements, a certain level of collaboration on a social
- level will be essential to the long term viability of the
- derivative.</p></div></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="idp32092080"></a>Conclusions</h2></div></div></div><p>As the techniques described in this paper evolve, the role
- that they play in free software development becomes increasingly
- prominent and increasingly important. Joining them will be other
- techniques and models that I have not described and cannot
- predict. Because of the size and usefulness of their code and
- the size of their development communities, large projects like
- Debian and Ubuntu have been forced into confronting and
- attempting to mediate the problems inherent in forking and
- deriving. However, as these problems are negotiated and tools
- and processes are advanced toward solutions, free software
- projects of all sizes will be able to offer users exactly what
- they want with minimal redundancy and little duplication of
- work. In doing this, free software will harness a power that
- proprietary models cannot compete with. They will increase their
- capacity to produce better products and better processes.
- Ultimately, it will help free software capture more users, bring
- in more developers, and produce more free software of a higher
- quality.</p></div><div class="footnotes"><br><hr style="width:100; text-align:left;margin-left: 0"><div id="ftn.idp26988128" class="footnote"><p><a href="#idp26988128" class="para"><sup class="para">[1] </sup></a>Information is listed on the distrowatch homepage
- here: <a class="ulink" href="http://distrowatch.com/dwres.php?resource=independence" target="_top">http://distrowatch.com/dwres.php?resource=independence</a></p></div><div id="ftn.idp26990736" class="footnote"><p><a href="#idp26990736" class="para"><sup class="para">[2] </sup></a>Netcraft posts yearly updates on the speed at which
- Linux distributions are growing. The one in question can be
- found at: <a class="ulink" href="http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html" target="_top">http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html</a></p></div><div id="ftn.idp26993168" class="footnote"><p><a href="#idp26993168" class="para"><sup class="para">[3] </sup></a>I spearheaded and help build a now mostly defunct
- derivation of Debian called Debian-Nonprofit (Debian-NP)
- geared for non-profit organizations by working within the
- Debian project.</p></div><div id="ftn.idp26998016" class="footnote"><p><a href="#idp26998016" class="para"><sup class="para">[4] </sup></a>Information Ubuntu can be found on the <a class="ulink" href="http://www.ubuntu.com" target="_top">Ubuntu homepage.</a>
- Information Canonical Limited can be found at <a class="ulink" href="http://www.canonical.com" target="_top">Canonical's
- homepage</a>.</p></div><div id="ftn.idp27004416" class="footnote"><p><a href="#idp27004416" class="para"><sup class="para">[5] </sup></a>Scott James Remnant maintains a list of these patches
- online here: <a class="ulink" href="http://people.ubuntu.com/~scott/patches/" target="_top">http://people.ubuntu.com/~scott/patches/</a></p></div><div id="ftn.idp27006336" class="footnote"><p><a href="#idp27006336" class="para"><sup class="para">[6] </sup></a>You can see that explicit statement on Ubuntu's
- website here: <a class="ulink" href="http://www.ubuntulinux.org/ubuntu/relationship/" target="_top">http://www.ubuntulinux.org/ubuntu/relationship/</a></p></div><div id="ftn.idp27008624" class="footnote"><p><a href="#idp27008624" class="para"><sup class="para">[7] </sup></a>The
- entire post can be read here: <a class="ulink" href="http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html" target="_top">http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html</a></p></div><div id="ftn.idp32057520" class="footnote"><p><a href="#idp32057520" class="para"><sup class="para">[8] </sup></a>More information on
- Debconf can be
- found online at: <a class="ulink" href="http://www.kitenet.net/programs/debconf/" target="_top">http://www.kitenet.net/programs/debconf/</a></p></div><div id="ftn.idp32069728" class="footnote"><p><a href="#idp32069728" class="para"><sup class="para">[9] </sup></a>The full message can be read online
- at: <a class="ulink" href="http://kerneltrap.org/mailarchive/1/message/48393/thread" target="_top">http://kerneltrap.org/mailarchive/1/message/48393/thread</a></p></div></div></div></body></html>
projects / to_fork_or_not / blobdiff