--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+
+<article id="paper-11194">
+ <articleinfo>
+ <title>To Fork or Not To Fork</title>
+ <subtitle>Lessons From Ubuntu and Debian</subtitle>
+ <author>
+ <firstname>Benjamin</firstname>
+ <othername>Mako</othername>
+ <surname>Hill</surname>
+ <affiliation>
+ <orgname>Canonical Limited</orgname>
+ </affiliation>
+ <affiliation>
+ <orgname>The Debian GNU/Linux Project</orgname>
+ </affiliation>
+ <affiliation>
+ <orgname>Software in the Public Interest, Inc.</orgname>
+ </affiliation>
+
+ <authorblurb>
+ <para>Benjamin Mako Hill is an intellectual property
+ researcher and activist and a professional Free/Open Source
+ Software (FOSS) advocate, developer, and consultant. He is
+ active participant in the Debian Project in both technical
+ and non-technical roles and a founder of Debian-Nonprofit
+ and other Free Software projects. He is the author of the
+ Free Software Project Management HOWTO and many published
+ works on Free and Open Source Software. He currently is
+ working full time for Canonical Ltd. on Ubuntu, a new
+ Debian-based distribution.</para>
+ </authorblurb>
+
+ </author>
+
+ <copyright>
+ <year>2005</year>
+ <holder>Benjamin Mako Hill</holder>
+ </copyright>
+ </articleinfo>
+
+ <abstract>
+ <para>This is where my abstract should go.</para>
+ </abstract>
+
+ <section>
+ <title>Introduction</title>
+
+ <para>The explosive growth of free and open source software over
+ the last decade has been mirrored by an equally explosive growth
+ in the ambitiousness of free software projects in choosing and
+ tackling problems. The free software movement approaches these
+ large problems with more code and with more expansive
+ communities than was even thinkable a decade ago. Example of
+ these massive projects include desktop environments — like
+ GNOME and KDE — and distributions like Debian.</para>
+
+ <para>These projects are leveraging the work of thousands of
+ programmers — both volunteer and paid — and are producing
+ millions of lines of code. Their software is being used by
+ millions of users with a diverse set of needs. This paper
+ focuses on two major effects of this situation:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>The communities that free software projects — and in
+ particular large projects — serve are increasingly diverse.
+ It is becoming increasingly difficult for a single large
+ project to release any single product that can cater to all
+ of its potential users.</para>
+ </listitem>
+ <listitem>
+ <para>It's becoming increasingly difficult to reproduce these
+ large projects. While reproducing entire project is
+ impossible for small groups of hackers, it is often not
+ substantially easier for small groups to even track and
+ maintain a fork of a large project over time.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>Taken together, these facts imply an increasingly realized
+ free software community in which programmers frequently derive
+ but where traditional forking is often untenable. "Forks," as
+ they are traditionally defined, will be improved upon.
+ Communities around large free software projects will be smarter
+ about the process of deriviation than they have been in the
+ past.</para>
+
+ <para>We are already seeing this with GNU/Linux distributions. New
+ distributions are rarely built from scratch today. Instead, they
+ adapted from and built on top of the work of existing projects.
+ As projects and userbases grow, these derived distributions are
+ increasingly common. Most of what I describe in this essay are
+ tools and experiences of derived distributions.</para>
+
+ <para>Software makers must pursue the idea of an
+ <emphasis>ecosystem</emphasis> of free software projects and
+ products that have forked but that maintain a close relationship
+ as they develop parallelly and symbiotically. To do this,
+ developers should:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Break down the process of derivation into a set of
+ different types of customization and derivation and
+ prioritize methods of derivation.</para>
+ </listitem>
+ <listitem>
+ <para>Create and foster social solutions to the social aspects
+ of the derivation problem.</para>
+ </listitem>
+ <listitem>
+ <para>Build and use new tools specifically designed to
+ coordinate development of software in the context of an
+ ecosystem of projects.</para>
+ </listitem>
+ <listitem>
+ <para>Distribute and utilize distributed version control tools
+ with an emphasis on maintaining differences over
+ time.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>This paper is an early analysis of this set of problems. As
+ such, it is highly focused on the experience of the Ubuntu
+ project and it's existence as a derived Debian distribution. It
+ also pulls from my experience with Debian-NP and the Custom
+ Debian Distribution (CDD) community. Since I am active member of
+ both the Ubuntu and Debian-NP projects, these are areas that I
+ can discuss with some degree of knowlege and experience.</para>
+ </section>
+
+ <section>
+ <title>"Fork" Is A Four Letter Word</title>
+
+ <para>The act of taking the code for a free software project and
+ bifurcating it to create a new project is called "forking."
+ There have been a number of famous forks in free software
+ history. One of the most famous was the schism that led to the
+ parallel development of two versions of the Emacs text editor:
+ GNU Emacs and XEmacs. This schism persists to this day.</para>
+
+ <para>Some forks, like Emacs and XEmacs, are permanent. Others are
+ relatively sort lived. An example of this is the GCC project
+ which saw two forks — EGCS and PGCC — that both eventually
+ merged back into GCC. Forking can happen for any number of
+ reasons. Often developers on a project develop political or
+ personal differences that keep them from continuing to work
+ together. In some cases, maintainers become unresponsive and
+ other developers on the project fork the project to keep the
+ project alive in some form.</para>
+
+ <para>Ultimately though, most forks occur because people do not
+ agree on the features, the mechanisms, or the technology at the
+ core of a project. People have different goals, different
+ problems, and want different tools. Often, these goals, problems
+ and tools are similar up until a certain point before the need
+ to part ways becomes essential.</para>
+
+ <para>A fork occurs on the level of code but a fork is not merely
+ — or even primarily — technical. Many projects create
+ "branches." Branches are alternative version of a piece of
+ software used to experiment with intrusive or unstable features
+ and bug fixes. Forks are distinguished from branches both in
+ that they are often more significant departures from a technical
+ perspective (i.e., more lines of code have been changed and/or
+ the changes are more invasive or represent a more fundamental
+ rethinking of the problem) and in that they are bifurcations
+ defined in social terms. Branches involve a
+ <emphasis>single</emphasis> developer or community of developers
+ — even if it does boil down to distinct subgroups within a
+ community — whereas forks are separate projects.</para>
+
+ <para>Forking has historically been viewed as a bad thing in free
+ software communities: they are seen to stem from people's
+ inability to work together and have ended in reproduction of
+ work. When I published the first version of the <ulink
+ url="http://mako.cc/projects/howto/">Free Software Project
+ Management HOWTO</ulink> more than four years ago, I included
+ a small subsection on forking which described forking to
+ prospective free software project leaders with this text:</para>
+
+ <blockquote>
+ <para>The short version of the fork section is, don't do them.
+ Forks force developers to choose one project to work with,
+ cause nasty political divisions, and redundancy of
+ work.</para>
+ </blockquote>
+
+ <para>In the <emphasis>best</emphasis> situations, a fork means
+ that two groups of people need to go on developing features and
+ doing work they would ordinarily do <emphasis>in addition
+ to</emphasis> tracking the forked project and having to
+ hand-select and apply features and fixes to their own code-base.
+ This level of monitoring and constant comparison can be
+ extremely difficult and time-consuming. The situation is not
+ helped substantially by traditional source control tools like
+ diff, patch, CVS and Subversion which are not optimized for this
+ task. The worse (and much more common) situation occurs when two
+ groups go about their work ignorant or partially ignorant of the
+ work done on the other side of the fork. Important features and
+ fixes are implemented twice — differently and
+ incompatibly.</para>
+
+ <para>The most substantial bright side to these drawbacks is that
+ the problems associated with forking are so severe and notorious
+ that, in most cases, the threat of a fork is enough to force
+ maintainers to work out solutions that keep the fork from
+ happening in the first place.</para>
+
+ <para>Before moving on, it is worth pointing out that fork is
+ something of a contested term. Because definitions of forks
+ involve, to one degree or another, statements about the
+ political, organization, and technical distinctions between
+ projects, bifurcations that many people call branches or
+ parallel trees are described as others as forks. Recently,
+ fueled by the advent of distributed version control systems, the
+ definition of what is and is not a fork has becoming
+ increasingly unclear. In part due to the same systems, the
+ benefits and drawbacks of what is increasingly problematically
+ called forking is equally debatable.</para>
+
+ </section>
+
+ <section>
+ <title>Case Study</title>
+
+ <para>In my introduction, I described how the growing scope of
+ free software projects and the rapidly increasingly size and
+ diversity of project's user communities is spearheading the need
+ for new type of derivation that avoids, as best as possible, the
+ drawbacks of forking. Nowhere is this more evident than in the
+ largest projects with the broadest scope: a small group of
+ projects that includes operating system distributions.</para>
+
+
+ <section>
+ <title>The Debian Project</title>
+
+ <para>The Debian project is a the largest, in terms of both code
+ and volunteers, free software distribution. It is the also,
+ arguably, the largest free software project in terms of the
+ number of volunteers. Debian includes more than 15,000
+ packages and the work of well over 1,000 official volunteers
+ and many more contributors without official membership status.
+ Projects without Debian's massive volunteer base cannot
+ replicate what Debian has accomplished; they can rarely hope
+ to even maintain what Debian currently has separately.</para>
+
+ <para>At the time that this paper was written, Distrowatch lists
+ 129 distributions based on Debian<footnote>
+ <para>Information is listed on the distrowatch homepage
+ here: <ulink
+ url="http://distrowatch.com/dwres.php?resource=independence">http://distrowatch.com/dwres.php?resource=independence</ulink></para>
+
+ </footnote> — most of them currently active to one degree or
+ another. Each distribution represents at least one person —
+ and in most cases a community of people — who disagreed with
+ Debian's vision or direction strongly enough to want to create
+ a new distribution <emphasis>and</emphasis> who had the
+ technical capacity to follow through with this goal. Despite
+ Debian's long-standing slogan — "the universal operating
+ system" — the fact that the Debian project has become the
+ fastest growing operating system while spawning so many
+ derivatives is testament to the fact that, as far as software
+ is concerned, one size does <emphasis>not</emphasis> fit
+ all.<footnote>
+ <para>Netcraft posts yearly updates on the speed at which
+ Linux distributions are growing. The one in question can
+ be found at <ulink
+ url="http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html">http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html</ulink></para>
+ </footnote>
+ </para>
+
+
+ <para>Organizationally, Debian derivers are located both inside
+ and outside of the Debian project. A group of derivers working
+ within the Debian project has labeled themselves "Custom
+ Debian Distributions" and has created nearly a dozen projects
+ customizing and deriving from Debian for specific groups of
+ users including non-profit organization, the medical
+ community, lawyers, children and many others.<footnote>
+ <para>I have spearheaded and built a derivation of Debian
+ called Debian-Nonprofit (Debian-NP) geared for non-profit
+ organizations working within the Debian project.</para>
+ </footnote> These projects build on the core distribution and
+ the canonical archive from <emphasis>within</emphasis> the
+ organizational and political limits of the Debian project and
+ constantly seek to minimize the delta by focusing on less
+ invasive changes and by advancing creative ways of building
+ the <emphasis>ability</emphasis> to make changes in the core
+ Debian code base through established and policy compliant
+ procedures.</para>
+
+<!-- http://linktocddinformation -->
+
+ <para>A second group of Debian customizers includes those
+ working outside of the Debian project organizationally.
+ Notable among this list are (in alphabetical order) Knoppix,
+ Libranet, Linspire (formerly Lindows), Progeny, MEPIS, Ubuntu,
+ Userlinux, and Xandros. With its strong technological base,
+ excellent package management, wide selection of packages to
+ choose from, and strong commitment to software freedom which
+ ensures derivability, Debian provides an ideal point from
+ which to create a GNU/Linux distribution.</para>
+
+ </section>
+
+
+ <section>
+ <title>Ubuntu</title>
+
+ <para>The Ubuntu project was started by Mark Shuttleworth in
+ April 2004 and the first version was executed almost entirely
+ by a small group of a Debian developers by Shuttleworth's
+ company Canonical Limited.<footnote>
+ <para>Information Ubuntu can be found on the <ulink
+ url="http://www.ubuntu.com">Ubuntu homepage.</ulink>
+ Information Canonical Limited can be found at <ulink
+ url="http://www.canonical.com">Canonical's
+ homepage</ulink>.</para>
+ </footnote> It was released to the world in the fall of 2004.
+ The second version was released six months later in April
+ 2005. The goals of Ubuntu are to provide a distribution based
+ on a subset of Debian with:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Regular and predictable releases — every six months
+ with support for eighteen months.</para>
+ </listitem>
+ <listitem>
+ <para>An emphasis on free software that will maintain the
+ derivability of the distribution.</para>
+ </listitem>
+ <listitem>
+ <para>An emphasis on usability and a consistent desktop
+ vision. As an example, this has translated into less
+ questions in the installer and a default selection and
+ configuration of packages that is usable for most desktop
+ users "out of the box."</para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>The Ubuntu project provides an interesting example of a
+ project that aims to derive from Debian to an extensive
+ degree. Ubuntu made code-level changes to nearly 1300 packages
+ in Debian at the time that this paper was written and the
+ speed of changes will only accelerate with time; the total
+ number of changes and the total size of the delta will
+ grow.<footnote>
+ <para>Scott James Remnant maintains a list of these patches
+ online here: <ulink
+ url="http://people.ubuntu.com/~scott/patches/">http://people.ubuntu.com/~scott/patches/</ulink></para>
+ </footnote> The changes that Ubuntu makes are primarily of the
+ most intrusive kind — changes to the code itself.</para>
+
+ <para>That said, the Ubuntu project is explicit about the fact
+ that it could not exist with the work done by the Debian
+ project before Ubuntu was created.<footnote>
+ <para>You can see that explicit statement on Ubunut's
+ website here: <ulink
+ url="http://www.ubuntulinux.org/ubuntu/relationship/">http://www.ubuntulinux.org/ubuntu/relationship/</ulink></para>
+ </footnote> More importantly, Ubutnu explains that it cannot
+ continue to provide the complete set of packages that its
+ users depend on without the ongoing work by the Debian
+ project. Even though Ubuntu has made changes to the nearly
+ 1300 packages, this is less than ten percent of the total
+ packages shipped in Ubuntu and pulled from Debian.</para>
+
+ <para>Scott James Remnant, a prominent Debian developer and a
+ hacker on Ubuntu who works for Canonical Ltd., described the
+ situation this way on his web log to introduce the Ubuntu
+ development methodology in the week after first public
+ announcement of Canonical and Ubuntu:<footnote>
+ <para>The entire post can be read here: <ulink
+ url="http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html">http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html</ulink></para>
+ </footnote>
+ </para>
+
+ <blockquote>
+
+ <para>I don't think Ubuntu is a "fork" of Debian, at least not
+ in the traditional sense. A fork suggests that at some
+ point we go our separate way from Debian and then
+ occasionally merge in changes as we carry on down our own
+ path.</para>
+
+ <para>Our model is quite different; every six months we take a
+ snapshot of Debian's unstable distribution, apply any
+ outstanding patches from our last release to it and spend a
+ couple of months testing and bug-fixing it.</para>
+
+
+ <para>
+ <inlinemediaobject>
+ <imageobject>
+ <imagedata fileref="picture-11194-01.png" format="PNG"/>
+ </imageobject>
+ </inlinemediaobject>
+ </para>
+
+ <para>One thing that should be obvious from this is our job is
+ a lot easier if Debian take all of our changes, the model
+ actually encourages us to give back to Debian.</para>
+
+ <para>That's why from the very first day we started fixing
+ bugs we began sending <ulink
+ url="http://www.no-name-yet.com/patches/">the
+ patches</ulink>_ back to Debian through the BTS. Not only
+ will it make our job so much easier when we come to freeze
+ for "hoary", our next release, but it's exactly what every
+ derivative should do in the first place.</para>
+
+ </blockquote>
+
+ <para>There is some debate on the degree to which Ubuntu
+ developers have succeeded in accomplishing the goals laid out
+ by Remnant. Ubuntu has filed hundreds of patches in the bug
+ tracking system although it has often run into problems in
+ deciding <emphasis>what</emphasis> constitutes something that
+ should be fed back to Debian. Many changes are simply not
+ relevant to upstream Debian developers. For example, they may
+ include changes to a package in response to another change
+ made in another package in Ubuntu that will not or has not
+ been taken by Debian.</para>
+
+ <para>The Ubuntu project's track record in working
+ constructively with Debian is, at the moment, decidedly mixed.
+ While an increasingly large number of Debian developers are
+ maintaining their packages actively within both projects, many
+ in both Debian and Ubuntu feel that Ubuntu has work left to do
+ in living up to its own goals of a smooth productive
+ relationship with Debian.</para>
+
+ </section>
+
+ <section>
+ <title>Applicability</title>
+
+ <para>Ubuntu and Debian are distributions and — as such —
+ operate on a different scale than the vast majority of free
+ software projects. Using a very simple metric, they include
+ more code and more people. As a result, there are questions as
+ to whether the experiences and lessons learned from these
+ projects are particularly applicable to the experience of
+ smaller free software projects.</para>
+
+ <para>Clearly, because of the difficulties associated with
+ forking massive amount of code and the problems associated
+ with duplicating the work of large volunteer bases,
+ distributions are forced into finding a way to balance the
+ benefits and drawbacks of forking. However, while the need is
+ stronger and more immediate in larger projects, the benefits
+ of their solutions will often be fully transferable.</para>
+
+ <para>Clearly, modifiability of free software to better fit the
+ needs of its users lies at the heart of the free software
+ movement's success. However, while modification usually comes
+ in the form of collaboration on a single code-base, this is
+ function of limitations in software development methodologies
+ and tools rather than the best response to the needs or
+ desires of users or developers.</para>
+
+ <para>I believe that the fundamental advantage of free software
+ in the next decade will be in the growing ability of any
+ single free software project to be multiple things to multiple
+ users simultaneously. This will translate into the fact that,
+ in the next ten years, technology and social processes will
+ evolve so that forking is increasingly less of a bad thing.
+ Free software development methodology will become less
+ dependent on a single project and begin to emphasize parallel
+ development within an ecosystem of software development
+ working on related projects. The result is that free software
+ projects will gain a competitive advantage over propriety
+ software projecrts through their ability to better serve the
+ increasingly diverse needs of increasingly large and
+ increasingly diverse user-bases. More projects will derive and
+ less redundant code will be written.</para>
+
+ <para>Projects more limited in code and scope may use the tools
+ and methods in different combinations, in different ways, and
+ to different degrees than the examples around distributions
+ introduced here. Different projects with different needs will
+ find that certain solutions work better than others. Because
+ communities of the size of Debian are difficult to fork in a
+ way that is beneficial to any party, it is in these
+ communities that the technology and development methodologies
+ will are first emerging. With time, these strategies and tools
+ will find themselves employed productively in a wide variety
+ of projects with a broad spectrum of sizes, needs, scopes and
+ descriptions.</para>
+
+ </section>
+
+ </section>
+
+ <section>
+ <title>Balancing Forking With Collaboration</title>
+
+ <section>
+ <title>Derivation and Problem Analysis</title>
+
+ <para>The easiest step in creating a productive derivative
+ software project is to break down the problems of deriviations
+ into a series of different classes of modification. Certain
+ types of modification is more easily done and are
+ intrinsically more maintainable.</para>
+
+ <para>In the context of distributions, the problem of derivation
+ can be broken down into the following types of changes (sorted
+ roughly according to the intrusiveness inherent in solving the
+ problem and the severity of the long-term maintainability
+ problems that they introduce):</para>
+
+ <orderedlist>
+ <listitem>
+ <para>Selection of individual pieces of software;a</para>
+ </listitem>
+ <listitem>
+ <para>Changes to the way that packages are installed or run
+ (e.g., in a Live CD type environment or using a different
+ installer);</para>
+ </listitem>
+ <listitem>
+ <para>Configuration of different pieces of software;</para>
+ </listitem>
+ <listitem>
+ <para>Changes made to the actual software package (made on
+ the level of changes to the packages code);</para>
+ </listitem>
+ </orderedlist>
+
+ <para>By breaking down the problem in this way. Debian derivers
+ have been able to approach deriviation in ways that focus
+ energy on the less intrusive problems first.</para>
+
+ <para>The first area that Ubuntu focused on was selecting a
+ subset of package that Ubuntu would support. Ubuntu selected
+ and supports approximate 2,000 packages. These became the
+ <command>main</command> component in Ubuntu. Other packages in
+ Debian were included in a separate section of the Ubuntu
+ archive called <command>universe</command> but were not
+ guaranteed to be supported with bug or security fixes. By
+ focusing on a small subset of packages, the Ubuntu team was
+ able to select a maintainable subsection of the Debian archive
+ that they could maintain over time.</para>
+
+ <para>The most simple derived distributions — often working
+ within the Debian project as CDDs but also including projects
+ like Userlinux — are merely lists of packages and do nothing
+ outside of package selection. The installation of lists of
+ packages and the maintenance of those lists over time can be
+ aided through the creation of what are called "metapackages:"
+ empty packages that are maintained over time.</para>
+
+ <para>The second item, configuration changes, are also
+ relatively low-impact. Focusing on moving as many changes as
+ possible into the realm of configuration changes is a
+ relatively low-impact strategy that derivers working within
+ the Debian project intent on a single code-base have pursued
+ actively. Their idea is that rather than forking a piece of
+ code due to disagreement in how the program should work, they
+ can leave the code intact but add the
+ <emphasis>ability</emphasis> to work in a different way. This
+ alternate functionality is made toggleable through a
+ configuration change of the distribution in much the same that
+ applications can be configured differently or shipped with
+ different configuration files. Since the Debian project has a
+ unified package configuration framework called Debconf,
+ derivers are able to configure an entire system in aa highly
+ centralized manner.<footnote>
+ <para>More information on Debconf can be found online at:
+ <ulink
+ url="http://www.kitenet.net/programs/debconf/">http://www.kitenet.net/programs/debconf/</ulink></para>
+ </footnote> This is not unlike RedHat's Kickstart although the
+ emphasis is on maintenance of those configuration changes over
+ the life and evolution of the package; Kickstart is focused
+ merely on installation of the package.</para>
+
+ <para>A third type of configuration is limited to changes in the
+ environment through which a system is run or installed. One is
+ example is Progeny's Anaconda-based Debian installer provides
+ which an alternate installer but installs an identical system.
+ Another example is the Knoppix project is famous for its Live
+ CD environments.<footnote>
+ <para>In reality, Knoppix makes a wide range of changes to a
+ default Debian installation that spam all items in my list
+ above.</para>
+ </footnote> Other Live CD projects, including Ubuntu's
+ Casper project, are purely a different
+ way of running the exact same code.</para>
+
+ <para>Because these three methods are relatively non-invasive,
+ they are reasonable strategies for small teams and individuals
+ working on creating a derived distribution. However, many
+ desirable changes — and in the case of some derived
+ distributions, most desirable changes — require more
+ invasive changes. The final and most invasive type of change
+ — changes to code — is the most difficult but also the most
+ promising and powerful if solved. Changes of this type involve
+ bifurcations of the code-base and will be the topic of the
+ remainder of this paper.</para>
+
+ </section>
+
+ <section>
+ <title>Distributed Source Control</title>
+
+ <para>One promising method of maintaining changes in forked or
+ branched problems lies in distributed version control systems
+ (VCS). Traditional VCS systems work in a highly centralized
+ fashion. CVS, the archetypal free software VCS and the basis
+ for many others, is based around the model of a single
+ centralized server. Anyone who wishes to commit to a project
+ must commit to the centralized repository. While CVS allows
+ users to create branches, anyone with commit rights has access
+ to the entire repository. The tools for branching and merging
+ over time are not particularly good.</para>
+
+ <para>The branching model is primarily geared toward a system
+ where development is bifurcated and then the branch is merged
+ completely back into the main tree. Normal use of a branch
+ might include creating a development branch, making a series
+ of development releases while maintaining and fixing important
+ bugs in the stable primary branch, and then ultimately
+ replacing the stable release with the development release. The
+ CVS model is <emphasis>not</emphasis> geared toward a system
+ where an arbitrary delta, or sets of deltas, is maintained
+ over time.</para>
+
+ <para>Distributed version control aims to solve a number of
+ problems introduced by CVS and alluded to above by:</para>
+
+ <itemizedlist>
+ <listitem>
+ <para>Allowing people to work disconnected from each other
+ and to sync with each other, in whole or in part, in an
+ arbitrary and ad-hoc fashion.</para>
+ </listitem>
+ <listitem>
+ <para>Allowing deltas to be maintained over time.</para>
+ </listitem>
+ </itemizedlist>
+
+ <para>Ultimately, this requires tools that are better at merging
+ changes and in <emphasis>not</emphasis> merging certain
+ changes when that is desirable. It also leads to tools capable
+ of history-sensitive merging.</para>
+
+ <para>The most famous switch to a distributed VCS model from a
+ centralized VCS model was the move by the Linux kernel
+ development community to the proprietary distributed version
+ control system BitKeeper. In his recent announcement of the
+ decision to part ways with BitKeeper, Linus Torvalds
+ said:</para>
+
+ <blockquote>
+ <para>In fact, one impact BK has had is to very fundamentally
+ make us (and me in particular) change how we do things. That
+ ranges from the fine-grained changeset tracking to just how
+ I ended up trusting sub-maintainers with much bigger things,
+ and not having to work on a patch-by-patch basis any
+ more.<footnote> <para>The full message can be read online
+ at: <ulink
+ url="http://kerneltrap.org/mailarchive/1/message/48393/thread">http://kerneltrap.org/mailarchive/1/message/48393/thread</ulink></para>
+ </footnote>
+ </para>
+ </blockquote>
+
+ <para>At the time of the switch, free distributed version
+ control tools were less advanced than they are today. At the
+ moment, an incomplete list of free software VCS tools includes
+ GNU Arch, Bazaar, Bazaar-NG, Darcs, Monotone, SVK (based on
+ Subversion), GIT (a system developed by Linus Torvlards as a
+ temporary replacement for BitKeeper) and others.</para>
+
+ <para>Each of these tools, at least after they reach a certain
+ level of maturity, allow or will allow its users to develop
+ software in a distributed fashion and to, over time, compare
+ their software and pull changes from others significantly more
+ easily than they could otherwise. The idea of parallel
+ development lies at the heart of the model, the tools for
+ merging and resolving conflicts over time, and the ability to
+ "cherry pick" certain patches or changes from a parallel
+ developer each make this type of development significantly
+ more useful than it has been in the past.</para>
+
+ <para>VCSs work entirely on the level of code. Due to the nature
+ of the types of changes that Ubuntu project is making to
+ Debian's code, Ubuntu has focused primarily on this model and
+ Canonical currently funds two major distributed control
+ products — the Bazaar and Bazaar-NG projects.</para>
+
+ <para>In many ways, employing distributed version control
+ effectively is a much easier problem to solve for small, more
+ traditional, free software development projects than it is for
+ GNU/Linux distributions. Because the problems with maintaining
+ parallel development of a single piece of software in a set of
+ related distributed repositories is primary use case for
+ distributed version control system, distributed VCS alone can
+ be a technical solution for certain types of parallel
+ development. As the tools and social processes for distributed
+ VCS evolve, they will become increasingly important tools in
+ the way that free software is developed.</para>
+
+ <para>Because the problems of scale associated with buildling an
+ entire derivative distribution are more complicated than those
+ associated with working with a single project, distributed
+ version control has not yet been widely deployed in the Ubuntu
+ project. Instead, the project is focusing on integrating these
+ into problem specific tools built on top of distributed
+ version control.</para>
+
+ </section>
+
+ <section>
+ <title>Problem Specific Tools</title>
+
+ <para>Another technique that Canonical Ltd. is experimenting
+ with is the creation of high level tools built on top of
+ distributed version control tools specifically designed for
+ maintaining difference between packages. Because packages are
+ usually distributed as a source file with a collection of one
+ or more patches, this introduces the unique possibility of
+ creating a limited high-level VCS system based on this
+ fact.</para>
+
+ <para>In the case of Ubuntu and Debian, he tool is creating one
+ branch per patch or feature and using heuristics to analyze
+ patch files and create these branches intelligently. The
+ package build system section of the total patch can also be
+ kept as a separate branch. Canonical's tool, called the
+ Hypothetical Changeset Tool (HCT) (although no longer
+ hypothetical), is one experimental way of creating a very
+ simple, very streamlined interface for dealing with a
+ particular type of source that is created and distributed in a
+ particular type of way with a particular type of
+ change.</para>
+
+ <para>While HCT promises to be very useful for people making
+ derived distributions based on Debian, its wider application
+ may be limited. That said, this provides an example of the way
+ that problem and context specific tools may play an essential
+ role in the maintenance of derived code more generally.</para>
+
+ </section>
+
+
+ <section>
+ <title>Social Solutions</title>
+
+ <para>It has been said that a common folly among technophiles is
+ based on the temptation to employ technical solutions toward
+ solving social problems. The problem of deriving software is
+ both a technical <emphasis>and</emphasis> a social problem and
+ adequately addressing the issue will require approaches that
+ take into consideration both type of solution.</para>
+
+ <para>Scott James Remnant compares the relationship between
+ distributions and derived distributions as not unlike the
+ relationship between distributions and upstream
+ maintainers:</para>
+ <blockquote>
+
+ <para>I don't think this is much different from how Debian
+ maintainers interact with their upstreams. As Debian
+ maintainers we take and package upstream software and then
+ act as a gateway for bugs and problems. Quite often we fix
+ bugs ourselves and apply the patch to the package and send
+ it upstream. Sometimes the upstream don't incorporate that
+ patch and we have to make sure we don't accidentally drop it
+ each subsequent release, we much prefer it if they take
+ them, but we don't get angry if they don't.</para>
+
+ <para>This is how I see the relationship between Ubuntu and
+ Debian, we're no more a fork of Debian than a Debian package
+ is a fork of its upstream.</para>
+ </blockquote>
+
+ <para>Scott alludes the fact that, at least in the world of
+ distributions, parallel development is already one way to view
+ the <emphasis>modus operandi</emphasis> of existing GNU/Linux
+ distributions. The relationship between a deriver and derivee
+ on the distribution level mirrors the relationship between the
+ distribution and the "upstream" authors of the packages that
+ make up the distribution. These relationships are rarely based
+ around technological tools but are entirely in the realm of
+ social solutions.</para>
+
+ <para>Ubuntu has pursued a number of different initiatives along
+ these lines. The first of these has been to regularly file
+ bugs in the Debian bug tracking system when bugs are fixed
+ that exist in Debian are fixed in Ubuntu. While this can be
+ partially automated, the choice to automate this is a purely
+ social one.</para>
+
+ <para>Ubuntu is still left with questions in regards to changes
+ that are made to packages that do not necessarily fix bugs or
+ that fix bugs that do not exist in Debian but may in the
+ future. Some Debian developers want to hear about the full
+ extent of changes made to their software in Ubuntu while
+ others do not want to be bothered. Ubuntu should continue to
+ work with Debian to find ways to allow developers to stay in
+ sync.</para>
+
+ <para>There is a recent initiative by some developers in Debian,
+ largely led by myself, to create a stronger relationship
+ between the Debian project and its ecosystem of derivers.
+ While the form that this will ultimately take is unclear,
+ projects existing within an ecosystem should explore the realm
+ of appropriate social relationships that will ensure that they
+ can work together and be informed of each others' work without
+ resorting to "spamming" each other with irrelevant or
+ unnecessary information.</para>
+
+ <para>Another issue that has recently played an important role
+ in the Debian/Ubuntu relationship is the importance of both
+ giving adequate credit to the authors or upstream maintainers
+ of software without implying a closer relationship than is the
+ case. Derivers must walk a file line where they credit others'
+ work on a project without implying that the others works for,
+ support, or are connected to the derivers project which, for
+ any number of reasons, the original author might not want to
+ be associated with.</para>
+
+ <para>In the case of Debian and Ubuntu, this has resulted in an
+ emphasis on keeping or importing changelog entries when
+ changes are imported and in noting the pedigree of changes
+ more generally. It has recently also been discussed in terms
+ of the "maintainer" field in each package in Ubuntu. Ubuntu
+ wants to avoid making changes to every unmodified source
+ package (and introducing an unnecessary delta) but does not
+ want to give the impression that the maintainer of the package
+ is someone unassociated with Ubuntu. While no solution has
+ been decided at the time of writing, one idea involved marking
+ the maintainer of the package explicitly as a Debian
+ maintainer at the time that the binary packages are built on
+ the Ubuntu build machines.</para>
+
+ <para>The emphasis on social solutions is also essential when
+ using distributed VCS technology. As Linus Torvalds alluded to
+ in the quote above, the importance of technological changes to
+ distributed VCS technology is only felt when people begin to
+ work in a different way — when they begin to employ differnet
+ social models of developer interaction.</para>
+
+ <para>While Ubuntu's experience can provide a good model for
+ tackling some of these source control issues, it can only
+ serve as a model and not as a fixed answer. Social solutions
+ must be appropriate for a given social relationship. Even in
+ situations where a package is branched because of social
+ incompatibility, a certain level of collaboration on a social
+ level will be essential to the long term viability of the
+ derivation.</para>
+
+ </section>
+
+ </section>
+
+ <section>
+ <title>Conclusions</title>
+
+ <para>As the techniques described in this paper evolve, the role
+ that they play in free software development becomes increasingly
+ prominent and increasingly important. Joining them will be other
+ techniques and models that I have not seen and cannot predict.
+ Because of the size and usefulness of their code and the size of
+ their development communities, large projects like Debian and
+ Ubuntu have been forced into confronting and attempting to
+ mediate the problems inherent in forking and deriving. However,
+ as these problems are negotiated and tools and processes are
+ advanced toward solutions, free software projects of all sizes
+ will be able to offer users exactly what they want with minimal
+ redundancy and little duplication of work. In doing this, free
+ software will harness a power that proprietary models cannot
+ compete with. They will increase their capacity to produce
+ better products and better processes. Ulimately, it will help
+ free software capture more users, bring in more developers, and
+ produce more free software of a higher quality.</para>
+
+ </section>
+
+</article>
+
+
+<!-- Keep this comment at the end of the file
+Local variables:
+mode: xml
+sgml-omittag:t
+sgml-shorttag:t
+sgml-namecase-general:t
+sgml-general-insert-case:lower
+sgml-minimize-attributes:nil
+sgml-always-quote-attributes:t
+sgml-parent-document:nil
+sgml-exposed-tags:nil
+sgml-local-catalogs:nil
+sgml-local-ecat-files:nil
+sgml-indent-step: 2
+sgml-indent-data: 2
+sgml-set-face: t
+End:
+-->
\ No newline at end of file
projects / to_fork_or_not / commitdiff
