<authorblurb>
<para>Benjamin Mako Hill is an intellectual property
researcher and activist and a professional Free/Open Source
- Software (FOSS) advocate, developer, and consultant. He is
- active participant in the Debian Project in both technical
- and non-technical roles and a founder of Debian-Nonprofit
- and other Free Software projects. He is the author of the
- Free Software Project Management HOWTO and many published
- works on Free and Open Source Software. He currently is
- working full time for Canonical Ltd. on Ubuntu, a new
- Debian-based distribution.</para>
+ Software (FOSS) advocate and developer. He is active
+ participant in the Debian Project in both technical and
+ non-technical roles. He is the author of the Free Software
+ Project Management HOWTO and many published works on Free
+ and Open Source Software. He currently is working full time
+ for Canonical Ltd. on Ubuntu, a new Debian-based
+ distribution.</para>
</authorblurb>
</author>
<year>2005</year>
<holder>Benjamin Mako Hill</holder>
</copyright>
- </articleinfo>
- <abstract>
- <para>This is where my abstract should go.</para>
- </abstract>
+
+ <legalnotice>
+ <para>This material is licensed under the <ulink
+ url="http://creativecommons.org/licenses/by-sa/2.0/">Creative
+ Commons Attribution-Sharealike 2.0 License</ulink>.</para>
+
+ <para>The canonical location for the most recent version of this
+ document is <ulink url="http://mako.cc/">at the author's
+ website</ulink>.</para>
+
+ </legalnotice>
+
+ <revhistory>
+ <revision>
+ <revnumber>0.2</revnumber>
+ <date>August 7, 2005</date>
+ <revremark>Correction and improvements.</revremark>
+ </revision>
+ <revision>
+ <revnumber>0.1</revnumber>
+ <date>May 15, 2005</date>
+
+ <revdescription>
+ <para>The first version of this paper was written to an
+ accepted talk given at Linuxtag 2005 given in Karlsruhe,
+ Germany.</para>
+ </revdescription>
+
+ </revision>
+ </revhistory>
+
+
+ </articleinfo>
<section>
<title>Introduction</title>
in the ambitiousness of free software projects in choosing and
tackling problems. The free software movement approaches these
large problems with more code and with more expansive
- communities than was even thinkable a decade ago. Example of
- these massive projects include desktop environments — like
- GNOME and KDE — and distributions like Debian.</para>
+ communities than was thinkable a decade ago. Example of these
+ massive projects include desktop environments — like GNOME
+ and KDE — and distributions like Debian, RedHat, and
+ Gentoo.</para>
<para>These projects are leveraging the work of thousands of
- programmers — both volunteer and paid — and are producing
- millions of lines of code. Their software is being used by
- millions of users with a diverse set of needs. This paper
- focuses on two major effects of this situation:</para>
+ programmers — both volunteer and paid — and are
+ producing millions of lines of code. Their software is being
+ used by millions of users with diverse sets of needs. This
+ paper focuses on two major effects of this situation:</para>
<itemizedlist>
<listitem>
- <para>The communities that free software projects — and in
- particular large projects — serve are increasingly diverse.
- It is becoming increasingly difficult for a single large
- project to release any single product that can cater to all
- of its potential users.</para>
+
+ <para>The communities that free software projects — and
+ in particular large projects — serve are increasingly
+ diverse. It is becoming increasingly difficult for a single
+ large project to release any single product that can cater
+ to all of its potential users.</para>
+
</listitem>
<listitem>
+
<para>It's becoming increasingly difficult to reproduce these
large projects. While reproducing entire project is
- impossible for small groups of hackers, it is often not
- substantially easier for small groups to even track and
- maintain a fork of a large project over time.</para>
+ impossible for small groups of hackers, it is often not even
+ possible for small groups to even track and maintain a fork
+ of a large project over time.</para>
+
</listitem>
</itemizedlist>
<para>Taken together, these facts imply an increasingly realized
free software community in which programmers frequently derive
but where traditional forking is often untenable. "Forks," as
- they are traditionally defined, will be improved upon.
- Communities around large free software projects will be smarter
- about the process of deriviation than they have been in the
+ they are traditionally defined, must be improved upon.
+ Communities around large free software projects must be smarter
+ about the process of derivation than they have been in the
past.</para>
<para>We are already seeing this with GNU/Linux distributions. New
distributions are rarely built from scratch today. Instead, they
adapted from and built on top of the work of existing projects.
- As projects and userbases grow, these derived distributions are
+ As projects and user-bases grow, these derived distributions are
increasingly common. Most of what I describe in this essay are
tools and experiences of derived distributions.</para>
<para>This paper is an early analysis of this set of problems. As
such, it is highly focused on the experience of the Ubuntu
- project and it's existence as a derived Debian distribution. It
+ project and its existence as a derived Debian distribution. It
also pulls from my experience with Debian-NP and the Custom
- Debian Distribution (CDD) community. Since I am active member of
- both the Ubuntu and Debian-NP projects, these are areas that I
- can discuss with some degree of knowlege and experience.</para>
+ Debian Distribution (CDD) community. Since I participate in both
+ the Ubuntu and CDD projects, these are areas that I can discuss
+ with some degree of knowledge and experience.</para>
</section>
<section>
GNU Emacs and XEmacs. This schism persists to this day.</para>
<para>Some forks, like Emacs and XEmacs, are permanent. Others are
- relatively sort lived. An example of this is the GCC project
- which saw two forks — EGCS and PGCC — that both eventually
- merged back into GCC. Forking can happen for any number of
- reasons. Often developers on a project develop political or
- personal differences that keep them from continuing to work
- together. In some cases, maintainers become unresponsive and
- other developers on the project fork the project to keep the
- project alive in some form.</para>
+ relatively short lived. An example of this is the GCC project
+ which saw two forks — EGCS and PGCC — that both
+ eventually merged back into GCC. Forking can happen for any
+ number of reasons. Often developers on a project develop
+ political or personal differences that keep them from continuing
+ to work together. In some cases, maintainers become unresponsive
+ and other developers fork to keep the software alive.</para>
<para>Ultimately though, most forks occur because people do not
agree on the features, the mechanisms, or the technology at the
<para>A fork occurs on the level of code but a fork is not merely
— or even primarily — technical. Many projects create
- "branches." Branches are alternative version of a piece of
+ "branches." Branches are alternative versions of a piece of
software used to experiment with intrusive or unstable features
- and bug fixes. Forks are distinguished from branches both in
+ and fixes. Forks are distinguished from branches both in
that they are often more significant departures from a technical
perspective (i.e., more lines of code have been changed and/or
the changes are more invasive or represent a more fundamental
rethinking of the problem) and in that they are bifurcations
- defined in social terms. Branches involve a
+ defined in social and political terms. Branches involve a
<emphasis>single</emphasis> developer or community of developers
— even if it does boil down to distinct subgroups within a
community — whereas forks are separate projects.</para>
work. When I published the first version of the <ulink
url="http://mako.cc/projects/howto/">Free Software Project
Management HOWTO</ulink> more than four years ago, I included
- a small subsection on forking which described forking to
- prospective free software project leaders with this text:</para>
+ a small subsection on forking which described the concept to
+ future free software project leaders with this text:</para>
<blockquote>
<para>The short version of the fork section is, don't do them.
diff, patch, CVS and Subversion which are not optimized for this
task. The worse (and much more common) situation occurs when two
groups go about their work ignorant or partially ignorant of the
- work done on the other side of the fork. Important features and
- fixes are implemented twice — differently and
+ code being cut on the other side of the fork. Important features
+ and fixes are implemented twice — differently and
incompatibly.</para>
<para>The most substantial bright side to these drawbacks is that
maintainers to work out solutions that keep the fork from
happening in the first place.</para>
- <para>Before moving on, it is worth pointing out that fork is
- something of a contested term. Because definitions of forks
- involve, to one degree or another, statements about the
- political, organization, and technical distinctions between
- projects, bifurcations that many people call branches or
- parallel trees are described as others as forks. Recently,
- fueled by the advent of distributed version control systems, the
- definition of what is and is not a fork has becoming
- increasingly unclear. In part due to the same systems, the
- benefits and drawbacks of what is increasingly problematically
- called forking is equally debatable.</para>
+ <para>Finally, it is worth pointing out that fork is something of
+ a contested term. Because definitions of forks involve, to one
+ degree or another, statements about the political, organization,
+ and technical distinctions between projects, bifurcations that
+ many people call branches or parallel trees are described by
+ others as forks. Recently, fueled by the advent of distributed
+ version control systems, the definition of what is and is not a
+ fork has become increasingly unclear. In part due to the same
+ systems, the benefits and drawbacks of what is increasingly
+ problematically called forking is equally debatable.</para>
</section>
<para>In my introduction, I described how the growing scope of
free software projects and the rapidly increasingly size and
- diversity of project's user communities is spearheading the need
- for new type of derivation that avoids, as best as possible, the
+ diversity of user communities is spearheading the need for new
+ type of derivation that avoids, as best as possible, the
drawbacks of forking. Nowhere is this more evident than in the
largest projects with the broadest scope: a small group of
projects that includes operating system distributions.</para>
<section>
<title>The Debian Project</title>
- <para>The Debian project is a the largest, in terms of both code
- and volunteers, free software distribution. It is the also,
+ <para>The Debian project is by many counts the largest free
+ software distribution in terms of code. It is the also,
arguably, the largest free software project in terms of the
number of volunteers. Debian includes more than 15,000
packages and the work of well over 1,000 official volunteers
- and many more contributors without official membership status.
+ and many more contributors without official membership.
Projects without Debian's massive volunteer base cannot
replicate what Debian has accomplished; they can rarely hope
- to even maintain what Debian currently has separately.</para>
+ to even maintain what Debian has produced.</para>
<para>At the time that this paper was written, Distrowatch lists
129 distributions based on Debian<footnote>
here: <ulink
url="http://distrowatch.com/dwres.php?resource=independence">http://distrowatch.com/dwres.php?resource=independence</ulink></para>
- </footnote> — most of them currently active to one degree or
- another. Each distribution represents at least one person —
- and in most cases a community of people — who disagreed with
- Debian's vision or direction strongly enough to want to create
- a new distribution <emphasis>and</emphasis> who had the
- technical capacity to follow through with this goal. Despite
- Debian's long-standing slogan — "the universal operating
- system" — the fact that the Debian project has become the
- fastest growing operating system while spawning so many
- derivatives is testament to the fact that, as far as software
- is concerned, one size does <emphasis>not</emphasis> fit
- all.<footnote>
- <para>Netcraft posts yearly updates on the speed at which
- Linux distributions are growing. The one in question can
- be found at <ulink
- url="http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html">http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html</ulink></para>
+ </footnote> — most of them
+ are currently active to varying degrees. Each distribution
+ represents at least one person — and in most cases a
+ community of people — who disagreed with Debian's vision
+ or direction strongly enough to want to create a new
+ distribution <emphasis>and</emphasis> who had the technical
+ capacity to follow through with this goal. Despite Debian's
+ long-standing slogan — "the universal operating system"
+ — the fact
+ that the Debian project has become the fastest growing
+ operating system while spawning so many derivatives is
+ testament to the fact that, as far as software is concerned,
+ one size <emphasis>can not</emphasis> fit all.<footnote>
+ <para>Netcraft posts yearly updates on the speed at which
+ Linux distributions are growing. The one in question can be
+ found at: <ulink
+ url="http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html">http://news.netcraft.com/archives/2004/01/28/debian_fastest_growing_linux_distribution.html</ulink></para>
</footnote>
</para>
customizing and deriving from Debian for specific groups of
users including non-profit organization, the medical
community, lawyers, children and many others.<footnote>
- <para>I have spearheaded and built a derivation of Debian
- called Debian-Nonprofit (Debian-NP) geared for non-profit
- organizations working within the Debian project.</para>
- </footnote> These projects build on the core distribution and
+ <para>I spearheaded and help build a now mostly defunct
+ derivation of Debian called Debian-Nonprofit (Debian-NP)
+ geared for non-profit organizations by working within the
+ Debian project.</para>
+ </footnote> These projects build on the core Debian distribution and
the canonical archive from <emphasis>within</emphasis> the
organizational and political limits of the Debian project and
constantly seek to minimize the delta by focusing on less
invasive changes and by advancing creative ways of building
- the <emphasis>ability</emphasis> to make changes in the core
+ the <emphasis>ability</emphasis> to alter the core
Debian code base through established and policy compliant
procedures.</para>
<title>Ubuntu</title>
<para>The Ubuntu project was started by Mark Shuttleworth in
- April 2004 and the first version was executed almost entirely
- by a small group of a Debian developers by Shuttleworth's
+ April 2004 and the first version was built almost entirely
+ by a small group of a Debian developers employed by Shuttleworth's
company Canonical Limited.<footnote>
<para>Information Ubuntu can be found on the <ulink
url="http://www.ubuntu.com">Ubuntu homepage.</ulink>
Information Canonical Limited can be found at <ulink
url="http://www.canonical.com">Canonical's
homepage</ulink>.</para>
- </footnote> It was released to the world in the fall of 2004.
+ </footnote> It was released to the world in late 2004.
The second version was released six months later in April
2005. The goals of Ubuntu are to provide a distribution based
on a subset of Debian with:</para>
project that aims to derive from Debian to an extensive
degree. Ubuntu made code-level changes to nearly 1300 packages
in Debian at the time that this paper was written and the
- speed of changes will only accelerate with time; the total
+ speed of changes will not decelerate with time; the total
number of changes and the total size of the delta will
grow.<footnote>
<para>Scott James Remnant maintains a list of these patches
most intrusive kind — changes to the code itself.</para>
<para>That said, the Ubuntu project is explicit about the fact
- that it could not exist with the work done by the Debian
- project before Ubuntu was created.<footnote>
- <para>You can see that explicit statement on Ubunut's
+ that it could not exist without the work done by the Debian
+ project.<footnote>
+ <para>You can see that explicit statement on Ubuntu's
website here: <ulink
url="http://www.ubuntulinux.org/ubuntu/relationship/">http://www.ubuntulinux.org/ubuntu/relationship/</ulink></para>
- </footnote> More importantly, Ubutnu explains that it cannot
+ </footnote> More importantly, Ubuntu explains that it cannot
continue to provide the complete set of packages that its
users depend on without the ongoing work by the Debian
project. Even though Ubuntu has made changes to the nearly
<para>Scott James Remnant, a prominent Debian developer and a
hacker on Ubuntu who works for Canonical Ltd., described the
situation this way on his web log to introduce the Ubuntu
- development methodology in the week after first public
- announcement of Canonical and Ubuntu:<footnote>
-
<para>The entire post can be read here: <ulink
- url="http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html">http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html</ulink></para>
+ development methodology in the week after the first public
+ announcement of Canonical and Ubuntu:<footnote> <para>The
+ entire post can be read here: <ulink
+ url="http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html">http://www.netsplit.com/blog/work/canonical/ubuntu_and_debian.html</ulink></para>
</footnote>
</para>
<para>
<inlinemediaobject>
<imageobject>
- <imagedata fileref="picture-01.png" format="PNG"/>
+ <imagedata fileref="tfontf-picture-01.png" format="PNG"/>
</imageobject>
</inlinemediaobject>
</para>
- <para>One thing that should be obvious from this is our job is
- a lot easier if Debian take all of our changes, the model
- actually encourages us to give back to Debian.</para>
+ <para>One thing that should be obvious from this is that our
+ job is a lot easier if Debian takes all of our changes. The
+ model actually encourages us to give back to
+ Debian.</para>
<para>That's why from the very first day we started fixing
bugs we began sending <ulink
url="http://www.no-name-yet.com/patches/">the
- patches</ulink>_ back to Debian through the BTS. Not only
+ patches</ulink> back to Debian through the BTS. Not only
will it make our job so much easier when we come to freeze
for "hoary", our next release, but it's exactly what every
derivative should do in the first place.</para>
<para>There is some debate on the degree to which Ubuntu
developers have succeeded in accomplishing the goals laid out
by Remnant. Ubuntu has filed hundreds of patches in the bug
- tracking system although it has often run into problems in
- deciding <emphasis>what</emphasis> constitutes something that
- should be fed back to Debian. Many changes are simply not
- relevant to upstream Debian developers. For example, they may
- include changes to a package in response to another change
- made in another package in Ubuntu that will not or has not
- been taken by Debian.</para>
+ tracking system but it has also run into problems in deciding
+ <emphasis>what</emphasis> constitutes something that should be
+ fed back to Debian. Many changes are simply not relevant to
+ Debian developers. For example, they may include changes to a
+ package in response to another change made in another package
+ in Ubuntu that will not or has not been taken by Debian. In
+ many other cases, the best action in regards to a particular
+ change, a particular package, and a particular upstream Debian
+ developer is simply unclear.</para>
<para>The Ubuntu project's track record in working
- constructively with Debian is, at the moment, decidedly mixed.
+ constructively with Debian is, at the moment, a mixed one.
While an increasingly large number of Debian developers are
maintaining their packages actively within both projects, many
in both Debian and Ubuntu feel that Ubuntu has work left to do
- in living up to its own goals of a smooth productive
+ in living up to its own goal of a completely smooth productive
relationship with Debian.</para>
+ <para>That said, the importance of the goals described by
+ Remnant in the context of of the Ubuntu development model
+ cannot be overstated. Every line of delta between Debian and
+ Ubuntu has a cost for Ubuntu developers. Technology, social
+ practices, and wise choices may reduce that cost but it cannot
+ eliminate it. The resources that Ubuntu can bring to bear upon
+ the problem of building a distribution are limited — far
+ more limited than Debian's. As a result, there is a limit to
+ how far Ubuntu can diverge; it is always in Ubuntu's advantage
+ to minimize the delta where possible.</para>
+
</section>
<section>
<title>Applicability</title>
- <para>Ubuntu and Debian are distributions and — as such —
- operate on a different scale than the vast majority of free
- software projects. Using a very simple metric, they include
- more code and more people. As a result, there are questions as
- to whether the experiences and lessons learned from these
- projects are particularly applicable to the experience of
- smaller free software projects.</para>
+ <para>Ubuntu and Debian are distributions and — as such
+ — operate on a different scale than the vast majority of
+ free software projects. They include more code and more
+ people. As a result, there are questions as to whether the
+ experiences and lessons learned from these projects are
+ particularly applicable to the experience of smaller free
+ software projects.</para>
<para>Clearly, because of the difficulties associated with
forking massive amount of code and the problems associated
needs of its users lies at the heart of the free software
movement's success. However, while modification usually comes
in the form of collaboration on a single code-base, this is
- function of limitations in software development methodologies
+ a function of limitations in software development methodologies
and tools rather than the best response to the needs or
desires of users or developers.</para>
single free software project to be multiple things to multiple
users simultaneously. This will translate into the fact that,
in the next ten years, technology and social processes will
- evolve so that forking is increasingly less of a bad thing.
+ evolve, so that forking is increasingly less of a bad thing.
Free software development methodology will become less
dependent on a single project and begin to emphasize parallel
- development within an ecosystem of software development
- working on related projects. The result is that free software
- projects will gain a competitive advantage over propriety
- software projecrts through their ability to better serve the
- increasingly diverse needs of increasingly large and
- increasingly diverse user-bases. More projects will derive and
- less redundant code will be written.</para>
+ development within an ecosystem of related projects. The
+ result is that free software projects will gain a competitive
+ advantage over propriety software projects through their
+ ability to better serve the increasingly diverse needs of
+ increasingly large and increasingly diverse user-bases.
+ Although it sounds paradoxical today, more projects will
+ derive and less redundant code will be written.</para>
<para>Projects more limited in code and scope may use the tools
- and methods in different combinations, in different ways, and
- to different degrees than the examples around distributions
- introduced here. Different projects with different needs will
- find that certain solutions work better than others. Because
- communities of the size of Debian are difficult to fork in a
- way that is beneficial to any party, it is in these
- communities that the technology and development methodologies
- will are first emerging. With time, these strategies and tools
- will find themselves employed productively in a wide variety
- of projects with a broad spectrum of sizes, needs, scopes and
+ and methods described in the remainder of this paper in
+ different combinations, in different ways, and to different
+ degrees than the examples around distributions introduced
+ here. Different projects with different needs will find that
+ certain solutions work better than others. Because communities
+ of the size of Debian are difficult to fork in a way that is
+ beneficial to any party, it is in these communities that the
+ technology and development methodologies are first
+ emerging. With time, these strategies and tools will find
+ themselves employed productively in a wide variety of projects
+ with a broad spectrum of sizes, needs, scopes and
descriptions.</para>
</section>
<title>Derivation and Problem Analysis</title>
<para>The easiest step in creating a productive derivative
- software project is to break down the problems of deriviations
+ software project is to break down the problems of derivations
into a series of different classes of modification. Certain
- types of modification is more easily done and are
+ types of modification are more easily done and are
intrinsically more maintainable.</para>
<para>In the context of distributions, the problem of derivation
<orderedlist>
<listitem>
- <para>Selection of individual pieces of software;a</para>
+ <para>Selection of individual pieces of software;</para>
</listitem>
<listitem>
<para>Changes to the way that packages are installed or run
</listitem>
</orderedlist>
- <para>By breaking down the problem in this way. Debian derivers
- have been able to approach deriviation in ways that focus
+ <para>By breaking down the problem in this way, Debian derivers
+ have been able to approach derivation in ways that focus
energy on the less intrusive problems first.</para>
<para>The first area that Ubuntu focused on was selecting a
- subset of package that Ubuntu would support. Ubuntu selected
+ subset of packages that Ubuntu would support. Ubuntu selected
and supports approximate 2,000 packages. These became the
<command>main</command> component in Ubuntu. Other packages in
Debian were included in a separate section of the Ubuntu
able to select a maintainable subsection of the Debian archive
that they could maintain over time.</para>
- <para>The most simple derived distributions — often working
- within the Debian project as CDDs but also including projects
- like Userlinux — are merely lists of packages and do nothing
- outside of package selection. The installation of lists of
- packages and the maintenance of those lists over time can be
- aided through the creation of what are called "metapackages:"
- empty packages that are maintained over time.</para>
+ <para>The most simple derived distributions — often
+ working within the Debian project as CDDs but also including
+ projects like Userlinux — are merely lists of packages
+ and do nothing outside of package selection. The installation
+ of lists of packages and the maintenance of those lists over
+ time can be aided through the creation of what are called
+ <emphasis>metapackages</emphasis>: empty packages with long
+ lists of "dependencies."</para>
- <para>The second item, configuration changes, are also
+ <para>The second item, configuration changes, is also
relatively low-impact. Focusing on moving as many changes as
possible into the realm of configuration changes is a
- relatively low-impact strategy that derivers working within
- the Debian project intent on a single code-base have pursued
- actively. Their idea is that rather than forking a piece of
- code due to disagreement in how the program should work, they
- can leave the code intact but add the
- <emphasis>ability</emphasis> to work in a different way. This
- alternate functionality is made toggleable through a
- configuration change of the distribution in much the same that
- applications can be configured differently or shipped with
- different configuration files. Since the Debian project has a
- unified package configuration framework called Debconf,
- derivers are able to configure an entire system in aa highly
- centralized manner.<footnote>
- <para>More information on Debconf can be found online at:
- <ulink
+ sustainable strategy that derivers working within the Debian
+ project intent on a single code-base have pursued actively.
+ Their idea is that rather than forking a piece of code due to
+ disagreement in how the program should work, they can leave
+ the code intact but add the <emphasis>ability</emphasis> to
+ work in a different way to the software. This alternate
+ functionality is made toggleable through a configuration
+ change in the same manner that applications are configured
+ through questions asked at install time. Since the Debian
+ project has a unified package configuration framework called
+ Debconf, derivers are able to configure an entire system in a
+ highly centralized manner.<footnote> <para>More information on
+ Debconf can be
+ found online at: <ulink
url="http://www.kitenet.net/programs/debconf/">http://www.kitenet.net/programs/debconf/</ulink></para>
</footnote> This is not unlike RedHat's Kickstart although the
emphasis is on maintenance of those configuration changes over
<para>A third type of configuration is limited to changes in the
environment through which a system is run or installed. One is
- example is Progeny's Anaconda-based Debian installer provides
- which an alternate installer but installs an identical system.
- Another example is the Knoppix project is famous for its Live
- CD environments.<footnote>
- <para>In reality, Knoppix makes a wide range of changes to a
- default Debian installation that spam all items in my list
- above.</para>
- </footnote> Other Live CD projects, including Ubuntu's
- Casper project, are purely a different
- way of running the exact same code.</para>
+ example is Progeny's Anaconda-based Debian installer which
+ provides an alternate installer but results in an identical
+ system. Another example is the Knoppix project which is famous
+ for its "Live CD" environments. While, Knoppix makes a wide
+ range of invasive changes that span all items in my list
+ above, other Live CD projects, including Ubuntu's "Casper"
+ project, are much closer to an alternate shell through which
+ the same code is run.</para>
<para>Because these three methods are relatively non-invasive,
they are reasonable strategies for small teams and individuals
working on creating a derived distribution. However, many
desirable changes — and in the case of some derived
- distributions, most desirable changes — require more
- invasive changes. The final and most invasive type of change
- — changes to code — is the most difficult but also the most
- promising and powerful if solved. Changes of this type involve
+ distributions, <emphasis>most</emphasis> desirable changes
+ — require more invasive techniques. The final and most
+ invasive type of change — changes to code — is the
+ most difficult but also the most promising and powerful if it
+ can be done sustainably. Changes of this type involve
bifurcations of the code-base and will be the topic of the
remainder of this paper.</para>
<section>
<title>Distributed Source Control</title>
- <para>One promising method of maintaining changes in forked or
- branched problems lies in distributed version control systems
+ <para>One promising method of maintaining deltas in forked or
+ branched projects lies in distributed version control systems
(VCS). Traditional VCS systems work in a highly centralized
fashion. CVS, the archetypal free software VCS and the basis
for many others, is based around the model of a single
bugs in the stable primary branch, and then ultimately
replacing the stable release with the development release. The
CVS model is <emphasis>not</emphasis> geared toward a system
- where an arbitrary delta, or sets of deltas, is maintained
+ where an arbitrary delta, or sets of deltas, are maintained
over time.</para>
<para>Distributed version control aims to solve a number of
<para>Ultimately, this requires tools that are better at merging
changes and in <emphasis>not</emphasis> merging certain
- changes when that is desirable. It also leads to tools capable
+ changes when that is the desired behavior. It also leads to tools capable
of history-sensitive merging.</para>
<para>The most famous switch to a distributed VCS model from a
control tools were less advanced than they are today. At the
moment, an incomplete list of free software VCS tools includes
GNU Arch, Bazaar, Bazaar-NG, Darcs, Monotone, SVK (based on
- Subversion), GIT (a system developed by Linus Torvlards as a
- temporary replacement for BitKeeper) and others.</para>
+ Subversion), GIT (a system developed by Linus Torvalds as a
+ replacement for BitKeeper) and others.</para>
<para>Each of these tools, at least after they reach a certain
- level of maturity, allow or will allow its users to develop
+ level of maturity, allow or will allow users to develop
software in a distributed fashion and to, over time, compare
their software and pull changes from others significantly more
easily than they could otherwise. The idea of parallel
- development lies at the heart of the model, the tools for
+ development lies at the heart of the model. The tools for
merging and resolving conflicts over time, and the ability to
"cherry pick" certain patches or changes from a parallel
developer each make this type of development significantly
<para>In many ways, employing distributed version control
effectively is a much easier problem to solve for small, more
traditional, free software development projects than it is for
- GNU/Linux distributions. Because the problems with maintaining
- parallel development of a single piece of software in a set of
- related distributed repositories is primary use case for
- distributed version control system, distributed VCS alone can
- be a technical solution for certain types of parallel
- development. As the tools and social processes for distributed
- VCS evolve, they will become increasingly important tools in
- the way that free software is developed.</para>
-
- <para>Because the problems of scale associated with buildling an
+ GNU/Linux distributions. Because the problems associated with
+ maintaining parallel development of a single piece of software
+ in a set of related distributed repositories is the primary
+ use case for distributed version control systems, distributed
+ VCS alone can be a technical solution for certain types of
+ parallel development. As the tools and social processes for
+ distributed VCS evolve, they will become increasingly
+ important tools in the way that free software is
+ developed.</para>
+
+ <para>Because the problems of scale associated with building an
entire derivative distribution are more complicated than those
- associated with working with a single project, distributed
- version control has not yet been widely deployed in the Ubuntu
- project. Instead, the project is focusing on integrating these
- into problem specific tools built on top of distributed
- version control.</para>
+ associated with working with a single "upstream" project,
+ distributed version control is only now being actively
+ deployed in the Ubuntu project. In doing so, the project is
+ focusing on integrating these into problem specific tools
+ built on top of distributed version control.</para>
</section>
maintaining difference between packages. Because packages are
usually distributed as a source file with a collection of one
or more patches, this introduces the unique possibility of
- creating a limited high-level VCS system based on this
- fact.</para>
-
- <para>In the case of Ubuntu and Debian, he tool is creating one
- branch per patch or feature and using heuristics to analyze
- patch files and create these branches intelligently. The
- package build system section of the total patch can also be
- kept as a separate branch. Canonical's tool, called the
- Hypothetical Changeset Tool (HCT) (although no longer
- hypothetical), is one experimental way of creating a very
- simple, very streamlined interface for dealing with a
+ creating a high-level VCS system based around this fact.</para>
+
+ <para>In the case of Ubuntu and Debian, the ideal tool creates
+ one branch per patch or feature and uses heuristics to
+ analyze patch files and create these branches
+ intelligently. The package build system section of the total
+ patch can also be kept as a separate branch. Canonical's tool,
+ called the Hypothetical Changeset Tool (HCT) (although no
+ longer hypothetical), is one experimental way of creating a
+ very simple, very streamlined interface for dealing with a
particular type of source that is created and distributed in a
particular type of way with a particular type of
change.</para>
<para>While HCT promises to be very useful for people making
- derived distributions based on Debian, its wider application
- may be limited. That said, this provides an example of the way
- that problem and context specific tools may play an essential
- role in the maintenance of derived code more generally.</para>
+ derived distributions based on Debian, its application outside
+ distribution makers will, in all likelihood, be limited. That
+ said, it provides an example of the way that problem and
+ context specific tools may play an essential role in the
+ maintenance of derived code more generally.</para>
</section>
<section>
<title>Social Solutions</title>
- <para>It has been said that a common folly among technophiles is
- based on the temptation to employ technical solutions toward
+ <para>It has been said that it is a common folly of a
+ technophile to attempt to employ technical solutions toward
solving social problems. The problem of deriving software is
- both a technical <emphasis>and</emphasis> a social problem and
- adequately addressing the issue will require approaches that
- take into consideration both type of solution.</para>
+ both a technical <emphasis>and</emphasis> social problem and
+ adequately addressing the larger problems requires approaches that
+ take into consideration both types of solution.</para>
<para>Scott James Remnant compares the relationship between
- distributions and derived distributions as not unlike the
+ distributions and derived distributions as similar to the
relationship between distributions and upstream
maintainers:</para>
<blockquote>
<para>Ubuntu has pursued a number of different initiatives along
these lines. The first of these has been to regularly file
- bugs in the Debian bug tracking system when bugs are fixed
- that exist in Debian are fixed in Ubuntu. While this can be
- partially automated, the choice to automate this is a purely
- social one.</para>
-
- <para>Ubuntu is still left with questions in regards to changes
- that are made to packages that do not necessarily fix bugs or
- that fix bugs that do not exist in Debian but may in the
- future. Some Debian developers want to hear about the full
- extent of changes made to their software in Ubuntu while
- others do not want to be bothered. Ubuntu should continue to
- work with Debian to find ways to allow developers to stay in
- sync.</para>
-
- <para>There is a recent initiative by some developers in Debian,
- largely led by myself, to create a stronger relationship
- between the Debian project and its ecosystem of derivers.
- While the form that this will ultimately take is unclear,
- projects existing within an ecosystem should explore the realm
- of appropriate social relationships that will ensure that they
- can work together and be informed of each others' work without
- resorting to "spamming" each other with irrelevant or
- unnecessary information.</para>
+ bugs in the Debian bug tracking system when bugs that exist in
+ Debian are fixed in Ubuntu. While this can be partially
+ automated, the choice to automate this and the manner in which
+ it it is set up is a purely social one.</para>
+
+ <para>However, as I alluded to above, Ubuntu is still left with
+ questions in regards to changes that are made to packages that
+ do not necessarily fix bugs or that fix bugs that do not exist
+ in Debian but may in the future. Some Debian developers want
+ to hear about the full extent of changes made to their
+ software in Ubuntu while others do not want to be
+ bothered. Ubuntu should continue to work with Debian to find
+ ways to allow developers to stay in sync.</para>
+
+ <para>There are also several initiatives by developers in
+ Debian, Ubuntu, and in other derivations to create a
+ stronger relationship between the Debian project and its
+ ecosystem of derivers and between Ubuntu and Debian in
+ particular. While the form that this will ultimately take is
+ unclear, projects existing within an ecosystem should explore
+ the realm of appropriate social relationships that will ensure
+ that they can work together and be informed of each others'
+ work without resorting to "spamming" each other with
+ irrelevant or unnecessary information.</para>
<para>Another issue that has recently played an important role
in the Debian/Ubuntu relationship is the importance of both
giving adequate credit to the authors or upstream maintainers
of software without implying a closer relationship than is the
case. Derivers must walk a file line where they credit others'
- work on a project without implying that the others works for,
- support, or are connected to the derivers project which, for
- any number of reasons, the original author might not want to
- be associated with.</para>
+ work on a project without implying that the others work for,
+ support, or are connected to the derivers project to which, for
+ any number of reasons, the "upstream" author might not want to
+ be associated.</para>
<para>In the case of Debian and Ubuntu, this has resulted in an
emphasis on keeping or importing changelog entries when
using distributed VCS technology. As Linus Torvalds alluded to
in the quote above, the importance of technological changes to
distributed VCS technology is only felt when people begin to
- work in a different way — when they begin to employ differnet
- social models of developer interaction.</para>
+ work in a different way — when they begin to employ
+ different social models of developer interaction.</para>
<para>While Ubuntu's experience can provide a good model for
tackling some of these source control issues, it can only
serve as a model and not as a fixed answer. Social solutions
must be appropriate for a given social relationship. Even in
situations where a package is branched because of social
- incompatibility, a certain level of collaboration on a social
+ disagreements, a certain level of collaboration on a social
level will be essential to the long term viability of the
- derivation.</para>
+ derivative.</para>
</section>
<para>As the techniques described in this paper evolve, the role
that they play in free software development becomes increasingly
prominent and increasingly important. Joining them will be other
- techniques and models that I have not seen and cannot predict.
- Because of the size and usefulness of their code and the size of
- their development communities, large projects like Debian and
- Ubuntu have been forced into confronting and attempting to
- mediate the problems inherent in forking and deriving. However,
- as these problems are negotiated and tools and processes are
- advanced toward solutions, free software projects of all sizes
- will be able to offer users exactly what they want with minimal
- redundancy and little duplication of work. In doing this, free
- software will harness a power that proprietary models cannot
- compete with. They will increase their capacity to produce
- better products and better processes. Ulimately, it will help
- free software capture more users, bring in more developers, and
- produce more free software of a higher quality.</para>
+ techniques and models that I have not described and cannot
+ predict. Because of the size and usefulness of their code and
+ the size of their development communities, large projects like
+ Debian and Ubuntu have been forced into confronting and
+ attempting to mediate the problems inherent in forking and
+ deriving. However, as these problems are negotiated and tools
+ and processes are advanced toward solutions, free software
+ projects of all sizes will be able to offer users exactly what
+ they want with minimal redundancy and little duplication of
+ work. In doing this, free software will harness a power that
+ proprietary models cannot compete with. They will increase their
+ capacity to produce better products and better processes.
+ Ultimately, it will help free software capture more users, bring
+ in more developers, and produce more free software of a higher
+ quality.</para>
</section>
projects / to_fork_or_not / blobdiff