To Fork or Not To Fork

To -Fork or Not To Fork

Lessons From Ubuntu and Debian

Benjamin -Mako Hill

Canonical -Limited

The Debian GNU/Linux -Project

Software in the -Public Interest, Inc.

This material is licensed under the Creative Commons Attribution-Sharealike 2.0 License.

The canonical location for the most recent version of this -document is at the author's website.

- - - - - - - - - - - - - - - - - - -

Revision -History
Revision 0.2	August 7, 2005
Correction and improvements.
Revision 0.1	May 15, 2005
- The first version of this paper was written to an accepted talk -given at Linuxtag 2005 given in Karlsruhe, Germany. -

Table of Contents

Introduction

"Fork" Is A Four -Letter Word

Case -Study

The Debian -Project
Ubuntu
Applicability

Balancing Forking -With Collaboration

Derivation and -Problem Analysis
Distributed Source -Control
Problem Specific -Tools
Social -Solutions

Conclusions

Introduction

+{% extends 'latex-like-layout.tmpl' %} +{% set page_title = 'To Fork or Not To Fork: Lessons From Ubuntu and Debian' %} + +{% block content_box %} + +

May 15 2005; Revised August 7, 2005

+ +

+
Translated into Serbo-Croatian (Srpskohrvatski) by Anja Skrba.
+ +
The first version of this paper was written to an accepted talk given at Linuxtag 2005 given in Karlsruhe, Germany.
+

+ +

Introduction

The explosive growth of free and open source software over the last decade has been mirrored by an equally explosive growth in the ambitiousness of free software projects in choosing and tackling @@ -137,8 +26,8 @@ programmers — both volunteer and paid — and are producing millions of lines of code. Their software is being used by millions of users with diverse sets of needs. This paper focuses on two major effects of this situation:

The communities that free software projects — and in particular large projects — serve are increasingly diverse. @@ -153,7 +42,7 @@ groups of hackers, it is often not even possible for small groups to even track and maintain a fork of a large project over time.

Taken together, these facts imply an increasingly realized free software community in which programmers frequently derive but where traditional forking is often untenable. "Forks," as they are @@ -171,8 +60,8 @@ tools and experiences of derived distributions.

products that have forked but that maintain a close relationship as they develop parallelly and symbiotically. To do this, developers should:

Break down the process of derivation into a set of different types of customization and derivation and prioritize methods of @@ -192,7 +81,7 @@ projects.
emphasis on maintaining differences over time.

This paper is an early analysis of this set of problems. As such, it is highly focused on the experience of the Ubuntu project and its existence as a derived Debian distribution. It also pulls @@ -200,16 +89,9 @@ from my experience with Debian-NP and the Custom Debian Distribution (CDD) community. Since I participate in both the Ubuntu and CDD projects, these are areas that I can discuss with some degree of knowledge and experience.

"Fork" Is A Four Letter Word

+ +

"Fork" Is A Four Letter Word

The act of taking the code for a free software project and bifurcating it to create a new project is called "forking." There have been a number of famous forks in free software history. One of @@ -251,13 +133,13 @@ published the first version of the more than four years ago, I included a small subsection on forking which described the concept to future free software project leaders with this text:

The short version of the fork section is, don't do them. Forks force developers to choose one project to work with, cause nasty political divisions, and redundancy of work.

In the best situations, a fork means that two groups of people need to go on developing features and doing work they would ordinarily do -

Case Study

+ +

Case Study

In my introduction, I described how the growing scope of free software projects and the rapidly increasingly size and diversity of user communities is spearheading the need for new type of @@ -304,15 +179,9 @@ derivation that avoids, as best as possible, the drawbacks of forking. Nowhere is this more evident than in the largest projects with the broadest scope: a small group of projects that includes operating system distributions.

The -Debian Project

+ +

The Debian Project

The Debian project is by many counts the largest free software distribution in terms of code. It is the also, arguably, the largest free software project in terms of the number of volunteers. @@ -362,16 +231,9 @@ With its strong technological base, excellent package management, wide selection of packages to choose from, and strong commitment to software freedom which ensures derivability, Debian provides an ideal point from which to create a GNU/Linux distribution.

Ubuntu

+ +

Ubuntu

The Ubuntu project was started by Mark Shuttleworth in April 2004 and the first version was built almost entirely by a small group of a Debian developers employed by Shuttleworth's company @@ -380,8 +242,8 @@ Canonical Limited. -

Regular and predictable releases — every six months with support for eighteen months.
@@ -397,7 +259,7 @@ and a default selection and configuration of packages that is usable for most desktop users "out of the box."

The Ubuntu project provides an interesting example of a project that aims to derive from Debian to an extensive degree. Ubuntu made code-level changes to nearly 1300 packages in Debian at the time @@ -425,7 +287,7 @@ methodology in the week after the first public announcement of Canonical and Ubuntu:^[7]

I don't think Ubuntu is a "fork" of Debian, at least not in the traditional sense. A fork suggests that at some point we go our @@ -448,7 +310,7 @@ much easier when we come to freeze for "hoary", our next release, but it's exactly what every derivative should do in the first place.

There is some debate on the degree to which Ubuntu developers have succeeded in accomplishing the goals laid out by Remnant. Ubuntu has filed hundreds of patches in the bug tracking system but @@ -478,15 +340,9 @@ Debian's. As a result, there is a limit to how far Ubuntu can diverge; it is always in Ubuntu's advantage to minimize the delta where possible.

Applicability

+ +

Applicability

Ubuntu and Debian are distributions and — as such — operate on a different scale than the vast majority of free software projects. They include more code and more people. As a @@ -533,26 +389,11 @@ methodologies are first emerging. With time, these strategies and tools will find themselves employed productively in a wide variety of projects with a broad spectrum of sizes, needs, scopes and descriptions.

Balancing Forking With Collaboration

Derivation and Problem Analysis

+ +

Balancing Forking With Collaboration

+ +

Derivation and Problem Analysis

The easiest step in creating a productive derivative software project is to break down the problems of derivations into a series of different classes of modification. Certain types of modification @@ -562,7 +403,7 @@ be broken down into the following types of changes (sorted roughly according to the intrusiveness inherent in solving the problem and the severity of the long-term maintainability problems that they introduce):

Selection of individual pieces of software;
@@ -579,7 +420,7 @@ a Live CD type environment or using a different installer);
of changes to the packages code);

By breaking down the problem in this way, Debian derivers have been able to approach derivation in ways that focus energy on the less intrusive problems first.

@@ -640,16 +481,9 @@ change — changes to code — is the most difficult but also the most promising and powerful if it can be done sustainably. Changes of this type involve bifurcations of the code-base and will be the topic of the remainder of this paper.

Distributed Source Control

+ +

Distributed Source Control

One promising method of maintaining deltas in forked or branched projects lies in distributed version control systems (VCS). Traditional VCS systems work in a highly centralized fashion. CVS, @@ -670,7 +504,7 @@ with the development release. The CVS model is

Distributed version control aims to solve a number of problems introduced by CVS and alluded to above by:

Allowing people to work disconnected from each other and to sync @@ -681,7 +515,7 @@ fashion.

Allowing deltas to be maintained over time.

Ultimately, this requires tools that are better at merging changes and in not merging certain changes when that is the desired behavior. It also leads to @@ -701,7 +535,7 @@ a patch-by-patch basis any more.^[9]

At the time of the switch, free distributed version control tools were less advanced than they are today. At the moment, an incomplete list of free software VCS tools includes GNU Arch, @@ -740,16 +574,9 @@ control is only now being actively deployed in the Ubuntu project. In doing so, the project is focusing on integrating these into problem specific tools built on top of distributed version control.

Problem Specific Tools

+ +

Problem Specific Tools

Another technique that Canonical Ltd. is experimenting with is the creation of high level tools built on top of distributed version control tools specifically designed for maintaining @@ -772,16 +599,9 @@ makers will, in all likelihood, be limited. That said, it provides an example of the way that problem and context specific tools may play an essential role in the maintenance of derived code more generally.

Social -Solutions

+ +

Social Solutions

It has been said that it is a common folly of a technophile to attempt to employ technical solutions toward solving social problems. The problem of deriving software is both a technical @@ -791,7 +611,7 @@ take into consideration both types of solution.

Scott James Remnant compares the relationship between distributions and derived distributions as similar to the relationship between distributions and upstream maintainers:

I don't think this is much different from how Debian maintainers interact with their upstreams. As Debian maintainers we take and @@ -805,7 +625,7 @@ they take them, but we don't get angry if they don't.
we're no more a fork of Debian than a Debian package is a fork of its upstream.

Scott alludes the fact that, at least in the world of distributions, parallel development is already one way to view the modus operandi of existing @@ -871,17 +691,9 @@ a given social relationship. Even in situations where a package is branched because of social disagreements, a certain level of collaboration on a social level will be essential to the long term viability of the derivative.