Subject: Re: quality, sharing, etc.
From: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp>
Date: Tue, 29 Aug 2000 13:21:34 +0900 (JST)

>>>>> "Brian" == Brian Behlendorf <brian@collab.net> writes:

    Brian> On Mon, 28 Aug 2000, Stephen J. Turnbull wrote:

    Brian> More broadly, if all the BSD's could be accomodated, then
    Brian> that might be extendable to other OS's too.

    >> How is this different from SourceForge in the end?

    Brian> *Much* different.  SF is a large repository for any open
    Brian> source project that wishes to use it, and is "where" the
    Brian> development is supposed to take place on that project.

    Brian> By contrast, the ports collection is not where the
    Brian> development of the code takes place, but where the
    Brian> information about how to integrate that package into the
    Brian> FreeBSD system (along some filesystem standards) is
    Brian> maintained.  It's useful to have one person maintain a
    Brian> large number of ports for a given platform, since

Conceded, but I don't see why they point to a per-distro ports
collection, as opposed to a little extra infrastructure (eg, a CVS
$PROJECT-port-$DISTRO module for each project) on SourceForge to help
support ports maintainers.  Even the cases you mention below could be
handled by the distributions cooperating create a project that is
nothing but a framework to hang ports on.

    Brian> consistancy is important; therefore the amount of time
    Brian> spent on each codebase needs to be as small as possible.

Would that the downstream "ports" maintainers thought that way!  The
reason that I think closer integration would be a good idea is that
often they do not limit themselves to enforcing consistency, but also
make significant functional changes to the code base---generating bug
reports to the upstream project.

    >> I see your *trusted* concept, but if the *BSD ports maintainers
    >> instead joined the upstream teams, wouldn't you then end up
    >> with a *trusted* upstream project?

    Brian> That presumes:

    Brian> a) those ports maintainers are able to join the "upstream
    Brian> team", which sometimes is just one person who's not
    Brian> interested in having others join (see qmail)

Conceded.  I don't suggest this colocation method has universal
applicability.  I'm proposing that it be recommended where feasible
and potential for upstream/downstream conflict is high.

    Brian> b) that port maintainer's concept of where things should go
    Brian> are accepted by the upstream project

I don't see why this is a problem; most projects already support
flexible installation via autoconf scripts, the $DESTDIR make
variable, and so on.  Debian demonstrates that it is practical to
completely separate this from the upstream code base for the vast
majority of packages.

    >> The separate ports maintainership is consistently a headache
    >> for XEmacs with respect to FreeBSD, as the FreeBSD patches
    >> often do _not_
    Brian> ...
    >> This is also consistently a headache with respect to Debian
    >> GNU/Linux, for the same reasons.

    Brian> Isn't open source wonderful?

It is.

    Brian> I'm not being sarcastic; I just don't see it as a bug that
    Brian> the ports modify the upstream code to be specific to the
    Brian> standards the distribution has.

I don't see that as a bug either.

    Brian> By the same token, though, *any* bug report on that
    Brian> modified codebase should be handled first by the
    Brian> distribution, then passed along to the upstream project
    Brian> once it's clear it's not related to the patches in the
    Brian> distro.  That would solve your headaches, I would think.

It would.  It also doesn't happen often enough, and IMHO will not
happen often enough without organization structure to encourage it.
That's human nature, and that's the bug.

This is a tension that is inherent in open source, as you imply.  But
just because I heartily approve of the _freedom_ that open source
engenders doesn't keep me from saying "bad practice" when I see
_licentiousness_.

It's not just a matter of who's ox is gored, either.  Eg, this is
particularly a problem in the XEmacs community because most externally
developed Lisp is developed on and for the mainline version of GNU
Emacs.  Many of those maintainers are averse to making changes in
their own code to conform to XEmacs API usages; some are pretty snotty
about it.  It's a natural response to think "it's free software" and
start hacking an XEmacs-specific version.  This is bad practice and
both annoys the upstream Lisp project and hurts XEmacs itself.

In the context of XEmacs, I'm pushing internally for clearer
documentation of these necessary changes, and to minimize them.  But
it would be better if closer cooperation could be achieved, and I
think colocation of the "port" code with the main repository could
help.  It wouldn't always be comfortable, I'll grant, especially where
there is a history of bad blood.

    >> I think this division of loyalty problem is natural for large
    >> subsystems, each with its own (sub-)community of developers,
    >> like X, Emacs, Perl, and Apache.

    Brian> True.  And if you care about bringing those back into the
    Brian> fold, you'll look at the changes they make and ask yourself
    Brian> if there isn't a more neutral third path that everyone can
    Brian> agree to.  I know we do that with the patches in the
    Brian> FreeBSD apache port from time to time.

Institutionalizing a "more neutral third path" is exactly what I'm
fumbling about.  The current organization of "sovereign projects" does
not encourage communication and compromise in precisely the cases
where they are most difficult and important, such as the relationships
between Emacsen and various other projects that must deal with the
Emacs Lisp API fork.

In these cases, it would be useful to have an institutional mechanism
so that well-behaved distros (and other downstream projects!) don't
get messed up by bilateral negotiations between a "difficult" distro
and the upstream project.  I'm thinking colocation of "port" code with
the main project might help.

    >> It's easy to say "add a standard that no patch to a distro's
    >> version of a package is admissible unless it's been vetted
    >> upstream," but this is just not feasible.

    Brian> Nor is that open source.

Sure it is.  I'm not proposing a clause in any license, rather self-
discipline by the distros/ports collections.

All of the Linux distros already try to maintain "pristine source."  I
suppose the BSD ports collections do, too.  Debian goes much further,
and strongly discourages its maintainers from adding patches unless
there is reason to believe they'll be integrated upstream.

I'm just suggesting wider adoption of a similar policy by all distros.
And an institution that would discourage aggressive ports maintainers
from "doing their own thing" in the name of maintaining a port.

Which even Debian maintainers do despite their own policy.  Most of
the bugs that have really annoyed me in individual Debian packages in
the last year or so have been Debian maintainer hubris, either fixing
things that weren't broke, or adding new breakage in the process of a
fix due to incomplete understanding of the upstream code.

The reason it's infeasible is not a conflict in principle with open
source; it's that port maintainers will not be willing to give up the
right to back-seat drive, explicitly granted by open source licenses.

After all, aren't all of them better-than-average programmers?  :-)

-- 
University of Tsukuba                Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
Institute of Policy and Planning Sciences       Tel/fax: +81 (298) 53-5091
_________________  _________________  _________________  _________________
What are those straight lines for?  "XEmacs rules."