Subject: Re: Pitching a "Big Tent" for Free Software
From: Rich Morin <rdm@cfcl.com>
Date: Tue, 14 Dec 1999 10:26:19 -0800

It is clear that I18N (and L12N and ... :-) is a big issue that we are
only beginning to address.  I will do a quick hand-wave, however, and
say that properly designed infrastructure will help us with them.

BT> Have you looked at http://sourceforge.net/?

No, I hadn't, and I'm very happy you pointed it out to me.  I knew VA
had started to offer hosting for Open Source projects, but I had no
idea that their efforts were this well developed.  I clearly need to
take a closer look at their efforts!

BT> A properly maintained project is designed to isolate the bits that
BT> need porting from the bits that don't. ...

True, but as you say, that's the ideal.  In practice, we see:

   *  limited platform standardization - even within OS sub-families
      (e.g., Linux and *BSD*), there are some differences.  As you
      move across the range of Unixen (let alone over to Windoze :-),
      standardization is (ahem) less than perfect.

   *  diverging and even separate development lines - A writes a
      package for the machines he uses.  B ports the package to
      another, but A doesn't want to take back the changes.  Or has
      moved on, or ...

   *  inappropriate abstracting of differences - ifdef's can and
      should be written in terms of key characteristics.  It is far
      easier, however, to write them in terms of particular
      environments.  Unfortunately, this becomes a problem when
      another environment shows up that only partially matches
      the preceding ones (A/UX was LOTS of fun on this issue :-).

      Ideally, I would claim, packages should automagically adapt
      themselves to their installation environments, including
      I18N, L12N, machine architecture, operating system, and a
      range of company/site/network/node/user preferences.  (We
      can't - and shouldn't - standardize all of these, so why not
      try for flexibility?)

Given all of these problems, we DO have a porting matrix to deal
with.  By building up infrastructure to deal with it, however, we
may be able to help our developers "fold down" the cases, getting
each package to that wonderful state where a single set of sources
"just runs" in every environment.

My first hack at such an infrastructure, FWIW, would look quite a
bit like sourceforge.  I would explicitly try to encourage the
developers to use CVS as a way to track the variant development
streams that may now exist, eventually merging them together.

By the way, it is interesting to contemplate the size of the Open
Source "literature" (code, docs, etc.).  My guess is that it is
less than 100 GB, which currently fits on $1K worth of disk space
(37 GB EIDE IBM DeskStars were $340 each, the last time I looked.)

Thus, storage is not the problem.  Nor, curiously, is getting all
the developers to "join in".  The FreeBSD Ports Collection doesn't
require developers to do anything special; it just uses their FTP
archives "as is".  A global Open Source CVS repository could (and
probably must) take a similar approach.

ILT> There is currently no way to deal with all packaging systems
ILT> as one. ... there needs to be one central set of sources for
ILT> any given package ... I would prefer to think in terms of
ILT> eliminating the matrix in software development, rather than
ILT> somehow exploiting it.

Well, the matrix exists, provided by our external environment.  We
can work, however, to "fold down" divergent responses to it into
unified, flexible code bases.  But, as long as we have no way to
deal with the matrix as a fact of life, our approaches will be
ad hoc and (I would maintain) less effective than they should be.

ILT> Cygnus has a range of hosts and a range of targets.

True.  I18N and L12N could also be said to provide dimensions to
the problem.  But my point is not that we should structure our
response as a matrix, merely that we should recognize that the
problem set can be viewed that way.  The same kinds of design
that help with OS/arch differences can help with I18N/L12N and
host/target challenges (he said, waving his hands wildly :-).

ILT> My impression is that porting experts have by and large
ILT> pooled their efforts.  Most people use autoconf/automake/
ILT> libtool.

To be sure, these tools are becoming increasingly popular and
they do solve a large class of issues.  Nonetheless, I can't
take a Ports Collection directory and do anything useful with
it on a Debian system.  About the best that I can hope for is
that I can hand-translate some of the common metadata (e.g.,
FTP sites) into the other system.  (Call this "packaging" if
you like, but the issues are thoroughly intertwingled).

ILT> My recollection is that RPM technology spread pretty
ILT> quickly once it was available.

Perhaps a bit TOO quickly (:-).  But yes, there is reason to
believe that better solutions could propagate very quickly.

ILT> So there is certainly work to be done here, but I'm not
ILT> sure how it should start.

Neither am I, but providing common infrastructure could be a
good start.  If we had, for instance, an XML definition for
the "interesting" metadata for each package, different sets
of packaging and indexing software could all use it.

There's a real danger, of course, in trying to get everyone
to "march to the same drummer".  If we pick CVS as THE WAY
to build source repositories, BitKeeper might never get a
real trial.  Fortunately, I think there is enough anarchy
around to keep this from being a real problem (:-).

-r
--
Rich Morin:          rdm@cfcl.com, +1 650-873-7841, http://www.ptf.com/~rdm
Prime Time Freeware: info@ptf.com, +1 408-433-9662, http://www.ptf.com
MacPerl: http://www.macperl.com,       http://www.ptf.com/ptf/products/MPPE
MkLinux: http://www.mklinux.apple.com, http://www.ptf.com/ptf/products/MKLP