Subject: Re: Meeting the MS challenge
From: ian@cygnus.com
Date: Sun, 14 Dec 1997 13:50:26 -0500 (EST)

In fsb adam@yggdrasil.com (Adam J. Richter) writes:

>	The Yggdrasil Plug-and-Play Linux releases use GNU Taylor UUCP.
>It is true that the configuration files that we ship are in traditional
>("honeydanber"?) format, which is not discussed in the Linux Bible
>or the uucp texinfo documentation (other than an explicity statement
>in the texinfo documentation to say that these formats are not
>discusssed), but it is certainly described in many general unix books.
>I do regard this as a bug, but it is not true to say that Taylor UUCP
>is not included in the Yggdrasil Plug and Play Linux distribution.

The discrepancy between the documentation and the behaviour of the
program is a serious problem for many users, and it is partially my
fault.  The situation is, perhaps, a cautionary tale of what can
happen when the developer and the distributor of a program are
different, a case which is obviously more likely to arise for free
software than for proprietary software.

I wrote the UUCP code in 1991 and 1992, at a time before the Linux
distributions existed (or, at least, at a time before I had heard of
them).  I assumed that everybody who installed the package would have
to at least read the README file, and would have the documentation
right there, and in particular would have to make considered choices
on matters such as which type of configuration file to support (which
is controlled by hand editing the policy.h file in the sources).

Because I assumed that level of knowledge, and because an explicit
goal of the package was compatibility at the user visible level with
existing Unix UUCP packages, I made the configuration choices very
flexible.  In particular, the installer can select which types of UUCP
configuration files to recognize, and where to find them.  This
flexibility was necessary because existing Unix systems used different
types of configuration files (V2 and HDB, with many variants) and
stored them in different places.

I increased the potential confusion by designing a new configuration
file format.  I did this for two reasons.  The first was that one of
the regular complaints concerning the older formats was that they were
incomprehensible.  The second was that the older formats were, in
general, not easily extensible, and I wanted to add more flexibility
in several areas without making the configuration files even more
cryptic.

I only documented the new configuration file format, since I reasoned
that anybody who built the code would have my documentation available,
and would only choose to support a different configuration file format
if they already had documentation for it.  Similarly, I only provided
sample configuration files for the new format.  These sample files
were part of the distribution but were not installed anywhere--I
thought it was sufficient to mention them in the documentation.

So, flexibility is usually good, and everything was fine until the
Linux binary distributions started to spread.  The people who first
distributed UUCP binaries and documentation for Linux--I no longer
remember who was involved--decided to distribute UUCP compiled to
support HDB configuration files, rather than the new format.  They did
this because most available Unix UUCP documentation, particularly
including the then-available edition of the O'Reilly UUCP book,
described HDB configuration files, and not the new format.  As I
recall I objected mildly, but mostly I didn't worry about--the Linux
users I was hearing from at the time, late 1992, were mostly prepared
to hack the kernel to fix their serial drivers, so I figured they
could handle rebuilding UUCP if they wanted to.

In retrospect several years later, that was the critical moment.  If I
had insisted that they distribute UUCP with the new configuration
files in standard locations, perhaps they would have done so.  I
didn't, and this set a pattern of distributing a Linux UUCP which was
compatible with documentation written for non-Linux systems.

Unfortunately, there are many non-Linux systems, and many slight
variants of UUCP, and my code could be configured to support most of
them.  Different binary distributions made different choices.

I had never made my documentation customizable in the way that the
sources were (INN is an example of how to do this well).  I didn't see
why I should bother; after all, again, the installer has the
documentation and policy.h right there, and it was pretty obvious how
changes to policy.h or the Makefile would affect the documentation.

UUCP is not, of course, the heart of a Linux system, so distributors
did not make any special effort for it.  They made some choices in
Makefile.in and policy.h, built the binaries, and distributed the
binaries and the documentation.  There was no good place to record the
choices they made when building the binaries, or, if there was, there
was no simple way to make the documentation link back to these
choices.  Distributors sometimes installed a set of sample UUCP
configuration files, and sometimes they didn't.  Distributors who
didn't use UUCP themselves didn't read the documentation, and didn't
know about the sample configuration files included in my distribution,
and those were only for the new configuration file format anyhow.

The effect was that people got binaries which were often configured
for compatibility with some other Unix system, and they got
documentation which described my chosen defaults, and they sometimes
got sample files for some configuration file format, but they didn't
know where the samples files were or whether they matched the
binaries.

As Linux snowballed in popularity, many people started running it with
no other Unix experience, and with no access to other Unix
documentation.  As the Internet spread, UUCP became less important, so
there was decreasing interest in cleaning up these problems.  I joined
Cygnus in summer 1992, and did little work on UUCP after early
1993--after all, the package really is pretty much feature
complete--so I didn't have much motivation to straighten out the
situation either.

The problems persist today.  Linux distributions still differ in how
UUCP is configured.  They still distribute my documentation, which
does not address how to handle a UUCP configured for anything other
than my defaults.  There are now other books describing the new UUCP
configuration file format, including the new edition of the O'Reilly
book, but these books can't tell you what you actually have.  There is
no way, given a set of UUCP binaries, to figure out how they were
built--another error on my part.

A final twist to the problem is that since the Internet has spread
widely, UUCP is increasingly being used primarily by people who do not
have access to the Internet.  These are generally people whose first
language is not English, and who have very limited access to
documentation other than what comes with their Linux distributions.
Because there are so many variantions in the types of problems they
can encounter, my stock answer to the regular mail messages I get is
to explain how to get the unmodified distribution, and suggest that
they build it themselves to match the documentation.  That appears to
be enough to solve most problems, but it's a terrible introduction to
getting mail on your Linux system.

Well, that was a much longer message than I intended.  One lesson I've
learned is that a free software author who wants an effective package
must ensure that the documentation always matches the distribution.
There are various possible approaches, including omitting flexibility
(difficult with autoconf options like --prefix), configuring the
documentation when building the code (the INN approach, but this
raises problems with web based documentation), and tightly controlling
distributions (difficult with free software).

UUCP, of course, has problems which other software packages do not
face, chiefly backward compatibility with existing code which itself
had many variations.  However, the general problem can at least
potentially arise for most popular free software packages.

I've always taken documentation seriously, and I always intended for
my UUCP package to be as easy as possible to use, given the
unfortunately complex niche it occupies (besides the problems
discussed here, installing it requires working with varying mail
packages, modems, and, at the other end of the UUCP link, system
administrators).  It's disappointing that UUCP is a poster child for
configuration and installation complexities, just as it was before I
started writing my package.

Ian