Subject: RE: standards & FS
From: Stephen Walli <stephen_walli@interix.com>
Date: Thu, 27 May 1999 10:04:11 -0700

Morning All,

I feel obliged to step into the fray to add some definitions (or request
them) wrt standards.  I've spent the past decade tied into the POSIX space
(IEEE PASC and ISO) and the XPG4/Single UNIX Spec space with X/Open (before
the merge with the OSF that created the Open Group.)  I spent a lot of time
during this period reporting on standards to USENIX and Europen (EUUG) and
representing them within the process.  I spent the decade before that
rewriting working software from platform to platform and see the customer
value in standards.  I'm also very interested in Open/Free source (for
reasons that will become apparent another day) and have begun thinking about
how standards and open source may interact.

De jure standards are created under well defined processes by formal
organizations.  There are rules in place to ensure that anti-competative
practices and agreements do not impact the process. CBEMA's X3 organization
(which gave us the C language standard) and IEEE's PASC (which gives us the
POSIX family of stds) are acreditted standards development organizations
(SDOs) that contribute to developing ANSI approved standards.  These
standards can then be forwarded into the international space through
appropriate avenues to ISO, another de jure SDO.  

There is often work done to try and co-ordinate efforts such that work is
done in one place to the benefit of all.  Two examples are the work in the C
language community where the ISO and X3 meetings were held together such
that a single C-language standard was produced, and within POSIX where the
IEEE is the development organization, and the ISO working group is more an
adminstrative and approval co-ordinating group.

These de jure processes are all "open" standards processes.  They are all
public processes with well defined rules to ensure a public decision making
paper trail. (Meetings must be announced in advance, meeting minutes taken,
and attendance lists available.)    Votes are cast in different ways: the
IEEE is by individual professional, X3 was by company, and ISO by country.
But each has multiple review check points where comments are heard and
recorded and can be acted upon by the approval authorities.  These approval
authorities are "just people" like you and I, individual professionals in
the industry representing ourselves, companies, or governments, but each
body is governed by rules and protocols precisely defined to ensure an open
public process free of anticompetative practice.

[I will not debate here and now the speed of development or its
appropriateness within SDOs, or the free availability of the published
specifications.  Those discussions are relevent here yet.]

I will be a little pedantic and state IMHO that the IETF process for
developing RFCs is NOT an SDO in the de jure rigorous sense of the previous
paragraphs, but will quickly agree that the IETF is an "open" specifications
body, with open public participation, and does brilliant work.

In all of the open de jure processes discussed, the market rules (the reason
we develop standards is driven by the market and the desire to expand
markets and interoperate within markets).  This also means that corporations
involve themselves to attempt to make their proprietary solution the
"standard", or to influence the standards such that they are not put at
competitive disadvantage.  The Java Wars are a brilliant illustration, but
so too are the C standards, the early POSIX standards, and the failure of
the OSI networking standards.  And I could talk at length about the tactics
and strategies used by large corporations to play these games, but it's not
actually relevant to this discussion.  Microsoft's statements in the
Halloween document about "extend and embrace" demonstrated an amazing lack
of understanding about how de jure standards processes actually work, and
how easy (or not) influence can actually be had, although one does need to
be very careful when dealing with such deep-pocketed bulls in the standards
china shop.

Large consumer organizations also involve themselves, with the U.S.
government being a notable example.  It also provides an excellent example
of some of the "market" aspects of standards.   The US National Institute of
Standards and Technology (NIST, formerly NBS) has been a huge participant in
the entire POSIX/UNIX standards efforts since the early 1984 /usr/group days
through IEEE and ISO and well into the Open Group.  In the public de jure
process of the IEEE, they flexed their muscles exactly once (in 1988) to
move forward on a stalemate issue, through their own procurement policies in
NIST.  While it would be GREAT to assume that the government as a "big
customer voice" could simply "declare" things to those pesky proprietary
vendors and make the "right" technical solution happen, NIST is part of the
Department of Commerce, whose mandate is to ensure there are lots of healthy
competing companies employing lots of tax-paying citizens.  "With great
power comes great responsibility" :-)   This was also back before the head
of NIST became a presidential appointment, therefore a grossly political
position, rather than a civil responsibility.  All bets are off now.

People also talk about de facto standards, but I tend to dislike the term as
it can apply to almost anything in large deployment.  TCP/IP is a de facto
standard, despite it's open creation process.  DOS disk formats are a de
facto standard, being replaced by ISO CD formats as a de facto standard.  I
would argue that "de facto" is more a reference to deployment and acceptance
than creation process, and applying the term to a standard is confusing if
one is discussing the opposite of a de jure standard.

Then there's the closed specifications.  Adobe PostScript.  Well published,
widely deployed and implemented, but essentially completely controlled and
licensed by a single vendor.  It might be unwise of them to gratuitously
change the specification, but nothing prevents it.

Industry consortia are a mixed bag.  Most of my experience centres around
X/Open (now the Open Group).  The core (80%) of it's largest specification
project, the Single UNIX Spec (tied to the UNIX95 brand) is POSIX.1-1990,
POSIX.2-1992, and ISO C (1990).  It's evolved version, the Single UNIX Spec,
Version 2 (tied to UNIX98) adds mainly POSIX threads and POSIX real-time
interfaces.  X/Open was created out of a very European co-operative vendor
base as a non-profit organization.  It has always been very de jure
standards focused as it's original sphere of influence was European
corporations and governments interested in those de jure standards.  That
said, it's process is relatively closed.  "Large" organizations (companies,
universities, government departments) participate to different levels
through different scaled fee structures, and often the public is not privy
to the outcome until it's published.  Meetings occur, open to members,  with
minutes, and votes recorded, but these documents are never public. 

So there's Stephe Walli's Standards Primer.

I was hugely uncomfortable with people's discussion of "free software
standards" and "proprietary standards".  While I would agree that PostScript
is a closed "proprietary standard", I'm not sure the author (craig?)
actually meant that.  POSIX and ISO/ANSI C are in no way "proprietary"
standards despite huge politics involved by large hardware OEMs pushing for
their OS extensions.

What's this have to do with Free/Open source?  I'm not sure, and I think
it's a fruitful area for discussion.  

Some observations based on this discussion to date would suggest that if
there's exactly one true implementation of a language (Perl in this
discussion) then no published standard is actually necessary.  There is
exactly one implementation (available under the artistic license) of the
language "specification" (which is itself well documented) for users.  It is
unlikely anyone would ever implement the language from scratch again.  Were
there to be competing development streams of a particular domain of
functionality, then a specification or standard may become warranted, but
based on Perl's community of interest (implementor and users), it is
unlikely.

Frank Hecker's statements about Mozilla were also appreciated.  There are
multiple implementations of browers and a huge community of users developing
content to be displayed.  Standards are essential here, and the HTML wars
and browser share battles are painful to watch.  But it's unclear that the
fact that Mozilla is published under the MPL is actually relevant to the
discussion.

It may be that for a particular project, if the free/open source base
defines the space and it is well specified (e.g. Perl) AND there exists a
strong central maintenance authority, no formal standard is required.  The
community of users' needs is already satisfied for portability of
scripts/programs, and a Perl script writer is not tied to a particular
vendors' Perl implementation.  

If there is already a standard in the domain of discussion, and the
free/open source base is to meet the standard (e.g. the Linux kernel and
POSIX.1-1990 API), and here I am assuming the libre software implementation
community or development authority sees the value in conforming to the
standard (e.g. Mozilla and HTML or XML), then work to achieve conformance
needs to be done and proof of conformance demonstrated.  This requires
sufficient knowledge, desire, and resources to accomplish the goal.  

One of the early debates about the Linux kernel and UNIX95 brand was
thoroughly screwed up because while there existed a knowledgeable resource
with a paying customer, the customer didn't understand their own requirement
for standards well enough in relationship to the libre software model.  The
customer assumed that instead of investing in the work, and reaping the
benefit of a "UNIX specification conforming Linux kernel", they would wait
for the "free software community" to make it conforming (of their own
volition of course) and the customer would "get the implementation for
free."  Whew -- did *that* customer ever confuse free speech and free beer.

Anyway, those are my initial observations on free software and standards.

stephe

Stephen R. Walli               http://www.interix.com          +1 519 571
0427
OPENNT is now INTERIX
The name in Open Systems solutions for Windows NT