Subject: Re: improving project maintainership
From: Tom Lord <lord@regexps.com>
Date: Sat, 9 Feb 2002 05:32:41 -0800 (PST)


[Various CC's removed as the thread is wandering off into
hopelessly-abstract-land.]


       Stephen:

       You're looking for an infrastructure to promote more collaborative
       development, especially cooperation on things like testing core
       components, without inhibiting the ability of individual projects or
       developers to vary in pursuit of their individual goals.  Close?

Neither right nor wrong.  Your summary is too far removed from
engineering issues to assess.

There are two views on what business the big FSBs and semi-FSBs are in
and really I think the truth is that they are in transition from A to
B:

	Business A:

	* Collect a bunch of publicly licensed pieces that are hard
	  for your customers to assemble.  You may be shooting
	  yourself in the foot because if all you ever do is collect a
	  fixed number of pieces, eventually it will become easy for
	  your customers to assemble those same pieces themselves.
	  Still, in the short term, you are definitely adding value.
	  You could look for ways to direct a lot of your revenue
	  towards expanding the number of pieces -- but right now your
	  money is tied up in the other aspects of "Business A".

	* Do some testing to decide which pieces to upgrade to and
	  which not.  You probably can't spend enough to do a really
	  thorough job at testing but you can somewhat make up for
	  that in a non-collaborative way by hiring some of the core
	  developers and aligning their priorities.  Those core
	  developers can focus and prune your testing efforts. This
	  may be shooting yourself in the foot, since those core
	  developers won't have as much time to enable collaboration
	  as well as they could have, but so long as you don't force
	  the issue, you can count on those developers to strike the
	  very best balance they can under the circumstances (until
	  they burn out or get rich and bored or become entrenched and
	  evil).  You could help eliminate much of the testing expense
	  by helping to make it more natural for the public projects
	  to maintain the testing infrastructure themselves -- but
	  right now your money is tied up.

	* Do a decent amount of release engineering.  If your
          distribution is popular, you can even get volunteers to do
          release engineering for you.  Again, you may be shooting
          yourself in the foot because, together with your
          competitors, you're just wasting in-house and volunteer
          effort as you partition employees and volunteers into those
          doing release engineering for your distribution vs. those
          doing release engineering for other distributions.
	  You could eliminate most of the expenses of release
          engineering by investing in tools that make it easier for
          the public projects to do release engineering themselves --
          but right now your money is tied up.

	* Build an installer and an upgrade-over-the-net tool.  As
          long as you're fighting with competitors over release
          engineering technology, you'll certainly want to have unique
          solutions in this area -- especially solutions that can
          subsume your competitors work.  Installers seem to be the
          dominant topic addressed by reviewers and, presumably, by
          equally shallow-looking purchasers.  Upgrade-over-the-net
          tools satisfy an IT dept's quest for lower cost of ownership
          and your quest for "ties".  Sure, a public project can come
          along, produce standard solutions in these areas and blow
          your effort out of the water, but until they do, isn't it
          rational to keep your solution as uniquely yours as
          possible, even if that means forsaking the benefits of
          collaboration on its development and extension?  Surely,
          none of the big hw vendors would have incentive to drive a
          standard solution in these areas: one can hardly imagine HP,
          IBM, and Sun finding a way to collaborate on an issue like
          this.  Besides, isn't your branded solution in these areas a
          way to leverage your specific release engineering
          technology?  And anyway, just now all your money is tied up.

	* Develop non-free documentation.  Look, its a revenue stream
          right?  And its necessary given your (effectively)
          proprietary install/upgrade technologies.  Sure, you could
          hire or free-lance writers to produce public documentation,
          taking care to develop standards for structure, formatting, and
          installation.  Doing that would allow more people to help
          develop the software.  But for now it gives you a
          competitive advantage.

	* Develop new markets.  Ok, you have very few (but not 0)
          opportunities to develop new components.  Coming up with and
          supporting a new configuration of existing components is
          hella expensive.  So, you've got to be shrewd and lucky to
          identify just those markets that are close enough to reach
          but large enough to be worth reaching.  


	Business B:

	* It's easy to put together the publicly available pieces.
	  What's hard is picking the pieces to put together to 
	  achieve particular goals.  It's hard because there are many
          solutions and evaluating them is a complex task.  That's
	  part of your job.

	* A lot of testing happens "for free".  Every significant
          package has an extensive test suite.  The GNU distros or the
          public maintainer distros (and candidate distros) are farmed
          out over a P2P network that automatically does a lot of
          testing on many different platforms.  For weird
          configurations that only your customers want, you run those
          same testing tools in house, very inexpensively.  Your
          larger customers run those same test in-house for confidence
          building and risk assessment.  You have a little server farm
          that contributes to the P2P testing network.

	* Release engineering happens "for free".  Every significant
          package conforms to the GNU coding standards version 121.
	  Those standards have enough features that the path from
          maintainer FTP sites to your CD-burner or OEMs is automated.

	* Installers and upgrade-over-the-net tools are solved.  Those
	  employees are working on something else now.

	* Develop new markets.  You have a lot of engineers
          participating in working groups to extend the architecture
          of the GNU/* platform for various market needs.  You have a
          lot engineers leading or contributing heavily to the
          implementation efforts.  Some of those same employees
          participate in customer working groups to develop the goals
          and communicate where the architecture is heading.  With
          some indirection the same pattern occurs with customers who
          are individual consumers buying toys, communication tools,
          and home management tools.  Your customers pay you to 
	  participate in those working groups.

So yeah, in the context of "Business B", public project maintainers
are dealing with distributions with varying goals and I'm trying to
help build tools for that -- but let's get down to specifics about
what that means.  To try and warp this into support for your critique
of the FSF's behavior toward non-Free software is, um, a bit of a
stretch.

-t