Subject: Re: rocket science
From: DV Henkel-Wallace <gumby@henkel-wallace.org>
Date: Sat, 26 Feb 2005 23:34:13 -0800

On 26 Feb 2005, at 18:48, Brian Behlendorf wrote:

> On Fri, 25 Feb 2005, Robin 'Roblimo' Miller wrote:
>> I've read elsewhere that the onboard computer group's software is not 
>> only the world's most error-free, but also the world's most 
>> expensive. This article didn't touch on cost.
>
> It does, at the end:
>
>   And money is not the critical constraint: the groups $35 million per
>   year budget is a trivial slice of the NASA pie, but on a
>   dollars-per-line basis, it makes the group among the nation's most
>   expensive software organizations.
>
> Um, yeah.  Dollars-per-line(-per-year!) is about as relevant as 
> lines-of-code-per-developer-hour in measuring anything, but relative 
> to other projects (commercial or FLOSS) this is insanely high.  And 
> having only one customer makes life pretty easy, too.

This points out an interesting issue, although one I admit is less than 
half-baked (because I'm thinking about it in work-avoidance-mode rather 
than just deleting the message and doing what I should).


Commercial software, generally, balances out the optimal spending on 
feature development, QA, etc (let's ignore transients at the extrema -- 
startups and monopoly providers -- and stick to a reasonable population 
of large commercial developers).  For example:

Avionics, telephony, trains, real parts of automotive, etc: expensive, 
super reliable.
Investment banks, CC providers, etc: risk-adusted they are pretty 
reliable but also more flexible.
Oracle, SAP, IBM, etc: looking at their 10-Ks I would say they've 
picked it pretty well.

Even ChoicePoint and those guys make the tradeoff "properly" -- they 
bear essentially zero risks in the case of poor security or processing 
errors, so are happy not investing in QA or security.

The question is: I would not be surprised (not on any evidence but off 
the top of my head) that Free Software Businesses can handle these 
risks poorly, since they don't get valid economic signals about this.

The usual model of commercial investment in free software is based on 
incrementalism (companies invest at the margins on the features that 
matter to them) which doesn't work in this case since the security or 
reliability decision has to be made on the entire system and not just 
on the new features being developed.

A FS support organization can manage this in a few rare cases.  Cygnus 
did some work for telephone switch providers.  IIRC these deals were 
more profitable than our other deals (the customer was willing to pay 
extra) but not especially so.  The advantages for us were not only the 
extra cost and the very low number of reported problems (since they 
started from a reliable base) but also we knew the software 
particularly well and had built a reliable system back when the code 
base was significantly smaller.  It's not clear a new entrant could do 
the same, or even that it could be done again now.

{GNU/,}Linux was lucky in that it had some serious interest in 
reliability back when the code base was small and that focus has 
generally remained as it has grown.  I can't say that's the same, for 
example for Mozilla (not to pick on them, just an example).  And Apache 
went the other way from super-flaky to usably reliable and has gotten 
better ever since, but I don't think it's yet as reliable as, say, 
Oracle and I can't imagine anybody knows if it will get there (or even 
if it should).

Thoughts?
-d