Subject: Re: Open letter to those who believe in a right to free software
From: Ben_Tilly@trepp.com
Date: Fri, 29 Oct 1999 13:40:14 -0400


> >>>>> "Ben" == Ben Tilly <Ben_Tilly@trepp.com> writes:
>
>     >> But I will keep the point in mind.  I suspect you are right
>     >> that if moderately strong it could affect the conclusion.
>
>     Ben> Compare learning Linux on your own.
>
>     Ben> Compare learning Linux with a friend you can call when the
>     Ben> going gets tough.
>
> Strongly negative, in my case.  My learning curve for Linux was like
> falling off a cliff:  more annoying than hives to quadruple my
> productivity under the previous OS in less than two weeks.  Calling
> friends generally results in talking about their problems far more
> than about mine.  Says more about my friends than me, I suppose  :-)
>
Are you a typical user then?

I doubt it!

>     Ben> Just how strong an effect are we talking?
>
> Depends on the parameterization of the model, I'm in no position to
> answer quantitative questions like that.
>
For most people the effect of having friends to get them over
the hump is both positive and very strong.  Network effects are
clearly significant in determining learning curves.

>     >> This is a good point.  However, it is not clear to me that it
>     >> is relevant to economics.  I think that for economic purposes
>     >> Perl 5 is _not_ a bigger Perl 4.  It is a new product.  I think
>     >> that this model, although unnatural for the developers
>     >> themselves, is accurate and would support the result I proposed
>     >> (a mixed regime is better).
>
>     Ben> Being a Perl programmer, I disagree.
>
> "Oh, ith _that_ what'th wrong with you," thaid the Lithper.  :-)
>
# Thpethially for Thtephen :-)
s/s/th/g;
s/S/Th/g;

> Speaking to the content, I already said it's not natural for you to
> think that way.  I don't see where you think the "new product" model
> is inaccurate, though.
>
I think that it is begging the question.  There are a number of
steps that can be taken with a project to allow its
development to scale better.  These steps are not
worthwhile for small projects, they are for big ones.  A
natural event when small projects grow is to adopt new
practices that are more appropriate to the new scale of
the project.  The process of adapting these changes will
reduce developer costs as and when they are made.

For your economic model it would be convenient to be
able to assume a convex cost model.  However the
above phenomena results in costs *not* being convex,
instead they ratchet up and down, granted it is more up,
but it is still far from convex.

However if you just redefine things so that each of these
natural stages is by definition a different product, you
can create an illusion that you are talking about
products, each of which is convex.  However the market
reality is much closer to there being one product, which
you have segmented into convex stages for
convenience, but which is not in fact convex!

>     >> Anything under GPL [will not produce a lock-in effect].
>
>     Ben> No, you are still locked into the basic product and
>     Ben> architecture.
>
> I don't see why this is interesting; it's surely irrelevent to free
> vs. proprietary.  If the customer is so worried about it, design the
> protocol/API so that it is efficiently extensible (a la X11's wire
> protocol or XML).

It is interesting because the wise consumer is making a
choice to be locked into a world view, and so the
consequences of that lock-in matter in their decision.
Questions about how well this architecture will evolve,
what future costs will look like, where you can get support
for it, etc will matter for the current decision.

Therefore, all else being equal, a consumer should pick
standards that are open, and extendable, and,
non-proprietary.  They should likewise, if applicable
prefer implementations that are free over proprietary.
This is, of course, "all else being equal".

After a certain complexity in your (combination of)
standards it is inevitable that there will be a divergence
between different implementations.  At this point, despite
your best efforts, you will likely wind up locked into a
particular implementation.  And yes, it makes a very
large difference whether or not that implementation is
proprietary.

>     Ben> No, it does not imply monopoly when the product comes out.
>     Ben> But the threatened future control does constain decisions.
>     Ben> In software purchasing people pay a lot of attention to
>     Ben> questions about whose products have the most "momentum".
>     Ben> People know that something will win, and there is a lot of
>     Ben> pressure to be on the right "bandwagon".
>
> I can't help it that Purchasing collects the least-capable employees
> after Personnel.  If all they can do is get on "bandwagons," they're
> going to be under pressure today and they'll be under pressure
> tomorrow.  If you want to get out from under, you step back, look at
> your organizational and systems architecture, and design them for
> flexible upgrades.  If that means commissioning a custom design for
> some component with full docs including source, so be it; you won't
> regret it.

You are implicitly assuming that trying to get on the
right bandwagon is an inherently bad thing to do.  But
given the presence of both strong network effects and
strong lock-in, is this really such an indefensible thing?
With the real costs of being second to market in a fast
moving area, is it really prudent to always value doing
things the right way?  I would *like* that to be true, but I
don't think that it always is...

> Think of it as evolution in action.  Economics is interested in
> decisions made by actors who are aware of and attempt to influence
> their environment.  By next week, the others aren't going to be of any
> interest except to economic paleontologists.

Given my observations from living in "the real world",
this statement is hopelessly naive.  Sure it is a great
party line, and it undoubtably simplifies your life a lot
to make that assumption, but it is clearly very wrong.

> >From the other side of the market, there is a lot of money to be made
> by people who give copies of S&V's chapters on lock-in to potential
> customers before making their sales pitch for flexible open source
> designs.
>
Advice.  Don't start paragraphs with the word "From".
Email systems get bothered when you do. :-)

Agreed on the profit potential there, I suspect that
some on the list (hi Rus, Tim:) have taken that to
heart...

>     Ben> Re-training is the least of lock-in worries.  I work for a
>     Ben> company which has a significant investment in a rather
>     Ben> complex VB front-end to their data.  (The company's basis
>     Ben> depends on its models of a specific type of financial
>     Ben> security.)
>
>     Ben> At the time they did this development they had to choose a
>     Ben> platform.  At this point to switch they have to build a
>     Ben> platform with similar functionality - while maintaining the
>     Ben> existing one because without it the company would quickly
>     Ben> fail.  This is rather greater lock-in than just, "Retrain a
>     Ben> couple of people."
>
> OK, so they're locked in.  Maybe they should have spent twice as much
> for a more modular, better-specified design which could be
> reimplemented and retargeted to a better platform.  Maybe not---then
> they're locked in.  But if so, we economists do not consider ourselves
> responsible for the preventable self-inflicted injuries of decision-
> makers.

Amusing.  Knowing the people who made the decision
I can honestly say that had they chosen to go to lengths
to avoid lock-in over the long term, the company would
currently only be of interest to economic paleontologists. :-)

Damned if you do, damned if you don't.  Those *are*
the appropriate terms here, and if that is the choice to
be made, then it is a failure of economics if it does not
try to account for choices between the rock and a hard
place.  Dismissing the inevitable wounds as being
merely "self-inflicted" is therefore a shortcoming in your
analysis.

And yes, every last word of this is reason to choose the
free solution over a proprietary one if you actually have
a choice.  Furthermore the choice is being presented
on these terms (thanks to ESR) and people are
responding to the implications.

>     Ben> Likewise the Word Perfect example that I gave with lawyers is
>     Ben> not a re-training issue.
>
> Lawyers, huh.  Lawyers are the beneficiaries of a government-
> franchised cartel and have plenty of room to afford inefficiency.  For
> that reason, lawyers are not businessmen.  It's going to take them a
> lot longer for them to get to a low-cost equilibrium than firms in
> more competitive lines of business.

Do you not like lawyers very much..?

Remind me to pass your comments on to my
brother-in-law at some point.  I am sure that he has a
rather different view of his role as a lawyer in the
aircraft insurance industry.  (He is currently working on
maintaining current exposure liability information.  In
particular exactly how much exposure does each
insurance company wind up with to any particular
crash under various scenarios...)

>     Ben> Sure, they could rebuild the system from scratch in Microsoft
>     Ben> Word, but the investment required is substantial, plus nobody
>     Ben> wants to go through the inevitable mistakes involved in doing
>     Ben> such a thing.
>
> Eventually some firm will do so.  They (or if they make big mistakes,
> the next one), will make out big.

Some firms have done so.  The difference is not
sufficient to justify the costs if you already have a
system up and working.

>     Ben> This is all theoretically true, however it merely supplies a
>     Ben> general theoretical explanation for my point.  In the
>     Ben> software market the supplier frequently does establish a
>     Ben> monopoly, and lock-in contributes to the ability of the
>     Ben> supplier to extract profits from this situation.
>
> True.  In general, however, the customers also extract profits from
> the situation.  Has anybody ever computed the percentage of cost
> reductions due to Microsoft products compared to not using the
> application at all (not compared to using the best alternative)
> captured by Microsoft?  Despite Microsoft's revenues, I'll bet it's in
> the single digits, and I'll offer good odds for less than 50%.
>
Let me ask a more basic question.  Has anybody
convincingly computed a productivity increase from
using computers in the office?  In case you wonder, the
answer is "no".

> The point is that even Microsoft is apparently quite constrained by
> something.

A shortage of minutes?  (After all they only get one per
minute! :-)

[...]
>     Ben> Proprietary formats are just the vendor helping the process
>     Ben> along.  There are still large factors creating lock-in.  Try
>     Ben> to switch a body of automated processes from Perl to Python
>     Ben> some time.  Even though the Perl processes may be using
>     Ben> non-proprietary files, and the code is available, the switch
>     Ben> is not very easy...
>
> So don't.  Why would you want to do such an insane thing?

Because someone thinks that Python with its cleaner syntax
and more complete OO support is better suited to large
projects?

I don't know.  But I think that I just showed that lock-in is a
real constraint even without proprietary formats.

> Lock-in is a problem when somebody comes along and wants to sell you a
> product to do the same thing at a price that would be paid for 5 times
> over in cost savings in 2 years _if you had no preexisting files_, but
> you can't afford to do it because the cost of translating file formats
> is prohibitive.

Or when your current system is tied to an unsupported
machine that will break when Y2K rolls around, and you
need to move the system now.  Or when you outgrow
the bounds of the original and you want to move to
bigger hardware.  Or when...

[...]
>     >> Yes.  It's worth noting that the lock-in here is generated by a
>     >> strong network externality in a "local" community, opposed by a
>     >> weaker global network externality.
>
>     Ben> Uh, no.  It is generated by the fact that legal firms have
>     Ben> implemented important infrastructures using features of Word
>     Ben> Perfect that would have to be rebuilt from scratch in
>     Ben> Microsoft Word.
>
> I fail to see where we differ.  You seem to think that only people can
> participate in network externalities?  Not true.  What I had in mind
> is that a law office is not composed of lawyers (irrelevant, they have
> no fingers), secretaries, and a monolithic WordPerfect application.
> Instead, that "infrastructure" you talk about is actually dozens or
> hundreds of document templates, macros, and whatever, that interact
> with the secretaries through WordPerfect.  All of that is made more
> valuable because they can interact through a single system.

Lawyers don't type?  That is a new one to me!

Perhaps I am unclear on what you mean by a "network
externality".  If having half of the templates in WordPerfect
and half in Microsoft Word creates constant internal
problems in a law office, in what way does this depend
on factors external to that office?

> If those templates and macros were all stored in "Active XML", though,
> the infrastructure problem goes away, doesn't it?

No.  You are now tied to ActiveXML.  If that falls out of
favor in the market, you again have hard choices to make.

So no matter how great an idea ActiveXML may be, if
it isn't popular, it could wind up being a really stupid
decision.  This is not an uncommon problem in the
software industry.

>     >> Unless the cost of time-to-market really dominates the cost of
>     >> production, this is convex costs.
>
>     Ben> But does it act like convex costs?
>
> As far as I can tell, this question is meaningless.  Does the
> subsystem act like convex costs?  Yes it does, because it possesses a
> convex cost function.

It is not meaningless.  If the total cost of production rises
with the addition of more participants, but this drops the
cost per participant, what happens?  If all participants are
part of the same economic unit (ie company) then they
will not be added.  But if they are independent of each
other then as long as the cost of participating is below
the benefit of joining participants will continue to join the
project.

Therefore free projects have the potential to realize
speed-ups from highly parallel approaches even when
those approaches are inefficient and would not be
chosen by a singule rational participant.

Is this an advantage or disadvantage?  I can argue
both sides.

>     >> No, this doesn't behave like convex costs.  This is precisely
>     >> why the
>
>     Ben> Apparently not... :-)
>
> Are you trying to score debating points, or are you trying to
> understand how modularizing this program will make it more
> undertandable and maintainable?

I am trying to point out a non-obvious difference
between proprietary and non-proprietary methods of
develop.  I think that it is an important point as well.  For
instance far more effort is being currently put into the
Linux UI than I think any one company could ever
justify.  Sure, it is inefficient.  It is also scarily effective,
and I think that it is sustainable as well.

The flip side is that unless development picks up
in Asia as well, much of this will make your life
harder.  Of course such events as official government
support in Korea may come into play here.

You tell me whether this is a relevant point...

>     Ben> I think that in a well-run company the costs of productions
>     Ben> need not be convex.  A sample example.  A real company that I
>     Ben> know took the C libraries and modified them so that a
>     Ben> core-dump, in the process of dumping core, would capture
>     Ben> various information including the command-line, location of
>     Ben> the core, and a stack-backtrace, and mail that to a list of
>     Ben> people.  What do you think this did to their development
>     Ben> costs for maintaining their ongoing crons?  (OK, *after*
>     Ben> development stopped for a month and a half. :-)
>
> It changed from the original convex cost function to a new convex cost
> function whose values are lower at every quantity.  What did you think
> I was going to say?  _It's not the same cost function, you just went
> and changed it._

I was hoping you would also notice that in the natural
course of a project growing it will find and implement such
improvements.  Therefore any valid projection of what the
development costs look like as it scales will have to take
into account that the development model will naturally
readjust, invalidating the naive projection of how it will
scale.

This is the basis of my assertion that software
development costs do not generally fit a convex model.
At any given time it is likely to be locally convex, but as
it scales there are downward drops.

>     >> This can, of course, be emulated by proprietary firms, although
>     >> their customers may tend to be more cranky about sitting
>     >> through the tests.
>
>     Ben> I think that having to fill in a bug report when they
>     Ben> *expected* to do an install might not fly too well either...
>
> A few messages ago (maybe it was private) you thought that was quite
> alright, at least when free software firms do it (the escaping betas
> thread).  I don't care whether it's "yes", "no", or "I don't know",
> but I don't see why the answer would be different for proprietary
> vs. free.

It was in a private exchange.  And yes, I thought that it
was quite acceptable when that is what the customer
expects.  (eg A certified beta.)  It is not acceptable if
the product is supposed to be a stable end-user
product.  To the extent that proprietary software tends
to sell itself as the latter, it has less freedom to demand
bug reports.

[...]
>     Ben> The introduction and acceptance of this technique will reduce
>     Ben> development costs.  Sure, once the improvement is
>     Ben> implemented, the costs again begin climbing (more shallowly
>     Ben> than before), And there is substantial cost in first creating
>     Ben> these suites.  But while you are creating this
>     Ben> infrastructure, the future per person development costs are
>     Ben> dropping.  This is true both per developer and per user.
>
> I saw you palm that card.  How _did_ that time argument get in there?
> I didn't put one in.
>
Do you think that development takes place instantaneously?

> Ben, an economic cost function is a function from the quantity of
> goods ("size") produced to the cost of production.  It can change over
> time, but it is a different cost function then.

Stephen, good developers vary the infrastructure used to
produce software significantly depending on the size of
project being tackled.  It is inappropriate to expect a
large project to cost what it would cost if developed using
methods appropriate to a personal project since the
developers are unlikely to so limit themselves.

Any cost function that does not recognize this is unrealistic.
It no would no more make sense to estimate the cost of
excavating a pit based on asking me to remove 5 pounds
dirt and then estimating how long it would take to excavate
it with a shovel!

>     Ben> I like that argument.  But how do you respond to the
>     Ben> assertion that many features which consumers want require
>     Ben> such tight integration, and as a result free software
>     Ben> projects will never be able to tackle projects above a
>     Ben> certain complexity?  (Say, any project on the order of
>     Ben> NT. :-)
>
> I was hoping _you_ would do that for me.

Sorry.  The assertion is debatable either way and I do not
currently see any way, other than faith, to come to final
conclusions yet.

> The basic argument is that if a customer can identify a feature, then
> it can probably be compartmentalized into a module.  If it can be
> compartmentalized into a module, you can hand it to a separate team to
> implement.

I want drag-n-drop to work the same in all my applications.

While we are at it, I want to be able to plug in my choice of
font, 1-byte or multi-byte, going right-left, left-right, alternating
(I believe Sanscrit does this) and going up or down.

Again I want that to work the same in all my applications.

And I want it all to be reconfigurable on the fly.

For all my applications at once of course.

How, praytell, do you compartmentalize requests like that?

Are these unrealistic requests?  (Yeah, I know... :-)

[...]
>     Ben> While this difference hands proprietary software a lot of
>     Ben> rope, it also puts a barrier on what sorts of techniques are
>     Ben> available to it.
>
> I just don't see that barrier.  I see biases on both sides, but no
> absolute barriers.

ARGH!!

My bad, I didn't mean, "it", I meant "free software".  My
claim is that free software is under much more pressure
to follow recognized best practices, but sometimes it
would be really nice to just forget about those...

[...]
>     Ben> Cool for helping people understand today's technologies some
>     Ben> 5 years too late.  What about the technologies that people
>     Ben> will be arguing about in 5 years?
>
> You really think it's going to change that much?  Maybe.  People will
> have to learn to abstract faster, is all.  People who can do that,
> even imperfectly, will make big bucks as consultants (or, with
> somewhat different personalities, chief executives).

How much has it changed over the last 5 years?  The 5 years
before that?

[...]
>     Ben> And they have not reached this awareness before?  How did the
>     Ben> OSI, POSIX, etc come into existence then?
>
> Do you mean for me to believe that OSI, POSIX, etc, were driven by
> _consumer demand_?  Next thing, you'll be selling shares in the
> Brooklyn Bridge by MLM....

No.  I mean for you to believe that recognition of these
factors lay behind their ability to market themselves both
to potential members and to consumers.  If these factors
were not real then they wouldn't have had a chance.

>     Ben> Plus people tend to forget the old lessons with each new
>     Ben> generation of types of devices...
>
> "Predictable, self-inflicted injuries."  They should buy insurance
> before taking a bath, is all I can say.

A company is interested in buying PDAs for their salesmen.
They have identified a number of advantages to doing so.
What do you recommend that they do?  Not buy?  Buy a
proprietary device?  Set out to design their own free device
whose design they will give away.  Which wound do you
recommend and in what words will you ridicule them for
their stupidity in 5 years?

In case you didn't realize, I think you are being unfair...

>     Ben> Thank you for confirming that most of corporate America is
>     Ben> run by idiots.  With that established, as these idiots
>     Ben> realize that the difference makes a difference to them in a
>     Ben> particular area and starts to act on that decision, the
>     Ben> result is economically important.
>
> You wanna parse that for me?

Do you read Dilbert?  It is a mild parody of reality.  What
you think of PHB is irrelevant.  They are exist and have a
real impact on markets.  Unless you think that Microsoft's
bottom line is economically uninteresting, I suggest that
you do not underestimate their importance.

[...]
>     Ben> Not if Microsoft can get the DOJ's budget slashed a few
>     Ben> times. :-/
>
> The DOJ will not go away, I assure you, and in 2020, when you talk to
> the college recruits about the bad old days with Microsoft they'll
> look at you blankly, wondering who's afraid of #100 on the Fortune
> 500, what with Cygnus Solutions at #3 and in the dock for
> planned obsolescence of glibjava....

You probably just insulted someone on the list, but I am not
quite sure who...

Ben