Subject: Re: near/medium future digital media economics
From: "Ben Tilly" <btilly@gmail.com>
Date: Sat, 20 May 2006 01:50:44 -0700

 Sat, 20 May 2006 01:50:44 -0700
On 5/19/06, Stephen J. Turnbull <stephen@xemacs.org> wrote:
> >>>>> "Ben" == Ben Tilly <btilly@gmail.com> writes:
>
>     Ben> Email has existed for over 3 decades and allows people to
>     Ben> communicate with strangers.  I submit that most of us mostly
>     Ben> talk with people we know.
>
> I'm not talking about "talking", I'm talking about connection.  You
> know, the kind of connection that associates you with the company that
> administers so-3-0-0.mpr2.iad5.us.above.net, or with the Tsuchiura
> office of Nomura Shoken.  Didn't know about those, did you?  So you
> obviously didn't "talk" to them personally, and those connections cost
> you no time, though you almost certainly connected to the former, and
> quite likely are fuzzily connected to the latter ("shoken" means
> "securities broker" in Japanese).

Are you counting connections as a cost, or as a value?  Of course
connections that I don't make don't cost me anything.  But neither do
I value them.  Nor do they generally value me.

> It's true that asymptotically the value of such networks will be
> limited by people talking to each other.  But today less than one
> percent of the world's population can say that their personally
> received value is dominated by their social relations.  Most people
> spend well over 50% of their time merely providing for the physical
> needs of themselves and the four, five, or ten people they most value
> talking with.

People talk for more than social reasons.  The kinds of connection
ratios that I'm pointing out come up in work-related communication as
well as social ones.  So even though the reasons for communicating
change, the value distribution tends to be similar.  In at least some
contexts.  That said, though, a brief business communication between
CEOs has sharply different value than a lengthy correspondence between
two rank-and-file employees.  (If the value of different
communications follows a power law, though, then the difference in
value of different communications matters far less than one would
expect.)

But as I've pointed out, it is very hard to generalize about all
networks.  What can you say in common about marketplaces, research
journals, telephones and telegraphs?  It is hard to say anything at
all - I've already pointed out that I think Metcalfe's Law is close to
holding for marketplaces and network effects were almost irrelevant
for telegraphs.  I have direct empirical evidence for those two claims
- namely the power (or non-power) of market leaders to maintain their
lead while faced with competition.  I believe that the other two
examples are more similar to each other, and most possible networks,
with scaling that is more like n log(n) than n*n.

> So I conjecture we have several generations of "mere" economic growth
> to pass through before the kinds of consumption activity you focus on
> present a binding constraint on network valuation.

The common result pops up in many different ways.  For instance let's
take mail.  This has, I think you'll grant, a variety of different
kinds of consumption activity, including business to business,
business to consumer, consumer to business, and consumer to consumer.
Well one line of argument that Andrew came up with involved studying
the volume of email delivered, and he found that as the number of
people available to send and receive email increases, the voume of
mail sent scales about n log(n).

What seems to matter here is not the type of consumption activity that
is happening, but rather the pattern of how connections are formed.
When people are using the network to connect with random strangers
with very different interests, then Metcalfe's Law will hold very
well.  When people use the network to communicate with the same people
day in and day out, then Metcalfe's Law doesn't appear to work so
well.

I submit that most kinds of networks fit the second description better
than the first.

>     Ben> If you disagree, then I expect you today to go out, find a
>     Ben> random email address, contact that person and open up a
>     Ben> conversation.  I am fairly confident in predicting that
>     Ben> you'll be disappointed in the result.
>
> You're paying no attention at all to what I've written, except those
> parts you find easy to disagree with.

I'm paying attention to what I've understood of what you've written.
I am choosing to respond to points where I feel that there is
disagreement, or where I don't understand what you meant.

>     >> What search technology *may* make possible is scaling up search
>     >> in a heterogeneous network to a billion participants or so.
>
>     Ben> It does.  However what do people search FOR?
>
> I'm not talking about people doing the searching.  Read what I wrote,
> I said that explicitly.

I am very confused about what you meant here.

If you mean that searches are conducted by computers, then we've just
talked past each other.  If I go into Google and type in a search, I
would say, "I searched for X" even though Google's computers did all
of the actual work.  So when I say "people search" I am not
disagreeing with the fact that computers do the work.  I am focusing
on the fact that there are people specifying the searches, people
interpreting the results of said searches, and humans deriving value
from being able to do those searches.

If you meant that people are not determining the searches that need to
happen, then you didn't say that explicitly.  At least not in the
quoted section, and not in the original email that I was responding to
(I went back and checked).

> This is really important.  I can't give crystal-clear examples,
> because what we're talking about is Tom's claim of a new emergent
> effect.  It is certainly true that historical claims of new emergent
> effects *that will transform society globally* have been nearly always
> bogus, but not quite so bogus as perpetual motion.  In particular, new
> emergent effects that support a profitable business model for a while
> are found regularly.

Tom's claim of an emergent effect is based on his estimates of
potential value which is in turn based on Metcalfe's Law.  However
he's talking about a kind of interaction where I believe Metcalfe's
Law fails, which is kind of an important wrench in his analysis.

> I can't be very optimistic about Tom's more grandiose claims, but this
> being FSB, I certainly do hope, and expect with positive probability,
> that there's a business opportunity in it for someone, maybe Tom
> himself.  It's worth being precise about our analysis for that
> reason.

If it is an emergent effect that is dependent on search technology,
then I'd bet that it will be Google who will succeed in delivering it.
:-)

Seriously, if there is a business opportunity, then I expect precision
from whoever tries to pursue the opportunity.  But not from anyone
else.  Certainly not from me - I've been thinking about this
particular problem off and on for a few years now and it has only
clarified for me exactly how much work it would take to try to make
any kind of precise estimates.  All that I hope for now is useful
generalizations, and an understanding of what you have to investigate
to get more precision.

> And there are such global innovations.  The market (bazaar) per se is
> one.  The very special markets for primary factors (ie, land, labor,
> and financial capital) are another.  I suspect that the Internet is a
> third.  We don't really understand the connection between the abstract
> economics of the market and its generalization to economics of more
> general networks, so it's hard to explain what's going on.  I can only
> humbly request that you try to figure out what I'm trying to say, help
> me to clarify it (including in my own mind), and disagree with *that*
> as appropriate (and of course correct my math mistakes!)

I sometimes have trouble figuring out what you've said, and now you
want me to try to figure out what you haven't said?  You aren't asking
for much, are you? :-/

Anyways I know from past conversations on this topic that sorting out
thoughts on this is surprisingly tricky.  Part of it is that we have
lots of experience at the "tree" level of an individual network
participant, and it is very hard to visualize how that works out at
the "forest" level of the network as a whole.

>     Ben> Andrew and I have been contacted since then by multiple
>     Ben> people who had come up with their own scaling estimates, it
>     Ben> is fascinating how many different approaches all come up with
>     Ben> n log(n) or something similar.
>
> Which is not Metcalfe's law, but it is still a *strong* network
> technology in economic terms.  Do you know what Walmart's margin on
> turnover is?  Around 2%.  Can you imagine what they could do with n
> log n returns to scale if they can dominate an inherently linear
> industry on that margin?

Of course it is.  People have long known that network effects are
real.  That much is obvious.  The important question is how big they
are.  All that n log(n) says that people have historically
significantly overestimated their impact, not that they don't exist.

Now I'll gladly grant you that this advantage is pretty nice.  If it
were coupled with truly being the best at what you're doing, it truly
becomes entirely insurmountable.  Even if it isn't, well compare, say,
the dominance that Walmart has in retailing with the dominance that
AT&T (in the world before government-mandated breakup and competition)
achieved in telephony.  (Admittedly AT&T was unable to achieve that
overwhelming dominance until the government intervened by forcing all
of its competitors to interconnect with it, while not requiring it to
interconnect with anyone else...)  Or Microsoft's market position in
desktop operating systems.  In short when network effects start to
matter, it is common to see someone achieve a level of dominance in
their area that Walmart cannot dream of matching in retail.

However I submit that this dominance is frequently not coupled with
being the best possible.  In something like retailing, exactly because
it is such a level playing fields, the best possible competitor can
emerge and prove itself.  However when network effects matter, the
market leader can afford to be worse than its competitors, and that
breeds complancy.  That doesn't mean that they are easy to beat, but
it is possible.

>     Ben> This is true, but people DON'T join networks at random.
>
> The networks I'm talking about, people don't join at all.  They *are*
> members; they participate as much as they please.  Thus the focus on
> marginal cost of participation per unit compared to value per unit.

Then I'm a member of many bulletin boards that I've never heard of!  I
am a member, I participate as much as I please (ie zero)!

There is a technical sense in which that's true, but nobody would
maintain that position in normal conversation.  Conversely of the
insane number of networks available on the Internet.  I consider
myself only a member of a handful.  (This being one of them.  Whether
to the pleasure or distress of the rest of network I am not entirely
sure.)

As to cost vs value, I grant that observing effort put out gives a
lower bound on the value of a network.  However I don't see how you're
going to readily get from that observation to a predction of how the
valuation scales with size.

>     >> communication (sorry, no examples---that's Tom's job! ;-) might
>     >> come pretty close, at least up to population levels feasible
>     >> for humans on this planet.
>
>     Ben> Whether it comes close depends on what you're communicating.
>     Ben> With markets I'm communicating something very simple that
>     Ben> machines can easily evaluate.  With content I'm communicating
>     Ben> something complex that machines cannot easily evaluate.  The
>     Ben> resulting dynamics are very different.
>
> The machines don't evaluate, the network does.  The market works for
> cars and homes just as it does West Texas Light and red wheat #2.

There is a trivial sense in which this is impossible for all but a few
networks.  There is a non-trivial way in which this is true for all
but a few networks.

First the trivial.  For something as fundamentally simple as buying
and selling items, it is easy to automate it and have the network do
the work.  For something complex like discussing network valuation,
the work needs to be done by humans.

Now the non-trivial.  Let's draw an analogy to how the brain works.
The brain works because relatively simple structures (neurons)
communicate with and react to each other along physical networks
(their synapses) and have rules for deciding when to pay attention to
each other.  The result is a brain that is capable of thought.  Well
in a parallel way we can think of ourselves as pieces of
organizations, both formal and informal, that think about different
things.  And the process of our communicating, reacting to, and
repeating (with modification) fairly complex thoughts is part of the
organizations thinking.  In that sense, thinking of ourselves as part
of the network, the network evaluates, modifies, selects what to
amplify, etc.  One can get very metaphysical about it all.

My reaction?  My reaction is that when complex human interaction is
inherent in participating in the network's operation, then the sum of
the individual values that humans draw from being part of such a
network is limited by human limitations on how we function.  So then,
Metcalfe's Law will (I believe) be overly optimistic in all but a few
situations.

>     >> I think the jury's out on just how far Metcalfe's Law can
>     >> extend its domain---but the financial markets are a pretty
>     >> compelling example for "hey, this really does work sometimes!"
>
>     Ben> So if Metcalfe's Law failed offline, for Usenet, and in Web
>     Ben> 1.0,
>
> Who said it has failed for Usenet and Web 1.0?  We're talking about a
> social phenomenon.  Internet time does not apply, it is way too early
> to talk about failure, even for Usenet (which might be worth
> considering a kind of Web 0.1 for this purpose).

Who said it has failed?  I did.  At this point in my life I
participate in only a few online forums, and those have been chosen
for density of interesting people rather than size.  (I abandoned
Usenet in 1997, and Slashdot in 2001 or so.)

>     Ben> It is a basic property of human nature that virtually
>     Ben> everything is junk to us, and the *meaningful* connections we
>     Ben> draw have very different topologies.
>
> Meaningful ! <=> valuable.  In fact, "meaningful" ! <=> "meaningful
> connection".  Sometimes the medium is the message, sometimes the
> message is valuable but otherwise not meaningful.

Given that humans value meaning, there is a strong correlation between
meaningfulness and value.

>     >> First, it's not an assumption of equal value, really.  It's an
>     >> assumption of equal expected value per connection, independent
>     >> of the number of connections.  Apply Law of Large Numbers, live
>     >> Metcalvian ever after.
>
>     Ben> Sorry, insert the word expected and my statement still
>     Ben> stands.  (It may be that my expectations are rather different
>     Ben> than yours, but I think that mine have a better grounding in
>     Ben> actual human behaviour...)
>
> Nevertheless, this stuff is very delicate.  I ask you to be precise,
> because there are lots of participants who are not going to make the
> automatic generalization from "equal value" to "equal expected/average
> value".

Point.

>     >> Second, consider Mr. Lynch's problem.  Have you ever watched a
>     >> headhunter network a LUG meeting? ;-) If such people (excuse
>     >> the exaggeration) are a constant fraction of the population,
>     >> whammer jammer, Metcalfe's Law!  (In the long run, nobody else
>     >> will matter if they obey Metcalfe's Law---their Metcalfe value
>     >> is a lower bound on social value.)
>
>     Ben> Yes I have.  It is the headhunter's job to facilitate finding
>     Ben> potential connections of value.  Their presence (like a
>     Ben> search engine online) therefore can bring the potential value
>     Ben> of the network from O(n) to O(n * log(n)).
>
> I don't understand that statement at all.  I'm now not sure my
> implicit model made sense at all, so I won't try to argue for
> Metcalfe's Law in the sense of a quadratic increase at this point.
> But the value of the match achieved is an order statistic and, unless
> you believe that people have unbounded potential value for a single
> connection, thus bounded above.  Since there's one match per person,
> the best *your* model of headhunting can achieve is O(n).

Let me explain my statement then.  I am hypothesizing from general
principles that the potential value of the connections to be made for
the average person are probably O(log(n)).  I am further hypothesizing
that most participants will spend their time talking with a small
number of connections that are guaranteed to be among their most
valuable - which are connections with people that they already know.
Thus most participants will naturally only find constant value in
their interactions.

Enter headhunters who, through talking with everyone, try to find some
what potentially valuable connections that are not being made and then
try to make them.  Therefore the value of headhunters is that they
make connections happen that otherwise would not.

About your "one match per person" comment, that ignores the fact that
employees are not necessarily loyal to employers, and further the fact
that a given potential employee sees value in having multiple
potential offers to compare.

>     Ben> Yes, this is true.  For instance in telephone networks, many
>     Ben> strangers will call the local pizza parlour.  However again
>     Ben> their potential value is bounded above by the potential value
>     Ben> that humans find from being on the network, which I believe
>     Ben> is generally far, far lower than Metcalfe's Law predicts.
>
> An economist only needs strictly superlinear to get seriously excited.
> I realize that from the point of view of corporate profit or rents
> earned by top programmers, quadratic is very nice, but the social
> organization effects are driven by small differences at the margin.

In that case you should be excited either way. :-)

>     Ben> I agree that Google increases the size of the network by
>     Ben> increasing the marginal value of participating on the network
>     Ben> to all participants.  However that's a second-order effect
>     Ben> and I think it far smaller than the primary benefit of
>     Ben> lowering the difference between the value that people
>     Ben> actually achieve and theoretically could in a perfect world.
>
> You are seriously confusing if you call an effect that is O(f(n)),
> f' > 0, "second-order" compared to an effect that is O(1).  What are
> you trying to say?

I am saying that the most direct way of seeing the value that Google
brings is to understand that it allows people to find what they are
looking for - so it narrows the gap between what people could
theoretically get from using the network, and what they're likely to
get in practice.

I'm further saying that if you use the above to do a
back-of-the-envelope estimate of the value that Google provides, your
simple estimate will be off by a bit, because the convenience of
Google also increases how much people want to use the Internet.  That
is a correction term.

I was saying "second order" when I merely meant "relatively small
correction term".  In other words I'm saying, "Yes, Google makes
people use the Internet more, but that's not the primary reason that
it is so valuable."

>     Ben> Oh no.  Reed's Law *explicitly* says that every new member
>     Ben> has that effect.  Which is why it doesn't pass the "smell"
>     Ben> test.
>
>     >> Strawman!  Strawman!  Out to the cornfields with you!
>
>     Ben> Strawman?  How so?
>
>     Ben> Reed's law says that the value of a network is exponential in
>     Ben> the size of that network.  At some point adding a new person
>     Ben> will, on average, increase the value of that network by more
>     Ben> than the current GNP.  This is not a strawman argument - it
>     Ben> is just a demonstration that Reed's law is not very
>     Ben> realistic.
>
> That's true, but that's not the argument you wrote above.

For the simplistic statement of Reed's Law, the argument that I first
wrote works.  For a slightly more sophisticated version you have to
add weasel words like "on average".  Either way the flaw is pretty
obvious.

>     >> Reed's Law would work if the fraction of great minds per
>     >> generation were constant regardless of size of generation, and
>     >> everybody else was worth precisely zero in this sense.  I think
>     >> that's unlikely but arguable.
>
>     Ben> I don't see that.
>
> You don't understand the "in a linear context substitute expected
> value for constant value, and get the same theorem" argument?  Above
> you claimed it goes without saying.  Or are you once again implicitly
> referring to the argument that I immediately proceed to make myself,
> but you haven't made yet?

I'm saying that I don't understand how the assumption that a fixed
fraction of minds are great and everyone else is worth zero should
lead to Reed's Law.

I *very* strongly suspect that it doesn't, and I'm suggesting that you
re-evaluate your reasoning for that point.  I have absolutely no idea
what you're referring to when you say that I claimed "it goes without
saying".  Because certainly nothing that I've said should lead to that
conclusion, and if it does then I said something I certainly don't
mean.

> I expect that of propaganda maestros like RMS, but not of you.

There has clearly been a miscommunication or miscalculation on one
side or both.  Hopefully I made it clear what I don't see and either
you'll tell me that you thought I was talking about something else, or
you'll supply me with the missing explanation.

>     >> The problem with Reed's law is elsewhere.  It is that it
>     >> assumes that all needed groups actually form.  If there's one
>     >> great mind, then she has to participate in all (N-1)(N-2)
>     >> possible groups for her to get Reed's law to work.  Again, you
>     >> could adjust this for some sort of average or probabilistic
>     >> formation, but I don't think that makes much more sense than
>     >> assuming all groups form---you still need the number of groups
>     >> forming to be proportional to the number of possible groups,
>     >> and eventually Ms. Great Mind will get tired of all that
>     >> networking.
>
>     Ben> Um, your math is off.  She has to participate in something
>     Ben> proportional to 2**(n-1) possible groups for her to get
>     Ben> Reed's law to work.  This is unrealistic.
>
> Yeah, well at least I was off in the direction such that correcting
> the math made the argument work better.  :-)

Kind of like how you hope to make 2 sign errors if you make one? :-)

Cheers,
Ben