Subject: Re: near/medium future digital media economics
From: "Ben Tilly" <btilly@gmail.com>
Date: Fri, 19 May 2006 10:48:44 -0700

 Fri, 19 May 2006 10:48:44 -0700
On 5/19/06, Stephen J. Turnbull <turnbull@sk.tsukuba.ac.jp> wrote:
> >>>>> "Ben" == Ben Tilly <btilly@gmail.com> writes:
>
>     Ben> Metcalfe's Law (BTW, note the spelling) is going to apply
>     Ben> best in networks where people actively wish to connect with
>     Ben> fairly random strangers.  Very few networks look like this.
>
> That's because historically commodity markets were the only practical
> place where that would work.  It is not an accident that the huge
> complex of networks we call the stock market are concentrated in maybe
> a dozen institutions worldwide, and they coalesced into an Intermarket
> so long ago that we no longer think of them when we talk about
> networks.

Email has existed for over 3 decades and allows people to communicate
with strangers.  I submit that most of us mostly talk with people we
know.  And virtually everyone detests the tiny minority that actively
seeks to talk to complete strangers.  (We call them spammers.)

Phones have been in common use for far longer than that.  Thanks to
phone books it is even easier than it is with email for anyone to
connect with a random person.  Again, most of us mostly talk with
people we know.  And virtually everyone detests the tiny minority that
actively seeks to talk with complete strangers.  (We call them
telemarketers.)

I submit that the value of communication largely stems from our human
nature, and so the medium matters far less to the dynamics than our
own makeup (which isn't changing any time soon).

If you disagree, then I expect you today to go out, find a random
email address, contact that person and open up a conversation.  I am
fairly confident in predicting that you'll be disappointed in the
result.

> What search technology *may* make possible is scaling up search in a
> heterogeneous network to a billion participants or so.

It does.  However what do people search FOR?

>     Ben> I suggest that you should look up Benford's Law and explain
>     Ben> why it doesn't apply to this type of content when it does to
>     Ben> other kinds.
>
> What in the world do you think Benford's Law has to do with anything
> here?  Benford's Law is about decimal representations of a
> monotonically increasing sequence, not about values.  Eg, if I were
> going to bet on leading "digits" in web content, I'd go for 60 base
> 256, wouldn't you?

Sorry, Bradford's Law.  There are so many power laws out there, I
named the wrong one.

>     Ben> This argument, like Metcalfe's original argument, has the
>     Ben> implicit assumption that everyone is equally interested in
>     Ben> all potential connections.  I'm not.  Nor is anyone else.
>
> Oh, come on, Ben, give us a little!  You're capable of presenting
> the math explicitly, and it's worth doing so here.

I did a bit of this in a different post.

http://www.dtc.umn.edu/~odlyzko/doc/metcalfe.pdf gives several
different arguments, based on several different lines of reasoning,
that all come up with similar scaling estimates.  Andrew and I have
been contacted since then by multiple people who had come up with
their own scaling estimates, it is fascinating how many different
approaches all come up with n log(n) or something similar.

> The question is "given an increasing size of network, how does the
> value distribution change?"  If we assume that to some agent (note the
> abstraction!) the probability distribution over (gross) value of
> connection to a random other agent is constant, then the expected
> (gross) value of the network *is* proportional to network size for
> each individual.

This is true, but people DON'T join networks at random.  In particular
people don't join a network until they have sufficient personal value
to justify doing so.  Which means in practice that people mostly join
networks because there are friends already on the network who they
wish to communicate with.  And once a group of friends has joined a
network together, they cease to gain significantly by the adding of
more people to it.

> So what you need is marginal cost of connecting bounded below that
> average value, and vol! net value per agent is linear in network
> size, and you got Metcalfe's Law.
>
> At one extreme of marginal cost, we have the marriage market.  For me
> it was zero for q=1 and infinite for q=2.  So then the value of the
> network to me was an order statistic (max Vi over i = 1, ..., N).
> Anything that increases network size increases personal value, but
> definitely not linearly (bounded above by "10", of course!)  So
> effectively we've increased the value of a rival good, but not changed
> it into a network good  l Metcalfe.

I'll randomly note that it was only infinite after you found the right #2. :-)

If you're like most of us, you found, evaluated, and discarded several
potential #2s before settling on the one that you married.  (Many of
us do not stop that cycle with marriage.)

> Can we get to the other extreme, constant marginal cost below average
> value?  If we're talking about communication networks among humans,
> no.  Social contact of positive value requires time bounded below on
> the order of a second or so (however long it takes to perceive your
> baby's smile), and there are only a finite number of seconds available
> to each of us.

Note that the fundamental constraint here is human.

> However, anonymous networks (eg, markets) or machine-mediated
> communication (sorry, no examples---that's Tom's job! ;-) might come
> pretty close, at least up to population levels feasible for humans on
> this planet.

Whether it comes close depends on what you're communicating.  With
markets I'm communicating something very simple that machines can
easily evaluate.  With content I'm communicating something complex
that machines cannot easily evaluate.  The resulting dynamics are very
different.

> I think the jury's out on just how far Metcalfe's Law can extend its
> domain---but the financial markets are a pretty compelling example for
> "hey, this really does work sometimes!"

Note that in the case where we agree that it works, it works both
online and off.  By contrast Tom is arguing that there are cases where
it doesn't work in offline analogs, but he thinks it will in online
ones.  What I'm saying is that Metcalfe's Law working or not is more a
function of how humans approach the interaction than a question of
what medium they interact over.  So if Metcalfe's Law failed offline,
for Usenet, and in Web 1.0, then waving the Web 2.0 (or 3.0 or
whatever) magic pixie dust won't make Metcalfe's Law start to work for
you.

> Note that a market is a pretty trivial network topology (star).  But
> then, the basic property of the Internet that these laws depend on is
> that it turns Life, the Universe, and Everything into a star topology.

It is the basic property of the internet that everything is
potentially a star topology.  It is a basic property of human nature
that virtually everything is junk to us, and the *meaningful*
connections we draw have very different topologies.  What that
topology is depends on what the context is - for markets it is closer
to * than for anything else.

>     Ben> You've just counted connections and assumed that they all
>     Ben> have equal value.  But that assumption flatly contradicts
>     Ben> observations of how people really network in social
>     Ben> situations.
>
> First, it's not an assumption of equal value, really.  It's an
> assumption of equal expected value per connection, independent of the
> number of connections.  Apply Law of Large Numbers, live Metcalvian
> ever after.

Sorry, insert the word expected and my statement still stands.  (It
may be that my expectations are rather different than yours, but I
think that mine have a better grounding in actual human behaviour...)

> Second, consider Mr. Lynch's problem.  Have you ever watched a
> headhunter network a LUG meeting? ;-)  If such people (excuse the
> exaggeration) are a constant fraction of the population, whammer
> jammer, Metcalfe's Law!  (In the long run, nobody else will matter if
> they obey Metcalfe's Law---their Metcalfe value is a lower bound on
> social value.)

Yes I have.  It is the headhunter's job to facilitate finding
potential connections of value.  Their presence (like a search engine
online) therefore can bring the potential value of the network from
O(n) to O(n * log(n)).  Their potential value to extract from their
role is therefore bounded above by the true potential value of the
network, which is far less than Metcalfe's Law would say.  (But they
can only achieve that value from visiting a large fraction of the
network.  Even so you'll note that technical recruiters only connect
like that at technical gatherings, you won't see them crashing the
florist convention to actively look for computer programmers...)

> Third, remember that many participants in networks are not people,
> they are firms.  To the extent that a firm can have a bounded labor
> force with the additional costs of networking borne by capital, a
> network of firms (or capital-enhanced humans, for that matter) could
> exhibit rather different networking behavior.  (Then again, maybe they
> wouldn't.  That was Jamie Zawinski's point about groupware.)

Yes, this is true.  For instance in telephone networks, many strangers
will call the local pizza parlour.  However again their potential
value is bounded above by the potential value that humans find from
being on the network, which I believe is generally far, far lower than
Metcalfe's Law predicts.

>     Ben>  And Google's power derives from the fact that good searching
>     Ben> allows people to come closer to perfectly extracting the
>     Ben> value of being on the network.
>
> Uh, what are you trying to say here?  I think it's arguable that
> Google increases the size of the network in some sense; I'd like to
> see a careful argument why that isn't so.

I agree that Google increases the size of the network by increasing
the marginal value of participating on the network to all
participants.  However that's a second-order effect and I think it far
smaller than the primary benefit of lowering the difference between
the value that people actually achieve and theoretically could in a
perfect world.

>     Ben> As for your defence of Reed's Law, well what you're saying is
>     Ben> that you think that it is reasonable that adding one person
>     Ben> to the internet could double everyone's perceived quality of
>     Ben> life.  If you think that that's reasonable, then I don't
>     Ben> really know what to tell you.
>
> "The King!"  ;-)
>
>     >> This is not to say that at some point, every new member has
>     >> that effect.  The constant factor on these approximations
>     >> matter and we are lucky/happy when a few great minds per
>     >> generation or 10 have such effect.
>
>     Ben> Oh no.  Reed's Law *explicitly* says that every new member
>     Ben> has that effect.  Which is why it doesn't pass the "smell"
>     Ben> test.
>
> Strawman!  Strawman!  Out to the cornfields with you!

Strawman?  How so?

Reed's law says that the value of a network is exponential in the size
of that network.  At some point adding a new person will, on average,
increase the value of that network by more than the current GNP.  This
is not a strawman argument - it is just a demonstration that Reed's
law is not very realistic.

> Reed's Law would work if the fraction of great minds per generation
> were constant regardless of size of generation, and everybody else was
> worth precisely zero in this sense.  I think that's unlikely but
> arguable.

I don't see that.

> The problem with Reed's law is elsewhere.  It is that it assumes that
> all needed groups actually form.  If there's one great mind, then she
> has to participate in all (N-1)(N-2) possible groups for her to get
> Reed's law to work.  Again, you could adjust this for some sort of
> average or probabilistic formation, but I don't think that makes much
> more sense than assuming all groups form---you still need the number
> of groups forming to be proportional to the number of possible groups,
> and eventually Ms. Great Mind will get tired of all that networking.

Um, your math is off.  She has to participate in something
proportional to 2**(n-1) possible groups for her to get Reed's law to
work.  This is unrealistic.

> And it needs to be the case that having formed the
> group-of-one-billion, *you* would get an equal increment of value from
> also forming the group-of-one-billion-less-*me*.  Hardly seems likely!
> :-)

Exactly.

> Please note, I agree with you, not Tom.  My intuition is that in the
> long run Metcalfe's law will fail, and network values will be
> sublinear.  But I strongly believe that the reason why they fail
> matters, both to the degree to which Metcalfe's Law approximates value
> for "small" networks, and to long-run industrial organization of
> networks that start out small enough for that approximation to be
> valid.

I agree with that.

Cheers,
Ben