Subject: Re: How accurate is Metcalfe's law? (Was: Ximian software)
From: Ben_Tilly@trepp.com
Date: Fri, 4 Jan 2002 13:32:59 -0500


Bernard Lang wrote:
> I do not know whether this is related to Odlyzko work, which I have
> not read (I know Odlyzko in interested in library business), but this
> reminds me of Bradford's law that is used by librarians to assess the
> worth of a collection.
>
> Here is the law as explained by jean-Claude Guedon in
> http://www.arl.org/arl/proceedings/138/guedon.html
>
> > Everyone in the library profession knows Bradford's distribution
> > law. It posits a multiplier, bm, actually derived from a ratio: if you
> > need 5 journals to survey the essential parts of your specialty and
> > these 5 journals yield, say, 12 interesting articles, and if to find
> > another 12 articles, you need 10 journals, then bm will be 2 (10
> > divided by 5). And if you want to find another collection of 12
> > articles, you will multiply the 10 journals by the multiplier and will
> > find 20.  Obviously, returns diminish rapidly, as the multiplier grows
> > exponentially, and it explains why scientists have long learned to
> > moderate their urge for exhaustive searches.
>
> It seems to me that a similar view can be taken to assess the value of
> a network to each customer.  At least of a phone network.

Fascinating.  I have no proof, of course, but I strongly suspect that
this principle is true of virtually any kind of information repository.
And given your figures, if your universe of journals is about 5 million
then there should be about 240 interesting articles in that lot - if you
can only find them.  (It would take 5 trillion more journals to double
that.  There are definitely diminishing returns here.)  The value of a
search engine is that it helps you find the articles which you would
care about if only you knew about them.  This suggests that the
potential value of the entire universe of possible information to
everyone scales like n*log(n), an unknown fraction of which is going to
be inaccessible for practical reasons.  But the distribution of our
value is highly uneven through that universe, and we need tools like
Slashdot and search engines to allow us to consistently extract more
than a constant times n of it.

Now what happens as we see a network grow to let us access that
information?  Well let us assume that resources in the section we care
about are being added at random.  Then what we perceive is a roughly
linear growth in value with the size of the network.  Of course with
distinct jumps when items near and dear to our hearts get added.  (As a
data point, my wife's enjoyment of email rose dramatically when her
father got an email account.)  In other words we see Metcalfe's law in
operation.  From which we predict amazing things!

But the part of the network that we care about is distinctly finite.
When that finite list has all been added, there is saturation.  The
growth in parts of the network far from our interests is of minimal
value.  Our having observed Metcalfe's law notwithstanding, the total
personal value to be reached is (barring the production of new
interesting content - which is a hard to evaluate factor) bounded by
the logarithm of the total size available.  The addition of many
people in China grows the network tremendously, and the value for
most of us very little.  (There is a node in Taiwan I cared about,
but now that my brother is online, I am unlikely to get much more
value in that billion people.)

Thus this predicts both the observations that lead to widespread
belief in Metcalfe's law, and the observation that Metcalfe's law is,
on the face of it, impossible.  Can we make more predictions?

Of course we can!  Assuming that the universe is relatively big, the
total personal value to be derived by an individual is strongly affected
by the value of bm.  If bm is large then almost all of the value that
person could get is concentrated in a small number of resources.  If
bm is small, then there is a "long tail" of value, and there will wind
up being such an embarrassment of riches that the happy individual
cannot consume them all, and is forced to only skim the cream of the
crop.

Therefore I predict that people for whom bm is small should enjoy the
net more than people for whom bm is large.  But what does the size of
bm mean?  It measure how specific your interests are.  If bm is large
then your interests are very specialized and narrow, outside of a
tightly defined band of interest you have trouble finding anything of
interest.  So if bm is small then you are naturally a generalist, you
are interested by a bit of anything.

So that prediction reads, "People who are inclined to be generalists
are likely to find the Internet particularly interesting."  That is
certainly true of me, and it fits the people I can think of off of the
top of my head.  (An unscientific sample, but what do you expect of a
theory that is 10 minutes old?:-)

Looks like we have a theory about the total value to be found in
networks which is based on already observed tendancies, explains all
of the starting facts, and generates testable predictions.  Not bad! :-)

> Now, I am not sure how that converts into value for the companies
> running them, since networks are interconnected.

I would call that the proverbial million dollar question, but that is
seriously underestimating the profit potential of a good answer.

Cheers,
Ben