Subject: Re: near/medium future digital media economics
From: "Ben Tilly" <btilly@gmail.com>
Date: Mon, 22 May 2006 18:37:26 -0700

 Mon, 22 May 2006 18:37:26 -0700
On 5/21/06, Stephen J. Turnbull <turnbull@sk.tsukuba.ac.jp> wrote:
> >>>>> "Ben" == Ben Tilly <btilly@gmail.com> writes:
>
>     Ben> On 5/19/06, Stephen J. Turnbull <stephen@xemacs.org> wrote:
>
>     >> I'm not talking about "talking", I'm talking about connection.
>     >> You know, the kind of connection that associates you with the
>     >> company that administers so-3-0-0.mpr2.iad5.us.above.net, or
>     >> with the Tsuchiura office of Nomura Shoken.  Didn't know about
>     >> those, did you?  So you obviously didn't "talk" to them
>     >> personally, and those connections cost you no time, though you
>     >> almost certainly connected to the former, and quite likely are
>     >> fuzzily connected to the latter ("shoken" means "securities
>     >> broker" in Japanese).
>
>     Ben> Are you counting connections as a cost, or as a value?  Of
>     Ben> course connections that I don't make don't cost me anything.
>     Ben> But neither do I value them.  Nor do they generally value me.
>
> Didn't I write that you *did* make connections to those nodes?  You
> just didn't "talk" to them.  You talked *through* one, and received
> option value aka network externality from the other.

That's technically true.  It is also true that my telephone calls go
through a variety of switching stations.  But ideally I do not know or
care about those stations, and under ordinary circumstances I do not
care which stations I go through or how many there are.  Those
technical details simply do not enter into my value judgements.

Stephen J. Turnbull *does* enter into my value judgements.

The specifics of how I am connected to you doesn't.

> As for the classification of "connection", I'm counting them as both
> cost and value, of course.  In the paragraph above, they appear as a
> "fact", and as a cost.

OK.

>     Ben> People talk for more than social reasons.
>
> I'm sorry.  Let me make it explicit: I'm defining any reason for
> *talking* as "social."  (If that's not acceptable, I can use other
> terminology.)  Communication in work is a social activity among the
> humans in the organization.  Non-social connections are connections
> that occur in the mechanism that do not necessarily connect two or
> more humans.  Connections may exist but not be used.  You and I are
> connected 24x7 by the email network.  Right now it may seem like we're
> using it 24x7, but we also go months without seeing each other's
> bylines, let alone communicating with each other.  On the other hand,
> I can send you a question quite out of the blue, and have done so,
> haven't I?  (n.b., yes, there was a personal connection, that was not
> random; but spam is, more or less.  I'm not talking about cost or
> value here, just connection.)

This is all true.  And were I to go to Japan (not in my plans any time
soon, but just supposing) then I'd check to see if we could physically
meet.  Conversely if you were passing through the Los Angeles area,
I'd hope that you'd return the favour.

> We *are* connected, and that connection has implicit value (option
> value, network externality) whether or not we actually use it.

True.  However it is the fact that it has been used that established
that value.  For instance I'm also potentially connected to your boss,
and that unused connection has almost no value to me.  No matter how
cool your boss may be, I've never made that connection.

>     Ben> That said, though, a brief business communication between
>     Ben> CEOs has sharply different value than a lengthy
>     Ben> correspondence between two rank-and-file employees.  (If the
>     Ben> value of different communications follows a power law
>     Ben> though, then the difference in value of different
>     Ben> communications matters far less than one would expect.)
>
> And if the value of communication has any relation to wage rates, the
> difference in value is exponential in number of levels of
> subordinates.

And the number of people at those higher exponents exponentially
decreases as you go up the pay scale.  I don't remember exactly how
unequal wealth distribution is in this country, but I guarantee that
the end result is that total actual value is worth a constant factor
more than what one would estimate looking at the median person.
Whether that constant is 2 or 10 is interesting, but does not affect
the scaling.

>     >> So I conjecture we have several generations of "mere" economic
>     >> growth to pass through before the kinds of consumption activity
>     >> you focus on present a binding constraint on network valuation.
>
>     Ben> The common result pops up in many different ways.
>
> Mostly asymptotic.  As Edward Lasker said, "Before the endgame, the
> gods have seen fit to place the middle game."  For the forseeable
> future we need to talk about the middle game.

I believe that the asymptotic behaviour generally dominates by the
time you have hundreds of thousands of participants.  (Possibly tens
of thousands.)

>     Ben> What seems to matter here is not the type of consumption
>     Ben> activity that is happening, but rather the pattern of how
>     Ben> connections are formed.  When people are using the network to
>     Ben> connect with random strangers with very different interests,
>     Ben> then Metcalfe's Law will hold very well.  When people use the
>     Ben> network to communicate with the same people day in and day
>     Ben> out, then Metcalfe's Law doesn't appear to work so well.
>
>     Ben> I submit that most kinds of networks fit the second
>     Ben> description better than the first.
>
> What-we-think-of-as-networks networks, maybe.  However, if you accept
> my thesis that markets are trivial networks, then most formalized
> existing networks are Metcalfian.  I'd also like to point out that
> you've made a typo.  Metcalfian networks arise when there are lots of
> random strangers with *closely aligned* interests (though the common
> interest may be very limited, you know that they're more or less
> aligned in advance).

That's a good point.  However then we encounter a significant problem.

Since I discovered the Internet, I have formed many connections with
former strangers who share some common interest.  You'd be among them.
 There is no question that I value these new relations more than I do
relations that I haven't maintained.

However in the absence of the Internet, I would have formed stronger
connections to people that I knew through happenstances of geography
and social circles.

How can one quantify how much better off I am because I've formed
connections with people that I share more interests with?  I can't
even quantify how much better off I think that *I* am, let alone
anyone else!

My *belief* is that the primary relationships that we form have a
personal value which is limited by our biological makeup.  So
opportunity the to form more congenial connections has limited
potential value.

Similarly if one wishes to measure value in money, the amount of money
that I have in these two scenarios is similar, therefore the
difference in how much I can spend because of changes in my personal
happiness is limited.  Therefore no matter how much happier I am, the
amount of money that I would spend on the improvement is limited.

However the point of fact is that every year I spend hundreds of
dollars to get an opportunity to see people that I know online.  The
only personal connections on which I have spent more over the years
are family.  Certainly ex-coworkers and old friends receive a far
smaller fraction of my attention.  So my behaviour shows that I care
far more about online connections than offline.  (With the exception
of family.)

[...]
>     >> I'm not talking about people doing the searching.  Read what I
>     >> wrote, I said that explicitly.
>
>     Ben> I am very confused about what you meant here.
>
>     Ben> If you mean that searches are conducted by computers, then
>     Ben> we've just talked past each other.
>
> No, that's not what I mean.  I'm distinguishing among whether the
> searches are explicitly initiated by a human, or whether they are
> implicit in a higher-level request.  Eg, suppose I asked a human
> secretary who was handling my FSB correspondence to "get me Ben
> Tilly's phone number."  He might search my emails; even if your public
> FSB persona doesn't provide it, your personal email might.  Google.
> Call the phone company.  Check the paper files.

OK.  So you're thinking of searches that are currently done by people
but which *could* be done by computers.

> There's no reason why a Sufficiently Smart Search Engine can't do the
> equivalent of all of that.  Of course "social engineering" the phone
> company into giving up your unlisted number would require more than
> Google (maybe an overclocked HOLMES IV could do it :-).  That requires
> *a lot* of connections over several conceptually different networks (I
> did mention "multigraphs", didn't I?)

Are you just guessing that my number is unlisted?

Incidentally Google is trying to be that Sufficiently Smart Search
Engine.  And the route that they are taking is to try to aggregate
everything into one search database, rather than trying to integrate
different databases.

>     Ben> If you meant that people are not determining the searches
>     Ben> that need to happen, then you didn't say that explicitly.  At
>     Ben> least not in the quoted section, and not in the original
>     Ben> email that I was responding to (I went back and checked).
>
> From the message you are quoting:
>
>     sjt> I'm not talking about "talking", I'm talking about
>     sjt> connection.  You know, the kind of connection that associates
>     sjt> you with the company that administers
>     sjt> so-3-0-0.mpr2.iad5.us.above.net, or with the Tsuchiura office
>     sjt> of Nomura Shoken.

I may have missed that section because I was scanning for "search".
And I was thinking (and still think) of connection as "end to end"
(ignoring the middle).

> From the "original email that [you were] responding to":
>
>     sjt> Can we get to the other extreme, constant marginal cost below
>     sjt> average value?  If we're talking about communication networks
>     sjt> among humans, no.  Social contact of positive value requires
>     sjt> time bounded below on the order of a second or so (however
>     sjt> long it takes to perceive your baby's smile), and there are
>     sjt> only a finite number of seconds available to each of us.
>
>     sjt> However, anonymous networks (eg, markets) or machine-mediated
>     sjt> communication (sorry, no examples---that's Tom's job! ;-)
>     sjt> might come pretty close, at least up to population levels
>     sjt> feasible for humans on this planet.

Which is a vision that I discount for the near future, except when
you're making very simple kinds of connections.  The ability to make
those connections seems highly dependent on the kind of network we
have.

> Granted, the connection to Google search isn't explicit.  But I think
> it should be clear that I'm not thinking about personal connections,
> whether purely "social" in the sense you were thinking about, or as
> role-based as arresting officer and suspect.

OK.

> For that to make sense in this context, there has to be an emergent
> effect based on "man Friday" software that "intelligently" generates
> lots of queries across the network.

Until we're dealing with networks where that is the case, I'm willing
to leave that factor out.

> Whether I can justify that in the sense of making more than pure magic
> designed to make Tom's dream come true is a different question.

Heh.

>     Ben> Tom's claim of an emergent effect is based on his estimates
>     Ben> of potential value which is in turn based on Metcalfe's Law.
>     Ben> However he's talking about a kind of interaction where I
>     Ben> believe Metcalfe's Law fails, which is kind of an important
>     Ben> wrench in his analysis.
>
> Sure.  And I agree with your analysis given the assumptions I
> understand you to be making.  But I've been doing this kind of
> analysis for far too long to trust my intuitions and analysis when
> we're talking about claims of "new" and "emergent."  I want to give
> Tom a fair amount of rope; he might just invent bungee jumping.

I like the Unix tradition of giving people enough rope to hang themselves. :-)

> Also, while Tom keeps saying "Metcalfe's Law," I think that n log n is
> quite sufficient to make his thesis interesting.  My feeling is that
> with current technology, we're talking something much lower than value
> = O(n log n) for wikipedia, etc.  And there's also the problem that
> there's actually growing content there, so it's not a pure matching
> problem---there's decreasing returns to scale for wikipedia content.
> And I think the frictions that make patents suck apply here, too.  But
> I certainly hope for something superlinear.

Well I believe that it is superlinear.  But not by much.

>     Ben> If it is an emergent effect that is dependent on search
>     Ben> technology, then I'd bet that it will be Google who will
>     Ben> succeed in delivering it.  :-)
>
> I'll take $100 of that bet, please.  Small, unmarked bills---you can
> just send it now.  If it's an "emergent effect", then no single
> company *can* deliver by definition.

Huh?  If there is an emergent effect in how people work that is
mediated by searching, I agree that Google will not capture the value
of that emergent effect.  But unless the current search landscape
changes radically, it will be Google that will deliver the searches
which that effect is based on.

>     Ben> it has only clarified for me exactly how much work it would
>     Ben> take to try to make any kind of precise estimates.  All that
>     Ben> I hope for now is useful generalizations, and an
>     Ben> understanding of what you have to investigate to get more
>     Ben> precision.
>
> Uh, a statement of what you have to investigate to get more precise
> estimates is what I mean by "precision".  Not the estimates themselves.

OK, to get more precise estimates what is needed is to select some
target networks and a measure of value, and then to do actual
measurements.  I would suggest comparing a number of activity based
measures (eg how many people logged in in the last month versus the
rate at which IMs are sent) where one can either compare competing
networks, or else compare one network with itself over time.  The goal
is to get a measure that is unambiguous, somewhat reasonable, and is
easy to measure.

The measurement problem is that if the network has significant size,
then I'm predicting that the scaling effect which one hopes to measure
is far smaller than the effect of even minor changes on usability.
What would be ideal is to pick a situation where you have many
independent networks to compare.

A good test case might be Jabber networks.  There are a lot of Jabber
servers out there that only a few people use, which don't bother
interconnecting with anyone else.  So you can find many very similar
networks of 10-2000 people.

Wikis make another good comparison case.

Unfortunately as the differences between markets, telephones and
telegraphs demonstrate, the scaling properties of different kinds of
networks will often differ sharply.

>     Ben> I sometimes have trouble figuring out what you've said, and
>     Ben> now you want me to try to figure out what you haven't said?
>     Ben> You aren't asking for much, are you? :-/
>
> I'm asking for one hell of a lot.  You can always say, no, I don't
> care enough to do that much work.  But ... you've already done a lot
> of it, and I can't ask much less because:
>
>     Ben> I know from past conversations on this topic that sorting out
>     Ben> thoughts on this is surprisingly tricky.
>
> Me too.  Until we get sorted, we need to avoid assuming that the other
> guy is using the same set of definitions we are.

Well a lot of my thinking is pre-definition.  As in, "What are the
qualitative properties that I expect to see from a reasonable
definition of value?"  As long as the actual definition of value is
reasonable and fits my expectations, the results will apply.  But I
haven't defined value there.

[...]
>     > The networks I'm talking about, people don't join at all.  They
>     > *are* members; they participate as much as they please.  Thus
>     > the focus on marginal cost of participation per unit compared to
>     > value per unit.
>
>     Ben> Then I'm a member of many bulletin boards that I've never
>     Ben> heard of!  I am a member, I participate as much as I please
>     Ben> (ie zero)!
>
> Exactly!
>
> But note, you may not be participating "zero".  Somebody may be
> republishing your words posted here, over there.  Isn't that precisely
> what free licenses are intended to enable?

This has certainly happened to me before.  However I still don't
consider myself to be a participant on that board.

>     Ben> There is a technical sense in which that's true, but nobody would
>     Ben> maintain that position in normal conversation.
>
> This is not a normal conversation.  We have to invent a terminology.
> I think "network member" is a lot easier to spell (not to mention
> pronounce) than "5102461a337bdbcef49211244946586860248c64".

I think that if we use a common phrase like network member, it should
have a definition that fits within the common understanding of that
phrase.  If you want to call me a "potential network member" of a
network I do not participate in, that fits.  But not "network member".

This is particularly important for this question - we're trying to
discuss how the value of a network changes depending on how many
members it has, and the discussion of network effects focus on the
value to one person of having other people pay attention to that
network.  Those arguments don't extend to the situation where there is
an effectively infinite number of "members" who pay absolutely no
attention to said network.

>     Ben> First the trivial.  For something as fundamentally simple as
>     Ben> buying and selling items, it is easy to automate it and have
>     Ben> the network do the work.  For something complex like
>     Ben> discussing network valuation, the work needs to be done by
>     Ben> humans.
>
> Discussing, yes.  Evaluating the value function, no.  Markets do that
> kind of thing all the time.  Houses, cars, even PCs are not
> commodities, yet markets do a tolerable job of assigning them values.
> Stock shares are commodities in the sense of all shares of common
> stock in a company being identical to each other, but humans don't do
> a very good job of assigning values to them.

We're in agreement here.

Markets do a better job than humans of evaluating the function.  (Not
necessarily great, but better.)  But the trick that makes that work
(ie making people put their money where their mouths are) doesn't work
for complex ideas.  Hence discussion requires human intervention.

Incidentally with some glaring exceptions (the bubble era comes to
mind), markets have evaluated valuations more in line with n log(n)
than with n*n.  For instance when two competitors interconnect, their
combined valuation doesn't sharply rise like Metcalfe's Law would
project.  And, as n log(n) would predict, when interconnections happen
the smaller participant pays the larger for the privilege.  (Metcalfe
predicts that the value of interconnecting is the same for both sides,
n log(n) predicts that it is greater for the smaller participant.)

>     Ben> My reaction?  My reaction is that when complex human
>     Ben> interaction is inherent in participating in the network's
>     Ben> operation, then the sum of the individual values that humans
>     Ben> draw from being part of such a network is limited by human
>     Ben> limitations on how we function.
>
> I can't argue with reductionism as a postulate.  I think it will have
> a hell of a time explaining the difference between the U.S. economy
> and that of China.  I know which people I think has smarter members on
> average.

I don't know that I would say that Chinese people have smarter members
on average.  More motivated and better educated, sure.  But native IQ?
 While current laziness hides it, there is a reason that "Yankee
ingenuity" was once respected worldwide.  (That said, the Chinese that
most Westerners will meet are those who've risen to the top of a very
competitive system.  They *will* be smarter than your average
Westerner.)

However let's avoid arguing The Bell Curve right now.

>     Ben> Who said [Usenet or the Web v1] has failed?  I did.  At this
>     Ben> point in my life I participate in only a few online forums,
>     Ben> and those have been chosen for density of interesting people
>     Ben> rather than size.
>
> Aside: Why do you call that a failure?  I'd call it "evolution".

To my eyes, Usenet no longer succeeds in its stated goal.  Which is to
provide a good forum for conversation.

>     > Meaningful ! <=> valuable.  In fact, "meaningful" ! <=>
>     > "meaningful connection".  Sometimes the medium is the message,
>     > sometimes the message is valuable but otherwise not meaningful.
>
>     Ben> Given that humans value meaning, there is a strong
>     Ben> correlation between meaningfulness and value.
>
> I haven't denied that, have I?  The whole point of "emergent
> phenomenon" is that humans can't detect valuable meaning in the
> individual pieces, but see it in the whole.

My understanding was that you were pointing out that I'd slipped from
talking about value to talking about meaning, and this wasn't a
transition that you were willing to accept.  My point is that meaning
and value (by any of a number of reasonable measures) are strongly
correlated in most networks, so there is frequently a strong enough
correspondence for one to apply to the other.

>     Ben> Enter headhunters who, through talking with everyone, try to
>     Ben> find some what potentially valuable connections that are not
>     Ben> being made and then try to make them.  Therefore the value of
>     Ben> headhunters is that they make connections happen that
>     Ben> otherwise would not.
>
> That's what I meant by "order statistic."  It's well know that the
> maximum (or second highest, which is important in economic
> applications) of a sample drawn from a given distribution will
> increase as sample size increases.

In particular the second highest becomes important when you're in a
negotiation situation.

>     Ben> About your "one match per person" comment, that ignores the
>     Ben> fact that employees are not necessarily loyal to employers,
>
> Yes.  That's the right thing to do in this context; look at the
> matching possibilities at an instant of time.  Note that the employer
> is automatically part of the employee's network, although not via the
> headhunter.
.
>     Ben> and further the fact that a given potential employee sees
>     Ben> value in having multiple potential offers to compare.
>
> No, the order statistic already accounts for that value, unless you're
> talking about a society like Japan in 1990 where having multiple
> offers gives bragging rights.

You're assuming here that the potential employee is very naive about
how to bargain.  (A safe assumption in many cases...)  If you're not
naive then two offers isn't just good for bragging rights.  It gives
you the opportunity to start going back and forth saying, "I am
considering this other job..."

In other words this is one of the cases where the value placed on you
by the second-best connection matters.

>     Ben> I was saying "second order" when I merely meant "relatively
>     Ben> small correction term".  In other words I'm saying, "Yes,
>     Ben> Google makes people use the Internet more, but that's not the
>     Ben> primary reason that it is so valuable."
>
> Aren't you embarrassed to admit that? :-)  Cheap shots aside, note

I'm confused about why I would be embarrassed here.

> that this is methodologically problematic when you're doing big O
> arithmetic, which implicitly assumes asymptotics.  Ie, what you're
> doing is saying "see all this cool math I can do?  Well, just forget
> that, because my intuition says the world is way way far from where
> that math matters.  Here's my best guess at what is true."  I value
> your best guess, but I'd rather have it supported by math, too. :-)

When you're talking big O, you get to ignore any term that is the same
size or smaller than the one you're focusing on.  I'm saying that I
think the effect of Google increasing how much time people spend on
the Internet is less important than its effect of enabling people to
find what they're looking for on the internet.  I admit that I don't
have quantitative results backing that up, but I know that for myself,
Google hasn't changed how much I use the internet.

>     Ben> I'm saying that I don't understand how the assumption that a
>     Ben> fixed fraction of minds are great and everyone else is worth
>     Ben> zero should lead to Reed's Law.
>
>     Ben> I *very* strongly suspect that it doesn't, and I'm suggesting
>     Ben> that you re-evaluate your reasoning for that point.
>
> No, you explicitly referred to the naive form and said that Reed's Law
> explicitly says every person adds that much value.  I pointed out that
> that's not true; as long as the distribution of those who add such
> value as a fraction of the population is constant as population
> increases, you can get Reed's Law.  You replied that this makes no
> sense because it means that the genius needs to connect with 2^(n-1)
> groups.  Sure, but Reed's Law works just as well as long as every
> subset has a resident genius, it doesn't need to be the same person
> for each subgroup.  At this point, you can reply that even with
> resident geniusness distributed evenly through the population you
> still need each genius to actively participate in an exponential
> number of groups, and I agree.  However, we're far enough away from
> ordinary intuition (at least mine!) at this point that I'll let you do
> the math; I'm not going to make another embarrassing off-the-cuff
> estimate.  :-)

Since the barrier is that people can only interact with a limited
number of subgroups, limiting the subgroups by saying, "Only the ones
with geniuses in them need apply" only puts off the limit to Reed's
law, it doesn't remove it.  Unless, that is, the geniuses are free
from ordinary human limits.

In any case I see why I didn't fill in this argument.

> My point is that it's very difficult to tell from your wording whether
> you're saying something based on a naive intuition, or whether you've
> done the math, at least roughly.  You don't need to show the math, I
> trust you to do it right (at least as often as I could, myself) and
> not to fudge the results.  But I do need some indication that it's
> been done, or I'm going to question it.  In many cases I'll want to
> see your assumptions and approximations anyway, because I've done my
> own math and gotten different answers.

Point.

>     > Yeah, well at least I was off in the direction such that
>     > correcting the math made the argument work better.  :-)
>
>     Ben> Kind of like how you hope to make 2 sign errors if you make
>     Ben> one? :-)
>
> In this case, yes; I just mistook the estimate badly.  But several
> fundamental ideas of economics depend on inherently biased estimates
> of that kind.  Eg, we know for sure that the average value of
> purchased goods is greater than the market price, but we (at least "we
> economists", but not necessarily limited to them) go ahead and call
> market price "value", anyway.

Which then makes it hard to figure out how to properly value something
like open source software. :-)

Cheers,
Ben