Subject: Re: is there a statistician in the house? (long)
From: "Marshall W. Van Alstyne" <marshall@MIT.EDU>
Date: Fri, 11 Mar 2005 01:56:16 -0500

At 02:25 PM 3/10/2005, Seth Gordon wrote:
>Now, if there were some way to *transfer reputation* from one person to 
>another, then the effects of (1) could be diluted.  If, for example, my 
>mother told me that a certain piece of free software made her life much 
>much easier and she regarded its author very highly, then even if I 
>personally had no use for that software, I would be favorably disposed 
>towards its author.  How can that transfer of reputation be formalized?
>
>With that introduction, I present The Kindness Of Strangers Game.

OK, there's a ton of interesting stuff here.  Let me try to reflect on 3 
points.

1) Reputation ranking scheme:  The iterative process described in the 
Kindness of Strangers Game already has a solution without the use of 
statistics.  The parallel concept comes from social network analysis.  In 
that context, the problem analog is to determine who is more prestigious 
based on "who talks to whom" instead of who rates whom highly.

The mathematical solution is called the "prestige rank index" and it 
depends on eigenvalues from a structural equation model not 
Kruskal-Wallis.  Define an N x N matrix of individuals in which elements 
represent the amount of communication from row person to column 
person.  The iterations of the game would come close to approximating the 
eigenvalues of this matrix, which can be used to give the "rank" or each 
player.

An advantage of the matrix formulation is that it does correct for one of 
the problems Karsten identified, namely you want to weight the rank of the 
person doing the ranking.  In the social network context, your rank rises 
with the rank of the people who want to talk to you.

Details can be found in "Social Network Analysis" by Wasserman & Faust. An 
implementation is available in the package UCINET.

2) Multi-attribute reputations -- an analog for this problem is 
multi-attribute utility rankings of goods in consumer economics.  In cars, 
you might value reliability while I might value speed (OK, for code we'd 
probably value both :) so the question is, in what sense can one car be 
said to be uniformly better than another?

In general, the answer is that you can't.  Different scenarios can render 
one or another product feature more critical.  This does not mean, however, 
that goods can never be sorted.  In M-dimensional space, use the highest 
scoring feature of each instance to define the efficient frontier.  Then 
goods farther out on a given convex hull will generally be preferred by 
consumers to goods inside the convex hull.

In the prestige rank index, this could be approximated by having M layers 
for each of the NxN communication links.  Of the N players, who talks to 
whom on each of the M topics.  Yeah, Bill's the guy for music but Ted's the 
guy for cars.

For consumer products, the original insight came from Lancaster, K. (1966). 
A New Approach to Consumer Theory. Journal of Political Economy, 74(2), 
132-157.

3) Gaming reputations.  The bad news is that this will always be a 
problem.  In fact, there's a nice theorem in info econ and mechanism design 
that states it is impossible so simultaneously achieve (i) truthful 
revelation of preferences (ii) voluntary participation of all players (iii) 
robustness to coalitions who seek to subvert the mechanism and (vi) budget 
balance (roughly meaning that if you want the truth, you may have to 
subsidize it in ways that generate losses).  I forget the details but 
they're available under "revelation mechanisms."

A very nice and recent summary of ways to manage the flaws in e-bay like 
reputation mechanisms can be found in Dellarocas, "The Digitization of 
Word-of-Mouth: Promise and Challenges of Online Reputation Mechanisms". 
Management Science, October 2003

Cheers,
MVA