Subject: Re: rocket science
From: Tom Lord <>
Date: Tue, 1 Mar 2005 09:45:07 -0800 (PST)

   From: "Stephen J. Turnbull" <>

       Tom> Please consider the example of the computing systems that run
       Tom> financial markets and the rise of intelligent consumerism in
       Tom> the markets that purchase those systems.

   Quite apart from Ben's points about "how financial markets actually
   work", those are not desktop systems.  I was quite clearly talking
   about desktop systems, the topic you selected; you have changed the

I don't think I have changed the subject.  Here is why:

A computing system viewed as a whole has several components.
In the cases of programmed trading servers and power control
grids, one way to draw the component boundaries is:

    whole system ==    server engine
		    +  control/monitoring interface
		    +  human decision-making context

The server engine controls an external process (trading in one case,
power generation and routing in another).

The control/monitoring interface controls the server.

The humans control that interface. 

A class of RISKs to the whole system could be characterized by
determining the potential value (or cost) of an exploit (or failure)
of the server engine via the control/monitoring interface and
comparing that to cost (or probability) of a successful attack (or a
failure) of either the control/monitoring interface or the human
decision-making context.

For example, as I recall, if you search the RISKs list archives, you
can find a report of a power-plant control system critically monitored
from an off-the-shelf, Internet-connected Windows PC.  [I recall this
report from around the time of one of the recent big blackouts in the
U.S.]  An attacker or failure of such an interface can influence the
decisions made by the humans operating the grid.  Thus, the
grid-control software is made vulnerable by the desktop interface for
monitoring it.

It's a "where is the weakest link in the chain" question and desktop
software appears in nearly all chains -- close, by definition, to humans
making critical decisions.

There is a general rule of thumb there: the engineering goal is to
reduce the RISKs to just one: deliberate, well informed, well
understood, appropriately delegated, yet utterly stupid decisions by
humans.  Such bogus decisions are the only risk which engineering
can't mediate (they are the province of evolution and cultural
innovation).  Trace out computing systems until you reach those
decision points; secure everything on the way; refuse to build (and
consider helping to destroy) any computing system than can't be thus
reduced; insist upon and make every effort to build systems that can
be thus reduced and that "cool down" the trouble spots --- and you're
ok in my book.

So I look to markets like deep-financial[*] as drivers: premium
purchasers leading the suppliers where they need to go.

In the FSB context I look to those markets as a hint about where we
need to collectively go (since we collectively share the development
process) if we are to retain and increase our competitiveness rather
than to fall collectively into irrelevance and ruin.


[*] deep-financial 

    Not so much your bank's or trader's web site or the terminals in
    their retail offices.  More like: the ones that are a couple of
    orders of magnitude more expensive than a bloomberg terminal.