Subject: Re: how to organize programmers
From: Bill White <bwhite@etnus.com>
Date: Fri, 24 Aug 2001 12:29:37 -0400

On Thu, Aug 23, 2001 at 07:31:02PM -0700, Tom Lord opined:
> 	Bill White writes:
> 	[a very funny rant ... with some good points mixed in]
Well, it wasn't meant as a funny rant with some good points
mixed in.  It was meant to be read literally and seriously.

>                                        I agree with you that, as
> a first approximation, as a group, we don't know what we're doing
> w.r.t. to software.
Not what I meant to say.  I think we know a lot about what
we're doing w.r.t. software.  I think we don't know much about
what our software is supposed to be doing, and that's a big
difference.  Let me give you two examples of what I mean.
 o I recently wrote a library to parse ISO8601 dates.  You
   would think it's easy, but it's not.  There are a bunch of
   abbreviations for different formats.  It took me a couple
   of days to get it right, and to get all of the regression
   tests written and working.  Unfortunately, due to brain
   cramp or perhaps alcohol abuse, I accidentally deleted
   all of it before I committed it to CVS.  But when I
   recreated it from memory, it only took a couple hours.
   I had solved all the hard problems, and the only thing
   I needed to do was recall the solutions from memory.
   I hadn't become an order of magnitude better programmer,
   I just knew what the problem was, and could solve it
   directly.
 o I currently work for a company that makes debuggers.
   I'm responsible for rewriting the symbol table.  I have
   worked on compilers before, and designed symbol tables
   for compilers, and the debugger's symbol table is just
   stuff from the compiler, so it should be close to the
   same thing, right?  Well, it turns out that all my compiler
   intuitions are wrong.  For example:
   - The compiler has essentially perfect information.  It
     has all of the program, and it gets to decide where
     everything goes.  
   - The debugger, however, gets information from the compiler,
     which sometimes lies, and sometimes talks in riddles.
     It sometimes gives us different inconsistent definitions
     of the same thing.  You can't say "The compiler is not
     right."  We don't really have any influence on the
     compiler group, which is either at another company, or,
     in the case of gcc, generally hostile to us anyway.
   - The compiler needs to process its input at computer
     speed.  The data structures have to be arranged so that
     it can process as much source text as rapidly as possible.
     The debugger has to operate at interactive human speeds.
     So, you can't build a big table hoping that it will
     speed things up by milliseconds later on, since you
     generally don't care about milliseconds, but you do
     care about how fast the debugger starts up.
     Furthermore, you may only make a few to a few dozen
     accesses to your table anyway.  The user is going to
     make a lot of access to a small number of variables
     and functions, whereas the compiler is going to make
     a small number of accesses to every object.
  All the hard thinking about data structures for the compiler
  gives you wrong answers for the debugger.

> In that context, "R&D" teams are focused on software basis sets -- not
> on a search for miraculous easy plays that can be pushed through
> otherwise broken development organizations.  I have a lot to say about
> software basis sets, but this isn't the thread for that.
I think your R&D end up being pulled from the basis set construction,
and set to putting out fires.  That's what usually happens, whether
it should or not.  In any case, how do we know that there even
are basis sets for software projects, or what they might be?
Remember, a basis in Linear Algebra is a different kind of thing
than a basis (of neighborhoods) in Topology.  If you try to
apply the first to the second, nothing good will come of it.

> The rules of the competition are determined largely by the rules of
> compensation and advancement (implicit or explicit, real or presumed).
> It isn't inevitable that employees will in-fight to the detriment of
> the larger organization, or the customer.
Sorry, but I disagree.  It is inevitable, as long as the
organization needs to evaluate their employees performance.

-- 
Just call me Svejk.
Bill White                 bill.white@grigggsinst.com
617-566-7461(Home)
617-699-8869(Cell)