Subject: Re: how to organize programmers
From: Bill White <bwhite@etnus.com>
Date: Thu, 23 Aug 2001 17:19:15 -0400

On Thu, Aug 23, 2001 at 03:56:54PM -0700, Tom Lord opined:
> 
> When programmers become dots on an org chart, or items in a budget,
> the people that manage org chats and budgets are treating them as
> abstract tools.  It is easy, in that circumstance, to make mistakes:
> for example, to abuse the programmers, or to organize programmers
> in an inefficient way that gives rise to poor engineering.
>               ...

I used to believe this kind of stuff, but I've changed my mind.
What you've suggested is in essence:
  o Developers
  o Maintenance
  o R&D
The problems with this are, in no particular order:
1.) The management will perceive Maintenance as less valuable,
    pay them less less, and try to work them harder.  It will
    be hard to recruit competent people to maintenance, and
    a lot of their time will be funnelled into supporting the
    marketing and field support teams.
2.) The management will perceive the R&D staff as the most
    valuable, pay them more, give them bigger offices and send
    them to tony conferences.  They will have a large
    amount of influence in decisions.  However, they will
    not be close enough to the Development organization to
    know what the Developers need.  Furthermore, in order to
    preserve their R&D cachet, they have to spend a lot of time
    theorizing in terms that the management will be impressed
    with.  Consequently, a lot of their recommendations will
    be nonsense.  By the time the Developers convince them
    that it's nonsense too many decisions will have been
    made, and all will have to live with it.
3.) At some point, each of the managers will realize:
      o She is in a competitive, free market, and her competitors
        are not other companies, but the managers in the other
        sections.
      o She owns something which the others cannot proceed without.
      o She can make the others look incompetent by withholding
        crucial support at the right moment.  This will create
        opportunities for her to step in in a public way, manage
        the ensuing crisis, save the day, and get very positive
        visibility from the upper management.  The very best
        way to do this in a SW organization is to own an interface,
        but not actually define it, or else to make frequent
        spelling changes in the operations.  Then people who
        need to use it are constantly thrown off balance, and
        need to spend time figuring out what's been changed.
    This point here is very important.  An employee's competition
    in any large organization are not other companies but the
    employee's coworkers.  As long as the entire company is put
    into rank order every year at review time, this will be
    true and an explicit if unstated part of managment policy.
    Only small companies truly work as teams.  Once a company
    gets big enough to have two middle managers, teamwork is
    not possible.  The really big companies I've worked for have
    been paralyzed paralyzed by this.  One graphics company
    successfully delayed the DirectX team for 6 months, by
    withholding the spec for communication between DirectX
    and the 2D driver.  This made the DirectX team (us)
    look like complete doofuses.  Of course, it helped them
    that the 2D driver team was the spec. owner, and was colocated
    with the upper management in Toronto, while the 3D
    driver team was located in Massachusetts.

These days, I'm thinking that the best way to organize software
is around the principle that we don't have much of any idea what
we are doing, and that any sort of design decision we make is
only an approximation of the right decision.  So, it's better to
make decisions early, find out why they are wrong, and make
better approximations.  The general strategy would be:
  o Form a kind of generally amorphous team.  This team
    will have areas of expertise, but everybody should be able
    to pull an oar in everybody else's area.
  o Get to code as quickly as possible.  You understand code
    in a way that is much more visceral than any kind of
    formal or informal specification.  You can run code.
  o Have lots of regression tests, and run them all the time.
    Make running the regression tests be part of the programmer's
    daily work.  Make writing regression tests and new test cases
    a large part of development.
  o Have more regression tests.
  o When you make mistakes, as you certainly will, fix them
    when you find them.  Don't be afraid of throwing away code,
    even code that you've spent a lot of time on, if it is not
    the right thing, or if there is a better way.  Fix design
    mistakes as vigorously as you fix programming mistakes.

The usual problem with the second bullet, Get to code as quickly
as possible, is that people tend to get to code before they
understand what the problem is, and then leave it as it is after
they understand the problem better, and understand that they have
screwed up.  This is especially true if the project has a sponsor
who pays in installments for different deliverables.  The management
doesn't want to say "We have to redesign the symbol table because
the chief scientist was a doofus, but he's much smarter now" when
they have already been given three quarters of a million dollars
and signed off on the symbol table.  Most especially if the
customer is the DOD, and they might be investigated by congress
if congress found out they hired a doofus team to design their
Ada compiler.
-- 
Just call me Svejk.
Bill White                 bill.white@grigggsinst.com
617-566-7461(Home)
617-699-8869(Cell)