Subject: Re: mechanised documentation and my business model solution
From: "Ben Tilly" <btilly@gmail.com>
Date: Sat, 25 Mar 2006 22:33:44 -0800

 Sat, 25 Mar 2006 22:33:44 -0800
On 3/25/06, Rich Morin <rdm@cfcl.com> wrote:
> At 6:55 PM -0800 3/25/06, Ben Tilly wrote:
> > On 3/25/06, Rich Morin <rdm@cfcl.com> wrote:
> >> Many existing wikis are clearly unsuitable for this use.
> >> For example, any wiki that isn't based on a DBMS is going
> >> to be hard-pressed to handle versioning, etc.  However, I
> >> think (hope :-) that useful exceptions can be found/built.
> > [...]
> >
> > Do you really believe that?  If so, then how do you explain
> > the existence of useful source control systems out there
> > which are not based on a DBMS?
>
> I certainly believe that "Many existing wikis are clearly
> unsuitable for this use."  For example, a wiki that stores
> each page in a topical "web" as a file in a single directory
> is going to have problems with scaling, versioning, etc.  It
> is possible to overcome these by brute force, but the basic
> structure is still "unsuitable" for, say, Wikipedia.

I didn't use wikis as an example.  I used source control systems, of
which there are many, and which generally support versioning pretty
well.

> Now for the second point.  The use of a DBMS (specifically an
> RDBMS) has numerous benefits for a wiki, including:
>
>   *  It makes it easy to SELECT data, based on arbitrary keys.
>
>   *  It supports "transactions", which allow sets of changes
>      to be applied atomically.
>
>   *  It makes "back link" management trivial.
>
>   *  It can enforce consistent keys, data types, etc.
>
> Can these things be done without an RDBMS?  Certainly, but it
> requires some heavy lifting.  If the wiki isn't built on top
> of an RDBMS, the needed functionality will have to be added.
> This is a lot of work; "hard-pressed" was my shorthand way of
> describing the situation.

The use of an RDBMS brings many disadvantages as well.  Including:

* Tree-like data structures don't play well with a RDBMS.

* RDBMS are usually not set up to efficiently store many versions of
the same document.

Several of the wins that you list also seem to me to be wins for
different kinds of applications than wikis.  For instance wikis are
pretty much all text, so consistent keys, data types, etc aren't a big
factor.  (I'm also not a big strong typing fan, but that is a side
issue that doesn't belong on this list.)

> > ... I'm amazed that nobody has yet linked to
> > http://www.cabochon.com/~stevey/blog-rants/nonesuch-beast.html
> > (or said what it says).  Namely it is easy to say, "I want a
> > documentation [system] that makes it easy to do everything
> > that needs to be done."  It is impossible to build one.
>
> Agreed.  And thanks for the pointer.  Like any good rant, it
> contains elements of truth.  Unfortunately, like many rants,
> it sets up and conclusively demolishes a straw man.  I agree
> that I don't know how to create the "all things to all people"
> documentation system his users are requesting.  But then, my
> aspirations were never that lofty.

They seemed pretty close to me.  The fundamental documentation barrier
seems to me to be human, not technical.  Which means that its solution
needs to be cultural.  In the Perl world, for instance, it is standard
to get fairly decent documentation.  Even though the mechanism used to
generate it (namely POD) is pretty bad.

But no matter how bad POD is, it has one huge advantage over a fancy
wiki system.  Namely that the documentation comes with the program, so
when you read the documentation you know that you are seeing
documentation for the program that you actually have.

> > It is possible to come up with some use cases, and build a
> > documentation system that is better for those than existing
> > ones.  But any solution is going to have serious drawbacks
> > that someone is not going to like much.
>
> Probably so.  Indeed, any program will have bugs, limitations,
> and so forth, but that is no reason to stop trying to improve
> the current situation.  Building better documentation systems
> is not (like the halting problem) impossible; it's merely hard.
> If it weren't, it would (most likely) have been done already.

It has been done already.  Multiple times.  For a famous instance,
look up "literate programming".

However such systems fall into two general categories.  The first are
clear improvements that you take for granted and use as a base for
your expectations and desires.  (Consider the lowly FAQ.)  The second
are ones that solve problems that someone else thought was important,
but you don't.

> Any "interesting" problem sits on the line between "trivial"
> and "impossible".  I think this one fits that definition.  I
> also think that mechanically-augmented documentation systems,
> like bug reporting and source code control systems, are part
> of the infrastructure that developers should expect to have.
> Making that belief a reality is, of course, the challenge...

I'd find this vision far more plausible if you had some clearly laid
out use cases that you want your system to be good at.  If, looking at
those use cases, I think that your proposed system would be better for
my needs, I might want it.  If I find it isn't, we at least have
something concrete to argue about.

Looking at your description, my reaction is to not believe that your
goal is viable.  A few messages ago you summarized what you want as a
system where the developer adds a few comments for the documentation
system, everything gets tied together into a model, and then great
documentation is produced.  I look at that vision and I think that
programmers are going to have to learn a new (and in the end probably
fairly large) meta language to figure out how the meta language works,
describing the programming model for any interesting system seems like
a huge amount of work, and the gap between the different kinds of
documentation that different audiences need does not seem to me to be
possible to bridge in any automated kind of way.

Looking at your essays, I get the sense that your documentation system
looks to be so complex that one would need a domain expert just to
understand what it was trying to do.  And it looks like it needs a lot
of information to be input into the system so it has something to work
with - enough that you'd cripple any agile team that tried to use that
as a documentation mechanism.

There is a fundamental issue here.  The simpler any system is to use,
the more likely people are to use it.  The more process that you
require from people adding to the system, the less likely people are
to use it.  That's the genius of a wiki - it hits a sweet spot where
it provides just enough to be useful, but simply enough that people
really use it. Reading through your descriptions I get the sense of
something that would be great if people used - but which people
probably won't use.

Cheers,
Ben