Subject: Re: Mechanizing community information resources
From: "Karsten M. Self" <>
Date: Fri, 20 Sep 2002 22:50:00 -0700
Fri, 20 Sep 2002 22:50:00 -0700
on Thu, Sep 19, 2002, Rich Morin ( wrote:

> The wiki discussion is OT to "Open Source shareware?", but I think it
> is interesting and (in the long term) important to FSBs.  So, I've put
> a new subject line on this response...
> At 1:49 AM -0700 9/19/02, Karsten M. Self wrote:
> >That said, as of earlier this year, I've launched, and am now
> >building a technically oriented, documentation-focused TWiki at
> >  Whether it takes off, or merely
> >encourages similar activities at LDP or iBiblio ultimately doesn't
> >matter.  What I'm trying to promote here is the process (though good
> >content to feed to whatever successor emerges won't hurt).
> There is a close relationship, as observed by Dr. Donald Knuth,
> between sorting and searching.  Simply put, sorting can reduce
> searching time.  In addition, turning raw text (e.g., usenet chatter)
> into structured information (e.g., wikis) can reduce ambiguity, noise,
> and errors.
> Consequently, the "librarians" among us spend time developing indexes,
> taxonomies, and other ways to make information "clean and organized".
> This takes human effort, however, so there are limits to how much of
> this can be performed.


Where Wiki wins is in allowing distribution of this effort -- load
balancing among many individuals.  Projects such as dmoz and Wikipedia
are among the more spectacularly successful.

I'll also note that TWiki in particular focuses on being a  corporate 
shared information resource.  Taking it to the web as a whole is a bit
of a stretch.  I think the fit, within a defined community, is

> Wikis and other collaborative systems can help in creating structured
> information.  There is a certain discipline to a well-run wiki; topics
> are discussed in localized areas, rather than splattered over a series
> of inter-related "messages".

Funny that.  Recently had to do some pruning (work in progress) of
growth into areas I don't feel are particularly fruitful.  Largely an
outgrowth of having adopted a topic structure based on the originating
organization.  Not quite the right fit, but a good first approximation.

> I have some problems with wikis, however.  Aside from the ugly naming
> system (e.g., ThisOrThatTopic) 

The naming convention can be awkward, and for certain uses, is
Considered Harmful.  There are ways around it -- TWiki lets you create
topics (and links) with arbitrary names.  However you lose some of the
automated power of Wiki, particularly in the searching and automated
page features.

This is one place where the Everything ( vibe has an

> and rather peculiar user interface for editing (actually, interfaces;
> each wiki has its own!), 

I have less a quibble with this (find "edit" link, edit, preview, post)
than the fact that web browser's built-in editors simply suck for more
than a few lines of content.  I actually prefer hitting a Wiki with w3m
or lynx, which allows me to shell out to a Real Editor.  It's also
possible to snarf'n'barf between a browser and a preferred editor, but
this can become tedious.

> there is the simple fact that I have to go TO the wiki to use it.  

In what sense?

There's actually a growing set of bidirectional content syndication.
This is IMO one of the more exciting features of Wikis and other P2P

At TWikIWeThey we have:

  - RSS -- content syndication.  Grab your feed at

  - Incoming headlines.  We use an RSS headline grabber to populate
    dynamic front-page content.

  - "InterWikis" -- a TWiki plugin which allows snarfing of specified
    content from an origin point.  This includes numerous other Wikis,
    Google, GoogleGroups, IETF RFCs, the Jargon file, and Powells books.
    Content may be either linked to or included in a page.

  - Sunir Shah's Meatball Wiki has taken this one step further with the
    MetaWiki search.  This is a search across a range of Wikis, though
    the primary search is merely on topic names.

Because a typical Wiki page is merely a URL prefix with a WikiWord
attached to it, it's possible to grab content with a pretty simplistic
utility (such as InterWikis).

As Rich noted, the editing interface varies.  For TWiki, however, the
current action is specified as part of the pathname:


...and a post method to commit changes.  This could readily be
incorporated into a local app.

> In addition, the discussion has no pre-existing structure; if I have a
> new topic, I have to dream up names for my topic and sub-topics, find
> places to link them into the rest of the wiki, etc.

It's worse than that.  There really *is* no structure.  The structure
that exists is via interrelationships between pages, references,
backtracking, and searching.

To paraphrase a college friend who was explaining why she wasn't in a
"relationship" at present:  TWiki transcends structure.

Again, Everything has done some really interesting work in this regard
as well.  What I dislike is the closed-web view of E2 (this dates from a
couple of years ago, I'm not sure if it still exists).  This is one of
the prime reasons I've picked up TWiki instead for both public use and
use at my employer.

> So, I have been musing about ways to integrate wiki-like interaction
> into a "browser" that actually sits on my own system.  

All browsers sit on your system.  The datastream is distributed.

For interactively syndicated and distributed file locking, I'd start by
looking at replacing the underlying RCS versioning engine of TWiki with
something similar to Bitkeeper.  This includes both distributed logic
and cache coherence, which are key to the problem.  I'd say that the
TWiki interface isn't the issue, ultimately it's just an input method.

When Rich was pursuing his Meta project more actively, I pointed out
that a virtual filesystem could be used as the local interface to such a
distributed documentation system.  File access methods -- read, write,
create, delete -- would be translated into the appropriate actions for
the distributed conent, and subject to externally imposed access
limitations.  In this case, Rich's comments on organization likely would
come into play, unless the repository was treated as a flat space onto
which an arbitrary mapping could be applied.

> Let's say that I have used this browser to look over the information
> available on the /etc/magic file.  If I realize that the information
> is incomplete or erroneous, I should be able to enter a comment or
> question right there.  If desired, I could limit the distribution of
> the note (e.g., personal, group, machine, site, developers).

...more or less as I've suggested above.

> In the FreeBSD Browser, I used a combination of mechanically-harvested
> information (e.g., FILES, and SEE ALSO references, file system links)
> human annotations ("this log file, if present, relates to that
> daemon") and rules ("files in these directories tend to have man pages
> in these other directories").  I would propose similar functionality
> for a new system, but allow (cajole?) users into adding their own
> information.
> In summary, I think that we need an integrated system to access OS and
> related information (including documentation and metadata).  It should
> provide ways for users to make (and, if desired, publish) notes on the
> material they see.  It should be distributed and highly mechanized,
> linking relevant information, bi-directionally and in a transitive
> manner, to related topics.

Wiki is most likely orthogonal to this ultimate end.  Wiki is one form
of distributed documentation; its scope is document management.
Specific implementations could support much of the functionality Rich
describes.  The local file / filesystem interface would supply mechanism
for browsing this repository (or repositories).  Specifying browsing
targets to me sounds like something akin to specifying package
repositories under Debian.   Relationships between data ultimately
becomes a class of data itself, subject to a different (higher) level of


Karsten M. Self <>
 What Part of "Gestalt" don't you understand?
   Hollings:  bought, paid for, but couldn't deliver the CBDTPA:

["application/pgp-signature" not shown]