Subject: Re: Performance metrics of Open Source software.
From: "William C. Cheng" <william@cs.umd.edu>
Date: Tue, 21 Sep 1999 07:40:07 -0400

Lacal Jose <Jose.Lacal@icn.siemens.de> writes:
  > 
  > During my by-now infamous "Open Source software is good for Siemens"
  > speeches, I am constantly being asked about the performance of Open Source
  > software vs. that of commercial packages.

Um...  For this list, Open Source software is commercial software.
So, your management probably mean software from companies that can't
get them fired if they buy from them. :-)

  > That is, is there any "official" set of measurements performed by a
  > "recognized" institution (meaning, something upper management here will
  > believe) that has compared the performance of the following software vs. its
  > commercial counterparts? Has anybody compiled such info somewhere?
  > 
  > - sendmail
  > - Apache
  > - WU-FTP
  > - squid
  > - INN
  > 
  > I know, I always say that "Apache runs 65% of all web sites, sendmail
  > transfers 70% of all e-mail, etc." but, still, management here needs some
  > hard facts. "IDC, in their latest benchmark studies, demonstrated that
  > Apache is ... faster than ..., and able to handle ...% more connections per
  > minute than ..." You get the picture.
  > 
  > It takes a lot of paperwork to convince a large company such as Siemens that
  > Open Source software should be officially supported. I keep on trying,
  > tough.

These are fare questions.  But running performance tests are very
expensive (especially when it comes to tweeking things to perform
well for benchmarks).  Recently, I was told about a performance
bug for a product from one of these large companies.  The only way
I can explain the bug is that it was tweeked for benchmarks and the
engineers couldn't really get it to perform well under "real" conditions!

On the other hand, I would think that IBM should be performance data
on Apache (and some FTP servers) since IBM is using it on commercial
products (although I don't know if IBM is selling software or just
selling services with it).
--
Bill Cheng // bill.cheng@acm.org <URL:http://bourbon.cs.umd.edu:8001/william/>