Response to the META Group White Paper, “Messaging Total Cost of Ownership: Microsoft Exchange 2003 and Lotus Domino in Small and Medium Organizations”

I was surprised to see another poorly done white paper on Exchange vs. Domino so close to the Radicati fiasco of last week. However, this one is from The META Group (www.metagroup.com), which has traditionally published very good work. I’m greatly surprised at the lack of thoroughness and due diligence in preparing this paper. It is not available on the META Group web site (so it is probably not written by one of the META analysts, eg, Matt Cain), but is available from Microsoft’s competitive intelligence site on Exchange, at www.microsoft.com/exchange/evaluation/compare/METAEx2k3vNotes.asp.

Why am I writing this?

• It has always been difficult, if not impossible, for an analyst to compare Microsoft Exchange and Lotus Domino.


• Comparative TCO studies are very important, but are hard to do well. Many different items across the two comparative products need to be looked at the same. For the comparisons to be valid, the author must apply the same standards and use a consistent approach with both products.


• The META group has a good reputation in the market, so its work carries weight.


• The author has prefaced the report with specifically chosen words about how careful he or she has been. A casual reader would take that at face value.


• I believe there are fundamental flaws in this TCO study, and they should be taken into consideration when reading it.

Here’s the points at which I think the author has displayed a “lack of thoroughness and due diligence”:

Comments

[Nathan T. Freeman]

Actually, Michael, the biggest thing I found myself asking when reading the report was...

Given that analysts claim that spam and email worms cost as much as $800/employee/year, (4 times the largest figure mentioned in the report) how can any TCO study NOT include the impact. There's no question as to the differences that email worms affect different platforms differently, nor to the differences in spam/virus-protection that the platforms offer. How is this not included? This is a classic example of analysts who don't even work in the real world.

[Michael Sampson]

Nathan, yes, there definitely seems like a mis-match here. p3 of the META report says that spam and virus filtering are part of the "Infrastructure Support" piece, it's mentioned on p7 under Hardware Servers, and on p8 under Software. Also, on p11 under Maintenance tasks ("Filtering and Forensics"). So, why was the financial impact of this so low?