Non-Profit Tech Blog

Those of you on the NOSI e-mail list are probably aware of the comments I’ve made concerning OSS CMSes. I made the following comment a couple days ago on the list talking about the OSS CMS Beauty Contest. I’m taking the time to answer here so that I don’t hog the list with my responses which tend to be fairly long. I’m really enjoying the back-and-forth despite the two flames I’ve received because I’m really learning more about how OSS consultants think than the merits of the case they’re arguing (and most of the time, very poorly I might add — note to developers: you do yourselves a disservice by flaming people who may well be your prospective client one day)

I’ve riled up a bit of criticism so after the jump is a post from the thread and at this point, I’ve already been flamed twice for my comments about Plone:

Wow, that’s the second flame directed at me in a month.

Put me down for the “He Who Most Resembles A Burnt Wooden Stick Award”…

And yes, I agree page views are not the only consideration for choosing CMSes but we have very few non-subjective criteria for OSS technology. In fact, I dare you to even try. And no, saying something is easier to code in or less complex to use or easier to maintain is all subjective until you start asking for objective proof like the number of deployments of that technology. We don’t have that number (yet) but we do have a handle of how many people are looking at those canonical sites.

I think choosing technology is more of an art and not at all like a science for the most part. This is because there’s a pretty serious lack of transparency in the non-profit world regarding the quality of any given software deployment. So given a paucity of data, we will have to turn to crude criterion until someone comes up with something better (which I fully expect they will). That graph shows that many more people are visiting the canonical sites for Joomla and Mambo than they are for Plone and Zope. It also shows that Drupal is becoming a big draw in its own universe. I think one can make inferences based on that data. Given that Plone and Zope have about an 18 month head start on Drupal and Joomla/Mambo, one would think that its page views would be higher but the graph indicates that acceptance of Plone/Zope is pretty much capped. Does it mean that Plone/Zope is inherently awful technology? No. Does it imply that for some reason X, many more developers are choosing some other platform? Probably, but the graph does not say what that X factor is. Does this have implications for those who are about to choose a new CMS for their non-profit? Of course it does. A larger and more popular (in terms of developer support) CMS will help keep down costs by giving an IT director more vendors to choose from.

Joel Burton responds:

On Wed, Mar 29, 2006 at 07:20:52PM -0500, Allan Benamer wrote:
> Wow, that’s the second flame directed at me in a month.
>
>
>
> Put me down for the “He Who Most Resembles A Burnt Wooden Stick
> Award”…
>
>
>
> And yes, I agree page views are not the only consideration for choosing
> CMSes but we have very few non-subjective criteria for OSS technology.
> In fact, I dare you to even try. And no, saying something is easier to
> code in or less complex to use or easier to maintain is all subjective
> until you start asking for objective proof like the number of
> deployments of that technology. We don’t have that number (yet) but we
> do have a handle of how many people are looking at those canonical
> sites.

A criticism of a pretty silly metric isn’t a flame, btw.

You can argue “but page views are better than *nothing*”,
but that’s a strawman argument: there’s much better out there than
nothing for comparing CMSes and figuring out the right one
for your organization.

There are case studies; white papers by consulting/anaylsis
firms, etc., which give much better answers to “what product is
right for what purpose”, not “what technology is most popular”?
Optaros, for instance, recently released a intelligent,
interesting white paper that compared the relative strengths and
featuresets of open source CMSs; that’s likely to help you
understand which one is right for *you*, not which one is
“best”–a purely silly notion, that one is right for everybody.
How many books are there? what kind of training is there?
Do the consultants you know and trust recommend it? These are
much better questions for most small/medium-sized firms that
popularity, even if you could measure that well.

Plone, for instance, is a significantly more complex, more flexible,
and more powerful product (uh, huh, yep, I’m biased). It has a
bigger learning curve, and sits on top of an enterprise-scaling
application server. It’s probably not right for the small
organization with an “accidental technologist”; and *that’s fine*. It
has other strengths, for larger or more complex sites. It has
different ideas about what content is, and how to manage it, than
Drupal or Joomla (or RedDot or Sharepoint, which tend to be our
competitors, anyway) do.

Also, page hits (especially via an external thing!) can be
tricky metrics: Plone doesn’t have forums on our site where
the user support takes place; we have our forums in email (which
are available via the web, but not at plone.org domain). So, comparing
domain page hits as “popularity” may, to a large extent, be comparing
apples and oranges–people downloading Plone & add-on products
vs. people talking in forums on other sites.

This list just had a kinda-helpful, kinda-flamey argument
about open source CMSes; I’m not hoping to start another one.

But, having just been a presenter at the Penguin Day in
Seattle, and hearing all the participants ask “which is the
*right CMS* for nonprofits?” puzzles me: nonprofits are not a
category of needs; they’re a tax status Some nonprofits
need very simple, small stacks. Some need lots of scaling
and flexibility. Some want to publish smart objects and
some want to public semi-static data, etc.

I hope this helps redirect the conversation somewhere useful

Cheers!

- j.

Joel Burton
Independent Zope/Plone/Knowledge Management Consultant
Chair of the Board, Plone Foundation

Joel makes a fairly standard argument here, that non-profit IT Directors, when choosing an open-source CMS should do the following:

1. Read white papers (usually by other consultants) and he points out the Optaros white papers in particular
2. Find out how many books there are about the technology
3. See what training is available
4. And lastly, what do your trusted consultants say about the subject?

Aside from the fact that these are fairly common practices for IT directors, let’s go down the list.

Points 1 and 4 are pretty much the same point. Consult the consultant white papers and the consultants themselves. It’s a good starting point but every IT director knows that consultants will always try to push the technology they’re more comfortable with. It would be better to look at more vendor-neutral or technology-neutral sites such as http://www.opensourcecms.com/. Also, those Optaros white papers are also grappling with analytical tools for assessing developer interest in particular OSS technologies and they also head in the same direction that I’m headed. Sooner or later, someone (perhaps me or Aspirationtech) will build a tool that will use site statistics as part of a metric for assessing adoption. O’Reilly

Point 2 is an interesting metric.This metric rides on the ability of publishers to accurately gauge the interest of developers in a particular technology. In fact, O’Reilly thinks this isn’t a bad metric at all. However, it’s clear that the O’Reilly metric is actually not tied to number of titles but to the sales volume for each particular volume. Also, one should expect a lag between publishing dates and actual interest in the title. There is also a secondary lag between new releases of the software and current books. Only books created in the last 2 years should be considered “usable” by web technologists because any particular CMS will probably have 1 or 2 major releases in that time. Amazon comes up with the following numbers for these technologies:

1. Plone 8 books (4 books published since 2004)
2. Zope 13 books (4 books published since 2004))
3. Joomla 3 books (all 3 published in 2006, 2 of which will be released later this year))
4. Drupal (1 book published in 2005 but it’s combined with phpBB and WordPress so it’s really 1/3 of a book))
5. Mambo 4 books (all 4 published since 2005))

I think that Mr. Burton, by suggesting this metric thought that Plone/Zope would benefit from the numerical comparison between Mambo/Joomla and Drupal. In fact, this was perhaps one of the most negative metrics Mr. Burton could have mentioned about Plone/Zope. According to Amazon, there are no books being published about Plone OR Zope this year. I also took a look at sales volume for titles about Plone or Zope vs. Joomla and Drupal. Sales rankings for Plone/Zope books are consistently low. One book, The Definitive Guide to Joomla!, isn’t even published yet (release date is August of this year) and it has HIGHER sales than it’s cousin, The Definitive Guide to Plone, which is published by the exact same publisher and with the exact same cover design. I knew Plone or Zope isn’t widely adopted by the Web community but I didn’t understand how low that adoption rate has dropped until I began to consider Mr. Burton’s comments about the books available to learn about Plone or Zope.

I think point 4 is the strongest point that Mr. Burton makes. It’s also one that IT directors follow as a matter of course.

I suspect that every non-profit IT director practices Burton’s method of due diligence as a matter of course. However, there is NO transparency regarding deployments of open-source technology in the non-profit sector. Since when has anyone heard of a nonprofit technology failure? Any case studies out there? A quick Google search turns up a very good article on this aspect of non-profit IT relations to OSS by Deborah Elizabeth Finn called Failure-Friendliness: An unexpected virtue in nonprofit technology. While we “know” of nonprofit IT failures (raising hand sheepishly here) who the heck wants to advertise them? Neither the nonprofits nor the consultants will do this.

Despite having followed all of Joel’s recommendations, Joe Q. Director is stuck with the following problems:

1. No metric of success extant for open-source deployments in the nonprofit sector
2. If a nonprofit IT director is choosing his or her FIRST OSS CMS consultant, there is NO trusted relationship and let me tell you, asking your current desktop support vendor will probably lead you down the wrong road
3. Issues of future sustainability are left unaddressed — will adoption of technology X result in a vicious cycle of consultant dependency for the nonprofit? And is this technology a dead-end in comparison with more widely adopted technologies?

There is also the 800-lb gorilla in the room — and that is, consultants and nonprofit IT directors are necessarily at odds over the choice of technology because neither population has the same agenda. Deborah Finn alludes to this when she points out the following:

Nonprofit workers need to understand how technology innovations and implementations happen in real life, and have a reasonable idea of what factors can lead to unexpected outcomes in technology projects. Progress is possible if – yes, you guessed it – the nonprofit workers listen, ask questions, and listen some more.

Well, I’m asking these questions of the OSS consultant community. Without transparency not on only on the part of nonprofits but OSS vendors, we do NOT have good metrics. Mr. Burton’s comments are well-meaning but somewhat self-serving. A better response was not to challenge the statistical analysis as presented by the Alexaholic chart but to present better and more concrete evidence of site traffic on plone.org I urge people like yourself, Mr. Burton, to be more transparent. Show people the site statistics for plone.org. We can ask the same for all the other open-source technologies. As an aside, according to Alexa, 32% of joomla.org’s site is directed to forums.joomla.org. Even if we accounted for visits to Joomla.org’s forum, it would STILL be substantially higher than traffic for plone.org’s site.