I found a very interesting blog post by Joe Gregorio entitled AtomPub is a failure. Well of course the title is there to get your attention, but the underlying argument is interesting : we now have other options for interchange formats.

We definitely live in interesting times, where so much focus on ease of integration is moving interoperability a *lot* faster. This can be a problem for standards that are trying to keep up, and mostly why I strongly believe that it can be a good thing implementation can drive specification like in the case of Apache Chemistry, the Apache project to build a strong CMIS implementation. Standards are very important, but in the end people also want and need strong reference implementations of those standards. After all, even HTTP was specified together with it's first implementation :)

JSON is incredibly powerful because it is so simple. I can explain it to a junior developer in a few minutes. I have trouble discussing XML formats with senior developers. Sure JSON is not as complete, but it also serves a lot of purposes. Don't get me wrong, I'm not saying that we must get rid of XML, it is here to stay and quite useful, but I'm mostly talking about simplicity.

Another aspect of JSON that most tend to ignore is that with it's simplicity also comes it's efficiency. It is very hard to find another interchange format that is more compact, and easier to parse. In web systems that are constantly trying to handle bigger loads, the cost of XML processing can be offset by using JSON.

A really hope that the CMIS committee will approve the upcoming proposal to introduce JSON as a binding, even if it means in a first version to use it as an AtomPub interchange format, but I would much prefer that they go the whole way of making it an extremely simple binding that can be implemented by many in very little time. They have so far had this focus, so I'm pretty sure they are interested in this idea.

Oh and yes I am thinking about joining the OASIS TC :)

The Apache Chemistry project, the incubator project that was just approved has an incredible potential. Started as a place to experiment with a Java implementation of the CMIS specification, it can become much more. There are already implementations out there in Javascript and C++, although not yet contributed to the project, but this might happen sooner than you think.

The hardest thing to do is to achieve in any standard is true interoperability. But what if everyone was using the same code base from the Apache Foundation, freely available to businesses ? What if all this code was fully tested by automated integration test matrixes ? Even achieving this is a challenge in itself but it is possible, and I really think it should happen. My dream, although probably unrealistic is that even major vendors such as Microsoft, IBM & EMC could use the code developed in Chemistry as the basis for their CMIS implementation, and contribute back to the project whenever they see problems.

A lot of what is happening with CMIS is reminiscent of the SOAP craze. After all SOAP is used as a binding for CMIS, so it is quite normal to see similarities. One of the biggest interoperability problems for SOAP lied in the potentially complex serialization of custom objects, and this proved in real life to be very difficult to get to work between implementations. It is a testament to the great work of the contributors to the Apache projects related to SOAP implementations (Axis, and before that Apache SOAP) that you could get web services to really talk to each other.

The Apache foundation is the perfect place for such standards to be implemented and grow as the basis for the network infrastructure of the entire industry. It is a place where even competing interests can find common ground for sharing development costs.

Of course it's not necessarily easy to directly use Apache-licensed code inside corporations such as Microsoft or IBM, as they are very concerned about code auditing, especially as they are often the target of copyright infringement lawsuits, but at least IBM is known to use such code, and in some cases simply packaging Apache products (such as the IBM HTTP Server). So we know that even if it is not trivial to achieve it is possible. And for Microsoft, well maybe this will convince you ? http://port25.technet.com/archive/2008/07/25/oscon2008.aspx

I'm dreaming of an Apache Chemistry project with the following implementations available to all : Java, PHP, C#, C++, Javascript. Then of course you could have more such as Ruby, Python or whatever else you love, but the initial list would be perfect for integration with most systems, and provide truly interoperable systems, not just at the specification level, but truly at the implementation level.

Maybe it's just a pipe dream, but it is possible, so maybe we should get together and try ?

The second day of the CMIS PlugFest was less about setting things up than actually trying to fill the grid of possible tests (over 28 possible tests) that cover all the combinations of client and server implementations that were available there.

I was myself hacking most of the day to get a brand new CMIS client implementation up and running, and I got the first unit tests working against both the Alfresco CMIS server and the Day CRX server. I even managed to navigate through the whole repositories. I then went on to work on the integration with Jahia, to be able to browse those repositories directly from Jahia's file manager. Unfortunately, despite the amount of code that was produced in the last 24 hours, it just wasn't possible. Despite this I believe this is really close to happening, and we should have something experimental to test against the various servers quite soon.

The good news of the day came from Jukka Zitting that was visiting the PlugFest, and that announced that Apache Chemistry was officially accepted by the Apache Foundation as an incubator project, and so over the course of the week-end Florent Guillaume, from Nuxeo, who contributed most of the code for Chemistry, will finally be able to commit all this work and let the community hack away at it.

We were also joined during the day by IBM who was also providing a server against which to test. We had very interesting discussions about what was needed in the CMIS specification. A few of us believed that CMIS should really make usage of JSON, and it seems that a proposal for going in that direction might just happen, but it must happen fast, as there is a lot of pressure to get CMIS 1.0 out the door soon.

The idea behind using JSON with CMIS is quite a natural one. Why have to deal with all the XML parsing involved in Atom Pub and Web services when you could use the extremely simple JSON format to exchange all the data you need ? I really hope that this will be added to CMIS as this could make it one of it's most flexible bindings. It is much simple to develop lightweight CMIS clients without the need for XML parsing. I'm mostly thinking of PHP, Javascript and other technologies.

In the interoperability tests, search was still a problem for many clients and servers, and this was of course expected as it is the most complex part to implement. This also means that it will be necessary in the near future to perform another interoperability PlugFest to go further into the testing and make sure that even search works properly.

The day was concluded with demonstrations of a few combinations of client and servers. Among the demonstrated clients were : Chemistry Javascript against Day CRX + Chemistry, SourceSense's CMIS Portlet against Alfresco, the CMIS Flex Explorer against Alfresco (which was also the client that worked against most of the servers !), OpenText's C++/Java client that integrates with Windows Explorer, Outlook and MS Word, and last but not least SAP's client that plugs into an existing infrastructure of explorer tools that plug into various SAP backend repositories (CMIS or not) and also exposes all the various back ends as WebDAV resources. This made me smile because there has been a lot of discussion wether CMIS should have a WebDAV binding, and basically SAP was demonstrating WebDAV on top of CMIS :)

During the demonstrations, a lot of discussions were focused on making sure that the points that were problematic to implementors were noted and then discussed in the OASIS TC. Among those points were the upload mechanism of binary data that is not very clear when using the Atom Pub binding, because there are two ways of doing this, and therefore this is confusing to client implementors. The second point concerned path navigation, and the fact that CMIS currently doesn't offer a way to lookup a content object by it's path. Currently, when given a path, the CMIS client implementations must navigate from the parent path down to all the descendants to find the object corresponding to the path, which is not very speed efficient.

It will be very interesting to see if these points can be adressed efficiently to avoid side tracking the specification. It seems that major vendors such as IBM, Microsoft and EMC want CMIS to be completed in Q3 2009, so this doesn't leave much time to solve issues.

All in all, I'd like to thank Day for hosting this PlugFest. This event, although not too certain how it would happen initially, turned out to be really necessary, and gave a really good picture of the real state of the CMIS interoperability. It is certain that this must happen again, when implementations are more mature, and especially when Apache Chemistry is fully available and ready to be tested. I would also like to thank Dave Caruana (Alfresco), who has been really great, helping me out with getting my tests up and running.

Day 1 started with some informal introductions, and setup of the existing servers. We now have OpenText, Alfresco and Jackrabbit+Chemistry servers up and running, and are running interoperability tests against those using a variety of clients, including OpenText that has a C++ plugin that connects Windows Explorer and as well as Office tools to CMIS back ends, SAP, Alfresco test units, two Flex-based clients (CMIS Explorer and CMIS Spaces), a Apache Chemistry CMIS Javascript client written by David Nuescheler at Day, an Apache Chemistry Java client that Florent Guillaume (Nuxeo software) is working hard on to commit hopefully before the end of the week.

It's really great to see so much effort going into interoperability tests. The most interesting thing about CMIS is the momentum behind it, more than the technology, that will probably still evolve over the year. It is also very important to get the first version of the specification out as early as possible, because so many specifications fall into tech-limbo, to never be completed (802.11n, etc...).

Apache Chemistry is also looking better than ever, having just been accepted into the incubator at Apache, and the code will probably be committed over the week-end. From then on hopefully the community will be able to have a look at it. This will also help interoperability, as even major vendors could use this code base to ensure compatibility.