Tuesday, April 24, 2007

Croquet and the Perils of Open Source

Croquet is Open Source. This has had both a positive impact on the project and in some ways a decidedly negative impact. The positive is the positives that most open source projects enjoy - lots of interest and creativity building around the system, visibility, strong academic relationships - it even helps allay concerns of larger enterprise customers knowing that there will not be a Microsoft-type lock-in with a single vender.

From the start Croquet was developed completely out in the open. Even the very first, barely useful versions of the system were readily available to download, deconstruct, and criticize. Nothing wrong with this - we invited it, and the project profited from it. The big problem we had to deal with is people were disappointed with how much of the system was or wasn't working. I have heard people say "Yeah, I tried Croquet a few years back, but it.... " (you can fill in the blank with a number of choices). I can understand why people would be irritated. Some critical problems in Croquet took much longer for us to solve than we ever thought they would, we had spent a minimal amount of time on UI, back-end infrastructure, performance, etc... Getting something like Croquet to work at all was a monumental task. We didn't exactly have a model to work from. We were inventing - not reverse engineering.

Of course, anyone that has ever developed a larger project has seen these kinds of issues. I certainly have dealt with them many times. The big difference was no one saw any of my project's warts except my colleagues and me. And we worked hard to remove, or at least hide them before we unveiled the final polished application. Croquet was different - people have been able to randomly sample the state of the project and critique it based upon that sample. Even today, the Croquet API is not quite a user-centric architecture. It is much more akin to the Linux kernel, looking for a front end to empower the user and a back-end to help users find each other and provide additional services. On the other hand, it is doing almost everything we said it would - and it really works well. We have even started a company Qwaq, Inc. based upon the same open source system that you can download today. We are getting great reviews for our first product Qwaq Forums, even though it is till in beta today.

What is my point? I have enjoyed this very open development process. I have learned a lot, and made a huge number of new friends and colleagues. Overall, it has been fun in spite of certain comments from people. I do think that next time - if there is one - I will probably hold off releasing a new system until it is a bit more mature and robust. In some ways that is sad - I think people that have stayed with the project from the beginning have learned a lot about how a complex system gets built and how it evolves. Most of this would be missed if you were to jump into a more complete end-user experience. On the other hand, it would probably increase the probability of success. In marketing, you only have one chance to make a first impression. Qwaq Forums is making a GREAT first impression. It would have been nice if Croquet had the same opportunity.


Extreme Designer said...

You mention in this post that you will wait on delivering a project until it is more polished. The eclipse project waited until it has a rather polished and usable system until they released and they have been successful. What other reasons do you have for waiting?

David A. Smith said...

I think perception is a good part of a software project's success. Establishing a very strong foundation or even better a very clean and complete system before announcing increases the probablility of success in the user community. The implication of this is of course that open source software projects are like the little red hen asking for help to make her bread. "Not I", said the dog. "Not I", said the sheep. I am not unhappy with the situation. It is just something to consider when you start up a new open project.

Paul said...

Hi David,

I hear what you say, and being a programmer I know how negative comments on "work in progress" can hurt.

For me, Croquet is a remarkable technology, and I'm yet to succcessfuly connect to a public space.

For the most part, the opinions that really count is that of normal everyday people, and for them Croquet isn't yet on their radar.

With Qwaq, it looks like the leading edge of the Croquet world is now private, which has advantages, but it also means that the best Croquet is less visible and less accesible to every day people.

Not sure how things will pan out, but both approaches (open source versus closed source) have their potential down-sides, and my concern as an outsider is that public Croquet may slow down as private Croquet speeds up.

KellyRued said...

What if your team had put as much polish into Croquet as Qwaq? I think Croquet will eventually make a good impression and become more relevant to everyday user types but you really can't look at what open croquet has released and by surprised that it hasn't been easy to work with. As you mentioned above, you're inventing a new kind of collaboration software, not implementing a small improvement on more widely distributed and familiar systems. The learning curve of squeak + the new ideas in croquet architecture itself = few developers able and willing to contribute to the project. I think that with Qwaq you knew no one would adopt it without some minimal level of hand-holding and clear explanations of value: this is how X works and this is why X is the solution for you. Open croquet doesn't establish that kind of introduction and hit people over the head with real value (how does croquet save time, enable innovation, save money, etc.). I just wonder how different open croquet adoption would be if Qwaq forums were but one part of open croquet. I think a lot of people are waiting for the next croquet release (hopefully with better docs). Hopefully the open croquet dev community will pick up and grow when there are more hands-on examples of exactly what this new technology offers people. Qwaq seems like it could have been that example or "killer app" for croquet.

David A. Smith said...

You are correct - Croquet needs a "front end" and a better way to ease people into the process. We worked hard on the stuff underneath, but have not had the time or resources to make it look the way it can and should.

Qwaq Forums certainly demonstrates what can be done with Croquet, and I suspect that many of these ideas and technologies will make their way back into the open source system. Everyone agrees that for Open Croquet to take off requires a critical mass of functionality that is simply not there yet.