Hi dev@, Chris, thanks for bobsledding this :)
-- I'd welcome all three projects as sub-projects. I'd also throw in Futon to become a sub project, too. My understanding of a sub-project is this: - The development community between the main project and the sub project should be overlapping significantly. - Everybody in the main project's committer list gets to commit to all projects. - The main project's PMC oversees releases of all projects. - Technically, a new release of the main project should be followed by compatible releases of sub projects. I think we're at a scale where we can keep this all in the CouchDB project. My experience with branching out sub-projects into separate groups comes from the PHP community (no flames please) where extensions to PHP were divided into "core" extensions and "PECL" extensions. PECL being an extension development and distribution project independent of the PHP core. Core extensions are maintained and kept up to date with the core project, PECL extensions have their own schedule and the individual authors are responsible for updates in time for new PHP releases. This structure was introduced with over 100 extensions being in core PHP and way over 1000 committers to the project (one of the cases where documenters got a separate VCS module to work with). This proved to be good for the timely release of PHP releases as core developers have significantly less extensions to take care of. In a project as big as PHP, this makes sense and was needed badly. CouchDB is not that big yet. I'd say let's figure out how to incorporate sub projects into our subversion tree (e.g. couchdb/trunk is couchdb-trunk, where do sub projects go? couchdb/projects/...?) and how to make releases (separate or bundled with CouchDB, Futon e.g. could be bundled and couchdb-lucene a separate package) and worry about splitting out independent sub projects when when the need arises. Rehashing the above points for the proposed projects: Futon: - The developers are currently the CouchDB committers and we got contributions small and big from users through patches. - A future Futon-only developer doesn't necessarily have to know any Erlang and I'd trust her to not touch that part of the code until everybody feeling comfortable with this. (Besides, Erlang is so simple on the couch_http_*-end that small fixes in the HTTP API could be done without much hassle). And we're using SVN, if any unwanted changes, accidental or deliberate are introduced, we'll revert them. - Futon always has to be current with CouchDB, even through trunk-only development phases. Splitting this out to a separate schedule doesn't make much sense to me. - Working on Futon is a great way for new developers to get into CouchDB via simpler, more familiar means (web development). couchdb-lucene: - CouchDB greatly benefits from an officially endorsed Lucene integration plugin. This is prone to bikeshedding and unnecessary for users to implement all over again and again. - A future couchdb-lucene developer doesn't necessarily have to know any Erlang and I'd trust her to not touch that part of the code until everybody feeling comfortable with this. - The CouchDB PMC doesn't have much love for Java. We'd need to overcome our differences to be able to support releases. I'm confident we can do this. - The plugin API is relatively stable, but improvements on the CouchDB end are ongoing. Having stable releases alongside CouchDB releases are desirable. However, technically, we are not bound to hold a CouchDB release if couchdb-lucene is not ready yet, we can always make a later release. Our users will apply the necessary pressure if they need synched releases. couchdb-lucene wouldn't have to part of the standard CouchDB release tarball. It could however (`./configure --with-couchdb-lucene`) in which case a CouchDB release could be blocked by a couchdb-lucene release. The PMC has to decide on the route. To get going, I propose to start with separated release tarballs. - Working on couchdb-lucene is a great way for new developers to get into CouchDB more familiar means (Java). couchdb-lounge: (see couchdb-lucene =~ s/ucene/ounge/g) - Porting couchdb-lounge to Erlang and obsoleting the plugin step-by-step sounds like a good solution. CouchApp: (see Futon =~ s/Futon/CouchApp/g). In addition: The couchdb-lounge and CouchApp developers (Benoit and Robert) have been contributing to CouchDB via patches in the past and I'd be confident to have them as CouchDB committers anyway. -- Next steps: - Figure out how to host sub-projects both bundled (Futon) and unbundled (couchdb-lucene) and how to accommodate them in SVN and the build system (if applicable). - Prepare PMC votes to integrate the sources. - On success, prepare IP-clearance through the incubator. - PMC vote on accepting the existing committers as CouchDB committers. - Relax. Cheers Jan -- On 14 Aug 2009, at 08:52, Chris Anderson wrote:
Many Apache projects have sub-projects, for two good example see: http://hadoop.apache.org/ which has 9 sub-projects http://lucene.apache.org/ which has 10 I think one benefit of having sub-projects is broadening the community. I think it also helps to give people looking at CouchDB for the first time an easier way to see some of the really cool tools and libraries it's offers. Also, I think it sounds relaxing. Being able to keep an eye on a more of the Apache-licensed CouchDB ecosystem in one repository I think will result in stronger code. I'd like to see a few projects out there become sub-projects, and maybe there are others we should include as well. Here's a list of 3: The CouchDB-Lounge project provides a CouchDB clustering via a smart HTTP proxy. I can see bringing that code in, and using it as a scaffold for our Erlang clustering infrastructure. If we do it right, deployments will have wide flexibility over which tools to use to scale CouchDB, being able to mix, say, CouchDB-Lounge's consistent hashing nginx-proxy for document CRUD, but use Erlang view merger or other cluster-optimized view engine. If someone is already a heavy nginx shop, but doesn't want to merge views in twisted python, they could see benefits to a mix and match architecture. Informally I asked Kevin Ferguson of CouchDB-Lounge if they'd be interested and he said it sounds great. CouchApp is a set of scripts to make deploying CouchDB design documents easy. I've been involved in it for a while, and Benoit has put a lot of time into it. The tool and the JavaScript framework it goes with are starting to have a community, and should gain more interest when the O'Reilly book goes to press. Benoit Chesneau is excited about bringing CouchApp into the CouchDB project. CouchDB-Lucene is another good candidate. I haven't asked Robert Newson yet what he thinks about it, but I think the project would be a good fit. There may be more candidates I'm missing, or maybe people will think I'm batty for having the idea in the first place... comments welcome. Cheers, Chris -- Chris Anderson http://jchrisa.net http://couch.io
