Hi Paul,
good points!
The questions boil down to:
- Is there a notion of core CouchDB that doesn't have cluster
features?
- Do we want to ship whatever release (cluster or not or both) of
CouchDB with a small ecosystem (Futon, CouchApp)?
Related:
- Do we want to foster plugins, extensions and other infrastructure
software or do we want to rely on the non CouchDB open source world to
come up with them?
--
My take:
- Is there a notion of core CouchDB that doesn't have cluster
features?
My understanding of CouchDB is it being a toolkit for users to build
their flavour of a distributed database with all the necessary bits in
place. CouchDB 0.9 and couchdb-lounge are a perfect example of that.
My understanding is also that we want an official distribution to
include couchdb-lounge style clustering. There's two ways to go about
it: 1) put it all into the source tree and disable and enable features
on build time (--enable-cluster) or 2) have separate trees (e.g. a
core and a cluster add on) that can be used to create two releases
(packaging time): couchdb-1.0.tar.gz and couchdb-with-cluster.tar.gz
Approach 2) could be generalized for other packaging-time plugins:
couchdb-with-lucene-1.0.tar.gz (including couchdb-lucene :) or couchdb-
no-futon-1.0.tar.gz We'd need to decide which and how many of these
distributions does the PMC want to release?
- Do we want to ship whatever release (cluster or not or both) of
CouchDB with a small ecosystem (Futon, CouchApp)?
Futon is already in "core", sub-projecting it would be merely done to
attract more Futon developers. Maybe that can be achieved in other
ways, too.
I think CouchApp should be a packaging-time part of CouchDB and
installed by default. This would add Python as a build dependency.
Maybe we can make ./configure smart enough to not install CouchApp and
give a warning when the necessary dependencies are not met. Maybe we
just add to the final `make install` output "Now go and install
CouchApp from ...".
- Do we want to foster plugins, extensions and other infrastructure
software or do we want to rely on the non CouchDB open source world to
come up with them?
I'd like to see a place like the Firefox extension development center
but for CouchDB plugins hosted on http://couchdb.apache.org.
--
Cheers
Jan
--
On 17 Aug 2009, at 08:12, Paul Davis wrote:
Sub-projects are a bad idea.
Been following this thread for awhile without being able to put my
concerns into small sentences. Each time I think about it I think
about how rapidly CouchDB is growing and how much that would hurt
sub-projects that are trying to keep up. And as others have said, we
should make sure that CouchDB doesn't turn into a namespace for
sub-projects.
Personally, I think CouchApp should be very frightened of becoming an
ASF project of any sort. My guess is that the stability/agility trade
off is just too serious. I think adding a CouchApp page on
http://couchdb.apache.org would be good, but adding CouchApp traffic
to the bug tracker or mailing lists would make me want to throw twice
as much stuff.
couchdb-lounge should never be a sub-project. Implementing it in
Erlang is going to touch more bits than most people consider. It'll
end up being unavoidable not having it part of the default
distribution. Trying to pull in the entire project as it is and the
replace it piece by piece is not going to work. We should keep it like
it is, a reference implementation that we hope to achieve in the
default CouchDB distribution.
The only project I could even consider being a sub-project is
couchdb-lucene. Though for roughly the same reasons as CouchApp I'd
probably rather see it as a separate project and just include it on
our website ecosystem. Both projects are public-api compatibile and as
others have stated, they'll either stay compatible or die.
And with all that, we're an Alpha (or Beta), pre-1.0 software project.
Now is not the time for adding bureaucracy to release procedures. We
should be focused on removing obstacles to making good software and
adding sub-projects seems like a good way to cause us a crap load of
pain in the future.
Faster not slower.
Paul Davis
On Fri, Aug 14, 2009 at 2:52 AM, Chris Anderson<[email protected]>
wrote:
Many Apache projects have sub-projects, for two good example see:
http://hadoop.apache.org/ which has 9 sub-projects
http://lucene.apache.org/ which has 10
I think one benefit of having sub-projects is broadening the
community. I think it also helps to give people looking at CouchDB
for
the first time an easier way to see some of the really cool tools and
libraries it's offers.
Also, I think it sounds relaxing. Being able to keep an eye on a more
of the Apache-licensed CouchDB ecosystem in one repository I think
will result in stronger code.
I'd like to see a few projects out there become sub-projects, and
maybe there are others we should include as well. Here's a list of 3:
The CouchDB-Lounge project provides a CouchDB clustering via a smart
HTTP proxy. I can see bringing that code in, and using it as a
scaffold for our Erlang clustering infrastructure. If we do it right,
deployments will have wide flexibility over which tools to use to
scale CouchDB, being able to mix, say, CouchDB-Lounge's consistent
hashing nginx-proxy for document CRUD, but use Erlang view merger or
other cluster-optimized view engine. If someone is already a heavy
nginx shop, but doesn't want to merge views in twisted python, they
could see benefits to a mix and match architecture.
Informally I asked Kevin Ferguson of CouchDB-Lounge if they'd be
interested and he said it sounds great.
CouchApp is a set of scripts to make deploying CouchDB design
documents easy. I've been involved in it for a while, and Benoit has
put a lot of time into it. The tool and the JavaScript framework it
goes with are starting to have a community, and should gain more
interest when the O'Reilly book goes to press. Benoit Chesneau is
excited about bringing CouchApp into the CouchDB project.
CouchDB-Lucene is another good candidate. I haven't asked Robert
Newson yet what he thinks about it, but I think the project would
be a
good fit.
There may be more candidates I'm missing, or maybe people will think
I'm batty for having the idea in the first place... comments welcome.
Cheers,
Chris
--
Chris Anderson
http://jchrisa.net
http://couch.io
