+1 for all the reasons stated. I think we need this sooner (i.e, immediately) to proceed with the 1843 bigcouch merge work, no?
The 1-to-1 parity will be particularly useful at helping pull in upstream changes (for mochi/ibrowse) while a) preserving local changes (like our SOCKS5 work) and b) avoiding dependencies on third party git hosts. B. On 14 Jan 2014, at 15:22, Paul Davis <paul.joseph.da...@gmail.com> wrote: > I've recently been having discussions about how to handle the > repository configuration for various bits of CouchDB post-merge. The > work that Benoit has been doing on the rcouch merge branch have also > touched on this topic as well. > > The background for those unfamiliar is that the standard operating > procedure for Erlang is to have a single Erlang application per > repository and then rely on rebar to fetch each dependency. > Traditionally in CouchDB land we've always just included the source to > all applications in a single monolithic repository and periodically > reimport changes from upstream dependencies. > > Recently rcouch changed from the monolithic repository to use external > repositories for some dependencies. Originally the BigCouch used an > even more federated scheme that had each Erlang application in an > external repository (and the core couch Erlang application was in the > root repository). When Bob Newson and I did the initial hacking on the > BigCouch merge we pulled those external dependencies into the root > repository reverting back to the large monolithic approach. > > After trying to deal with the merge and contemplating how various > Erlang release things might work it's become fairly apparent that the > monolithic approach is a bit constrictive. For instance, part of > rebar's versioning abilities lets you tag repositories to generate > versions rather than manually updating versions in source files. > Another thing I've found on other projects is that having each > application in a separate repository requires developers to think a > bit more detailed about the public internal interfaces used through > out the system. We've done some work to this extent already with > separating source directories but forcing commits to multiple > repositories shoots up a big red flag that maybe there's a high level > of coupling between two bits of code. > > Other benefits of having the multiple repository setup is that its > possible that this lends itself to being integrated with the proposed > plugin system. It'd be fairly trivial to have a script that went and > fetched plugins that aren't developed at Apache (as a ./configure time > switch type of thing). Having a system like this would also allow us > to have groups focused on particular bits of development not have to > concern themselves with the unrelated parts of the system. > > Given all that, I'd like to propose that we move to having a > repository for each application/dependency that we use to build > CouchDB. Each repository would be hosted on ASF infra and mirrored to > GitHub as expected. This means that we could have the root repository > be a simple repo that contains packaging/release/build stuff that > would enable lots of the ideas offered on configurable types of > release generation. I've included an initial list of repositories at > the end of this email. Its basically just the apps that have been > split out in either rcouch or bigcouch plus a few other bits from > CouchDB master. > > I would also point out that even though our main repo would need to > fetch other dependencies from the internet to build the final output, > we fully intend that our release tarballs would *not* have this > requirement. Ie, when we go to cut a release part of the process the > RM would run would be to pull all of those dependencies before > creating a tarball that would be wholly self contained. Given an > apache-couchdb-x.y.z.tar.gz release file, there won't be a requirement > to have access to the ASF git repos. > > I'm not entirely sure how controversial this is for anyone. For the > most part the reactions I remember hearing were more concerned on > whether the infrastructure team would allow us to use this sort of > configuration. I looked yesterday and asked and apparently its > something we can request but as always we'll want to verify again if > we have consensus to move in this direction. > > Anyone have comments or flames? Right now I'm just interested in > feeling out what sort of (lack of?) consensus there is on such a > change. If there's general consensus I'd think we'd do a vote in a > couple weeks and if that passes then start on down this road for the > two merge projects and then it would become part of master once those > land (as opposed to doing this to master and then attempting to merge > rcouch/bigcouch onto that somehow). > > > This is a quick pass at listing what extra repositories I'd have > created. Some of these applications only exist in the bigcouch and/or > rcouch branches so that's where the unfamiliar application names are > from. I'd also point out that the documentation and fauxton things are > just on a whim in that we could decouple that development from the > erlang development. I can see arguments for an against those. I'm much > less concerned on that aspect than the Erlang parts that are directly > affected by rebar/Erlang conventions. > > chttpd > config > couch > couch_collate > couch_dbupdates > couch_httpd > couch_index > couch_mrview > couch_plugins > couch_replicator > documentation > ddoc_cache > ets_lru > fabric > fauxton > ibrowse > jiffy > mem3 > mochiweb > oauth > rebar > rexi > snappy > twig
