I'm claiming 2nd person added! On 17 Jan 2014, at 1:28 PM, Noah Slater <[email protected]> wrote:
> Psst. A little birdy tells me that if you ask nicely, the infra folks > will add you to the Apache GitHub org too, so you can show off your > Apache affiliation. I was the first person added. Because I may have > been the first to ask. ;) > > On 17 January 2014 11:56, Noah Slater <[email protected]> wrote: >> Awesome, thanks Paul. >> >> Note to all devs: if you want your contributions to CouchDB to show up >> on your GitHub profile, you have to star each of the repositories. >> (That's just how GitHub mechanics work for repo mirrors.) >> >> You can find them all here: >> >> https://github.com/apache >> >> On 17 January 2014 00:00, Paul Davis <[email protected]> wrote: >>> New repos are up: https://git-wip-us.apache.org/repos/asf?s=couchdb >>> >>> I'm gonna go through and initialize them with history from master or >>> one of the bigcouch and rcouch branches as appropriate. >>> >>> On Thu, Jan 16, 2014 at 2:12 PM, Paul Davis <[email protected]> >>> wrote: >>>> Infrastructure ticket opened: >>>> https://issues.apache.org/jira/browse/INFRA-7203 >>>> >>>> On Thu, Jan 16, 2014 at 1:42 PM, Jan Lehnardt <[email protected]> wrote: >>>>> >>>>> On 16 Jan 2014, at 20:42 , Paul Davis <[email protected]> wrote: >>>>> >>>>>> It doesn't appear that this is objectionable to anyone. Does anyone >>>>>> have an objection to us having infra/me create these repos to use for >>>>>> the bigcouch/rcouch merge work? This won't affect master or releases >>>>>> until those merges finish. >>>>> >>>>> no objections. >>>>> >>>>> Jan >>>>> -- >>>>> >>>>>> >>>>>> On Tue, Jan 14, 2014 at 11:02 PM, Paul J Davis >>>>>> <[email protected]> wrote: >>>>>>> >>>>>>> >>>>>>>> On Jan 14, 2014, at 8:37 PM, Benoit Chesneau <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> On Wed, Jan 15, 2014 at 12:22 AM, Paul Davis >>>>>>>> <[email protected]>wrote: >>>>>>>> >>>>>>>>> I've recently been having discussions about how to handle the >>>>>>>>> repository configuration for various bits of CouchDB post-merge. The >>>>>>>>> work that Benoit has been doing on the rcouch merge branch have also >>>>>>>>> touched on this topic as well. >>>>>>>>> >>>>>>>>> The background for those unfamiliar is that the standard operating >>>>>>>>> procedure for Erlang is to have a single Erlang application per >>>>>>>>> repository and then rely on rebar to fetch each dependency. >>>>>>>>> Traditionally in CouchDB land we've always just included the source to >>>>>>>>> all applications in a single monolithic repository and periodically >>>>>>>>> reimport changes from upstream dependencies. >>>>>>>>> >>>>>>>>> Recently rcouch changed from the monolithic repository to use external >>>>>>>>> repositories for some dependencies. Originally the BigCouch used an >>>>>>>>> even more federated scheme that had each Erlang application in an >>>>>>>>> external repository (and the core couch Erlang application was in the >>>>>>>>> root repository). When Bob Newson and I did the initial hacking on the >>>>>>>>> BigCouch merge we pulled those external dependencies into the root >>>>>>>>> repository reverting back to the large monolithic approach. >>>>>>>>> >>>>>>>>> After trying to deal with the merge and contemplating how various >>>>>>>>> Erlang release things might work it's become fairly apparent that the >>>>>>>>> monolithic approach is a bit constrictive. For instance, part of >>>>>>>>> rebar's versioning abilities lets you tag repositories to generate >>>>>>>>> versions rather than manually updating versions in source files. >>>>>>>>> Another thing I've found on other projects is that having each >>>>>>>>> application in a separate repository requires developers to think a >>>>>>>>> bit more detailed about the public internal interfaces used through >>>>>>>>> out the system. We've done some work to this extent already with >>>>>>>>> separating source directories but forcing commits to multiple >>>>>>>>> repositories shoots up a big red flag that maybe there's a high level >>>>>>>>> of coupling between two bits of code. >>>>>>>>> >>>>>>>>> Other benefits of having the multiple repository setup is that its >>>>>>>>> possible that this lends itself to being integrated with the proposed >>>>>>>>> plugin system. It'd be fairly trivial to have a script that went and >>>>>>>>> fetched plugins that aren't developed at Apache (as a ./configure time >>>>>>>>> switch type of thing). Having a system like this would also allow us >>>>>>>>> to have groups focused on particular bits of development not have to >>>>>>>>> concern themselves with the unrelated parts of the system. >>>>>>>>> >>>>>>>>> Given all that, I'd like to propose that we move to having a >>>>>>>>> repository for each application/dependency that we use to build >>>>>>>>> CouchDB. Each repository would be hosted on ASF infra and mirrored to >>>>>>>>> GitHub as expected. This means that we could have the root repository >>>>>>>>> be a simple repo that contains packaging/release/build stuff that >>>>>>>>> would enable lots of the ideas offered on configurable types of >>>>>>>>> release generation. I've included an initial list of repositories at >>>>>>>>> the end of this email. Its basically just the apps that have been >>>>>>>>> split out in either rcouch or bigcouch plus a few other bits from >>>>>>>>> CouchDB master. >>>>>>>>> >>>>>>>>> I would also point out that even though our main repo would need to >>>>>>>>> fetch other dependencies from the internet to build the final output, >>>>>>>>> we fully intend that our release tarballs would *not* have this >>>>>>>>> requirement. Ie, when we go to cut a release part of the process the >>>>>>>>> RM would run would be to pull all of those dependencies before >>>>>>>>> creating a tarball that would be wholly self contained. Given an >>>>>>>>> apache-couchdb-x.y.z.tar.gz release file, there won't be a requirement >>>>>>>>> to have access to the ASF git repos. >>>>>>>>> >>>>>>>>> I'm not entirely sure how controversial this is for anyone. For the >>>>>>>>> most part the reactions I remember hearing were more concerned on >>>>>>>>> whether the infrastructure team would allow us to use this sort of >>>>>>>>> configuration. I looked yesterday and asked and apparently its >>>>>>>>> something we can request but as always we'll want to verify again if >>>>>>>>> we have consensus to move in this direction. >>>>>>>>> >>>>>>>>> Anyone have comments or flames? Right now I'm just interested in >>>>>>>>> feeling out what sort of (lack of?) consensus there is on such a >>>>>>>>> change. If there's general consensus I'd think we'd do a vote in a >>>>>>>>> couple weeks and if that passes then start on down this road for the >>>>>>>>> two merge projects and then it would become part of master once those >>>>>>>>> land (as opposed to doing this to master and then attempting to merge >>>>>>>>> rcouch/bigcouch onto that somehow). >>>>>>>>> >>>>>>>>> >>>>>>>>> This is a quick pass at listing what extra repositories I'd have >>>>>>>>> created. Some of these applications only exist in the bigcouch and/or >>>>>>>>> rcouch branches so that's where the unfamiliar application names are >>>>>>>>> from. I'd also point out that the documentation and fauxton things are >>>>>>>>> just on a whim in that we could decouple that development from the >>>>>>>>> erlang development. I can see arguments for an against those. I'm much >>>>>>>>> less concerned on that aspect than the Erlang parts that are directly >>>>>>>>> affected by rebar/Erlang conventions. >>>>>>>>> >>>>>>>>> chttpd >>>>>>>>> config >>>>>>>>> couch >>>>>>>>> couch_collate >>>>>>>>> couch_dbupdates >>>>>>>>> couch_httpd >>>>>>>>> couch_index >>>>>>>>> couch_mrview >>>>>>>>> couch_plugins >>>>>>>>> couch_replicator >>>>>>>>> documentation >>>>>>>>> ddoc_cache >>>>>>>>> ets_lru >>>>>>>>> fabric >>>>>>>>> fauxton >>>>>>>>> ibrowse >>>>>>>>> jiffy >>>>>>>>> mem3 >>>>>>>>> mochiweb >>>>>>>>> oauth >>>>>>>>> rebar >>>>>>>>> rexi >>>>>>>>> snappy >>>>>>>>> twig >>>>>>>> >>>>>>>> >>>>>>>> I also contemplated this and and I am generally +1 on this. And >>>>>>>> definitely >>>>>>>> +1 to mirror them on the apache git if possible. I have a couple of >>>>>>>> comments though. >>>>>>>> >>>>>>>> Initially I also had everything separated in its own source >>>>>>>> repository. 1 >>>>>>>> year ago I merged back as one core repo the couchdb erlang >>>>>>>> applications and >>>>>>>> put all the dependencies in the refuge repository or in the refuge CDN >>>>>>>> for >>>>>>>> the spidermonkey and ICU sources. >>>>>>>> >>>>>>>> I merged back as one core repo the couchdb erlang applications because >>>>>>>> they >>>>>>>> were a little too much dependant. Especially couch_httpd, couch_index >>>>>>>> and >>>>>>>> couch_mrview. These applications are not yet enough by themselves. >>>>>>>> >>>>>>>> Imo if we split everything in their own apps, then we should make sure >>>>>>>> that couch_httpd can be used without couch_index and couch_mrview >>>>>>>> (which >>>>>>>> means that "all_docs" is available in couch_httpd). Also we should be >>>>>>>> able >>>>>>>> to just launch couch without any of the above. And probably without the >>>>>>>> need of an ini. The couch_query_server module thing is an interesting >>>>>>>> case. >>>>>>>> bigcouch is also introducing `ddoc_cache` which I am not sure why it is >>>>>>>> provided as a standalone app. Does it means it can be replaced by >>>>>>>> another >>>>>>>> application eventually? Why not having it simply in the couch >>>>>>>> application? >>>>>>>> Does it needs to be updated separately? >>>>>>>> >>>>>>>> Also all our base applications should also be named spaced correctly >>>>>>>> so >>>>>>>> they will be strictly identified as erlang modules: "config" is too >>>>>>>> generic, "ddoc_cache" too. Others are probably OK. >>>>>>>> >>>>>>>> There are probably other things that we could provide as apps: >>>>>>>> >>>>>>>> - couch_daemon, >>>>>>>> - couch_js >>>>>>>> - couch_external >>>>>>>> - couch_stats >>>>>>>> - couch_compaction_daemon >>>>>>>> - couch_httpd_proxy >>>>>>>> >>>>>>>> Anyway again i'm +1 for this move, I really think it's a good idea. >>>>>>>> >>>>>>>> - benoit >>>>>>> >>>>>>> I agree on most of this. Roughly I see three general points. >>>>>>> >>>>>>> First, deciding on whether some things are external deps is definitely >>>>>>> up for discussion. Whether couch_mrview is a different app/repo is not >>>>>>> necessarily clear cut. Personally I think I over engineered couch_index >>>>>>> which blurs the lines a bit. If I could wave a wand I'd have just >>>>>>> couch_mrview and it'd be separate. More importantly I think the >>>>>>> separate repos makes these things more apparent. The fact were >>>>>>> discussing this sort of architecture thing is suggestive that it's >>>>>>> forcing us to think a bit harder. >>>>>>> >>>>>>> Second is the aspect of composability. For instance the mrview thing to >>>>>>> me is obviously a different repo precisely so a user could import couch >>>>>>> (_core?) directly without requiring the spider monkey dependency. The >>>>>>> monolithic repo doesn't allow this without some very non-standard >>>>>>> tooling. >>>>>>> >>>>>>> Thirdly, app naming is always a contention. The config name was >>>>>>> actually a hot code upgrade concern. We couldn't reuse couch_config >>>>>>> directly at the time. And Adam was also hopeful we could the it into a >>>>>>> useful non-specific config app. >>>>>>> >>>>>>> Fourthly, and related to secondly, we'll also want to look at splitting >>>>>>> other apps out as necessary. The ones you listed I think aren't >>>>>>> controversial it's just that no one has done it yet. My list was purely >>>>>>> what existed so far without attempting to carve things up more. I >>>>>>> definitely agree we should carve more in just wanted to cover consensus >>>>>>> that carving is the right direction. >>>>>>> >>>>>>> Fifthly, I'm done typing on my phone. I'll fill in more thoughts >>>>>>> tomorrow. >>>>>>> >>>>> >> >> >> >> -- >> Noah Slater >> https://twitter.com/nslater > > > > -- > Noah Slater > https://twitter.com/nslater
