Good plan. What's your wiki username? I'll add you. On 17 January 2014 20:01, Mutton, James <[email protected]> wrote: > Might I suggest that this sounds like good info to document on the wiki > for committers getting started. I'd add it but I'm not in the allow-list. > :) > > </JamesM> > > > > > On 1/17/14 4:00 AM, "Garren Smith" <[email protected]> wrote: > >>I'm claiming 2nd person added! >> >>On 17 Jan 2014, at 1:28 PM, Noah Slater <[email protected]> wrote: >> >>> Psst. A little birdy tells me that if you ask nicely, the infra folks >>> will add you to the Apache GitHub org too, so you can show off your >>> Apache affiliation. I was the first person added. Because I may have >>> been the first to ask. ;) >>> >>> On 17 January 2014 11:56, Noah Slater <[email protected]> wrote: >>>> Awesome, thanks Paul. >>>> >>>> Note to all devs: if you want your contributions to CouchDB to show up >>>> on your GitHub profile, you have to star each of the repositories. >>>> (That's just how GitHub mechanics work for repo mirrors.) >>>> >>>> You can find them all here: >>>> >>>> https://github.com/apache >>>> >>>> On 17 January 2014 00:00, Paul Davis <[email protected]> >>>>wrote: >>>>> New repos are up: https://git-wip-us.apache.org/repos/asf?s=couchdb >>>>> >>>>> I'm gonna go through and initialize them with history from master or >>>>> one of the bigcouch and rcouch branches as appropriate. >>>>> >>>>> On Thu, Jan 16, 2014 at 2:12 PM, Paul Davis >>>>><[email protected]> wrote: >>>>>> Infrastructure ticket opened: >>>>>>https://issues.apache.org/jira/browse/INFRA-7203 >>>>>> >>>>>> On Thu, Jan 16, 2014 at 1:42 PM, Jan Lehnardt <[email protected]> wrote: >>>>>>> >>>>>>> On 16 Jan 2014, at 20:42 , Paul Davis <[email protected]> >>>>>>>wrote: >>>>>>> >>>>>>>> It doesn't appear that this is objectionable to anyone. Does anyone >>>>>>>> have an objection to us having infra/me create these repos to use >>>>>>>>for >>>>>>>> the bigcouch/rcouch merge work? This won't affect master or >>>>>>>>releases >>>>>>>> until those merges finish. >>>>>>> >>>>>>> no objections. >>>>>>> >>>>>>> Jan >>>>>>> -- >>>>>>> >>>>>>>> >>>>>>>> On Tue, Jan 14, 2014 at 11:02 PM, Paul J Davis >>>>>>>> <[email protected]> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Jan 14, 2014, at 8:37 PM, Benoit Chesneau >>>>>>>>>><[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> On Wed, Jan 15, 2014 at 12:22 AM, Paul Davis >>>>>>>>>><[email protected]>wrote: >>>>>>>>>> >>>>>>>>>>> I've recently been having discussions about how to handle the >>>>>>>>>>> repository configuration for various bits of CouchDB >>>>>>>>>>>post-merge. The >>>>>>>>>>> work that Benoit has been doing on the rcouch merge branch have >>>>>>>>>>>also >>>>>>>>>>> touched on this topic as well. >>>>>>>>>>> >>>>>>>>>>> The background for those unfamiliar is that the standard >>>>>>>>>>>operating >>>>>>>>>>> procedure for Erlang is to have a single Erlang application per >>>>>>>>>>> repository and then rely on rebar to fetch each dependency. >>>>>>>>>>> Traditionally in CouchDB land we've always just included the >>>>>>>>>>>source to >>>>>>>>>>> all applications in a single monolithic repository and >>>>>>>>>>>periodically >>>>>>>>>>> reimport changes from upstream dependencies. >>>>>>>>>>> >>>>>>>>>>> Recently rcouch changed from the monolithic repository to use >>>>>>>>>>>external >>>>>>>>>>> repositories for some dependencies. Originally the BigCouch >>>>>>>>>>>used an >>>>>>>>>>> even more federated scheme that had each Erlang application in >>>>>>>>>>>an >>>>>>>>>>> external repository (and the core couch Erlang application was >>>>>>>>>>>in the >>>>>>>>>>> root repository). When Bob Newson and I did the initial hacking >>>>>>>>>>>on the >>>>>>>>>>> BigCouch merge we pulled those external dependencies into the >>>>>>>>>>>root >>>>>>>>>>> repository reverting back to the large monolithic approach. >>>>>>>>>>> >>>>>>>>>>> After trying to deal with the merge and contemplating how >>>>>>>>>>>various >>>>>>>>>>> Erlang release things might work it's become fairly apparent >>>>>>>>>>>that the >>>>>>>>>>> monolithic approach is a bit constrictive. For instance, part of >>>>>>>>>>> rebar's versioning abilities lets you tag repositories to >>>>>>>>>>>generate >>>>>>>>>>> versions rather than manually updating versions in source files. >>>>>>>>>>> Another thing I've found on other projects is that having each >>>>>>>>>>> application in a separate repository requires developers to >>>>>>>>>>>think a >>>>>>>>>>> bit more detailed about the public internal interfaces used >>>>>>>>>>>through >>>>>>>>>>> out the system. We've done some work to this extent already with >>>>>>>>>>> separating source directories but forcing commits to multiple >>>>>>>>>>> repositories shoots up a big red flag that maybe there's a high >>>>>>>>>>>level >>>>>>>>>>> of coupling between two bits of code. >>>>>>>>>>> >>>>>>>>>>> Other benefits of having the multiple repository setup is that >>>>>>>>>>>its >>>>>>>>>>> possible that this lends itself to being integrated with the >>>>>>>>>>>proposed >>>>>>>>>>> plugin system. It'd be fairly trivial to have a script that >>>>>>>>>>>went and >>>>>>>>>>> fetched plugins that aren't developed at Apache (as a >>>>>>>>>>>./configure time >>>>>>>>>>> switch type of thing). Having a system like this would also >>>>>>>>>>>allow us >>>>>>>>>>> to have groups focused on particular bits of development not >>>>>>>>>>>have to >>>>>>>>>>> concern themselves with the unrelated parts of the system. >>>>>>>>>>> >>>>>>>>>>> Given all that, I'd like to propose that we move to having a >>>>>>>>>>> repository for each application/dependency that we use to build >>>>>>>>>>> CouchDB. Each repository would be hosted on ASF infra and >>>>>>>>>>>mirrored to >>>>>>>>>>> GitHub as expected. This means that we could have the root >>>>>>>>>>>repository >>>>>>>>>>> be a simple repo that contains packaging/release/build stuff >>>>>>>>>>>that >>>>>>>>>>> would enable lots of the ideas offered on configurable types of >>>>>>>>>>> release generation. I've included an initial list of >>>>>>>>>>>repositories at >>>>>>>>>>> the end of this email. Its basically just the apps that have >>>>>>>>>>>been >>>>>>>>>>> split out in either rcouch or bigcouch plus a few other bits >>>>>>>>>>>from >>>>>>>>>>> CouchDB master. >>>>>>>>>>> >>>>>>>>>>> I would also point out that even though our main repo would >>>>>>>>>>>need to >>>>>>>>>>> fetch other dependencies from the internet to build the final >>>>>>>>>>>output, >>>>>>>>>>> we fully intend that our release tarballs would *not* have this >>>>>>>>>>> requirement. Ie, when we go to cut a release part of the >>>>>>>>>>>process the >>>>>>>>>>> RM would run would be to pull all of those dependencies before >>>>>>>>>>> creating a tarball that would be wholly self contained. Given an >>>>>>>>>>> apache-couchdb-x.y.z.tar.gz release file, there won't be a >>>>>>>>>>>requirement >>>>>>>>>>> to have access to the ASF git repos. >>>>>>>>>>> >>>>>>>>>>> I'm not entirely sure how controversial this is for anyone. For >>>>>>>>>>>the >>>>>>>>>>> most part the reactions I remember hearing were more concerned >>>>>>>>>>>on >>>>>>>>>>> whether the infrastructure team would allow us to use this sort >>>>>>>>>>>of >>>>>>>>>>> configuration. I looked yesterday and asked and apparently its >>>>>>>>>>> something we can request but as always we'll want to verify >>>>>>>>>>>again if >>>>>>>>>>> we have consensus to move in this direction. >>>>>>>>>>> >>>>>>>>>>> Anyone have comments or flames? Right now I'm just interested in >>>>>>>>>>> feeling out what sort of (lack of?) consensus there is on such a >>>>>>>>>>> change. If there's general consensus I'd think we'd do a vote >>>>>>>>>>>in a >>>>>>>>>>> couple weeks and if that passes then start on down this road >>>>>>>>>>>for the >>>>>>>>>>> two merge projects and then it would become part of master once >>>>>>>>>>>those >>>>>>>>>>> land (as opposed to doing this to master and then attempting to >>>>>>>>>>>merge >>>>>>>>>>> rcouch/bigcouch onto that somehow). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> This is a quick pass at listing what extra repositories I'd have >>>>>>>>>>> created. Some of these applications only exist in the bigcouch >>>>>>>>>>>and/or >>>>>>>>>>> rcouch branches so that's where the unfamiliar application >>>>>>>>>>>names are >>>>>>>>>>> from. I'd also point out that the documentation and fauxton >>>>>>>>>>>things are >>>>>>>>>>> just on a whim in that we could decouple that development from >>>>>>>>>>>the >>>>>>>>>>> erlang development. I can see arguments for an against those. >>>>>>>>>>>I'm much >>>>>>>>>>> less concerned on that aspect than the Erlang parts that are >>>>>>>>>>>directly >>>>>>>>>>> affected by rebar/Erlang conventions. >>>>>>>>>>> >>>>>>>>>>> chttpd >>>>>>>>>>> config >>>>>>>>>>> couch >>>>>>>>>>> couch_collate >>>>>>>>>>> couch_dbupdates >>>>>>>>>>> couch_httpd >>>>>>>>>>> couch_index >>>>>>>>>>> couch_mrview >>>>>>>>>>> couch_plugins >>>>>>>>>>> couch_replicator >>>>>>>>>>> documentation >>>>>>>>>>> ddoc_cache >>>>>>>>>>> ets_lru >>>>>>>>>>> fabric >>>>>>>>>>> fauxton >>>>>>>>>>> ibrowse >>>>>>>>>>> jiffy >>>>>>>>>>> mem3 >>>>>>>>>>> mochiweb >>>>>>>>>>> oauth >>>>>>>>>>> rebar >>>>>>>>>>> rexi >>>>>>>>>>> snappy >>>>>>>>>>> twig >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> I also contemplated this and and I am generally +1 on this. And >>>>>>>>>>definitely >>>>>>>>>> +1 to mirror them on the apache git if possible. I have a >>>>>>>>>>couple of >>>>>>>>>> comments though. >>>>>>>>>> >>>>>>>>>> Initially I also had everything separated in its own source >>>>>>>>>>repository. 1 >>>>>>>>>> year ago I merged back as one core repo the couchdb erlang >>>>>>>>>>applications and >>>>>>>>>> put all the dependencies in the refuge repository or in the >>>>>>>>>>refuge CDN for >>>>>>>>>> the spidermonkey and ICU sources. >>>>>>>>>> >>>>>>>>>> I merged back as one core repo the couchdb erlang applications >>>>>>>>>>because they >>>>>>>>>> were a little too much dependant. Especially couch_httpd, >>>>>>>>>>couch_index and >>>>>>>>>> couch_mrview. These applications are not yet enough by >>>>>>>>>>themselves. >>>>>>>>>> >>>>>>>>>> Imo if we split everything in their own apps, then we should >>>>>>>>>>make sure >>>>>>>>>> that couch_httpd can be used without couch_index and >>>>>>>>>>couch_mrview (which >>>>>>>>>> means that "all_docs" is available in couch_httpd). Also we >>>>>>>>>>should be able >>>>>>>>>> to just launch couch without any of the above. And probably >>>>>>>>>>without the >>>>>>>>>> need of an ini. The couch_query_server module thing is an >>>>>>>>>>interesting case. >>>>>>>>>> bigcouch is also introducing `ddoc_cache` which I am not sure >>>>>>>>>>why it is >>>>>>>>>> provided as a standalone app. Does it means it can be replaced >>>>>>>>>>by another >>>>>>>>>> application eventually? Why not having it simply in the couch >>>>>>>>>>application? >>>>>>>>>> Does it needs to be updated separately? >>>>>>>>>> >>>>>>>>>> Also all our base applications should also be named spaced >>>>>>>>>>correctly so >>>>>>>>>> they will be strictly identified as erlang modules: "config" is >>>>>>>>>>too >>>>>>>>>> generic, "ddoc_cache" too. Others are probably OK. >>>>>>>>>> >>>>>>>>>> There are probably other things that we could provide as apps: >>>>>>>>>> >>>>>>>>>> - couch_daemon, >>>>>>>>>> - couch_js >>>>>>>>>> - couch_external >>>>>>>>>> - couch_stats >>>>>>>>>> - couch_compaction_daemon >>>>>>>>>> - couch_httpd_proxy >>>>>>>>>> >>>>>>>>>> Anyway again i'm +1 for this move, I really think it's a good >>>>>>>>>>idea. >>>>>>>>>> >>>>>>>>>> - benoit >>>>>>>>> >>>>>>>>> I agree on most of this. Roughly I see three general points. >>>>>>>>> >>>>>>>>> First, deciding on whether some things are external deps is >>>>>>>>>definitely up for discussion. Whether couch_mrview is a different >>>>>>>>>app/repo is not necessarily clear cut. Personally I think I over >>>>>>>>>engineered couch_index which blurs the lines a bit. If I could >>>>>>>>>wave a wand I'd have just couch_mrview and it'd be separate. More >>>>>>>>>importantly I think the separate repos makes these things more >>>>>>>>>apparent. The fact were discussing this sort of architecture thing >>>>>>>>>is suggestive that it's forcing us to think a bit harder. >>>>>>>>> >>>>>>>>> Second is the aspect of composability. For instance the mrview >>>>>>>>>thing to me is obviously a different repo precisely so a user >>>>>>>>>could import couch (_core?) directly without requiring the spider >>>>>>>>>monkey dependency. The monolithic repo doesn't allow this without >>>>>>>>>some very non-standard tooling. >>>>>>>>> >>>>>>>>> Thirdly, app naming is always a contention. The config name was >>>>>>>>>actually a hot code upgrade concern. We couldn't reuse >>>>>>>>>couch_config directly at the time. And Adam was also hopeful we >>>>>>>>>could the it into a useful non-specific config app. >>>>>>>>> >>>>>>>>> Fourthly, and related to secondly, we'll also want to look at >>>>>>>>>splitting other apps out as necessary. The ones you listed I think >>>>>>>>>aren't controversial it's just that no one has done it yet. My >>>>>>>>>list was purely what existed so far without attempting to carve >>>>>>>>>things up more. I definitely agree we should carve more in just >>>>>>>>>wanted to cover consensus that carving is the right direction. >>>>>>>>> >>>>>>>>> Fifthly, I'm done typing on my phone. I'll fill in more thoughts >>>>>>>>>tomorrow. >>>>>>>>> >>>>>>> >>>> >>>> >>>> >>>> -- >>>> Noah Slater >>>> https://twitter.com/nslater >>> >>> >>> >>> -- >>> Noah Slater >>> https://twitter.com/nslater >> >
-- Noah Slater https://twitter.com/nslater
