Ace, Paul :) Just some bikeshedding as I agree with most that is said here.
I'm wondering if the OS process handling could be its own module or part of couch_core. While the View Server is the primary user, it is not the only one and having OS process management in the view server module just feels wrong to me :) Cheers Jan -- On 14 Oct 2010, at 20:54, Paul Davis wrote: > On Wed, Oct 13, 2010 at 5:23 PM, Benoit Chesneau <[email protected]> wrote: >> In an attempt to start some merging with cloudant I would like to >> start by using rebar in our install process. >> >> Like i see it, we could continue to use autotools to create the >> rebar.config files and other templates an then rebar for the final >> build and dependencies management. This changes as noticed by @davisp >> also imply we make our tree a little more OTP compliant. I would like >> to start this work asap. >> >> Thoughts ? >> >> - benoit >> > > So there's a couple issues at hand here which seem to be motivated by > the desire to start using tools like rebar. > > Our current source tree is not compliant with some of the basic > Erlang/OTP conventions. This is both bad technically and socially. > Technically, it prevents us from easily integrating tools like rebar > that would help advanced users with things like making Erlang reltools > packages. Socially, it doesn't reflect well on us to members of the > Erlang community that may have otherwise become contributors. All > languages have a standard package layout and Erlang is no different. > > The current CouchDB Erlang app has grown considerably. There's been > general consensus that we need to start splitting it up into smaller > applications that encompass specific functionality. There's been a bit > of effort in this direction, but its such a major change to source > file location it needs to have a community consensus to really start > working on seriously. > > I don't think we should focus directly on the issue of integrating > rebar. It should definitely be a goal, but not at the cost of our > current situation. Noah Slater has maintained an excellent build > system for us as is shown by the number of people building CouchDB > from source and the number of packages available. While I have argued > with him on numerous occasions about details, I have come to the > conclusion that it is not possible for him to be wrong. I personally > attribute this to the fact that he's most likely an advanced robot > from the future. That said, Noah has voiced concerns to various ideas > and we should make sure that any of his concerns are fully addressed. > > We should attempt to make sure that any tool support doesn't morph > into tool requirement. For instance, I think we should make sure that > its possible to keep compiling CouchDB without rebar and not come to > rely on it. > > While I'd be more than happy to start in on this and handle all of the > build system refactoring to make this happen, I'm not going to start > until there's a community consensus on what needs to be done. There > are a couple paths that I could see us taking to make this happen. We > could just make the current source tree be rebar compatible and figure > out the build system to do the optional rebar build or we could also > take this chance to split the source code into multiple applications. > Personally, I'd prefer to take this opportunity to organize the code > with multiple erlang apps. > > Too get the conversation rolling here's a first pass at a new app proposal: > > etap: > > Nick Gerakines now releases etap as a single .erl file that can be > dropped into the test directory. This app should be removed in favor > of that method. > > erlang-oauth: > > Should be renamed to just oauth. That erlang- prefix has bugged me > fore entirely too long. > > mochiweb, ibrowse, oauth: > > Refactored to use standard src, include, ebin, priv directories to > be OTP compliant. This results in directories like > > ./src/$APP/ebin > ./src/$APP/incldue > ./src/$APP/priv > ./src/$APP/src > > couchdb: > > Each proposed app will be structured as described above. Proposed apps: > > couch_core: The core Erlang modules for storing docs and managing > "internal infrastructure" > couch_view: The view engine as well as the holder for managing OS > processes. > couch_rep: couch_rep*.erl > couch_externals: couch_external*.erl > couch_httpd: couch_http*.erl > > At the bottom of this email I made an initial pass through the > ./src/couchdb tree to classify file by file into the described apps. > There are also some minor warts in this split. Things like the core > couchdb classes currently require calling into the couch_view app and > what not. Given the "interesting" call graph for CouchDb internals, I > think we should avoid addressing this and aim to minimize the actual > change to source file contents during this move. After modules have > been moved we can then start on teasing out the various cyclic > dependencies as necessary. > > Also, on a last note, seeing as this is a major change to the source > tree (and ideally 0 change to source code) it might be a good idea to > start this work just after we release 1.1.0. > > > What do you guys think? > > Paul Davis > > > List of files by per app: > > couch_core: > > ./src/couchdb/couch.erl > ./src/couchdb/couch_app.erl > ./src/couchdb/couch_auth_cache.erl > ./src/couchdb/couch_btree.erl > ./src/couchdb/couch_changes.erl > ./src/couchdb/couch_config.erl > ./src/couchdb/couch_config_writer.erl > ./src/couchdb/couch_db.erl > ./src/couchdb/couch_db.hrl > ./src/couchdb/couch_db_update_notifier.erl > ./src/couchdb/couch_db_update_notifier_sup.erl > ./src/couchdb/couch_db_updater.erl > ./src/couchdb/couch_doc.erl > ./src/couchdb/couch_event_sup.erl > ./src/couchdb/couch_file.erl > ./src/couchdb/couch_key_tree.erl > ./src/couchdb/couch_log.erl > ./src/couchdb/couch_ref_counter.erl > ./src/couchdb/couch_server.erl > ./src/couchdb/couch_server_sup.erl > ./src/couchdb/couch_stats_aggregator.erl > ./src/couchdb/couch_stats_collector.erl > ./src/couchdb/couch_stream.erl > ./src/couchdb/couch_task_status.erl > ./src/couchdb/couch_util.erl > ./src/couchdb/couch_uuids.erl > ./src/couchdb/priv/icu_driver/couch_icu_driver.c > ./src/couchdb/priv/stat_descriptions.cfg.in > > couch_view: > > ./src/couchdb/couch_native_process.erl > ./src/couchdb/couch_os_process.erl > ./src/couchdb/couch_query_servers.erl > ./src/couchdb/couch_view.erl > ./src/couchdb/couch_view_compactor.erl > ./src/couchdb/couch_view_group.erl > ./src/couchdb/couch_view_updater.erl > ./src/couchdb/couch_work_queue.erl > ./src/couchdb/priv/couch_js/http.c > ./src/couchdb/priv/couch_js/http.h > ./src/couchdb/priv/couch_js/main.c > ./src/couchdb/priv/couch_js/utf8.c > ./src/couchdb/priv/couch_js/utf8.h > ./src/couchdb/priv/spawnkillable/couchspawnkillable.sh > ./src/couchdb/priv/spawnkillable/couchspawnkillable_win.c > > couch_rep: > > ./src/couchdb/couch_js_functions.hrl > ./src/couchdb/couch_rep.erl > ./src/couchdb/couch_rep_att.erl > ./src/couchdb/couch_rep_changes_feed.erl > ./src/couchdb/couch_rep_db_listener.erl > ./src/couchdb/couch_rep_httpc.erl > ./src/couchdb/couch_rep_missing_revs.erl > ./src/couchdb/couch_rep_reader.erl > ./src/couchdb/couch_rep_sup.erl > ./src/couchdb/couch_rep_writer.erl > > couch_externals: > > ./src/couchdb/couch_external_manager.erl > ./src/couchdb/couch_external_server.erl > > couch_httpd: > > ./src/couchdb/couch_httpd.erl > ./src/couchdb/couch_httpd_auth.erl > ./src/couchdb/couch_httpd_db.erl > ./src/couchdb/couch_httpd_external.erl > ./src/couchdb/couch_httpd_misc_handlers.erl > ./src/couchdb/couch_httpd_oauth.erl > ./src/couchdb/couch_httpd_rewrite.erl > ./src/couchdb/couch_httpd_show.erl > ./src/couchdb/couch_httpd_stats_handlers.erl > ./src/couchdb/couch_httpd_vhost.erl > ./src/couchdb/couch_httpd_view.erl
