Add this one to the pile: Improve config file write-back behavior. https://issues.apache.org/jira/browse/COUCHDB-1086
On Mon, Jan 23, 2012 at 01:03, Randall Leeds <[email protected]> wrote: > On Sat, Dec 31, 2011 at 10:34, Jan Johannes <[email protected]> wrote: >> Dear favorite Document-database developer community :-) >> This is my first message to the mailing list, so please forgive me and >> gently correct me if i am breaking any communication standards whatsoeverŠ > > Nope! You're just bringing up a deep topic with a lot of text. > I hope you didn't expect a speedy response! :) > >> >> I am currently working on a WebDAV interface for CouchDB, that now gets >> closer to its release date, somewhere in February or March and as i >> approach that, I want to package everything in a proper CouchDB Module >> format but the CouchDB Module system is, as i understood, in early >> development or pre-development phase and needs a bit of work. >> >> So as i work on a CouchDB Module anyways i would like to contribute to the >> Module system or start kicking the work on that of. >> >> Hence the following questions: >> >> Who started working on such a system or would be interested in talking >> about the design decisions that need to be done before actual coding can >> start? >> (I know at least Jan Lehnardt expressed interest, but i can't quite >> remember if he started actual work or just had plans for that.) > > Benoit is particularly interested and there is activity on the refuge > project mailing list about it from Nicolas Dufour. > I believe Jason Smith might be pretty interested since he's authored > some "plugins" already. > Others on the list are certainly interested. > I'm keen to get a good module system and it's why I was in favor of > the effort to get "couch-config" shipped as part of the install. > >> >> If there is any documentation on planning such a system, could someone >> please point me to the recourses? > > There's an umbrella ticket I made for modularizing the code base more: > https://issues.apache.org/jira/browse/COUCHDB-1251 > > And these: > > Utility to help plugin developers manage paths > https://issues.apache.org/jira/browse/COUCHDB-1012 > > support load of external erlang modules in couchdb. > https://issues.apache.org/jira/browse/COUCHDB-1046 > >> >> Lastly here is my two cents on a few topics that need to be addressed: >> >> - We should build the best module system there is for a database or >> content management system to gain back a bit of the traction we lost >> during the recent plateau phase of couchdb enthusiasm > > I find it hard to judge enthusiasm from the inside, but, yeah... a > proper module system would rock. > One of the things CouchDB struggles with in terms of image, but excels > at in terms of function is scaling up and down. The only way to really > make one project move from mobile phone to datacenter cluster is to > modularize the build and deployment of key components. I think > everyone agrees about this. CouchDB is the Gentoo of document > databases in this way. > >> >> - I think "module" is the preferable term, as extension, plugin or the >> like sound like second class citizens, whereas module implies, that even a >> core component of the system can be made into such an instance (Any other >> suggestions?) > > Module is a little bit confusing because it conflicts with Erlang's > definition. While the two would often be synonymous for many add-ons, > it's not necessarily the case. > I would go with calling them "applications" maybe. I've often thought > that each such thing could correspond well to a single Erlang > application and .app file, which Erlang has significant support and > prior art for loading and unloading on the fly. But really, I don't > care what color we paint this and long as it's neon awesome. > >> >> - A CouchDB module should be packaged as a stand alone couchdb-document, >> with the yet to be specified manifest beeing the json structure of the >> document and the files beeing the attachment. > > I'm pretty sure I disagree here. I think this conflates two issues. > The first, to me, is a good hooks system from which more interactions > in the core could be scripted with sandboxed user code. Once again, I > know I've talked to Benoit about this. The second, though, is the more > first-class type of application that requires compiled Erlang > bytecode. While these have some degree of portability[1] I > instinctively cringe, from a security standpoint, at the idea of hot > loading code from a document into the VM. While the document-module > idea I find attractive from a share-ability and install-ability > standpoint, I don't think it befits the general purpose module system. > Although, a good hooks system may make certain kinds of modules work > this way. > > [1] > http://stackoverflow.com/questions/2255658/how-portable-are-erlang-beam-files > >> >> - Discovery, Installation, Enabling, Disabling, Module-Specific >> Configuration and Distribution should be part of Futon and introduce as >> little new interface concepts as possible, also the futon interface >> section "configuration" may be needed to be integrated and streamlined >> quite nicely > > Again, this works the dynamic-language, hooks-based module, but I'm > not sure it works for all modules. Kanso [2] is doing this to a large > extent already, under the CouchApp frame, but I see no reason why a > hooks system couldn't accommodate a new, more powerful kind of Kanso > package. > > [2] http://kan.so/ > >> >> - Even if a module is not possible to completely install via this method >> (e.g. It depends on a os component, that is too low level etc.) such >> modules should be as tightly integrated into the system as possible and >> the manual admin installation steps should be documented in a way, so that >> in a normal case no place but the module system needs to be accessed > > Agree. > >> >> - on the top level the system should be the same for all OSes >> >> - The work packages i see at the moment for the structure i have in mind >> would be: >> >> - Module repository: A CouchDB Instance, that hosts a collection of >> CouchDB Modules > > See Kanso, above. > >> - Futon User Interface to browse available Modules in the configured >> repositories, should be close to the Firefox plugin paradigms > > That would be neat. Or a separate 'shopping' utility for CouchApps. > Sounds a bit like a combination between JChris' CouchApp Garden [3] > and Max Ogden's Chrome Couch application [4]. Crossing these would > Kanso would be really neat. > > [3] https://github.com/jchris/garden > [4] https://github.com/maxogden/chromecouch > >> - Module Manifest: Descriptive CouchDoc, that references the attached >> files or git repository, how files need to be fetched, packaged and what >> goes where, also version dependencies, description, rating, >> Module >> Category, needed system privileges (basically if you need to be a couch >> admin or if you need to be root on the os), needed >> configuration, and >> basic description of the configuration parameters > > I'd really prefer to punt on low-level privileges and just call this > two separate things. The way to unify them, in my mind, is to create a > hooks system and then to make a native module that listens for all > hooks and delegates processing to design documents that indicate they > handle certain hooks. In other words, build the un-privileged module > system on top of the privileged one. Let administrators worry about > file systems when installing the privileged ones. It's really not so > bad. Sysadmins have been installing things for years, and actually > probably prefer to install stuff on the command line than with a POST > to /_replicate. The couch-config utility is there to facilitate this. > What we probably need is an extra location for user-installed add-on > applications that need to be added to the Erlang library path and used > for loading configured applications. > >> - Futon User Interface to the installed modules, with options to >> enable, >> disable, uninstall, rate and configure > > Yes again. I like the gist, but I don't know if it should be bound up > in Futon or not. > >> - The tools for installation/uninstallation and enabling/disabling as >> well as packaging and dependency resolution > > That sounds like figuring out a good way to structure the > configuration sections for modules. See the JIRA tickets I referenced > for some discussion of that. > >> >> Open questions: What existing projects could be made use of, maybe >> there >> is something great for the general erlang world, maybe there is a project >> like zero install, that could be made use of, maybe that is not >> necessary, what are your thoughts? > > I would also check out Agner: http://erlagner.org/ > > Basically, if we could get to the point where CouchDB administrators > could use Agner to install new pieces I'd consider that a huge win. > If, in the process, we get a modularized core and support for a > design-document-level hooks system I'd consider that an epic win. I > never say epic win. I would say it three times out loud. > >> >> I wish you all a happy new year! >> >> > > Thanks :) > -Randall
