wow - awesome - just wanted to let you know that this will be really cool - having plugins will increase the CouchDB users a lot! Great job!
On 4 August 2013 17:39, Jan Lehnardt <j...@apache.org> wrote: > > On Aug 4, 2013, at 17:29 , Octavian Damiean <mainer...@gmail.com> wrote: > >> Signed PGP part >> Great work! >> >> Got a quick question. In the notice you mention that the instructions >> work only with the 1867-feature-plugin branch but you didn't add steps >> to checkout said branch in the preparation step. >> >> Want me to add that bit of information or was that deliberate? > > Total oversight, you are more then welcome to add this :) > > Jan > -- > > > >> >> Cheers, >> Octavian >> >> On 2013-08-03 22:15, Jan Lehnardt wrote: >> > More update: >> > >> > I started producing a plugin template repo that people can clone >> > to build their own plugins along with a comprehensive README.md: >> > >> > https://github.com/janl/my-first-couchdb-plugin >> > >> > The idea is to move the README to the CouchDB docs eventually and >> > ship the plugin template with CouchDB, so people can get started >> > easily. >> > >> > Best Jan -- >> > >> > On Aug 3, 2013, at 17:53 , Simon Metson <si...@cloudant.com> >> > wrote: >> > >> >> :) >> >> >> >> >> >> On Saturday, 3 August 2013 at 14:21, Jan Lehnardt wrote: >> >> >> >>> Couldn’t help but implement it. It’s in the branch now. >> >>> >> >>> Jan -- >> >>> >> >>> On Aug 3, 2013, at 08:12 , Simon Metson <si...@cloudant.com> >> >>> wrote: >> >>> >> >>>> Sounds good to me. >> >>>> >> >>>> >> >>>> On Saturday, 3 August 2013 at 00:56, Jan Lehnardt wrote: >> >>>> >> >>>>> >> >>>>> On Aug 3, 2013, at 00:02 , Russell Branca >> >>>>> <chewbra...@apache.org (mailto:chewbra...@apache.org)> >> >>>>> wrote: >> >>>>> >> >>>>>> This is fantastic, Jan! Glad to see this coming along. >> >>>>>> >> >>>>>> One of the goals with Fauxton has always been to make it >> >>>>>> easy for plugins to extend the interface and provide new >> >>>>>> functionality. I've been toying with the idea of having a >> >>>>>> _fauxton db that plugins install to as docs with >> >>>>>> attachments, but that's for a different thread. The >> >>>>>> general idea here is that a plugin will be able to extend >> >>>>>> Fauxton by adding a new page with it's own functionality, >> >>>>>> or hook into existing pages to extend other areas. >> >>>>>> >> >>>>>> For instance, you could have a couchdb-lucene plugin that >> >>>>>> hooks into the databases list and allows you to add >> >>>>>> interfaces for building full text indexes and searching >> >>>>>> on existing indexes. Or you could have a dedicated page >> >>>>>> for Geocouch, or whatever. >> >>>>>> >> >>>>>> The functionality is there, but it's still a bit of a >> >>>>>> manual process, so we'll need to make it more dynamic and >> >>>>>> smooth out the rough edges. >> >>>>>> >> >>>>>> I'm very excited to see progress being made on plugins, >> >>>>>> great work! >> >>>>> >> >>>>> Thanks, I’m glad you like this! :) >> >>>>> >> >>>>> Another way to get the Fauxton plugin loaded would be to >> >>>>> extend the /_plugins API endpoint, so Fauxton could request >> >>>>> GET /_plugins/<pluginname>/ and it would serve >> >>>>> <couchdblibdir>/plugins/<pluginname/priv/www which is just >> >>>>> a place for Fauxton-enabled plugins. >> >>>>> >> >>>>> Fauxton would walk /_config/plugins/ to get to a list of >> >>>>> plugins. >> >>>>> >> >>>>> In fact that should be pretty simple to set up. >> >>>>> >> >>>>> For now I am trying to avoid having a custom database for >> >>>>> this, mostly because I don’t think there are many >> >>>>> advantages (e.g. replication of plugins?) and code >> >>>>> complexity. These priorities might change in the future, >> >>>>> but for now I am happy to get this working at all :) >> >>>>> >> >>>>> If you are okay with the above plan of serving plugin >> >>>>> HTML/JS/CSS from /_plugins/<pluginname>, I’m happy to add >> >>>>> this to the branch. >> >>>>> >> >>>>> Best Jan -- >> >>>>> >> >>>>> >> >>>>> >> >>>>>> >> >>>>>> >> >>>>>> -Russell >> >>>>>> >> >>>>>> >> >>>>>> On Fri, Aug 2, 2013 at 2:17 PM, Jan Lehnardt >> >>>>>> <j...@apache.org (mailto:j...@apache.org)> wrote: >> >>>>>> >> >>>>>>> And a few more (from COUCHDB-1867): >> >>>>>>> >> >>>>>>> - Add uninstall, incl. Futon UI. - Only install a >> >>>>>>> plugin if the source and target CouchDB version >> >>>>>>> matches. - Rebase against master. >> >>>>>>> >> >>>>>>> * * * >> >>>>>>> >> >>>>>>> This concludes my list for a Minimally Viable Plugin >> >>>>>>> feature. (See the original email or README.md >> >>>>>>> (http://README.md)* for the roadmap) >> >>>>>>> >> >>>>>>> I’d appreciate some more reviews & feedback**, but >> >>>>>>> other than that, I’d be happy to ship this as an >> >>>>>>> experimental feature in any next release. >> >>>>>>> >> >>>>>>> * >> >>>>>>> https://github.com/janl/couchdb/blob/1867-feature-plugins/src/couch_plugins/README.md#roadmap >> >>>>>>> >> >>>>>>> >> ** >> >>>>>>> https://github.com/janl/couchdb/compare/apache:master...1867-feature-plugins >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> Best >> >>>>>>> Jan -- >> >>>>>>> >> >>>>>>> On Aug 1, 2013, at 19:34 , Jan Lehnardt <j...@apache.org >> >>>>>>> (mailto:j...@apache.org)> wrote: >> >>>>>>> >> >>>>>>>> A few updates: >> >>>>>>>> >> >>>>>>>> By Bob Ippolito / @etrepum: - Plugins are now >> >>>>>>>> installed in libdir (instead of /tmp). - Config >> >>>>>>>> loading is now done with proper .ini files. - Various >> >>>>>>>> cleanups and code review (Thanks!). >> >>>>>>>> >> >>>>>>>> Mine (most suggested by Bob): - `plugins.html` now >> >>>>>>>> shows you if a plugin is already installed. and which >> >>>>>>>> version, if it doesn’t match the installable one. - >> >>>>>>>> The Install button now disables after an >> >>>>>>>> installation. - Plugins are now registered with >> >>>>>>>> couch_config as /_config/plugins/name = version - >> >>>>>>>> Updated `couch-config` to print --erlang-version and >> >>>>>>>> --erl-bin - Updated the geocouch plugin to use the >> >>>>>>>> new options in `couch-config`. - Added Bob Ippolito’s >> >>>>>>>> couchperuser plugin to Futon. >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> Best Jan -- >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> On Jul 31, 2013, at 19:07 , Jan Lehnardt >> >>>>>>>> <j...@apache.org (mailto:j...@apache.org)> wrote: >> >>>>>>>> >> >>>>>>>>> Heya, >> >>>>>>>>> >> >>>>>>>>> I couldn’t help myself thinking about plugin stuff >> >>>>>>>>> and ended up whipping up a proof of concept. >> >>>>>>>>> >> >>>>>>>>> Here’s a <1 minute demo video: >> >>>>>>>>> >> >>>>>>>>> https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.mov >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> Alternative encoding: >> >>>>>>>>> >> >>>>>>>>> https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.m4v) >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> In my head the whole plugin idea is a very wide area, but I was so >> >>>>>>>>> intrigued by the idea of getting something running >> >>>>>>>>> with a click on a button in Futon. So I looked for >> >>>>>>>>> a minimally viable plugin system. >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> ## Design principles >> >>>>>>>>> >> >>>>>>>>> It took me a day to put this all together and this >> >>>>>>>>> was only possible because I took a lot of >> >>>>>>>>> shortcuts. I believe they are all viable for a >> >>>>>>>>> first iteration of a plugins system: >> >>>>>>>>> >> >>>>>>>>> 1. Install with one click on a button in Futon (or >> >>>>>>>>> HTTP call) 2. Only pure Erlang plugins are >> >>>>>>>>> allowed. 3. The plugin author must provide a binary >> >>>>>>>>> package for each Erlang (and, later, each CouchDB >> >>>>>>>>> version). 4. Complete trust-based system. You trust >> >>>>>>>>> me to not do any nasty things when you click on the >> >>>>>>>>> install button. No crypto, no nothing. Only people >> >>>>>>>>> who can commit to Futon can release new versions of >> >>>>>>>>> plugins. 5. Minimal user-friendlyness: won’t >> >>>>>>>>> install plugins that don’t match the current Erlang >> >>>>>>>>> version, gives semi-sensible error messages >> >>>>>>>>> (wrapped in a HTTP 500 response :) 6. Require a >> >>>>>>>>> pretty strict format for binary releases. >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> ## Roadmap >> >>>>>>>>> >> >>>>>>>>> Here’s a list of things this first iterations does >> >>>>>>>>> and doesn’t do: >> >>>>>>>>> >> >>>>>>>>> - Pure Erlang plugins only. No C-dependencies, no >> >>>>>>>>> JavaScript, no >> >>>>>>> nothing. >> >>>>>>>>> - No C-dependencies. - Install a plugin via Futon >> >>>>>>>>> (or HTTP call). Admin only. - A hardcoded list of >> >>>>>>>>> plugins in Futon. - Loads a pre-packaged, >> >>>>>>>>> pre-compiled .tar.gz file from a URL. - Only >> >>>>>>>>> installs if Erlang version matches. - No security >> >>>>>>>>> checking of binaries. - No identity checking of >> >>>>>>>>> binaries. >> >>>>>>>>> >> >>>>>>>>> Here are a few things I want to add before I call >> >>>>>>>>> it MVP*: >> >>>>>>>>> >> >>>>>>>>> - Uninstall a plugin via Futon (or HTTP call). >> >>>>>>>>> Admin only. - Only installs if CouchDB version >> >>>>>>>>> matches. - Binaries must be published on >> >>>>>>>>> *.apache.org (http://apache.org). - Register >> >>>>>>>>> installed plugins in the config system. - Make sure >> >>>>>>>>> plugins start with the next restart of CouchDB. - >> >>>>>>>>> Show when a particular plugin is installed. >> >>>>>>>>> >> >>>>>>>>> *MVP hopefully means you agree we can ship this >> >>>>>>>>> with a few warnings so people can get a hang of >> >>>>>>>>> it. >> >>>>>>>>> >> >>>>>>>>> Here is a rough list of features squared against >> >>>>>>>>> future milestones: >> >>>>>>>>> >> >>>>>>>>> Milestone 2: Be creator friendly - Make it easy to >> >>>>>>>>> build a CouchDB plugin by providing one or more >> >>>>>>>>> easy to start templates. - Make it easy to publish >> >>>>>>>>> new plugins and new versions of existing >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> plugins. >> >>>>>>>>> - Make it easy to supply packages for multiple >> >>>>>>>>> Erlang & CouchDB >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> versions. >> >>>>>>>>> >> >>>>>>>>> Milestone 3: Public registry - Instead of >> >>>>>>>>> hardcoding a list of plugins into Futon/Fauxton, we >> >>>>>>>>> load a list of applicable plugins from a central >> >>>>>>>>> (and configurable) plugins repository. - This >> >>>>>>>>> allows plugin authors to publish new plugins and >> >>>>>>>>> new versions of existing plugins independently. >> >>>>>>>>> >> >>>>>>>>> Milestone 4: Other Languages - Figure out how to >> >>>>>>>>> handle C-dependencies for Erlang plugins. - Figure >> >>>>>>>>> out how to allow other language plugins (c.f. >> >>>>>>>>> non-JS query servers) >> >>>>>>>>> >> >>>>>>>>> Milestone X: Later - Add some >> >>>>>>>>> account/identity/maybe crypto-web-of-trust system >> >>>>>>>>> for authors to publish “legit” plugins. - Sign & >> >>>>>>>>> verify individual releases. >> >>>>>>>>> >> >>>>>>>>> A few more things that can happen concurrently >> >>>>>>>>> depending on what plugins require: - Integrate >> >>>>>>>>> Erlang/JS tests in the installation - Integrate >> >>>>>>>>> docs >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> ## How it works >> >>>>>>>>> >> >>>>>>>>> This plugin system lives in `src/couch_plugins` and >> >>>>>>>>> is a tiny CouchDB module. >> >>>>>>>>> >> >>>>>>>>> It exposes one new API endpoint `/_plugins` that an >> >>>>>>>>> admin user can POST to. >> >>>>>>>>> >> >>>>>>>>> The additional Futon page lives at >> >>>>>>>>> /_utils/plugins.html it is hardcoded. >> >>>>>>>>> >> >>>>>>>>> Futon (or you) post an object to `/_plugins` with >> >>>>>>>>> four properties: >> >>>>>>>>> >> >>>>>>>>> { "name": "geocouch", // name of the plugin, must >> >>>>>>>>> be unique "url": "http://people.apache.org/~jan";, >> >>>>>>>>> // “base URL” for plugin >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> releases (see below) >> >>>>>>>>> "version": "couchdb1.2.x_v0.3.0-11-gd83ba22", // >> >>>>>>>>> whatever version >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> internal to the plugin >> >>>>>>>>> "checksums": { "R15B03": >> >>>>>>>>> "ZetgdHj2bY2w37buulWVf3USOZs=" // base64’d sha >> >>>>>>>>> hash >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> over the binary >> >>>>>>>>> } } >> >>>>>>>>> >> >>>>>>>>> `couch_plugins` then attempts to download a .tar.gz >> >>>>>>>>> from this location: >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> http://people.apache.org/~jan/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03.tar.gz >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>> >> It should be obvious how the URL is constructed from the POST data. >> >>>>>>>>> (This url is live, feel free to play around with >> >>>>>>>>> this tarball). >> >>>>>>>>> >> >>>>>>>>> Next it calculates the sha hash for the downloaded >> >>>>>>>>> .tar.gz file and matches it against the correct >> >>>>>>>>> version in the `checksums` parameter. >> >>>>>>>>> >> >>>>>>>>> If that succeeds, we unpack the .tar.gz file >> >>>>>>>>> (currently in `/tmp`, need to find a better place >> >>>>>>>>> for this) and adds the extracted directory to the >> >>>>>>>>> Erlang code path >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> (`code:add_path("/tmp/couchdb_plugins/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03/ebin")`) >> >>>>>>>>> >> >>>>>>> >> and loads the included application (`application:load(geocouch)`). >> >>>>>>>>> >> >>>>>>>>> Then it looks into the `./config` directory that >> >>>>>>>>> lives next to `ebin/` in the plugin directory for a >> >>>>>>>>> file `config.erlt` (“erl-terms”). with a list of >> >>>>>>>>> configuration parameters to load. We parse the file >> >>>>>>>>> and set the config directives one by one. >> >>>>>>>>> >> >>>>>>>>> If that all goes to plan, we report success back to >> >>>>>>>>> the HTTP caller. >> >>>>>>>>> >> >>>>>>>>> That’s it! :) >> >>>>>>>>> >> >>>>>>>>> It’s deceptively simple, probably does a few things >> >>>>>>>>> very wrong and leaves a few things open (see >> >>>>>>>>> above). >> >>>>>>>>> >> >>>>>>>>> One open question I’d like an answer for is finding >> >>>>>>>>> a good location to unpack & install the plugin >> >>>>>>>>> files that isn’t `tmp`. If the answer is different >> >>>>>>>>> for a pre-BigCouch/rcouch-merge and >> >>>>>>>>> post-BigCouch/rcouch- merge world, I’d love to know >> >>>>>>>>> :) >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> ## Code >> >>>>>>>>> >> >>>>>>>>> The main branch for this is 1867-feature-plugins: >> >>>>>>>>> >> >>>>>>>>> ASF: >> >>>>>>> https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=log;h=refs/heads/1867-feature-plugins >> >>>>>>>>> >> >>>>>>> >> GitHub: https://github.com/janl/couchdb/compare/1867-feature-plugins >> >>>>>>>>> >> >>>>>>>>> I created a branch on GeoCouch that adds a few >> >>>>>>>>> lines to its `Makefile` that shows how a binary >> >>>>>>>>> package is built: >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> https://github.com/janl/geocouch/compare/couchbase:couchdb1.3.x...couchdb1.3.x-plugins >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>> >> * * * >> >>>>>>>>> >> >>>>>>>>> I hope you like this :) Please comment and improve >> >>>>>>>>> heavily! >> >>>>>>>>> >> >>>>>>>>> Let me know if you have any questions :) >> >>>>>>>>> >> >>>>>>>>> If you have any criticism, please phrase it in a >> >>>>>>>>> way that we can use to improve this, thanks! >> >>>>>>>>> >> >>>>>>>>> Best, Jan -- >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>>> >> >>> >> >>> >> >>> >> >> >> >> >> > >> > -- Andy Wenk Hamburg - Germany RockIt! "CouchDB - Das Praxisbuch für Entwickler und Administratoren" http://www.galileocomputing.de/2462 http://www.couchdb-buch.de "PostgreSQL 8.4: Das Praxisbuch" http://www.galileocomputing.de/2008 http://www.pg-praxisbuch.de -----BEGIN PGP PUBLIC KEY BLOCK----- Version: OpenPGP.js v.1.20130712 Comment: http://openpgpjs.org xo0EUeJpHgED/34kUBUQNNT+3fcc621CLjzQZsuwYajo7Pj1hxtTcPbOo6Ci UbyGOlIhlSDBaiXGXsFKxtdp4z/os7NdFQstzh6QpHzppjbGzGkv/om49jJM SYLYkXyMDquhEQO47ovgOQUwJeO5qSzKE5fxftJQUjzHY1K673aA9D80uREM Jc1tABEBAAHNF0FuZHkgV2VuayA8YW5keUBubXMuZGU+wpwEEAEIABAFAlHi aR8JEAhxaGu1XIB2AAAfuAP/ZJXbv5wxAGCPridI/8Za9fXcccM0GmsG5ciH bkhE9bakLlexclv3Jb+iQ2Cyp2FFo1wzLSADRRMEz1EvFFUoDo/Wj2SUQnaq LNA8tKkRBuW0tUf88aK66TcdRINghhAcEqVJtwRIXF7fI5Arv6N+ql8heD3O 4/jA6c/8iExS0dE= =6ftE -----END PGP PUBLIC KEY BLOCK-----
