Re: [ANNOUNCE] Sue Lockwood elected as CouchDB committer
Awesome, welcome Sue. On 15 Aug 2013, at 2:40 AM, Jason Smith jason.h.sm...@gmail.com wrote: Welcome, Sue! On Thu, Aug 15, 2013 at 12:50 AM, Octavian Damiean odami...@linux.comwrote: Congratulations Sue! Well done. On Aug 14, 2013 6:13 PM, Noah Slater nsla...@apache.org wrote: Dear community, I am pleased to announce that the CouchDB Project Management Committee has elected Sue Lockwood as a CouchDB committer. Apache ID: deathbear IRC nick: deathbear Twitter: deathbearbrown By default, external contributions to the project follow the Review-Then-Commit model. Being a committer means you can follow the Commit-Then-Review model. In other words, Sue can now make changes at will. This election was made in recognition of Sue's existing contributions and commitment to the project. Please join me in extending a warm welcome to Sue! On behalf of the CouchDB PMC, -- Noah Slater https://twitter.com/nslater
Re: [REPORT] CouchDB
On Aug 14, 2013, at 22:24 , Noah Slater nsla...@apache.org wrote:
Should we blog this? Happy to run with that.
yeah, why not, it could be the first iteration of a monthly status report :)
Good idea!
Jan
--
On 14 August 2013 20:55, Jan Lehnardt j...@apache.org wrote:
Hey everyone,
I’ve just sent this report to the ASF board :)
Thanks to Noah for compiling this!
* * *
Apache CouchDB is a database that uses JSON for documents, JavaScript
for MapReduce queries, and regular HTTP for an API.
Releases
* 1.3.1 (27 June 2013)
*
http://www.apache.org/dist/couchdb/notes/1.3.1/apache-couchdb-1.3.1.html
Recent Activity
* Support added for latest Erlang/OTP releases.
* Preparation for 1.4.0 release cycle has begun.
* Work is underway to document an official project vision.
* Work is underway to release in Cloudant's Fauxton overhaul of the
CouchDB admin interface.
* Work is underway to merge in Cloudant's BigCouch fork which will
add clustering capabilities to CouchDB.
* Work is underway to merge in parts of Benoît Chesneau's rcouch fork
which improves the build system, refactored internal applications,
and additional user-facing functionality such as bonjour/zeroconf
awareness, global db changes feeds, replications and changes feeds
across views.
* Work is underway on a plugin system, with early results already
shared with the community.
* Cloudant announced the BigCouch merge, generating lots of press
activity.
* Cloudant hosted a CouchDB booth and CouchDB party at OSCON.
* Cloudant are organising 'Apache CouchDB Conf, Vancouver' on November
13th, 2013 (in line with ASF branding and CouchDB PMC cooperation).
Our GitHub comment notifications have not been set up yet, due to lack
of progress with necessary infrastructure tasks.
Community
Including the following additions, CouchDB has 28 committers and 9 PMC
members.
New committers:
* Dirk Ochtman
* Sue 'deathbear' Lockwood
No new PMC members.
Mailing list stats:
* announce
* 94 subscribers (''+38'')
* 1 message since May (''-1'')
* user
* 1441 subscribers (''-15'')
* 496 messages since May (''-561'')
* erlang
* 121 subscribers (''+16'')
* 7 messages since May (''-7'')
* dev
* 599 subscribers (''-2'')
* 1010 messages since May (''-996'')
* commits
* 107 subscribers (''+6'')
* 697 messages since May (''-262'')
Issues
No issues for the board at this time.
* * *
Jan
--
--
Noah Slater
https://twitter.com/nslater
signature.asc
Description: Message signed with OpenPGP using GPGMail
Re: Erlang vs JavaScript
On Aug 15, 2013, at 10:09 , Robert Newson rnew...@apache.org wrote: A big +1 to Jason's clarification of erlang vs native. CouchDB could have shipped an erlang view server that worked in a separate process and had the stdio overhead, to combine the slowness of the protocol with the obtuseness of erlang. ;) Evaluating Javascript within the erlang VM process intrigues me, Jens, how is that done in your case? I've not previously found the assertion that V8 would be faster than SpiderMonkey for a view server compelling since the bottleneck is almost never in the code evaluation, but I do support CouchDB switching to it for the synergy effects of a closer binding with node.js, but if it's running in the same process, that would change (though I don't immediately see why the same couldn't be done for SpiderMonkey). Off the top of my head, I don't know a safe way to evaluate JS in the VM. A NIF-based approach would either be quite elaborate or would trip all the scheduling problems that long-running NIF's are now notorious for. At a step removed, the view server protocol itself seems like the thing to improve on, it feels like that's the principal bottleneck. The code is here: https://github.com/couchbase/couchdb/tree/master/src/mapreduce I’d love for someone to pick this up and give CouchDB, say, a ./configure --enable-native-v8 option or a plugin that allows people to opt into the speed improvements made there. :) The choice for V8 was made because of easier integration API and more reliable releases as a standalone project, which I think was a smart move. IIRC it relies on a change to CouchDB-y internals that has not made it back from Couchbase to CouchDB (Filipe will know, but I doubt he’s reading this thread), but we should look into that and get us “native JS views”, at least as an option or plugin. CCing dev@. Jan -- B. On 15 August 2013 08:22, Jason Smith j...@apache.org wrote: Yes, to a first approximation, with a native view, CouchDB is basically running eval() on your code. In my example, I took advantage of this to build a nonstandard response to satisfy an application. (Instead of a 404, we sent a designated fallback document body.) But, if you accumulate the list in a native view, a JavaScript view, or a hypothetical Erlang view (i.e. a subprocess), from the operating system's perspective, the memory for that list will be allocated somewhere. Either the CouchDB process asks for X KB more memory, or its subprocess will ask for it. So I think the total system impact is probably low in practice. So I guess my point is not that native views are wrong, just they have a cost so you should weigh the cost/benefit for your own project. In the case of manage_couchdb, I wrote a JavaScript implementation; but since sometimes I have an emergency and I must find conflicts ASAP, I made an Erlang version because it is worth it. On Thu, Aug 15, 2013 at 2:05 PM, Stanley Iriele siriele...@gmail.comwrote: Whoa...OK...that I had no idea about...thanks for taking the time to go to that granularity, by the way. So does this mean that the process memory is shared? As apposed to living in its own space?.so if someone accumulates a large json object in a list function its chewing up couchdb's memory?... I guess I'm a little confused about what's in the same process and what isn't now On Aug 14, 2013 11:57 PM, Jason Smith j...@apache.org wrote: To me, an Erlang view is a view server which supports map, reduce, show, update, list, etc. functions in the Erlang language. (Basically it is implemented in Erlang.) A view server is a subprocess that runs beneath CouchDB which communicates with it over standard i/o. It is a different process in the operating system and only interfaces with the main server using the view server protocol (basically a bunch of JSON messages going back and forth). I do not know of an Erlang view server which works well and is currently maintained. A native view (shipped by CouchDB but disabled by default) is some corner-cutting. Code is evaluated directly by the primary CouchDB server. Since CouchDB is Erlang, the native query server is necessarily Erlang. The key difference is, your code is right there in the eye of the storm. You can call couch_server:open(some_db) and completely circumvent security and other invariants which CouchDB enforces. You can leak memory until the kernel OOM killer terminates CouchDB. It's not about the language, it's that is is running inside the CouchDB process. On Thu, Aug 15, 2013 at 1:36 PM, Stanley Iriele siriele...@gmail.com wrote: WaitI'm a tad confused here..Jason what is the difference between native views and Erlang views?... On Aug 14, 2013 11:16 PM, Jason Smith j...@apache.org wrote: Oh, also: They are **not** Erlang views. They are **native** views. We should emphasize the latter to remind ourselves about the security and
Re: Erlang vs JavaScript
On Thu, Aug 15, 2013 at 11:38 AM, Jan Lehnardt j...@apache.org wrote: On Aug 15, 2013, at 10:09 , Robert Newson rnew...@apache.org wrote: A big +1 to Jason's clarification of erlang vs native. CouchDB could have shipped an erlang view server that worked in a separate process and had the stdio overhead, to combine the slowness of the protocol with the obtuseness of erlang. ;) Evaluating Javascript within the erlang VM process intrigues me, Jens, how is that done in your case? I've not previously found the assertion that V8 would be faster than SpiderMonkey for a view server compelling since the bottleneck is almost never in the code evaluation, but I do support CouchDB switching to it for the synergy effects of a closer binding with node.js, but if it's running in the same process, that would change (though I don't immediately see why the same couldn't be done for SpiderMonkey). Off the top of my head, I don't know a safe way to evaluate JS in the VM. A NIF-based approach would either be quite elaborate or would trip all the scheduling problems that long-running NIF's are now notorious for. At a step removed, the view server protocol itself seems like the thing to improve on, it feels like that's the principal bottleneck. The code is here: https://github.com/couchbase/couchdb/tree/master/src/mapreduce I’d love for someone to pick this up and give CouchDB, say, a ./configure --enable-native-v8 option or a plugin that allows people to opt into the speed improvements made there. :) The choice for V8 was made because of easier integration API and more reliable releases as a standalone project, which I think was a smart move. IIRC it relies on a change to CouchDB-y internals that has not made it back from Couchbase to CouchDB (Filipe will know, but I doubt he’s reading this thread), but we should look into that and get us “native JS views”, at least as an option or plugin. CCing dev@. Jan -- Well on the first hand nifs look like a good idea but can be very problematic: - when the view computation take time it would block the full vm scheduling. It can be mitigated using a pool of threads to execute the work asynchronously but then can create other problems like memory leaking etc. - nifs can't be upgraded easily during hot upgrade - when a nif crash, all the vm crash. (Note that we have the same problem when using a nif to decode/encode json, it only works well with medium sized documents) One other way to improve the js handling would be removing the main bottleneck ie the serialization-deserialization we do on each step. Not sure if it exists but feasible, why not passing erlang terms from erlang to js and js to erlang? So at the end the deserialization would happen only on the JS side ie instead of having get erlang term encode to json send to js decode json process encode json send json decode json to erlang term store we sould just have get erlang term send over STDIO decode erlang term to JS object process encode to erlang term send erlang term store Erlang serialization is also very optimised. Both solutions could co-exist, that may worh a try and benchmark each... - benoit
Re: Erlang vs JavaScript
On Aug 15, 2013, at 11:53 , Benoit Chesneau bchesn...@gmail.com wrote: On Thu, Aug 15, 2013 at 11:38 AM, Jan Lehnardt j...@apache.org wrote: On Aug 15, 2013, at 10:09 , Robert Newson rnew...@apache.org wrote: A big +1 to Jason's clarification of erlang vs native. CouchDB could have shipped an erlang view server that worked in a separate process and had the stdio overhead, to combine the slowness of the protocol with the obtuseness of erlang. ;) Evaluating Javascript within the erlang VM process intrigues me, Jens, how is that done in your case? I've not previously found the assertion that V8 would be faster than SpiderMonkey for a view server compelling since the bottleneck is almost never in the code evaluation, but I do support CouchDB switching to it for the synergy effects of a closer binding with node.js, but if it's running in the same process, that would change (though I don't immediately see why the same couldn't be done for SpiderMonkey). Off the top of my head, I don't know a safe way to evaluate JS in the VM. A NIF-based approach would either be quite elaborate or would trip all the scheduling problems that long-running NIF's are now notorious for. At a step removed, the view server protocol itself seems like the thing to improve on, it feels like that's the principal bottleneck. The code is here: https://github.com/couchbase/couchdb/tree/master/src/mapreduce I’d love for someone to pick this up and give CouchDB, say, a ./configure --enable-native-v8 option or a plugin that allows people to opt into the speed improvements made there. :) The choice for V8 was made because of easier integration API and more reliable releases as a standalone project, which I think was a smart move. IIRC it relies on a change to CouchDB-y internals that has not made it back from Couchbase to CouchDB (Filipe will know, but I doubt he’s reading this thread), but we should look into that and get us “native JS views”, at least as an option or plugin. CCing dev@. Jan -- Well on the first hand nifs look like a good idea but can be very problematic: - when the view computation take time it would block the full vm scheduling. It can be mitigated using a pool of threads to execute the work asynchronously but then can create other problems like memory leaking etc. - nifs can't be upgraded easily during hot upgrade - when a nif crash, all the vm crash. Yeah totally, hence making the whole thing an option. (Note that we have the same problem when using a nif to decode/encode json, it only works well with medium sized documents) One other way to improve the js handling would be removing the main bottleneck ie the serialization-deserialization we do on each step. Not sure if it exists but feasible, why not passing erlang terms from erlang to js and js to erlang? So at the end the deserialization would happen only on the JS side ie instead of having get erlang term encode to json send to js decode json process encode json send json decode json to erlang term store we sould just have get erlang term send over STDIO decode erlang term to JS object process encode to erlang term send erlang term store Erlang serialization is also very optimised. Both solutions could co-exist, that may worh a try and benchmark each... I think we just want both solutions period, the embedded one will still be faster, but potentially a little less stable, and the external view server one will be slower but extremely robust. Users should be able to choose between them :) Best Jan -- signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [REVIEW] Docs update
Octavian, what do you think about: https://github.com/iriscouch/build-couchdb On 13 August 2013 09:58, Octavian Damiean odami...@linux.com wrote: On Fri, Aug 9, 2013 at 4:04 PM, Filippo Fadda filippo.fa...@programmazione.it wrote: Actually the Installation of Unix-like systems is full of you are driving me crazy things. I didn't try, but I'm pretty sure to install on RedHat, CentOS, Fedora you just need yum install couchdb like apt-get install couchdb on Debian style distro like Ubuntu. Let's apply the KIS principle, even to the documentation. It's OK having a detailed section that list all the dependencies but that's something, probably, used only by developers or people having troubles on their systems. So, also the UNIX section should be revised with the minimum required steps to install CouchDB on the various distro. All the Unix stuff should be moved to Install from source section, where find all the dependencies you need to compile CouchDB, tips and tricks, etc. It's really not that simple. For the bleeding edge distros a package manager solution might be enough, as they might have CouchDB in a relatively new version in their repository but most don't. Here is the current CouchDB version situation in different distros and versions of that distro. Arch Linux: 1.4.0 Debian Wheezy (stable): 1.2.0 Debian Jessie (testing): 1.2.0 Debian Sid (unstable): 1.2.0 Fedora 18: 1.2.1 Fedora 19: 1.2.2 Ubuntu 12.04 Precise Pangolin: 1.0.1 Ubuntu 12.10 Quantal Quetzal: 1.2.0 Ubuntu 13.04 Raring Ringtail: 1.2.0 Ubuntu 13.10 Saucy Salamander (still in development, might change): 1.2.0 So the overall picture doesn't look that good to me. None of those distributions provide an up-to-date stable version of CouchDB, only Arch Linux provides a bleeding edge version which is better than nothing but still not really satisfying in my opinion. The build from source procedures are required and we should probably try to create a build tool like the Mozilla guys have for Firefox. I'm talking about mach if anyone is wondering. That tools makes it really easy to build from source using one command. Just my 2cents on the Linux part as that's pretty much my area of expertise. Cheers, -- Octavian Damiean GitHub: https://github.com/mainerror -- Noah Slater https://twitter.com/nslater
Re: [REPORT] CouchDB
Done!
https://blogs.apache.org/couchdb/entry/board_report_august_2013
Being a volunteer organisation, we rely on you to help us promote releases.
Here is the official tweet:
https://twitter.com/CouchDB/status/368154559488610304
And Google+ posts:
https://plus.google.com/109226482722655790973/posts/biAq1iqfxfS
https://plus.google.com/109226482722655790973/posts/LkkruL5N2Ni
We also have a Hacker News item:
https://news.ycombinator.com/item?id=6221309
And several Reddit items:
http://www.reddit.com/r/programming/comments/1kgcar/couchdbs_board_report_august_2013/
http://www.reddit.com/r/Database/comments/1kgcm9/couchdbs_board_report_august_2013/
http://www.reddit.com/r/nosql/comments/1kgcni/couchdbs_board_report_august_2013/
http://www.reddit.com/r/CouchDB/comments/1kgcoe/couchdbs_board_report_august_2013/
Please take a moment to share these links with your friends.
Retweet us if you use Twitter, and upvote us if you use Google+, Hacker
News, or Reddit.
(Note that Hacker News users may want to navigate to the item from the
front page. I believe up votes are not counted when you link to an item
directly.)
If you like, you could monitor these threads and respond to people with
questions. This is a great opportunity for community outreach.
If you have any other promotion you'd like to highlight, please reply to
this thread.
P.S. Dirkjan, sorry for getting your name wrong in the report! :/ Can't
change that now.
But at least you have my apology on record. ⬏
On 15 August 2013 10:30, Jan Lehnardt j...@apache.org wrote:
On Aug 14, 2013, at 22:24 , Noah Slater nsla...@apache.org wrote:
Should we blog this? Happy to run with that.
yeah, why not, it could be the first iteration of a monthly status report
:)
Good idea!
Jan
--
On 14 August 2013 20:55, Jan Lehnardt j...@apache.org wrote:
Hey everyone,
I’ve just sent this report to the ASF board :)
Thanks to Noah for compiling this!
* * *
Apache CouchDB is a database that uses JSON for documents, JavaScript
for MapReduce queries, and regular HTTP for an API.
Releases
* 1.3.1 (27 June 2013)
*
http://www.apache.org/dist/couchdb/notes/1.3.1/apache-couchdb-1.3.1.html
Recent Activity
* Support added for latest Erlang/OTP releases.
* Preparation for 1.4.0 release cycle has begun.
* Work is underway to document an official project vision.
* Work is underway to release in Cloudant's Fauxton overhaul of the
CouchDB admin interface.
* Work is underway to merge in Cloudant's BigCouch fork which will
add clustering capabilities to CouchDB.
* Work is underway to merge in parts of Benoît Chesneau's rcouch fork
which improves the build system, refactored internal applications,
and additional user-facing functionality such as bonjour/zeroconf
awareness, global db changes feeds, replications and changes feeds
across views.
* Work is underway on a plugin system, with early results already
shared with the community.
* Cloudant announced the BigCouch merge, generating lots of press
activity.
* Cloudant hosted a CouchDB booth and CouchDB party at OSCON.
* Cloudant are organising 'Apache CouchDB Conf, Vancouver' on November
13th, 2013 (in line with ASF branding and CouchDB PMC cooperation).
Our GitHub comment notifications have not been set up yet, due to lack
of progress with necessary infrastructure tasks.
Community
Including the following additions, CouchDB has 28 committers and 9 PMC
members.
New committers:
* Dirk Ochtman
* Sue 'deathbear' Lockwood
No new PMC members.
Mailing list stats:
* announce
* 94 subscribers (''+38'')
* 1 message since May (''-1'')
* user
* 1441 subscribers (''-15'')
* 496 messages since May (''-561'')
* erlang
* 121 subscribers (''+16'')
* 7 messages since May (''-7'')
* dev
* 599 subscribers (''-2'')
* 1010 messages since May (''-996'')
* commits
* 107 subscribers (''+6'')
* 697 messages since May (''-262'')
Issues
No issues for the board at this time.
* * *
Jan
--
--
Noah Slater
https://twitter.com/nslater
--
Noah Slater
https://twitter.com/nslater
