Re: CouchDB 2.0 Release Candidate 3

[Jan Lehnardt]

> On 04 Aug 2016, at 14:01, Alexander Shorin <kxe...@gmail.com> wrote:
> 
> On Thu, Aug 4, 2016 at 2:58 PM, Jan Lehnardt <j...@apache.org> wrote:
>>  - specifically, Alexander Shorin is working on pulling together a 1.7.0 
>> release, that should include any deprecation warnings for things that are 
>> gone in 2.0 and onwards (either temporarily or permanently). We should get 
>> 1.7.0 out at the same time or before 2.0. — Alexander, what’s the state here?
> 
> Slowly in progress, I think we'll have something on this weekends. Stay tuned!

Awesome, thank you! :)

Jan
--

CouchDB 2.0 Release Candidate 3

[Jan Lehnardt]

Hi everyone,

I’m happy to announce release candidate 3 (three) of CouchDB 2.0 \o/

Up top on http://couchdb.apache.org/release-candidate/2.0/

Testing documentation here: 
https://docs.google.com/document/d/1BtndYr-0KDQTqBSLVdJoR_8C5ObYjT1RBo_Qyh5ykdQ/edit

Changes since RC2:

0b88400 update tests for COUCHDB-3084
e5e540f Fix dev/run: "content" should be "contents"
6415cd1 Remove incorrect logfile sed attempt in dev/run
4f6d586 Don't set "authenticated" property in _session unless we authenticated
99be496 [couch_log] Add format character for process exit traces
1ad1f0f Bump couch_replicator for rep_start_pids items arity fix
449aa87 Bump couch_mrview dep
7c66e23 Bump couch dep for logging fix in unit tests
6a2ffe5 Bump chttpd to fix multi-query type overrides
b424a4a Bump chttpd dep for logging updates
80dfcd5 Fix logging for eunit tests
7b3f5d0 Merge branch '3067-make-couch-log-smarter'
f7823b8 Use new couch_log version
094ceae Add EPL to LICENSE for trunc_io modules
4c972ab Add config file notes for new logging options
595 Remove lager and its dependencies

A milestone not exactly noticeable from the commits: Fully working Windows 
Build curtesy of Joan  — Thanks so much! Link forthcoming.

Joan and Klemens: start your machines for the Windows build of RC3 and the 
Docker image respectively, thanks! :)

* * *

As a reminder from last time: A few things that we have to do during the 
RC-phase:

- double, triple, and quadruple-check all LICENSE and NOTICE files to adhere to 
ASF policy and to make sure we don’t pull in any incompatible licenses (e.g. 
*GPL) as dependencies.

- docs: make sure we get docs.couchdb.org up to speed with 2.0 so it makes 
sense to people who get started. This is mainly adding cluster bits, as most of 
the rest stays the same.

- blog post series: is going well, thanks everybody! <3

- integrate the new logo everywhere (fauxton, docs, installers, website)

- prepare website update (should make big splash about cluster and replication, 
and replication-family projects)

- update breaking changes docs

- update deprecations warnings and docs
  - specifically, Alexander Shorin is working on pulling together a 1.7.0 
release, that should include any deprecation warnings for things that are gone 
in 2.0 and onwards (either temporarily or permanently). We should get 1.7.0 out 
at the same time or before 2.0. — Alexander, what’s the state here?


Best
Jan
-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB 2.0 Release Candidate 2

[Jan Lehnardt]

> On 21 Jul 2016, at 11:06, Jan Lehnardt <j...@apache.org> wrote:
> 
> Hi everyone,
> 
> I’m happy to announce release candidate 2 of CouchDB 2.0 \o/
> 
> Up top on http://couchdb.apache.org/release-candidate/2.0/
> 
> Testing documentation here: 
> https://docs.google.com/document/d/1BtndYr-0KDQTqBSLVdJoR_8C5ObYjT1RBo_Qyh5ykdQ/edit
> 
> Changes since RC1:
> 
>> bdb68b9 feat: improve `make rc`: show list of commits since last RC
>> 9b7511a feat: add `make rc` target. Requires `COUCH_RC` env var set to an 
>> int.
>> b68e094 Revise Windows installation readme
>> 79d9aaf Quote application paths on Windows
>> e07dfde Use couch_epi:decide for validate_dbname
>> dd8943e Copy LICENSE file into release
>> d40ce3e Copy couchjs to rel for Windows
>> 18d1764 Pull in replicator fix for connection:close handling
>> 7b06c98 More Windows fixes; make release now works
>> 9620ff6 Clean up ./dev/run output
>> d0b8d79 dep(couch): pull in compaction test fixes
>> 660c32f Remove unbuilt doc targets from Win build
>> 33c8ca6 Bump docs repo for Windows changes
>> 47a65c0 Add Windows Makefile, bump docs repo to match
>> 212ac60 pull fix for sorted=false view feature (COUCHDB-3060)
> 
> * * *
> 
> A few notes on the “RC process” that I’ve just made up on the spot
> (e.g. happy to hear about improvements =).
> 
> We’ll publish a new RC every Thursday

Sorry for the late email, but I’m holding RC3 for a little more cleanup work 
stopped by less individual availability than we had anticipated. I’ll publish 
it next Thursday the latest.

Best
Jan
--


> before the weekly news come out
> that includes any commits on master since the last RC and announce it on
> dev@, user@ and marketing@ like In this email above.
> 
> We do this until we are comfortable that everything is in a state that
> we’re happy to call a release.
> 
> * * *
> 
> A few things that we have to do during the RC-phase:
> 
> - double, triple, and quadruple-check all LICENSE and NOTICE files to adhere 
> to ASF policy and to make sure we don’t pull in any incompatible licenses 
> (e.g. *GPL) as dependencies.
> 
> - docs: make sure we get docs.couchdb.org up to speed with 2.0 so it makes 
> sense to people who get started. This is mainly adding cluster bits, as most 
> of the rest stays the same.
> 
> - blog post series: as outlined by Jenn, thanks to all the volunteers who 
> signed up already <3
> 
> - integrate the new logo everywhere (fauxton, docs, installers, website)
> 
> - prepare website update (should make big splash about cluster and 
> replication, and replication-family projects)
> 
> - update breaking changes docs
> 
> - update deprecations warnings and docs
> - specifically, Alexander Shorin is working on pulling together a 1.7.0 
> release, that should include any deprecation warnings for things that are 
> gone in 2.0 and onwards (either temporarily or permanently). We should get 
> 1.7.0 out at the same time or before 2.0.
> 
> - figure out a way for more people to test release candidates more 
> thoroughly. Elasticsearch has had good experience with an idea I suggested to 
> them: send swag to anyone who reports an issue on a release candidate. Maybe 
> Cloudant can spring for cool and exclusive CouchDB coffee/tea mugs with the 
> new logo via Spreadshirt/Cafepress/etc… (so we don’t have to worry about 
> fulfilment).
> 
> * * *
> 
> Let’s do this! :)
> 
> Jan
> -- 
> Professional Support for Apache CouchDB:
> https://neighbourhood.ie/couchdb-support/
> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Sharding question for clustered CouchDB 2.0

[Jan Lehnardt]

> On 25 Jul 2016, at 16:35, Peyton Vaughn  wrote:
> 
> I'm afraid I must echo Teo's question: how do I run compaction at the shard
> level?
> 
> Fauxton lists all of my shards as:
> 
> shards/-1fff/_global_changes.1469456629 This database failed to
> load.
> 
> So interaction there doesn't seem to be an option.
> I attempted to use curl, as outlined in the documentation:
> 
> curl -XPOST  http://localhost:15984//_compact
> 
> But I cannot figure out the correct database name to provide. All of my
> attempts result in a "not_found" or a "illegal_database_name" error
> returned.

The answer is the same, quoth @rnewson:

> You'll need to do so on port 5986, the node-local interface. 

That is [1-3]5986 in your dev cluster case.

Best
Jan
-- 

> 
> peyton
> 
> 
> 
> On Sat, Jul 23, 2016 at 2:32 PM, Robert Newson  wrote:
> 
>> You'll need to do so on port 5986, the node-local interface.
>> 
>> Sent from my iPhone
>> 
>>> On 23 Jul 2016, at 07:15, Constantin Teodorescu 
>> wrote:
>>> 
 On Sat, Jul 23, 2016 at 12:47 AM, Robert Newson 
>> wrote:
 
 Are you updating one doc over and over? That's my inference. Also you'll
 need to run compaction on all shards then look at the distribution
 afterward.
>>> 
>>> How do I run compaction on all shards?
>>> On Fauxton UI I didn't found anywhere any button for database or view
>>> compaction! :-(
>>> 
>>> Teo
>> 
>> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

CouchDB 2.0 Release Candidate 2

[Jan Lehnardt]

Hi everyone,

I’m happy to announce release candidate 2 of CouchDB 2.0 \o/

Up top on http://couchdb.apache.org/release-candidate/2.0/

Testing documentation here: 
https://docs.google.com/document/d/1BtndYr-0KDQTqBSLVdJoR_8C5ObYjT1RBo_Qyh5ykdQ/edit

Changes since RC1:

> bdb68b9 feat: improve `make rc`: show list of commits since last RC
> 9b7511a feat: add `make rc` target. Requires `COUCH_RC` env var set to an int.
> b68e094 Revise Windows installation readme
> 79d9aaf Quote application paths on Windows
> e07dfde Use couch_epi:decide for validate_dbname
> dd8943e Copy LICENSE file into release
> d40ce3e Copy couchjs to rel for Windows
> 18d1764 Pull in replicator fix for connection:close handling
> 7b06c98 More Windows fixes; make release now works
> 9620ff6 Clean up ./dev/run output
> d0b8d79 dep(couch): pull in compaction test fixes
> 660c32f Remove unbuilt doc targets from Win build
> 33c8ca6 Bump docs repo for Windows changes
> 47a65c0 Add Windows Makefile, bump docs repo to match
> 212ac60 pull fix for sorted=false view feature (COUCHDB-3060)

* * *

A few notes on the “RC process” that I’ve just made up on the spot
(e.g. happy to hear about improvements =).

We’ll publish a new RC every Thursday before the weekly news come out
that includes any commits on master since the last RC and announce it on
dev@, user@ and marketing@ like In this email above.

We do this until we are comfortable that everything is in a state that
we’re happy to call a release.

* * *

A few things that we have to do during the RC-phase:

- double, triple, and quadruple-check all LICENSE and NOTICE files to adhere to 
ASF policy and to make sure we don’t pull in any incompatible licenses (e.g. 
*GPL) as dependencies.

- docs: make sure we get docs.couchdb.org up to speed with 2.0 so it makes 
sense to people who get started. This is mainly adding cluster bits, as most of 
the rest stays the same.

- blog post series: as outlined by Jenn, thanks to all the volunteers who 
signed up already <3

- integrate the new logo everywhere (fauxton, docs, installers, website)

- prepare website update (should make big splash about cluster and replication, 
and replication-family projects)

- update breaking changes docs

- update deprecations warnings and docs
 - specifically, Alexander Shorin is working on pulling together a 1.7.0 
release, that should include any deprecation warnings for things that are gone 
in 2.0 and onwards (either temporarily or permanently). We should get 1.7.0 out 
at the same time or before 2.0.

- figure out a way for more people to test release candidates more thoroughly. 
Elasticsearch has had good experience with an idea I suggested to them: send 
swag to anyone who reports an issue on a release candidate. Maybe Cloudant can 
spring for cool and exclusive CouchDB coffee/tea mugs with the new logo via 
Spreadshirt/Cafepress/etc… (so we don’t have to worry about fulfilment).

* * *

Let’s do this! :)

Jan
-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Using CouchDB as a file system server side

[Jan Lehnardt]

> On 22 Jun 2016, at 10:20, Jason Smith  wrote:
> 
> On Wed, Jun 22, 2016 at 3:29 AM, Alexander Harm  wrote:
> 
>> I especially love the quote of Laurie Voss:
>> 
>> "One of the big things that everybody who's spent a lot of time with
>> databases knows is that you should never put your binaries in the database.
>> It's a terrible idea. It always goes wrong. I have never met a database in
>> 15 years of which it is not true, and it's definitely not true of CouchDB.
>> You are taking this thing which is meant to sort and organize data, and
>> you're giving it binary data, which it can neither sort nor organize. It
>> can't do anything with that data, other than get really fat.”
>> 
> 
> This is a pet peeve of mine.
> 
> "Do not store files in the database."
> 
> That simplistic mantra is up there with "[about regular expressions], now
> you have two problems." It is so simple, so naïve. It can be over-done, or
> under-analyzed.
> 
> Some thoughts.
> 
> 1. All large systems experience growing pains from off-the-shelf tools. All
> large projects must become custom. Twitter's growth is not an indictment of
> Ruby on Rails. Google's growth is not an indictment of the ext filesystem.
> 
> 2. Attachments in CouchDB are very simple. Simple is easy to ship. Simple
> is easy to maintain. Simple projects allow you to focus on other problems.
> 
> 3. Attachments in CouchDB make less sense as a project matures. Yes, I said
> it. But just be careful not to prematurely optimize.
> 
> Once you really grow, you will be investigating CDNs, and caching tools,
> and all sorts of alternatives. But how did you become that successful? How
> did you come to this "good problem"? Because you shipped, and you
> delivered, and you scaled. And at long last, you have earned the privilege
> of building something large and complex.
> 
> Laurie Voss joined npm and made that quote some years after npm came
> online. It had been growing exponentially for a few years. I remember
> clearly: npm was already processing 1,000 requests per second, all on
> plain-Jane CouchDB, with a very simple design.
> 
> "You didn't build that," as the man says. In my opinion, attachments played
> a part in npm's early success, and moving away from them played a part in
> npm's latter success. Both were wise decisions.

I second Jason’s observation. Attachment support can be crucial in getting
something off the ground and proving an idea. That will then usually unlock
the resources to scale things up as needed.

As a rule of thumb:

  Don’t ever trust one-line truisms in technology (including this one).

Best
Jan
-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB 1.6.1 on Debian 8

[Jan Lehnardt]

> On 03 Jun 2016, at 16:12, Sinan Gabel <sinan.ga...@gmail.com> wrote:
> 
> Hi!
> 
> I tried to install 1.x.x on Ubuntu 16.04 Desktop and it failed with roughly

What is the error precisely?

Best
Jan
--



> the same info as given above.
> 
> Br,
> Sinan
> 
> On 3 June 2016 at 05:32, Frédéric Audon <chave...@gmail.com> wrote:
> 
>> 
>> 
>> On 1 Jun 2016, at 7:43, Jan Lehnardt wrote:
>> 
>> On 01 Jun 2016, at 00:09, Frédéric Audon <chave...@gmail.com> wrote:
>>>> 
>>>> 
>>>> Le 31 mai 2016 à 18:59, Jan Lehnardt <j...@apache.org> a écrit :
>>>>> 
>>>>> 
>>>>> On 31 May 2016, at 18:57, Frédéric Audon <chave...@gmail.com> wrote:
>>>>>> 
>>>>>> 
>>>>>> Le 31 mai 2016 à 16:46, Jan Lehnardt <j...@apache.org> a écrit :
>>>>>>> 
>>>>>>> 
>>>>>>> On 31 May 2016, at 16:06, Frédéric Audon <chave...@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Le 30 mai 2016 à 22:59, Jan Lehnardt <j...@apache.org> a écrit :
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 30 May 2016, at 18:53, Frédéric Audon <chave...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hello
>>>>>>>>>> 
>>>>>>>>>> I try to install CouchDB 16.1 on Debian 8 ( jessie )
>>>>>>>>>> 
>>>>>>>>>> I stuck
>>>>>>>>>> checking Erlang version of compatibility ... Erlang detected
>>>>>>>>>> Version: 7.3.0 ... configure: error : The installed version of 
>>>>>>>>>> Erlang must
>>>>>>>>>> be> = R14B ( erts - 5.8.1 ) and = <17 ( erts -6.0 ) major_version 
>>>>>>>>>> does not
>>>>>>>>>> match
>>>>>>>>>> 
>>>>>>>>>> What should I do ?
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Best option probably is to build CouchDB from source, the 1.x.x
>>>>>>>>> branch is slightly ahead of 1.6.1 but safe to use in production, and
>>>>>>>>> compatible with newer Erlang releases.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> ok
>>>>>>>> 
>>>>>>>> wget https://github.com/apache/couchdb/archive/1.6.x.zip
>>>>>>>> unzip 1.6.x.zip
>>>>>>>> cd couchdb-1.6.x/
>>>>>>>> 
>>>>>>>> How do I get ./configure ?
>>>>>>>> 
>>>>>>> 
>>>>>>> ./bootstrap
>>>>>>> 
>>>>>> 
>>>>>> Unable to find any variant: glibtoolize libtoolize
>>>>>> 
>>>>>> Have you installed a version of this package?
>>>>>> 
>>>>> 
>>>>> You need to install libtool :)
>>>>> 
>>>>> Dev dependencies are explained in DEVELOPERS (
>>>>> https://github.com/apache/couchdb/blob/1.x.x/DEVELOPERS)
>>>>> 
>>>>> (you probably want `./configure --disable-docs` to avoid the LaTeX etc.
>>>>> dependency chain)
>>>>> 
>>>>> 
>>>> Thanks Jan
>>>> 
>>>> ./bootstrap OK
>>>> ./configure --prefix=/usr/local --with-js-lib=/usr/lib
>>>> --with-js-include=/usr/include/mozjs --enable-init OK
>>>> make && make install -->KO
>>>> 
>>>> make[2]: Leaving directory '/root/couchdb-1.6.x/utils’
>>>> 
>>> 
>>> looks like you are building 1.6.x and not 1.x.x?
>>> 
>>> Can you try 1.x.x?
>>> 
>>> 
>> gzip -9 < README.rst > README.gz
>> make[2]: *** No rule to make target 'THANKS', needed by 'THANKS.gz'.  Stop.
>> make[2]: Leaving directory '/var/lib/couchdb/couchdb-1.x.x'
>> Makefile:555: recipe for target 'all-recursive' failed
>> make[1]: *** [all-recursive] Error 1
>> make[1]: Leaving directory '/var/lib/couchdb/couchdb-1.x.x'
>> Makefile:453: recipe for target 'all' failed
>> 
>> make: *** [all] Error 2
>> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB 1.6.1 on Debian 8

[Jan Lehnardt]

> On 01 Jun 2016, at 00:09, Frédéric Audon <chave...@gmail.com> wrote:
> 
> 
>> Le 31 mai 2016 à 18:59, Jan Lehnardt <j...@apache.org> a écrit :
>> 
>> 
>>> On 31 May 2016, at 18:57, Frédéric Audon <chave...@gmail.com> wrote:
>>> 
>>> 
>>>> Le 31 mai 2016 à 16:46, Jan Lehnardt <j...@apache.org> a écrit :
>>>> 
>>>> 
>>>>> On 31 May 2016, at 16:06, Frédéric Audon <chave...@gmail.com> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>>> Le 30 mai 2016 à 22:59, Jan Lehnardt <j...@apache.org> a écrit :
>>>>>> 
>>>>>> 
>>>>>>> On 30 May 2016, at 18:53, Frédéric Audon <chave...@gmail.com> wrote:
>>>>>>> 
>>>>>>> Hello
>>>>>>> 
>>>>>>> I try to install CouchDB 16.1 on Debian 8 ( jessie )
>>>>>>> 
>>>>>>> I stuck 
>>>>>>> checking Erlang version of compatibility ... Erlang detected Version: 
>>>>>>> 7.3.0 ... configure: error : The installed version of Erlang must be> = 
>>>>>>> R14B ( erts - 5.8.1 ) and = <17 ( erts -6.0 ) major_version does not 
>>>>>>> match 
>>>>>>> 
>>>>>>> What should I do ?
>>>>>> 
>>>>>> Best option probably is to build CouchDB from source, the 1.x.x branch 
>>>>>> is slightly ahead of 1.6.1 but safe to use in production, and compatible 
>>>>>> with newer Erlang releases.
>>>>>> 
>>>>> 
>>>>> ok
>>>>> 
>>>>> wget https://github.com/apache/couchdb/archive/1.6.x.zip
>>>>> unzip 1.6.x.zip
>>>>> cd couchdb-1.6.x/
>>>>> 
>>>>> How do I get ./configure ?
>>>> 
>>>> ./bootstrap
>>> 
>>> Unable to find any variant: glibtoolize libtoolize
>>> 
>>> Have you installed a version of this package?
>> 
>> You need to install libtool :)
>> 
>> Dev dependencies are explained in DEVELOPERS 
>> (https://github.com/apache/couchdb/blob/1.x.x/DEVELOPERS)
>> 
>> (you probably want `./configure --disable-docs` to avoid the LaTeX etc. 
>> dependency chain)
>> 
> 
> Thanks Jan
> 
> ./bootstrap OK
> ./configure --prefix=/usr/local --with-js-lib=/usr/lib 
> --with-js-include=/usr/include/mozjs --enable-init OK
> make && make install -->KO
> 
> make[2]: Leaving directory '/root/couchdb-1.6.x/utils’

looks like you are building 1.6.x and not 1.x.x?

Can you try 1.x.x?

Best
Jan
--

> make[2]: Entering directory '/root/couchdb-1.6.x'
> make[2]: *** No rule to make target 'THANKS', needed by 'THANKS.gz'.  Stop.
> make[2]: Leaving directory '/root/couchdb-1.6.x'
> Makefile:551: recipe for target 'all-recursive' failed
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory '/root/couchdb-1.6.x'
> Makefile:451: recipe for target 'all' failed
> make: *** [all] Error 2
> 
> Every time, I use a clean install Debian 8

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB 1.6.1 on Debian 8

[Jan Lehnardt]

> On 31 May 2016, at 18:57, Frédéric Audon <chave...@gmail.com> wrote:
> 
> 
>> Le 31 mai 2016 à 16:46, Jan Lehnardt <j...@apache.org> a écrit :
>> 
>> 
>>> On 31 May 2016, at 16:06, Frédéric Audon <chave...@gmail.com> wrote:
>>> 
>>> 
>>> 
>>>> Le 30 mai 2016 à 22:59, Jan Lehnardt <j...@apache.org> a écrit :
>>>> 
>>>> 
>>>>> On 30 May 2016, at 18:53, Frédéric Audon <chave...@gmail.com> wrote:
>>>>> 
>>>>> Hello
>>>>> 
>>>>> I try to install CouchDB 16.1 on Debian 8 ( jessie )
>>>>> 
>>>>> I stuck 
>>>>> checking Erlang version of compatibility ... Erlang detected Version: 
>>>>> 7.3.0 ... configure: error : The installed version of Erlang must be> = 
>>>>> R14B ( erts - 5.8.1 ) and = <17 ( erts -6.0 ) major_version does not 
>>>>> match 
>>>>> 
>>>>> What should I do ?
>>>> 
>>>> Best option probably is to build CouchDB from source, the 1.x.x branch is 
>>>> slightly ahead of 1.6.1 but safe to use in production, and compatible with 
>>>> newer Erlang releases.
>>>> 
>>> 
>>> ok
>>> 
>>> wget https://github.com/apache/couchdb/archive/1.6.x.zip
>>> unzip 1.6.x.zip
>>> cd couchdb-1.6.x/
>>> 
>>> How do I get ./configure ?
>> 
>> ./bootstrap
> 
> Unable to find any variant: glibtoolize libtoolize
> 
> Have you installed a version of this package?

You need to install libtool :)

Dev dependencies are explained in DEVELOPERS 
(https://github.com/apache/couchdb/blob/1.x.x/DEVELOPERS)

(you probably want `./configure --disable-docs` to avoid the LaTeX etc. 
dependency chain)

Best
Jan
-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: final long term deletion,cleanup database, and id generation

[Jan Lehnardt]

CouchDB only rebuilds view indexes when two purge operations happen WITHOUT a 
view index update in between.

If you can guarantee that you have a vie index update after each purge, using 
purge for removing tombstones is a viable solution.

I’m short on time right now, but happy to write up how it all works tomorrow.

Best
Jan
--

> On 31 May 2016, at 17:26, Dale Harvey  wrote:
> 
> Does mango share the same disadvantages as map reduce with indexes having
> to be rebuilt on purge? In PouchDB it currently does however we are looking
> to change the pouchdb-find implementation so that purge has no negative
> effect on the query performance, at which point we will implement and be
> happy pointing users towards using purge (with the caveats about their
> affect on replication).
> 
> With PouchDB we have to be very aware about keeping local disk size to a
> minimum and recommending to users that they replicate to new databases in
> order to save disk size is not user friendly imo, so we are definitely
> interested in looking at ways to improve disk space usage. Moving purge
> away from 'avoid this' territory is one way.
> 
> On 31 May 2016 at 16:12, Stefan Klein  wrote:
> 
>> Hi,
>> 
>> 2016-05-31 17:00 GMT+02:00 david rene comba lareu <
>> shadow.of.sou...@gmail.com>:
>> 
>>> HI,
>>> 
>> 
>> [ ... ]
>> 
>> 
>>> 
>>> 1) when you delete a document, is supposed that revisions are only
>>> deleted after the cleanup, but the tombstones are still there, and it
>>> increase the amount of size exponentially. i found this post about it:
>>> 
>>> 
>> http://eclipsesource.com/blogs/2015/04/20/how-to-finally-delete-documents-in-couchdb/
>>> with a sort of solution, that i find tricky and insecure. i don't want
>>> be manipulating live DB's on live servers. any other option?
>>> 
>> 
>> I haven't double checked, but if i remember correctly purging 2 documents
>> will cause all views to be re-created.
>> Purge is really not to be meant for normal operation.
>> 
>> So the replication of non deleted documents to a new DB is - to my
>> knowledge - the only option.
>> 
>> Do you have a proxy in front of your couchdb?
>> If so, you do not even need to shut down and copy DB files, but you could
>> simply point the proxy to the new DB.
>> Remember to refresh the views before switching DBs.
>> 
>> --
>> Stefan
>> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB 1.6.1 on Debian 8

[Jan Lehnardt]

> On 31 May 2016, at 16:06, Frédéric Audon <chave...@gmail.com> wrote:
> 
> 
> 
>> Le 30 mai 2016 à 22:59, Jan Lehnardt <j...@apache.org> a écrit :
>> 
>> 
>>> On 30 May 2016, at 18:53, Frédéric Audon <chave...@gmail.com> wrote:
>>> 
>>> Hello
>>> 
>>> I try to install CouchDB 16.1 on Debian 8 ( jessie )
>>> 
>>> I stuck 
>>> checking Erlang version of compatibility ... Erlang detected Version: 7.3.0 
>>> ... configure: error : The installed version of Erlang must be> = R14B ( 
>>> erts - 5.8.1 ) and = <17 ( erts -6.0 ) major_version does not match 
>>> 
>>> What should I do ?
>> 
>> Best option probably is to build CouchDB from source, the 1.x.x branch is 
>> slightly ahead of 1.6.1 but safe to use in production, and compatible with 
>> newer Erlang releases.
>> 
> 
> ok
> 
> wget https://github.com/apache/couchdb/archive/1.6.x.zip
> unzip 1.6.x.zip
> cd couchdb-1.6.x/
> 
> How do I get ./configure ?

./bootstrap

> 
> --
> Best
> F.

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Deletion of documents

[Jan Lehnardt]

Heya Oleg,

thanks for trying 2.0! :)

Deleted docs are not a bad thing in general, but they can cause performance 
issues in certain cases if they appear in large numbers (100s of 1000s).

It might just be that Fauxton’s algorithm for showing the warning needs a bit 
relaxing.

There is no need to purge anything during normal operations. Purge is reserved 
for things like accidentally stored social security numbers or other 
immutable/hard to change items.

Best
Jan
--



> On 12 May 2016, at 21:22, Oleg Cohen  wrote:
> 
> Greetings,
> 
> I have a CouchDB 2.0 cluster. I notice when I delete documents, Fauxton 
> displays a warning next to the # of Docs. Here is an example of the warning: 
> "This database has just 1 docs and 3 deleted docs”. What does it mean and is 
> this a problem?
> 
> Also, how do I remove deleted documents from the database? So far the only 
> way I found is to run a _purge command on each shard on each node in the 
> cluster. Is this the correct way? Are there better ways to do it?
> 
> Thank you,
> Oleg
> 
> 
> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB 2.0 cluster problem when first node is down

[Jan Lehnardt]

Oleg, thanks for the report! This was reported before, but only very
recently and we are still looking into it. Can you maybe add your setup
details to the ticket at:

https://issues.apache.org/jira/browse/COUCHDB-3009?

Thank you!

Best
Jan
--

> On 06 May 2016, at 20:18, Oleg Cohen  wrote:
> 
> Greetings,
> 
> We ran into an issue when testing CouchDB 2.0 clustering. We ran a 2-node
> cluster using the dev/run -n 2 command.
> 
> If we bring node2 down, all on node1 is still working fine. When we bring
> down node1 using remsh and issuing init:stop(). command, the databases on
> node2 are no longer readable.
> 
> We test this by trying to read the _users database using the following
> command:
> 
> curl -X GET "http://127.0.0.1:25984/_users; --user admin:xxx
> 
> The get the following error:
> 
> {"error":"nodedown","reason":"progress not possible"}
> 
> If node1 is restarted, the problem goes away.
> 
> We experienced the same issue running a 3-node cluster across 3 different
> servers.
> 
> Wondering if anyone ran into the same issue and if there is a workaround or
> a way to fix the issue.
> 
> Thank you,
> Oleg
> 
> 
> -- 
> *Oleg Cohen  |  Principal  |  **A S S U R E B R I D G E*
> *Office: +1 617 564 0737  |  Mobile: +1 617 455 7927  |  Fax: +1 888 409
> 6995*
> *Email: oleg.co...@assurebridge.com   **|
> www.assurebridge.com *

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: How can I unsubscribe?

[Jan Lehnardt]

To unsubscribe, send an email to user-ubsubscr...@couchdb.apache.org from the 
email address that you have subscribed with.

Best
Jan
--

> On 28 Apr 2016, at 18:20, Diego .  wrote:
> 
> How can I unsubscribe?
> 
> Thanks.

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: ReduceMap

[Jan Lehnardt]

Heya,

the order of values is actually not defined, the decending parameter
only affects the map part of views.

Best
Jan
--
 
> On 21 Apr 2016, at 23:28, José Esteves  wrote:
> 
> Hi, i’m having some trouble understanding reduce value array parameter is
> ordered.
> 
> Having this data (the view called with reduce=false)
> 
> 
> {"total_rows":7,"offset":0,"rows":[
> {"id":"1","key":"1","value":{"anid":1,"dest":"A"}},
> {"id":"1","key":"1","value":{"anid":2,"dest":"A"}},
> {"id":"1","key":"1","value":{"anid":3,"dest":"A"}},
> {"id":"1","key":"1","value":{"anid":4,"dest":"A"}},
> {"id":"1","key":"1","value":{"anid":5,"dest":"A"}},
> {"id":"2","key":"2","value":{"anid":1,"dest":"B"}},
> {"id":"2","key":"2","value":{"anid":2,"dest":"B"}} ]}
> 
> with this functions
> 
> {
>  "_id": "_design/movs",
>  "_rev": "131-bfc19def2bea6bda0784a609b2ba3f25",
>  "views": {
>"unwind": {
>  "map": "function(doc)
> {doc.animals_ids.forEach(function(id){emit(doc._id, {anid: id, dest:
> doc.destiny}) })} ",
>  "reduce": "function(key,values){var res = [];var ac=[];var rt=
> {}; 
> values.forEach(function(i){if(ac.indexOf(i.anid)==-1){ac.push(i.anid);rt[i.dest]?rt[i.dest].push(i.anid):rt[i.dest]=[i.anid]
> }}); for(key in rt){res.push({dest: key, ids:rt[key],
> cant:rt[key].length}) }; return res }"
>}
>  },
>  "language": "javascript"
> }
> 
> I’m getting this result when calling the view without params
> 
> 
> {"rows":[ 
> {"key":null,"value":[{"dest":"B","ids":[1,2],"cant":2},{"dest":"A","ids":[3,4,5],"cant":3}]}
> ]}
> 
> But I was expecting to get to this result after passing the parameter
> descending=true not before. The only explanation I’ve arrived is that the
> value array in reduce function is generated on the inverse order of the map
> order.
> 
> The reduce function objective is to pass only the first  repetition of each
> id. When the same id appears on the loop it is ignore.   But newer
> documents, with higher id are taken precedence, over olders docs
> 
> 
> if someone could take a look to see if im doing something wrong I'll
> appreciate
> 
> 
> thanks,
> 
> 
> 
> 
> 
> 
> 
> ​

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Improve documentation shipped with CouchDB?

[Jan Lehnardt]

Heya Matthew,

thank you for offering to help. The docs live at 
https://github.com/apache/couchdb-documentation now :)

Best
Jan
-- 

> On 20 Apr 2016, at 11:35, Matthew Buckett  wrote:
> 
> Today I was trying to find out what version of JavaScript I could use with
> CouchDB 1.6.1 and in the end I downloaded the source and seeing that it
> required SpiderMonkey 1.8.5 to build, and looking on Mozilla's page saw it
> supports ECMAScript Edition 5 (I was hoping for 6).
> 
> https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Releases/1.8.5
> 
> I was going to quickly add this fact to the documentation about the
> JavaScript query engine, but the instructions for doing so in:
> 
> http://docs.couchdb.org/en/1.6.1/contributing.html?highlight=spidermonkey
> 
> point to a folder that doesn't currently exist in git (share/doc/src). I'm
> guessing that this is because the changes involved in CouchDB 2.0 but
> wondered where I should look to improve the documentation on this?
> 
> Thanks.

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Help with CouchDB Crash logs

[Jan Lehnardt]

** {{badmatch,{error,eacces}},

this means permission issues. Make sure everything in CouchDB’s database_dir 
and view_index–dir is read/writeable by the user that your CouchDB instance 
runs under.

Best
Jan
--
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/


> On 15 Apr 2016, at 17:02, Gene  wrote:
> 
> We are using CouchDB 1.6.1/CentOS Linux release 7.0.1406. CouchDB was 
> installed using `yum`.
> 
> We tried to run data conversion on some 100 databases. Most databases have 
> less than 1500 documents (around 1MB) except for 3 which have around 200,000 
> documents (around 250 MB). Conversion ran fine on few databases then we 
> started seeing `Error: connect ECONNREFUSED 127.0.0.1:5984` errors. 
> 
> Conversion steps:
> 
> Replicate `database_1` to `database_1_backup`.
> Delete `database_1`.
> Recreate `database_1`.
> Read documents from `database_1_backup` in memory.
> Write to `database_1` using bulkDocs.
> 
> Crash log:
> 
> [Wed, 13 Apr 2016 21:05:06 GMT] [info] [<0.2715.524>] starting new 
> replication `27dd24d1bd28e13225559e3e0a6c275a` at <0.5681.524> (`database_1` 
> -> `database_1_backup`)
> [Wed, 13 Apr 2016 21:05:07 GMT] [info] [<0.5681.524>] recording a checkpoint 
> for `database_1` -> `database_1_backup` at source update_seq 2209
> [Wed, 13 Apr 2016 21:05:07 GMT] [info] [<0.2715.524>]  - - POST 
> /_replicate 200
> [Wed, 13 Apr 2016 21:05:07 GMT] [info] [<0.31752.523>]  - - GET 
> /database_1_backup/ 200
> [Wed, 13 Apr 2016 21:05:07 GMT] [info] [<0.10914.524>]  - - GET 
> /database_1/ 200
> [Wed, 13 Apr 2016 21:05:07 GMT] [info] [<0.2623.524>]  - - GET 
> /database_1/ 200
> [Wed, 13 Apr 2016 21:05:07 GMT] [info] [<0.7567.524>]  - - DELETE 
> /database_1/ 200
> [Wed, 13 Apr 2016 21:05:07 GMT] [error] [<0.137.0>] ** Generic server 
> couch_index_server terminating
> ** Last message in was {'$gen_cast',{reset_indexes,<<"database_1">>}}
> ** When Server state == {st,"/var/lib/couchdb"}
> ** Reason for termination ==
> ** {{badmatch,{error,eacces}},
>[{couch_file,nuke_dir,2,[{file,"couch_file.erl"},{line,237}]},
> {couch_file,'-nuke_dir/2-fun-0-',3,[{file,"couch_file.erl"},{line,228}]},
> {lists,foreach,2,[{file,"lists.erl"},{line,1323}]},
> {couch_file,nuke_dir,2,[{file,"couch_file.erl"},{line,236}]},
> {couch_index_server,hafndle_cast,2,
> [{file,"src/couch_index_server.erl"},{line,117}]},
> {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,604}]},
> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}
> 
> Our seconds attempt to re-run the conversion completely crashed couchDB.
> 
> [Wed, 13 Apr 2016 22:17:19 GMT] [info] [<0.19197.0>] starting new replication 
> `6fe446668153db8635e9f49ddd8895f2` at <0.20012.0> (`database_2` -> 
> `database_2`)
> [Wed, 13 Apr 2016 22:17:19 GMT] [info] [<0.20012.0>] recording a checkpoint 
> for `database_2` -> `database_2` at source update_seq 1631
> [Wed, 13 Apr 2016 22:17:19 GMT] [error] [<0.20012.0>] Replication 
> `6fe446668153db8635e9f49ddd8895f2` (`database_2` -> `database_2`) failed: 
> {checkpoint_commit_failure,<<"Error updating the target checkpoint document: 
> conflict">>}
> [Wed, 13 Apr 2016 22:17:19 GMT] [error] [<0.20012.0>] ** Generic server 
> <0.20012.0> terminating
> ** Last message in was {'EXIT',<0.20027.0>,normal}
> ** When Server state == {rep_state,
> {rep,
>  {"6fe446668153db8635e9f49ddd8895f2",[]},
>  <<"database_2">>,<<"database_2">>,
>  [{checkpoint_interval,5000},
>   {connection_timeout,3},
>   {http_connections,20},
>   {retries,10},
>   {socket_options,[{keepalive,true},{nodelay,false}]},
>   {use_checkpoints,true},
>   {worker_batch_size,500},
>   {worker_processes,4}],
> 
> 
> erl_crash.dump - https://paste.ee/r/EWRYV 
> 
> SeLinux is not an issue here, at least not this time. 
> 
> Any help would be greatly appreciated debugging this crash log.
> 
> Thanks,
> 
> Sajin Shrestha
> 

Re: [NEWS] Your links for the CouchDB Weekly News?

[Jan Lehnardt]

Hi Glen,

you can unsubscribe by sending an email to user-unsubscr...@couchdb.apache.org 
:)

Best
Jan
--

> On 04 Apr 2016, at 14:55, Glen Aidukas <gaidu...@behaviormatrix.com> wrote:
> 
> I love and use CouchDB but I don't have a need for all these emails so please 
> unsubscribe me please.
> 
> Thanks,
> 
> Glen 
> 
> 
> -Original Message-
> From: Jenn Turner [mailto:j...@thehoodiefirm.com] 
> Sent: Sunday, April 3, 2016 1:50 PM
> Subject: [NEWS] Your links for the CouchDB Weekly News?
> 
> Happy April ☔️ CouchDB friends!
> 
> It’s that time again. If you want to submit a link for this week’s CouchDB 
> Weekly News, please don’t hesitate and send it to this thread until Thursday, 
> April 7, 11 am UTC+2. 
> 
> Thanks, have a great rest of your day 
>  Jenn 
> 
> —
> Jenn Turner
> The Neighbourhoodie Software GmbH
> Adalbertstr. 7-8, 10999 Berlin
> neighbourhood.ie
> 
> Handelsregister HRB 157851 B Amtsgericht Charlottenburg
> Geschäftsführung: Jan Lehnardt
> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Couchdb 2.0 start error

[Jan Lehnardt]

> On 15 Mar 2016, at 23:38, Ying Bian <yb...@me.com> wrote:
> 
> That did the trick! The command returned {ok: true} and now the errors are 
> gone. Thanks a lot, Jan!
> 
> But what caused this? I don’t think this is a normal step for users to do 
> during installation. Will we fix this or at least add it in the installation 
> instruction?  Do I need to log an issue?

creating system databases needs to be done after the cluster is set up, so all 
nodes get a copy, that’s why 2.0 doesn’t create _users on startup unlike 1.x. 
We should definitely create a better error message there.

If you set up a cluster, the Fauxton setup wizard takes care of this for you. 
If you run 2.0 as a single node, you have to do this manually and I just 
realised this is missing from 
https://docs.google.com/document/d/1BtndYr-0KDQTqBSLVdJoR_8C5ObYjT1RBo_Qyh5ykdQ/edit#
 — so if anyone feels like adding this, please go ahead! <3


> Thanks to you guys again! The timely response I got here makes me more 
> confident to choose couchdb. ;-)

Thank you! And thanks for testing 2.0, please let us know how it goes!

Best
Jan
--

> 
> -Ying
> 
>> 在 2016年3月15日，18:16，Jan Lehnardt <j...@apache.org> 写道：
>> 
>> When you get this error, CouchDB should still be running. If that’s the 
>> case, can you try this: curl -X PUT http://127.0.0.1:5984/_users
>> 
>> Best
>> Jan
>> --
>>> On 14 Mar 2016, at 23:56, Ying Bian <yb...@me.com> wrote:
>>> 
>>> I did everything using a normal user so I can just run couchdb using that 
>>> user. I corrected the permissions of some files caused by sudo and then ran 
>>> ‘couchdb’ using the normal user. Now I have the same errors as root:
>>> 
>>> 06:46:01.455 [error] gen_server chttpd_auth_cache terminated with reason: 
>>> {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,282}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,270}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,286}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,85}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},...]}
>>> 06:46:01.457 [error] CRASH REPORT Process chttpd_auth_cache with 0 
>>> neighbours exited with reason: 
>>> {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,282}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,270}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,286}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,85}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},...]}
>>>  in gen_server:terminate/7 line 826
>>> 06:46:01.458 [error] Supervisor chttpd_sup had child undefined started with 
>>> chttpd_auth_cache:start_link() at <0.312.0> exit with reason 
>>> {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,282}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,270}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,286}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,85}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},...]}
>>>  in context child_terminated
>>> 
>>> Anything that I missed? The same used to work. Again, my system is OS X 
>>> 10.11.3.
>>> 
>>> -Ying
>>> 
>>>> 在 2016年3月14日，23:15，Alexander Shorin <kxe...@gmail.com> 写道：
>>>> 
>>>> On Mon, Mar 14, 2016 at 6:11 PM, Ying Bian <yb...@me.com> wrote:
>>>>> {badmatch,{error,eacces}
>>>> 
>>>> Permissions. Don't run anything with sudo. Setup couchdb user and use
>>>> it instead.
>>>> 
>>>> --
>>>> ,,,^..^,,,
>>> 
>> 
>> -- 
>> Professional Support for Apache CouchDB:
>> https://neighbourhood.ie/couchdb-support/
>> 
> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Thank you Jan, Garren and Jenn

[Jan Lehnardt]

Thanks for the shout-out James! :)

Best
Jan
--
> On 16 Mar 2016, at 17:04, James Bertino <jbert...@royalpalmsoftware.com> 
> wrote:
> 
> To all,
> For those that did not know and have a little time the link below is to a 
> podcast that Jan Lehnardt and Garren Smith discuss CoudhDb 2.0. I am too busy 
> with 1.6.1 to play with 2.x so it was neat for me to sit back with some 
> popcorn and learn about 2.0.
> https://twit.tv/shows/floss-weekly/episodes/378?autostart=false
> 
> Then I had an added bonus...
> Yesterday Jan Lehnardt and Jenn Turner discussed Hoodie.
> https://twit.tv/shows/floss-weekly/episodes/379?autostart=false
> 
> I am not affiliated in any way with Twit.tv. I just like to watch the 
> podcasts they produce. It was quite surprising to me to see the folks that I 
> feel I already know from CouchDb.
> 
> Regards,
> James Bertino
> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Couchdb 2.0 start error

[Jan Lehnardt]

When you get this error, CouchDB should still be running. If that’s the case, 
can you try this: curl -X PUT http://127.0.0.1:5984/_users

Best
Jan
--
> On 14 Mar 2016, at 23:56, Ying Bian  wrote:
> 
> I did everything using a normal user so I can just run couchdb using that 
> user. I corrected the permissions of some files caused by sudo and then ran 
> ‘couchdb’ using the normal user. Now I have the same errors as root:
> 
> 06:46:01.455 [error] gen_server chttpd_auth_cache terminated with reason: 
> {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,282}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,270}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,286}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,85}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},...]}
> 06:46:01.457 [error] CRASH REPORT Process chttpd_auth_cache with 0 neighbours 
> exited with reason: 
> {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,282}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,270}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,286}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,85}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},...]}
>  in gen_server:terminate/7 line 826
> 06:46:01.458 [error] Supervisor chttpd_sup had child undefined started with 
> chttpd_auth_cache:start_link() at <0.312.0> exit with reason 
> {database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,282}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,270}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,286}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,85}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,38}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,187}]},...]}
>  in context child_terminated
> 
> Anything that I missed? The same used to work. Again, my system is OS X 
> 10.11.3.
> 
> -Ying
> 
>> 在 2016年3月14日，23:15，Alexander Shorin  写道：
>> 
>> On Mon, Mar 14, 2016 at 6:11 PM, Ying Bian  wrote:
>>> {badmatch,{error,eacces}
>> 
>> Permissions. Don't run anything with sudo. Setup couchdb user and use
>> it instead.
>> 
>> --
>> ,,,^..^,,,
> 

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: Multiple database backup strategy

[Jan Lehnardt]

> On 09 Mar 2016, at 21:29, Nick Wood  wrote:
> 
> Hello,
> 
> I'm looking to back up a CouchDB server with multiple databases. Currently
> 1,400, but it fluctuates up and down throughout the day as new databases
> are added and old ones deleted. ~10% of the databases are written to within
> any 5 minute period of time.
> 
> Goals
> - Maintain a continual off-site snapshot of all databases, preferably no
> older than a few seconds (or minutes)
> - Be efficient with bandwidth (i.e. not copy the whole database file for
> every backup run)
> 
> My current solution watches the global _changes feed and fires up a
> continuous replication to an off-site server whenever it sees a change. If
> it doesn't see a change from a database for 10 minutes, it kills that
> replication. This means I only have ~150 active replications running on
> average at any given time.

How about instead of using continuous replications and killing them,
use non-continuous replications based on _db_updates? They end
automatically and should use fewer resources then.

Best
Jan
--



> 
> I thought this was a pretty clever approach, but I can't stabilize it.
> Replications hang frequently with crashes in the log file. I haven't yet
> tracked down the source of the crashes. I'm running the official 1.6.1
> docker image as of yesterday so I don't think it would be an erlang issue.
> 
> Rather than keep banging my head against these stability issues, I thought
> I'd ask to see if anyone else has come up with a clever backup solution
> that meets the above goals?
> 
>  Nick

-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/

Re: CouchDB crash during compaction with no log messages

[Jan Lehnardt]

nup 202
> [debug] [<0.987.0>] CouchDB swapping files 
> /usr/local/var/lib/couchdb/tobacco_cessation.couch and 
> /usr/local/var/lib/couchdb/tobacco_cessation.couch.compact.
> [info] [<0.987.0>] Compaction for db "tobacco_cessation" completed.
> [debug] [<0.815.0>] 'POST' /trackers/_compact {1,1} from "127.0.0.1"
> Headers: [{'Accept',"*/*"},
>  {'Content-Type',"application/json"},
>  {'Host',"localhost:5984"},
>  {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
> NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}]
> [debug] [<0.815.0>] OAuth Params: []
> [info] [<0.1011.0>] Starting compaction for db "trackers"
> [debug] [<0.1016.0>] Compaction process spawned for db "trackers"
> [info] [<0.815.0>] 127.0.0.1 - - POST /trackers/_compact 202
> [debug] [<0.866.0>] 'POST' /trackers/_view_cleanup {1,1} from "127.0.0.1"
> Headers: [{'Accept',"*/*"},
>  {'Content-Type',"application/json"},
>  {'Host',"localhost:5984"},
>  {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
> NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}]
> [debug] [<0.866.0>] OAuth Params: []
> [info] [<0.866.0>] 127.0.0.1 - - POST /trackers/_view_cleanup 202
> [debug] [<0.867.0>] 'POST' /users/_compact {1,1} from "127.0.0.1"
> Headers: [{'Accept',"*/*"},
>  {'Content-Type',"application/json"},
>  {'Host',"localhost:5984"},
>  {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
> NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}]
> [debug] [<0.867.0>] OAuth Params: []
> [info] [<0.1029.0>] Starting compaction for db "users"
> [debug] [<0.1034.0>] Compaction process spawned for db "users"
> [info] [<0.867.0>] 127.0.0.1 - - POST /users/_compact 202
> [debug] [<0.87.0>] New task status for <0.1034.0>: [{changes_done,0},
>{database,<<"users">>},
>{progress,0},
>{started_on,1457453395},
>{total_changes,0},
>{type,database_compaction},
>{updated_on,1457453395}]
> [debug] [<0.884.0>] 'POST' /users/_view_cleanup {1,1} from "127.0.0.1"
> Headers: [{'Accept',"*/*"},
>  {'Content-Type',"application/json"},
>  {'Host',"localhost:5984"},
>  {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
> NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}]
> [debug] [<0.884.0>] OAuth Params: []
> [info] [<0.884.0>] 127.0.0.1 - - POST /users/_view_cleanup 202
> [debug] [<0.1029.0>] CouchDB swapping files 
> /usr/local/var/lib/couchdb/users.couch and 
> /usr/local/var/lib/couchdb/users.couch.compact.
> [info] [<0.1029.0>] Compaction for db "users" completed.
> [debug] [<0.885.0>] 'POST' /weight/_compact {1,1} from "127.0.0.1"
> Headers: [{'Accept',"*/*"},
>  {'Content-Type',"application/json"},
>  {'Host',"localhost:5984"},
>  {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
> NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}]
> [debug] [<0.885.0>] OAuth Params: []
> [info] [<0.1053.0>] Starting compaction for db "weight"
> [debug] [<0.1058.0>] Compaction process spawned for db "weight"
> [info] [<0.885.0>] 127.0.0.1 - - POST /weight/_compact 202
> [debug] [<0.902.0>] 'POST' /weight/_view_cleanup {1,1} from "127.0.0.1"
> Headers: [{'Accept',"*/*"},
>  {'Content-Type',"application/json"},
>  {'Host',"localhost:5984"},
>  {'User-Agent',"curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 
> NSS/3.18 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2"}]
> [debug] [<0.902.0>] OAuth Params: []
> [info] [<0.902.0>] 127.0.0.1 - - POST /weight/_view_cleanup 202
> 
> Crash dump was written to: erl_crash.dump
> eheap_alloc: Cannot allocate 156725600 bytes of memory (of type "old_heap").
> 
> [gtarsa@prod-db01 ~]$ 
> 
> 
> 
>> On Mar 7, 2016, at 4:09 PM, Jan Lehnardt <j...@apache.org> wrot

Re: CouchDB crash during compaction with no log messages

[Jan Lehnardt]

Heya Greg,

this should definitely not happen at all, regardless of AWS storage type.

Are there any other things going on on the VM, when you do this?

Can you reliably reproduce this behaviour?

Are there other correlating factors (like does this always happen at the same 
time / due to a cronjob, etc)?

Can you set your CouchDB log level to debug and see if that gets you more info? 
(curl -X PUT http://[user:pass@]127.0.0.1:5984/_config/log/level -d '"debug"').

Is it possible for you to share these database files (publicly or in private)?

What are your disk usage levels before/during compaction?

Are you getting anything in the system log(s)?

Best
Jan
-- 
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/


> On 07 Mar 2016, at 21:27, Greg Tarsa  wrote:
> 
> We have a set of couchdb databases that we use to collect user information 
> for various purposes.  I am inheriting this configuration from a predecessor 
> and am relatively new to couchdb.
> 
> Whenever we attempt to compact the databases, the server crashes without any 
> messages either in the couchdb log or the system logs.  This is running in an 
> AWS instance with an EBS volume.
> 
> Experiments have shown that if the instance is configured with instance 
> storage (ephemeral storage that disappears when the instance disappears) then 
> this operation works properly.   But we would like to use larger volumes and 
> have persistence.
> 
> When the instance is configured with an external EBS volume, then we see the 
> server crash described above.
> 
> I have searched the web for “couchdb compaction crash no log” and not found 
> anything helpful.
> 
> It seems like compacting while running should not be failing at all, much 
> less silently, so I am looking for insights to the problem, or solutions if 
> such exist.
> 
> Configuration and log info is below.
> 
> Any help would be appreciated.
> 
> Thanks,
> Greg
> 
> 
> -
> 
> CouchDB version: 1.6.1
> OS: RHEL 6.6
> 
> -
> 
> Here is a directory of the databases as the time of the crash:
> 
> cat bad.couch.dbinfo.txt 
> total 15400740
> 12 -rw-r--r--. 1 couchdb couchdb   8297 Jan 20 16:31 _users.couch
> 16 -rw-r--r--. 1 couchdb couchdb  12393 Jan 20 16:33 _replicator.couch
>  21060 -rw-r--r--. 1 couchdb couchdb   21557368 Mar  7 11:57 biometrics.couch
> 781136 -rw-r--r--. 1 couchdb couchdb  799875192 Mar  7 12:00 fitness.couch
> 954244 -rw-r--r--. 1 couchdb couchdb  977137784 Mar  7 12:05 nutrition.couch
> 8419624 -rw-r--r--. 1 couchdb couchdb 8621678721 Mar  7 12:06 routine.couch
> 390796 -rw-r--r--. 1 couchdb couchdb  400167032 Mar  7 12:06 sleep.couch
> 217932 -rw-r--r--. 1 couchdb couchdb  223154296 Mar  7 12:06 weight.couch
> 4614884 -rw-r--r--. 1 couchdb couchdb 4725629060 Mar  7 12:06 trackers.couch
>  4 -rw-r--r--. 1 couchdb couchdb 79 Mar  7 12:41 
> fitness.couch.compact
>  4 -rw-r--r--. 1 couchdb couchdb 79 Mar  7 12:41 
> nutrition.couch.compact
>  4 -rw-r--r--. 1 couchdb couchdb 79 Mar  7 12:41 
> routine.couch.compact
> 64 -rw-r--r--. 1 couchdb couchdb  61551 Mar  7 12:41 diabetes.couch
>  4 -rw-r--r--. 1 couchdb couchdb 79 Mar  7 12:41 
> sleep.couch.compact
> 12 -rw-r--r--. 1 couchdb couchdb   8300 Mar  7 12:41 
> tobacco_cessation.couch
>  4 -rw-r--r--. 1 couchdb couchdb 79 Mar  7 12:41 users.couch
>  4 -rw-r--r--. 1 couchdb couchdb 79 Mar  7 12:41 
> weight.couch.compact
>152 -rw-r--r--. 1 couchdb couchdb 151797 Mar  7 12:42 
> trackers.couch.compact
>784 -rw-r--r--. 1 couchdb couchdb 801865 Mar  7 12:42 
> biometrics.couch.compact
> 
> -
> 
> Here is the contents of the log at the time of the crash:
> 
> [Mon, 07 Mar 2016 17:25:32 GMT] [info] [<0.31.0>] Apache CouchDB has started 
> on http://0.0.0.0:5984/
> [Mon, 07 Mar 2016 17:25:32 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /_active_tasks 200
> [Mon, 07 Mar 2016 17:25:33 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /favicon.ico 200
> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /_active_tasks 200
> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /favicon.ico 200
> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /_active_tasks 200
> [Mon, 07 Mar 2016 17:25:34 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /favicon.ico 200
> [Mon, 07 Mar 2016 17:25:35 GMT] [info] [<0.123.0>] 10.1.1.12 - - GET /users/ 
> 200
> [Mon, 07 Mar 2016 17:25:37 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /_active_tasks 200
> [Mon, 07 Mar 2016 17:25:37 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /favicon.ico 200
> [Mon, 07 Mar 2016 17:25:38 GMT] [info] [<0.108.0>] 127.0.0.1 - - GET 
> /_active_tasks 200
> [Mon, 07 Mar 2016 

Re: CouchDB 2.0 performance issue

[Jan Lehnardt]

> On 04 Mar 2016, at 16:52, Peyton Vaughn  wrote:
> 
> Thanks for the help! Turning delayed_commits back on did indeed make a big
> difference. I had run across that setting already in the docs, but just
> assumed it was set to false in 1.6 as well (I see in the docs sometimes
> there are annotations like "Changed in version 1.2" - this might be a good
> candidate for a similar footnote :)
> 
> One follow-up question: will couch attempt to flush the write buffer on
> receiving a TERM signal? (which is a typical way to shutdown in a docker
> environment)

It will not :)

Best
Jan
--

> 
> On Thu, Mar 3, 2016 at 4:30 PM, Alexander Shorin  wrote:
> 
>> Hi,
>> 
>> The main reason is that 1.6 has couchdb/delayed_commits[1] = true by
>> default while 2.0 has it false instead to safe you from fun of
>> distributed issues. No delayed commits means that every write
>> operation hits the disk with fsync() call without any intermediate
>> buffering which delayed commits create (for 5 seconds). Add here a
>> short overhead for cluster wide communications and here is your
>> difference.
>> 
>> P.S. Yes, that's annoying drawback, but so far there is nothing to do
>> with it, except setup real cluster and balance the load over it.
>> 
>> [1]:
>> http://docs.couchdb.org/en/1.6.1/config/couchdb.html#couchdb/delayed_commits
>> --
>> ,,,^..^,,,
>> 
>> 
>> On Fri, Mar 4, 2016 at 12:24 AM, Peyton Vaughn 
>> wrote:
>>> Hello,
>>> 
>>> I've been using 1.6 for several months now, and wanted to try out 2.0.
>> But
>>> from the start, I'm experiencing much slower performance with 2.0. In
>> both
>>> cases I'm using the docker images (klaemo/couchdb:1.6 and
>>> klaemo/couchdb:2.0-dev), with a small program that pushes a few thousand
>>> documents to benchmark. The same program takes significantly longer to
>>> finish when I push to the couch 2.0 instance (~45 seconds vs several
>>> minutes).
>>> 
>>> I run the containers on the same 4-cpu virtual machine (not at the same
>>> time of course). I run the 2.0 instance as instructed on docker hub:
>>> docker run -p 5984:5984  --rm klaemo/couchdb:2.0-dev
>>> --with-admin-party-please   --with-haproxy -n 1
>>> 
>>> The difference in performance is big enough that I figure it must only be
>>> me, but I don't know what I might be doing wrong.
>>> 
>>> Any insight would be greatly appreciated!
>>> 
>>> Peyton Vaughn
>> 

The 2016 Annual* Apache CouchDB User Survey

[Jan Lehnardt]

Hey everyone,

The Apache CouchDB project is inviting its user base to participate in the 2016 
Annual* Apache CouchDB User Survey.

The development team is requesting your input to get an idea about the use of 
CouchDB in the field. The more we know about our users, the better we can make 
CouchDB for you.

Thank you in advance for taking the time to help the CouchDB project! <3


  
http://blog.couchdb.org/2016/03/01/the-2016-annual-apache-couchdb-user-survery/


Best
Jan
--
* This is the first year ;)

Re: Contract Dev Help

[Jan Lehnardt]

Heya Jeff, dear list,

I replied in private. It is totally fine to ask here for this kind of help!

And if any one here is ever in trouble, my company is available for hire to do 
troubleshooting and support for CouchDB. :)

Best
Jan
--
> On 13 Feb 2016, at 00:06, Jeff Charette  wrote:
> 
> Hi everyone,
> 
> Not sure if this is the right place to ask, but we are running CouchDB in a 
> production environment and are having serious performance issues on only 7000 
> documents.  I am looking for someone I can pay to take a look, as even after 
> 6 years of Couchdb experience I am at my wits end as to what the issue is. 
> I’m happy to copy and paste multiple verbose logs, but I believe in the end 
> we may need someone with a bit more experience to take a look.
> 
> We are using the changes feed with event source for syncing approximately 300 
> devices and are crashing a 16GB 8 core instance on Digital ocean.  Beam.smp 
> seams to eat all the memory over a 24 hour period.  Any help or guidance 
> would be greatly appreciated.
> 
> Best,
> 
> 
> Jeff Charette | Principal 
> We Are Charette
> web / identity / packaging
> 
> w  wearecharette.com 
> 
> IRS CIRCULAR 230:  Unless explicitly stated to the contrary, this 
> communication (including any attachments) is not intended or written to be 
> used, and cannot be used, for the purpose of (i) avoiding penalties under the 
> Internal Revenue Code or (ii) promoting, marketing, or recommending to 
> another party any transaction or matter addressed herein.
> 
> CONFIDENTIALITY:  This email and any attachments may be confidential and/or 
> privileged and are therefore protected against copying, use, disclosure or 
> distribution.  If you are not the intended recipient, please notify us 
> immediately by replying to the sender and double deleting this copy and the 
> reply from your system.
> 
> 

CouchDB 2.0 ALPHA Releases Available

[Jan Lehnardt]

Hey everyone,

I used the downtime before the holidays to set up CouchDB 2.0
alpha releases that are ready for wider testing.

And I wrote up a guide for people who want to give 2.0 a spin: 

  
https://docs.google.com/document/d/1BtndYr-0KDQTqBSLVdJoR_8C5ObYjT1RBo_Qyh5ykdQ/edit#

You don’t have to be an expert in CouchDB or anything,
and if you just want to play with the new features,
that’s totally cool, too.

Just let us know how it goes :)

Enjoy your holiday hacking time!

Best
Jan
-- 

Re: Updating lost user passwords

[Jan Lehnardt]

 On 28 Jul 2015, at 03:05, jumbo jim jumboji...@gmail.com wrote:
 
 A comment regarding the anonymous creation feature for _users
 database - while anonymous users are unable to get a list of all
 _users, they are able to see Number of documents (users). Can this be
 disabled?

good call, however, this is not possible at the moment.

 
 On Tue, Jul 28, 2015 at 10:45 AM, jumbo jim jumboji...@gmail.com wrote:
 
 Couchdb has a nice feature that allows anonymous users to write to the
 _users database. This allows new users to sign up. A logged in user (ie
 valid session) has the ability to update/change their password as they have
 update privileges to their *own* _user document.
 
 I envisage a scenario when I need to reset the password for a user as they
 have forgotten it.
 
 Is it possible to have a privileged user (but not admin) user that is
 permitted to make changes to other _user documents? (password changes)...
 but, at the same time still permitting anonymous creation of new documents?

only admins can edit other user’s _users docs.

Best
Jan
--


 
 
 
 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: Hide all _all_docs and _changes

[Jan Lehnardt]

 On 28 Jul 2015, at 02:39, jumbo jim jumboji...@gmail.com wrote:
 
 Hi,
 
 It is possible to disable the _all_docs feature by editing the local.ini
 and entering the following -
 
 [httpd_db_handlers]
 _all_docs =
 
 However, I then realised that a user could basically get a full listing of
 all documents by requesting _changes. So, I now use -
 
 [httpd_db_handlers]
 _all_docs =
 _changes =
 
 
 Do I need to close anything else off if I want to prevent a particular user
 from seeing a list of all documents?

This seems like a brittle approach, what are you trying to accomplish?

 Also - is it possible to do disable _all_docs/ and _changes for only
 specific databases? I would still like the admin to use _all_docs/ and
 _changes

That’s not possible.

Best
Jan
--


 
 Thanks

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: A big Thumbs Up for CouchDB!

[Jan Lehnardt]

Thank you Kiril! You email made my day and I’m happy CouchDB is working so well 
for you :)

Best
Jan
--

 On 25 Jul 2015, at 01:32, Kiril Stankov ki...@open-net.biz wrote:
 
 Hi all,
 
 I am working on a pretty complex project for the last 7 months.
 Since the beginning I was researching a lot what DB to use. NoSQL was obvious 
 choice, but I wondered between several options, MongoDB and CouchDB being the 
 obvious leaders.
 Now, half an year later, I am very glad for choosing CouchDB!
 
 It is an incredible product. Replication and views are the best features! My 
 app is a distributed one and I use CouchDB for the back-end synchronization 
 of data. My server is node.js and 'follow' gives me a perfect way to monitor 
 for changes.
 
 For now, CouchDB is very stable, fast and reliable. It is the heart of a 
 complex solution and there are no bad surprises until now. Updates to the DB 
 are very intensive and we have more than 1000 documents inserted per day. 
 Replication works great across continents and two operating systems. HTTP 
 API's make it so easy to monitor it and to offload part of the load from 
 node.js and retrieve data directly from Couch. We do use a lot of List and 
 Reduce functions to make life easier.
 
 Another great think about Couch is the community. I found so many useful 
 posts and articles. I got a lot of help from people on this list. Thank you.
 
 Well, there are still some issues, but there are not that important and I am 
 sure they'll be solved soon. Unfortunately I didn't have the time to check 
 v2.0, but I'll surely do so, once it is released officially.
 
 Finally, a big THANK YOU to all developers and supporters of CouchDB. I think 
 that with every open source project this is the least we, the users, can do 
 to cheer up the people behind such a great software!
 
 Regards and Relax,
 
 Kiril.

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: High CPU after loading 15K documents

[Jan Lehnardt]

Do you see anything in /_active_tasks?

 On 21 Jul 2015, at 12:37, Kiril Stankov ki...@open-net.biz wrote:
 
 It is constantly high, not some peak moments.
 I am monitoring htop with interval 1 sec and the CPU is above 12-15% with 
 most of this taken by erlang.
 
 *With best regards,*
 Kiril Stankov,
 CEO
 
 
   This Email disclaimer
   http://open-net.biz/emailsignature.html is integral part
   of this message.
 
 On 21-Jul-15 12:49 PM, Jan Lehnardt wrote:
 Do you see anything in /_active_tasks while the CPU is up?
 
 Best
 Jan
 --
 
 On 21 Jul 2015, at 10:36, Kiril Stankov ki...@open-net.biz wrote:
 
 Hi,
 
 I am monitoring CouchDB all the time and the CPU keeps around 10-12% all 
 the time from the erlang processes.
 I do not think this is normal.
 A new document is added to the DB each 5-6 minutes, but what takes CPU time 
 when the DB is idle?
 There are very few read requests, may be 1 each 1-2 minutes or so.
 
 Here is a screenshot from htop:
 http://pho.to/9aTTK
 
 Any ideas how can I understand what goes on?
 
 *With best regards,*
 Kiril Stankov,
 CEO
 
 
   This Email disclaimer
   http://open-net.biz/emailsignature.html is integral part
   of this message.
 
 On 08-Jul-15 12:35 AM, Kiril Stankov wrote:
 Hi,
 
 sorry, I saw the couchdb user, mistaking it for a process. Actually what 
 takes most of the cpu is beam.smp.
 Yes, I do have some reduce functions, 90% of them are simply _count.
 On the Futon status screen I see only replication jobs, which are 100% 
 completed.
 There is nothing special in the couch.log, either.
 Database sizes are very small, few Mb.
 Now, it's midnight here, there is no activity whatsoever in the 
 application and beam.smp swings between 3 and 15% cpu.
 For the last 10 hours it has 8+ hours TIME in 'top'.
 Host is latest Ubuntu, Couch version is 1.6.1.
 
 On 07-Jul-15 10:08 PM, Adam Kocoloski wrote:
 Hi Kiril, no it’s not normal. Did you mean `couchjs` when you said 
 `couchdb`? If so it’s likely related to indexing. Do your views have 
 custom reduce functions? A poorly-behaved custom reduce function (e.g. 
 one that doesn’t really “reduce” its output) could yield very slow 
 indexing. Does the dashboard indicate any indexing jobs ongoing?
 
 Adam
 
 On Jul 7, 2015, at 1:42 PM, Kiril Stankov ki...@open-net.biz wrote:
 
 Hi,
 
 I've loaded 15K docs to a single DB, with 7 views, which do not emit 
 docs (just id's).
 Since then (4 hours now) the CPU is constantly high (~15-20%) with most 
 of it divided between couchdb and beam.smp.
 In addition couch became significantly slower than before (when there 
 were ~ 3K docs).
 Is that normal? Can it be related to indexing? How long can it take?
 
 Thanks in advance.
 
 Kiril.
 
 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: High CPU after loading 15K documents

[Jan Lehnardt]

Do you see anything in /_active_tasks while the CPU is up?

Best
Jan
--

 On 21 Jul 2015, at 10:36, Kiril Stankov ki...@open-net.biz wrote:
 
 Hi,
 
 I am monitoring CouchDB all the time and the CPU keeps around 10-12% all the 
 time from the erlang processes.
 I do not think this is normal.
 A new document is added to the DB each 5-6 minutes, but what takes CPU time 
 when the DB is idle?
 There are very few read requests, may be 1 each 1-2 minutes or so.
 
 Here is a screenshot from htop:
 http://pho.to/9aTTK
 
 Any ideas how can I understand what goes on?
 
 *With best regards,*
 Kiril Stankov,
 CEO
 
 
   This Email disclaimer
   http://open-net.biz/emailsignature.html is integral part
   of this message.
 
 On 08-Jul-15 12:35 AM, Kiril Stankov wrote:
 Hi,
 
 sorry, I saw the couchdb user, mistaking it for a process. Actually what 
 takes most of the cpu is beam.smp.
 Yes, I do have some reduce functions, 90% of them are simply _count.
 On the Futon status screen I see only replication jobs, which are 100% 
 completed.
 There is nothing special in the couch.log, either.
 Database sizes are very small, few Mb.
 Now, it's midnight here, there is no activity whatsoever in the application 
 and beam.smp swings between 3 and 15% cpu.
 For the last 10 hours it has 8+ hours TIME in 'top'.
 Host is latest Ubuntu, Couch version is 1.6.1.
 
 On 07-Jul-15 10:08 PM, Adam Kocoloski wrote:
 Hi Kiril, no it’s not normal. Did you mean `couchjs` when you said 
 `couchdb`? If so it’s likely related to indexing. Do your views have custom 
 reduce functions? A poorly-behaved custom reduce function (e.g. one that 
 doesn’t really “reduce” its output) could yield very slow indexing. Does 
 the dashboard indicate any indexing jobs ongoing?
 
 Adam
 
 On Jul 7, 2015, at 1:42 PM, Kiril Stankov ki...@open-net.biz wrote:
 
 Hi,
 
 I've loaded 15K docs to a single DB, with 7 views, which do not emit docs 
 (just id's).
 Since then (4 hours now) the CPU is constantly high (~15-20%) with most of 
 it divided between couchdb and beam.smp.
 In addition couch became significantly slower than before (when there were 
 ~ 3K docs).
 Is that normal? Can it be related to indexing? How long can it take?
 
 Thanks in advance.
 
 Kiril.
 
 
 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: CouchDB 2.0 automatic sharding and replication of all databases

[Jan Lehnardt]

 On 17 Jul 2015, at 04:21, Vladimir Ralev vladimir.ra...@gmail.com wrote:
 
 Hi guys,

we are not all guys here :)

 I am trying to figure out if couchdb 2.0 will be capable of automatic
 scaling like bigcouch. Just put some nodes out there, configure the
 _nodes and how many replicas you want in the cluster and it takes care
 of everything automatically.
 
 There is this document
 http://docs.couchdb.org/en/latest/cluster/sharding.html#scaling-out
 which suggests I have to configure everything manually per-database
 and there is no resharding.
 
 Is there something in place to allow easier replication and migration
 like in bigcouch?

Yes, CouchDB 2.0 will include a newer version of the BigCouch code.

Best
Jan
-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: parsing json

[Jan Lehnardt]

 On 02 Jul 2015, at 12:23, Jan Lehnardt j...@apache.org wrote:
 
 
 On 02 Jul 2015, at 12:19, Hermann Norpois hnorp...@gmail.com wrote:
 
 Hello,
 
 how does couchdb parse the json-documents? Which tool is used? Is it a
 known library or is it a couchdb built in function?
 
 We use https://github.com/apache/couchdb-jiffy a C-implementation of a JSON
 parser.
 
 Although there is now a discussion to move back to mochijson2, which is
 native Erlang: https://issues.apache.org/jira/browse/COUCHDB-2732

In addition: view results are serialised by SpiderMonkey’s native 
JSON.stringify();

Best
Jan
--
 
 Best
 Jan
 --
 
 
 Thanks
 Hermann
 
 -- 
 Professional Support for Apache CouchDB:
 http://www.neighbourhood.ie/couchdb-support/
 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: Sanitize before saving to couch document?

[Jan Lehnardt]

 On 13 May 2015, at 05:33, jumbo jim jumboji...@gmail.com wrote:
 
 I am thinking about user-submitted data that I want to save. I currently
 validate and submit data directly to couch, and when I want to display the
 data back to users.. I escape it first to avoid xss. Validate on input,
 escape on output.
 
 My question relates specifically to the step when I save the user-submitted
 data to couch. Ie - they submit a string.. and I want to save that to a
 field within a document. Can couchdb accept any arbitrary string for a
 field within a document? I validate against string length. I have observed
 that that special characters (quotes, newlines etc) are automatically
 escaped etc (ie \\n). Couchdb also accepts unicode. Are there any
 situations where I need to be wary about what type of data I save to a
 document that may have been submitted by an evil user?

Ah, you want a validate_doc_update function: 
http://docs.couchdb.org/en/1.6.1/couchapp/ddocs.html?highlight=validate_doc_update#validate-document-update-functions

Best
Jan
-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: Sanitize before saving to couch document?

[Jan Lehnardt]

 On 11 May 2015, at 06:55, jumbo jim jumboji...@gmail.com wrote:
 
 Hi,
 
 When saving to couch, is there any requirement to sanitize before saving?
 From my testing is appears that everything is automatically encoded to json
 correctly.
 
 Are there any scenarios when it can be dangerous to save certain types of
 data in couchdb?

What types are you thinking of? CouchDB only accepts application/json for
documents. Attachments can be of any content-type.

Best
Jan
--
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: Jan bias for Hoodie brought him to eradicate couchapps?!?!

[Jan Lehnardt]

Giovanni,

I'd appreciate it if you'd lay off the personal attacks. I have nothing but the 
best interests of CouchDB in mind. If you are arguing against me, start from 
that premise.

Best
Jan
--

 On 07.05.2015, at 05:13, Giovanni Lenzi g.le...@smileupps.com wrote:
 
 Jan.. don't you think you are simply too biased for external projects, such
 as your hoodie and pouchdb, to choose by your own, on a topic like
 'retiring couchapps', so important to couchdb users and devs?
 
 You REALLY can't mask it behind a marketing@ decision, of finding a story
 you have already written, since a long time, and that you simply don'want
 to change. Your story of the data where you need it, does not have users
 and industries consensus. You can't hide it!
 
 You started with changing tooling names, sutting down couchapp domains and
 forcing our company to change our old couchappy name (even if couchapp
 wasn't even a trademark). We did it, because you promised visibility to us
 and couchapps, which has never happened. We believed in your words and in
 an healty pmc, but after this last move, we really can't think it anymore.
 
 Couchapps are now much more powerful than they were in the past, also
 thanks to our smileupps tutorials and app store platform, and to many other
 improvements proposal already advanced by users, who instead would like to
 improve them, not the contrary!!!
 
 Your chance now is to completely eradicate a main recruiting couchdb
 feature, before users can get aware of its potential, leading couchdb
 towards a probable self-killing action. And the worst is you are doing
 this, without any proven evidence and motivation, withouth asking devs of
 its development complexity, and users what they think about it.
 
 I definetely think you should:
 1. HEAR YOUR COMMUNITY AND DEVS.. and if the consensus is 'don't remove
 couchapps, but push them with aggressive marketing'.. well then.. just do
 it!
 2. Don't mark it absolutely as a marketing issue
 
 And you sould do this, before this could really be faced in front of the
 ASF board.

Re: the future of couchapp

[Jan Lehnardt]

I never said CouchApps should be removed. Can everybody please stop refuting a 
point that I never made. And lay off the personal attacks while you are at it.

Best
Jan
--

 On 07.05.2015, at 11:16, Harald Kisch haraldki...@gmail.com wrote:
 
 Sorry guys, my eyes dont believe what they see..
 @Jan: How are you? long time not spoken to eachother. How is Lena? What she
 is thinking about couchapps?
 
 Why not to make a list of pros and cons for couchapps?
 Did you ask jchris about removing couchapps?
 Or why you don't directly ask Damien aboout the cause he implements it?
 
 What is your benefit removing it?
 
 For my part I have been successfully using it since 2008 and this is my
 favorite use case for building secure, anywhere applications for the
 industry and I would apreciate improvements on couchapps regarding
 Authentication, LDAP and Active Directory erlang modules like built in 2010
 and never improved.
 
 We should not forget what CouchDB is all about. And for me the guys who
 claim out to remove couchapp simply forget the power Damien have built in
 and the power who made CouchDB proud to kick-off the no-sql area.
 Dont forget who Damien really is. He is one of the leading developers of
 Domino Notes aka Lotus Notes aka IBM Notes, registering 3 patents in USA
 for it. Because only older guys know it, Lotus Notes was the previously
 business standard groupware software for large scale companies before SAP
 was destroying it's reputations with the help of IBM (which wanted only to
 sell their DB2 Database). Everybody who was working with Lotus Notes begun
 to love it. Not so SAP-Users, they are hating their daily work with SAP
 because it simply don't work (complexity).
 
 Couchapp is not complex and still have the power lotus notes has. Remember:
 Damien has built CouchDB because everybody was talking about it, and
 nobody did it, until Damien got it done with CouchDB.
 
 In my opinion, and there are much more people who think in this way,
 Couchapp is the most important and game-changing feature in CouchDB. But
 also most misunderstood and most criticised, partly because of the fear it
 creates amoung other database vendors who always want exactly this:
 Application execution directly out of the database core. Yes, exactly this
 is Couchapp! And it does it without CLR (Microsoft SQLSERVER) or JAVA
 (Oracle Forms) and its terribly complex architecture.
 
 Please stop using CouchDB as stupid data container and think more about it
 as Web-Application-Engine!
 
 Cheers
 Harald Kisch
 
 ---
 
 Given the importance of this topic: the future of couchapp... I'm moving
 this to the user@ mailing list.
 
 This should be definitely something @users should be involved in.. at least
 those interested in Couchapps.
 
 To recap:
 Jan: wants to remove Couchapp name and design doc functions because it
 finds them to be source of confusion
 
 ermouth: pointed out Couchapps are not silver bullet but they cover many
 use cases and there are rooms for improvements
 
 giovanni: pointed out many users and industries today are using couchapps
 successfully, withouth such this confusion between what couchdb is and what
 couchapps are, and they won't simply understand couchapps removal, leading
 couchdb to a second-death. Moreover couchapps potential has increased a lot
 within the last months: now they are secure and has support for features
 like e-mail, sms, paypal/stripe integration, events scheduling.
 
 johs: pointed out couchapps are big recruiter for couchdb and they should
 not be dropped: a fadeout of couchapp name withouth any removal should be
 sufficient
 
 andy: was not aware of the name confusion, and wanted to keep the name
 
 What you all think about it?
 
 
 2015-05-05 22:35 GMT+02:00 Giovanni Lenzi g.le...@smileupps.com:
 
 Agree with Andy.. why change a name of something that is born with
 couchdb, lives in couchdb and runs only inside of it?
 
 Dropping name or removing it won't simply be understood by users and
 industries already relying on it. imho negative impact could be very
 high..
 and I'm afraid this could really lead to a new second death for the
 project, after the first one with the damien katz retirement issue...
 
 all of the above can't be justified with only some naming conflicts, even
 considered that couchapp tools and also couchappy project have changed
 their name just to prevent it
 
 More than a naming confusion, i'm aware of a lack of clarification about
 what can and cannot be done, supported by facts, real examples and
 eventually benchmarks
 
 Furthermore, so far on social networks I have seen more focus on what
 cannot be done, instead of the contrary.. and I can well understand users
 can be afraid and confused by this.
 Il giorno 05/mag/2015 20:33, Andy Wenk andyw...@apache.org ha scritto:
 
 On 5 May 2015 at 18:44, Jan Lehnardt j...@apache.org wrote:
 
 
 On 05 May 2015, at 18:37, Andy Wenk andyw...@apache.org wrote:
 
 As often, here are many truths in all the replies

Re: the future of couchapp

[Jan Lehnardt]

Funny how this proves my point about CouchApps being confusing. We can't even 
talk about their future without talking past each other.

In addition, cherry-picking quotes from my emails won't help to clarify things. 
I understand you *can* read a position of let's remove CouchApps into what I 
wrote, by only if you selectively ignore some of the things I also said. I 
don't think that is useful.

Please join marketing@ to join the uncut discussion there.

Best
Jan
--

Best
Jan
--
 On 07.05.2015, at 15:10, Harald Kisch haraldki...@gmail.com wrote:
 
 Sorry Jan, please don't take it personally but in your both emails you
 definitely claimed out, that couchapps does'nt fit in YOUR CouchDB-Story.
 
 In
 
 http://markmail.org/message/no3jfksdxjcgxz4d
 
 you personally said:
 Technically, that would have meant CouchApps had to grow a lot more and I
 realised quickly that CouchDB is not the right place to grow such a thing.
 
 Sorry but for me it means you don't want CouchDB to have an App-Engine
 inside. The only confusion is: What could be the issue for that? Every
 database vendor works for decades on it? And why peer-to-peer should be
 bad? Think on Git, torrent network, etc. pp. are these projects not the
 leading inventions of the last decades based on peer-to-peer? And yes,
 CouchDB is NOW extremely powerful with Apps executed out of its database
 core and replicated anywhere without the need of permanent internet
 connection!
 
 Also in
 
 http://markmail.org/message/xla3hulea4lo5duw
 
 you personally said:
 figure out a plan to retire “CouchApp”
 
 Sorry this words are not misunderstandable for me. And I am wondering why
 and how you can say that. And if you really want to retire CouchApp
 because of confusions (for me it is the other way around - the storage is
 part of the App-Engine because stupid containers you can find everywhere
 based on node.js etc. to support the same as CouchDB's native App
 Core-Feature called Couchapp)
 
 CouchApps for me are the CouchDB Story. There is no confusion about that.
 Please do not claim that somone has something against you and please take
 it objective not emotional. But if you take such desruptive things into the
 community, you should also stand for it to explain them accordingly and not
 start to change the basics to loose all the current users of that
 game-changing core-feature.
 
 Best
 Harald
 
 On Thu, May 7, 2015 at 1:14 PM, Jan Lehnardt j...@apache.org wrote:
 
 I never said CouchApps should be removed. Can everybody please stop
 refuting a point that I never made. And lay off the personal attacks while
 you are at it.
 
 Best
 Jan
 --
 
 On 07.05.2015, at 11:16, Harald Kisch haraldki...@gmail.com wrote:
 
 Sorry guys, my eyes dont believe what they see..
 @Jan: How are you? long time not spoken to eachother. How is Lena? What
 she
 is thinking about couchapps?
 
 Why not to make a list of pros and cons for couchapps?
 Did you ask jchris about removing couchapps?
 Or why you don't directly ask Damien aboout the cause he implements it?
 
 What is your benefit removing it?
 
 For my part I have been successfully using it since 2008 and this is my
 favorite use case for building secure, anywhere applications for the
 industry and I would apreciate improvements on couchapps regarding
 Authentication, LDAP and Active Directory erlang modules like built in
 2010
 and never improved.
 
 We should not forget what CouchDB is all about. And for me the guys who
 claim out to remove couchapp simply forget the power Damien have built in
 and the power who made CouchDB proud to kick-off the no-sql area.
 Dont forget who Damien really is. He is one of the leading developers of
 Domino Notes aka Lotus Notes aka IBM Notes, registering 3 patents in USA
 for it. Because only older guys know it, Lotus Notes was the previously
 business standard groupware software for large scale companies before SAP
 was destroying it's reputations with the help of IBM (which wanted only
 to
 sell their DB2 Database). Everybody who was working with Lotus Notes
 begun
 to love it. Not so SAP-Users, they are hating their daily work with SAP
 because it simply don't work (complexity).
 
 Couchapp is not complex and still have the power lotus notes has.
 Remember:
 Damien has built CouchDB because everybody was talking about it, and
 nobody did it, until Damien got it done with CouchDB.
 
 In my opinion, and there are much more people who think in this way,
 Couchapp is the most important and game-changing feature in CouchDB. But
 also most misunderstood and most criticised, partly because of the fear
 it
 creates amoung other database vendors who always want exactly this:
 Application execution directly out of the database core. Yes, exactly
 this
 is Couchapp! And it does it without CLR (Microsoft SQLSERVER) or JAVA
 (Oracle Forms) and its terribly complex architecture.
 
 Please stop using CouchDB as stupid data container and think more about
 it
 as Web-Application-Engine!
 
 Cheers

Re: the future of couchapp

[Jan Lehnardt]

 On 07.05.2015, at 17:05, Miles Fidelman mfidel...@meetinghouse.net wrote:
 
 Speaking as someone who writes proposals for a living, what I find confusing 
 are terms that sound very clear - such as retire CouchApps - that require 
 digging through lots of distributed materials to find clarification of what 
 you really mean.

I didn't bring this to user@ for that exact reason. Let's continue this 
discussion on marketing@.

Can you repost your thoughts there?

Best
Jan
--




 
 I.e., it's not CouchApps that are confusing - it's the verbiage that's 
 confusing.
 
 Personally, I'd like to see more language like:
 
 CouchDB includes application server functionality, that supports both 
 client-side and server side native applications, which we call CouchApps.
 or maybe,
 CouchApps are CouchDB's equivalent of Java applets and servelets.
 
 I think those are pretty clear descriptions of CouchApps, at a conceptual 
 level.  (If not, then maybe CouchApps really are confusing.)
 
 Miles Fidelman
 
 
 
 Jan Lehnardt wrote:
 Funny how this proves my point about CouchApps being confusing. We can't 
 even talk about their future without talking past each other.
 
 In addition, cherry-picking quotes from my emails won't help to clarify 
 things. I understand you *can* read a position of let's remove CouchApps 
 into what I wrote, by only if you selectively ignore some of the things I 
 also said. I don't think that is useful.
 
 Please join marketing@ to join the uncut discussion there.
 
 Best
 Jan
 --
 
 Best
 Jan
 --
 On 07.05.2015, at 15:10, Harald Kisch haraldki...@gmail.com wrote:
 
 Sorry Jan, please don't take it personally but in your both emails you
 definitely claimed out, that couchapps does'nt fit in YOUR CouchDB-Story.
 
 In
 
 http://markmail.org/message/no3jfksdxjcgxz4d
 
 you personally said:
 Technically, that would have meant CouchApps had to grow a lot more and I
 realised quickly that CouchDB is not the right place to grow such a thing.
 
 Sorry but for me it means you don't want CouchDB to have an App-Engine
 inside. The only confusion is: What could be the issue for that? Every
 database vendor works for decades on it? And why peer-to-peer should be
 bad? Think on Git, torrent network, etc. pp. are these projects not the
 leading inventions of the last decades based on peer-to-peer? And yes,
 CouchDB is NOW extremely powerful with Apps executed out of its database
 core and replicated anywhere without the need of permanent internet
 connection!
 
 Also in
 
 http://markmail.org/message/xla3hulea4lo5duw
 
 you personally said:
 figure out a plan to retire “CouchApp”
 
 Sorry this words are not misunderstandable for me. And I am wondering why
 and how you can say that. And if you really want to retire CouchApp
 because of confusions (for me it is the other way around - the storage is
 part of the App-Engine because stupid containers you can find everywhere
 based on node.js etc. to support the same as CouchDB's native App
 Core-Feature called Couchapp)
 
 CouchApps for me are the CouchDB Story. There is no confusion about that.
 Please do not claim that somone has something against you and please take
 it objective not emotional. But if you take such desruptive things into the
 community, you should also stand for it to explain them accordingly and not
 start to change the basics to loose all the current users of that
 game-changing core-feature.
 
 Best
 Harald
 
 On Thu, May 7, 2015 at 1:14 PM, Jan Lehnardt j...@apache.org wrote:
 
 I never said CouchApps should be removed. Can everybody please stop
 refuting a point that I never made. And lay off the personal attacks while
 you are at it.
 
 Best
 Jan
 --
 
 On 07.05.2015, at 11:16, Harald Kisch haraldki...@gmail.com wrote:
 
 Sorry guys, my eyes dont believe what they see..
 @Jan: How are you? long time not spoken to eachother. How is Lena? What
 she
 is thinking about couchapps?
 
 Why not to make a list of pros and cons for couchapps?
 Did you ask jchris about removing couchapps?
 Or why you don't directly ask Damien aboout the cause he implements it?
 
 What is your benefit removing it?
 
 For my part I have been successfully using it since 2008 and this is my
 favorite use case for building secure, anywhere applications for the
 industry and I would apreciate improvements on couchapps regarding
 Authentication, LDAP and Active Directory erlang modules like built in
 2010
 and never improved.
 
 We should not forget what CouchDB is all about. And for me the guys who
 claim out to remove couchapp simply forget the power Damien have built in
 and the power who made CouchDB proud to kick-off the no-sql area.
 Dont forget who Damien really is. He is one of the leading developers of
 Domino Notes aka Lotus Notes aka IBM Notes, registering 3 patents in USA
 for it. Because only older guys know it, Lotus Notes was the previously
 business standard groupware software for large scale companies before SAP
 was destroying it's reputations

Re: How do CouchApps fit into the CouchDB story? (Was: CouchDB Articles, Pills and Tutorials Ideas)

[Jan Lehnardt]

 On 06 May 2015, at 18:18, Alexander Shorin kxe...@gmail.com wrote:
 
 On Wed, May 6, 2015 at 6:49 PM, Giovanni Lenzi g.le...@smileupps.com wrote:
 What you all think about it?
 
 I think we need just clarify what CouchApp is, define it's show cases
 and area where they are works perfectly.
 First step is to back couchapp.org online (almost done). Then, just
 fill it the right content. That will solve any confusions.

Before we fix anything, we need to understand how to fix it. To understand
how to fix it we need to understand what we do. To understand what we do,
we need to understand why we do it. — Hence, we first need to find a “The
Why of CouchDB” story that reflects our intentions before any of the other
decisions can happen. That’s what my original message is about.


 I'm -1 for removal CouchApps as a since it's not possible as CouchApps
 are not a some entity, but a way of organizing design documents in
 order to solve some specific problem.

The problem is, as outlined in my emails on marketing@ that I linked earlier
that “CouchApp” means at least eight different things to different groups of
people. That is terribly confusing and makes people new to CouchDB leave us
in spades because it is just too complicated.

I never meant for the actual technical features in CouchDB to be removed and
I never said so.

 We just need to find a sweet spot, a right vector for their future since as
 for now I feel CouchApp as a technology is just lost in space.

Exactly, and its harming CouchDB.

To get this rolling, we need to understand what we do and why we do it. Then
we can decide if and how “CouchApp” in whatever form fits into this.

Best
Jan
--

Re: How do CouchApps fit into the CouchDB story? (Was: CouchDB Articles, Pills and Tutorials Ideas)

[Jan Lehnardt]

 On 06 May 2015, at 18:54, Miles Fidelman mfidel...@meetinghouse.net wrote:
 
 Jan Lehnardt wrote:
 On 06 May 2015, at 17:57, Giovanni Lenzig.le...@smileupps.com wrote:
 
 Given the importance of the topic: the future of couchapp... I'm moving
 this from the @marketing to the user@ mailing list.
 I’d say this is too early, would prefer we keep this on marketing@ until
 we have the messaging right. Everything else follows from them. Given that
 user@ is missing the entire “the why of CouchDB” context that marketing@
 has, I’d say moving this is premature.
 
 I also *could* see this as a cheap ploy to quickly garner public pressure
 against my position, but in good faith, I won’t assume this is your motive.
 
 If this is the kind of discussion going on, on the marketing list, then you 
 REALLY need to open it up to user feedback.  I think you're going off track.

Noted.

 
 Re. a couple of things below:
 This should be definitely something @users should be involved in.. at least
 those interested in Couchapps.
 
 To recap:
 Jan: wants to remove Couchapp name and design doc functions because it
 finds them to be source of confusion
 This does not adequately reflects my position. I don’t suggest to remove
 any of the things that make CouchApps possible.
 
 My larger argument can be foundhttp://markmail.org/message/no3jfksdxjcgxz4d 
 andhttp://markmail.org/message/xla3hulea4lo5duw
 
 tl;dr: I’d like us to think about how the CouchApp (or whatever the
 final name might be) story fits into the larger CouchDB story of “Data where
 you need it.” — Not necessarily how the slogan made be “true” in the context
 of CouchApps (e.g. “Data (and logic) where you need it.”, but more:
 
 - CouchDB’s core feature is geographically distributed replication.
 
 Really?  That's the argument that lead to CouchBase.

I’m a Couchbase co-founder and I can assure you you are mistaken.

 
 Yes, MVCC w/ replication is A core feature, but, at least to me, Couch's core 
 feature is a full-featured HTTP interface -- everything is a resource, 
 accessed through RESTful HTTP operations.

What is the one thing that make CouchDB unique? REST or Replication?

To wit, this isn’t about removing everything that isn’t replication. REST will 
stay around, we may have other interfaces too at some point, but not anytime 
soon. Either way, REST is not what makes CouchDB unique.

I understand it is an important feature to you, much like many other features 
are very important to other people. But overall, CouchDB’s unique feature is 
replication.


 
 
 - We also have somewhat quirky, while technically neat, applications that can
 live inside CouchDB.
 
 I don’t think that last bit helps paint the big picture CouchDB story at all.
 Granted, I’m making a deliberately weak attempt of tying things together
 (because I don’t know any better), but this is to get you thinking about how
 this could fit in our larger narrative, if at all.
 
 From this user's perspective, you're 100% wrong on this.

In your opinion.


 The fact that you can use CouchDB as an App Server is of particularly high 
 value; the more so that the Apps can run in a browser.  More to the point - 
 it's an HTML5 WebApp server, making it a full platform for RESTful 
 applications.

This is a great approach to get this into the core narrative of CouchDB, would 
love for you to join the discussion on marketing@.


 
 My current thinking is that we can’t make it clear and thus should figure 
 out a
 plan to retire “CouchApp”, while still allowing all the cool tech, and find a
 new name for the concept that can live on without making CouchDB’s core 
 message
 unclear.
 
 
 Hell no!

Hell yes, let’s keep CouchDB a confusing mess.


 Maybe the tooling for CouchApps is a bit funky, but the support for them is 
 (IMHO) core features of Couch.  And the notion of Couch as an App Server is 
 pretty straightforward - and the term fits quite nicely.

I’ve spent most of my time since 2007 advocating CouchDB to thousands of people 
in person and even more online. CouchDB’s “AppServer” features are neither 
straightforward nor does the term fit nicely. People are confused by the fact 
that there are three different “couchapp” tools in various languages, that you 
need one of them to manage design docs, just to query CouchDB (I’m sure glad we 
have mango now, where the steps are “create index, start querying”, like in 
every other database). There is confusion of using “couchapp” to manage views, 
when CouchApps mean so much more than just managing map/reduce functions. I can 
go on for hours how this is confusing for first-time experiences that people 
have told me and keep telling me.


 
 Rather than retire the term, or relegate it to obsucrity - market it 
 aggressively!  Perhaps to the the extent of changing the current tag line on 
 couchdb.apache.org from A Database for the Web to A Database and 
 Integrated App Server for the Web.”

I’ll refer you to the “The Why of CouchDB” discussion

Re: How do CouchApps fit into the CouchDB story? (Was: CouchDB Articles, Pills and Tutorials Ideas)

[Jan Lehnardt]

 On 06 May 2015, at 17:57, Giovanni Lenzi g.le...@smileupps.com wrote:
 
 Given the importance of the topic: the future of couchapp... I'm moving
 this from the @marketing to the user@ mailing list.

I’d say this is too early, would prefer we keep this on marketing@ until
we have the messaging right. Everything else follows from them. Given that
user@ is missing the entire “the why of CouchDB” context that marketing@
has, I’d say moving this is premature.

I also *could* see this as a cheap ploy to quickly garner public pressure
against my position, but in good faith, I won’t assume this is your motive.


 This should be definitely something @users should be involved in.. at least
 those interested in Couchapps.
 
 To recap:
 Jan: wants to remove Couchapp name and design doc functions because it
 finds them to be source of confusion

This does not adequately reflects my position. I don’t suggest to remove
any of the things that make CouchApps possible.

My larger argument can be found http://markmail.org/message/no3jfksdxjcgxz4d 
and http://markmail.org/message/xla3hulea4lo5duw

tl;dr: I’d like us to think about how the CouchApp (or whatever the
final name might be) story fits into the larger CouchDB story of “Data where
you need it.” — Not necessarily how the slogan made be “true” in the context
of CouchApps (e.g. “Data (and logic) where you need it.”, but more:

- CouchDB’s core feature is geographically distributed replication.

- Replication becomes really powerful with projects like PouchDB and TouchDB
that take individual user data offline on their browsers and phones.

- The industry’s current battle is vendor-lock-in with proprietary backend as a
service solutions that may or may not have offline capabilities.

- CouchDB is the only serious Open Source contender in this.

- We also have somewhat quirky, while technically neat, applications that can
live inside CouchDB.

I don’t think that last bit helps paint the big picture CouchDB story at all.
Granted, I’m making a deliberately weak attempt of tying things together
(because I don’t know any better), but this is to get you thinking about how
this could fit in our larger narrative, if at all.

My current thinking is that we can’t make it clear and thus should figure out a
plan to retire “CouchApp”, while still allowing all the cool tech, and find a
new name for the concept that can live on without making CouchDB’s core message
unclear.

* * *

Please join us on the market...@couchdb.apache.org list for this discussion.

Best
Jan
-- 



 
 ermouth: pointed out Couchapps are not silver bullet but they cover many
 use cases and there are rooms for improvements
 
 giovanni: pointed out many users and industries today are using couchapps
 successfully, withouth such this confusion between what couchdb is and what
 couchapps are, and they won't simply understand couchapps removal, leading
 couchdb to a second-death. Moreover couchapps potential has increased a lot
 within the last months: now they are secure and has support for features
 like e-mail, sms, paypal/stripe integration, events scheduling.
 
 johs: pointed out couchapps are big recruiter for couchdb and they should
 not be dropped: a fadeout of couchapp name withouth any removal should be
 sufficient
 
 andy: was not aware of the name confusion, and wanted to keep the name
 
 What you all think about it?
 
 
 2015-05-05 22:35 GMT+02:00 Giovanni Lenzi g.le...@smileupps.com:
 
 Agree with Andy.. why change a name of something that is born with
 couchdb, lives in couchdb and runs only inside of it?
 
 Dropping name or removing it won't simply be understood by users and
 industries already relying on it. imho negative impact could be very high..
 and I'm afraid this could really lead to a new second death for the
 project, after the first one with the damien katz retirement issue...
 
 all of the above can't be justified with only some naming conflicts, even
 considered that couchapp tools and also couchappy project have changed
 their name just to prevent it
 
 More than a naming confusion, i'm aware of a lack of clarification about
 what can and cannot be done, supported by facts, real examples and
 eventually benchmarks
 
 Furthermore, so far on social networks I have seen more focus on what
 cannot be done, instead of the contrary.. and I can well understand users
 can be afraid and confused by this.
 Il giorno 05/mag/2015 20:33, Andy Wenk andyw...@apache.org ha scritto:
 
 On 5 May 2015 at 18:44, Jan Lehnardt j...@apache.org wrote:
 
 
 On 05 May 2015, at 18:37, Andy Wenk andyw...@apache.org wrote:
 
 As often, here are many truths in all the replies. I see myself just
 jumping in from the side because I don't actually use CouchApps. I
 have
 full respect for people like Giovanni  who want to keep CouchApps
 'alive'.
 So I think the plan Jan wrote done can work quite good also for me.
 Here
 are my comments:
 
 On 5 May 2015 at 17:39, Jan Lehnardt j...@apache.org wrote:
 
 Thanks

Re: [BLOG] The CouchDB Weekly News, April 23, is out!

[Jan Lehnardt]

Outstanding job for the first run! Thank you Katharina! :)

Best
Jan
--

 On 23 Apr 2015, at 18:07, Katharina Jockenhöfer ka...@thehoodiefirm.com 
 wrote:
 
 Hi everyone,
 
 this week's CouchDB Weekly News is out: 
 http://blog.couchdb.org/2015/04/23/couchdb-weekly-news-april-23-2015/ 
 http://blog.couchdb.org/2015/04/23/couchdb-weekly-news-april-23-2015/
 
 Highlights:
 - Weekly meeting
 - Logo vote
 - New mailing list @notifications
 - News from Fauxton
 - Interesting use cases 
 … as well as tons of community releases and questions, events,
 jobs and an extensive relax! section and more.
 
 Thanks to Robert, Paul and Garren for submitting event news, interesting 
 reads and other news!
 
 Please help us promote the Weekly News here:
 https://twitter.com/CouchDB/status/591260321932832768 
 https://twitter.com/CouchDB/status/591260321932832768
 http://www.reddit.com/r/CouchDB/comments/33lq1s/couchdb_weekly_news_april_23_2015/
  
 http://www.reddit.com/r/CouchDB/comments/33lq1s/couchdb_weekly_news_april_23_2015/
 https://www.linkedin.com/company/apache-couchdb?trk=company_logo 
 https://www.linkedin.com/company/apache-couchdb?trk=company_logo
 https://www.facebook.com/pages/CouchDB/507603582605204?ref=hl 
 https://www.facebook.com/pages/CouchDB/507603582605204?ref=hl
 
 We'd invite you to share the link in the social networks you use to help us
 promote CouchDB.
 
 Thank a lot and a good soon-to-be weekend to all of you. Talk next week!
 Katharina
 
 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: CouchDB 2.0 release date

[Jan Lehnardt]

Hi Ron,

we don’t have a release date yet. Upgrading from 1.6.1 to 2.0 will be 
straightforward (via replication), so no harm in starting on 1.6.1 and 
upgrading when 2.0 s out. The APIs will be (nearly) identical.

Best
Jan
--

 On 22 Apr 2015, at 00:26, Ronald Lepper ron.lep...@gmail.com wrote:
 
 We are currently starting out a project with Couchdb 1.6.1 but are having
 to currently work around some things that Couchdb 2.0 looks like it will
 solve (specifically, some of the clustering features).  Is there any update
 on when the release date is for Couchdb 2.0?
 
 Sincerely,
 Ron

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: How to cancel a non-continous replication

[Jan Lehnardt]

 On 20 Apr 2015, at 19:38, Roger Sindreu ro...@empaytech.com wrote:
 
 Hi
 
 We have looked at the documentation and we have seen ways to cancel
 continuous replication by removing the documents in the database.
 
 My question is how to cancel a replication if there is no document in the
 _replication database?
 
 We are in 1.6.1 and the replication never ends for some reason (which we
 would like to solve BTW).

http://docs.couchdb.org/en/1.6.1/replication/intro.html#triggering-replication

Last sentence.

Best
Jan
--

 
 Thanks.

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: can replication be setup between different versions?

[Jan Lehnardt]

 On 18 Apr 2015, at 12:36, Roger Sindreu ro...@empaytech.com wrote:
 
 Hi Alexander
 
 Should we expect any significant improvement in performance among these two
 versions?

Not at this point. Greater reliability in 1.6.1 though.

Best
Jan
--

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: CouchDB 1.6.1 crashes when compacting 5GB database

[Jan Lehnardt]

Heya Jerry,

can you post the full log somewhere?

Best
Jan
-- 

 On 09 Apr 2015, at 08:34, Jerry w...@everymatrix.com wrote:
 
 Hi
 
 When compacting a 5GB database and it proceeded for a while then exited.
 No error from the log.
 
 And I have no view in my database
 
 
 Apache CouchDB 1.6.1 (LogLevel=info) is starting.
 Apache CouchDB has started. Time to relax.
 [info] [0.32.0] Apache CouchDB has started on http://0.0.0.0:5984/
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 [info] [0.105.0] 192.168.1.2 - - GET 
 /cms/_changes?limit=10include_docs=truefeed=longpolltimeout=18since=11799
  200
 
 Br

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: replication continuosly restarts

[Jan Lehnardt]

 On 09 Apr 2015, at 19:34, Chris Thro chris.t...@citrix.com wrote:
 
 We are constantly seeing the following in the log:
 [Thu, 09 Apr 2015 11:03:30 GMT] [error] [0.105.0] Error in replication 
 `64f5d1684f0e22273f165de8b44893fc+continuous` (triggered by document 
 `docs-g2w_couchdb3_las`): {checkpoint_commit_failure,Error updating the 
 source checkpoint document: conflict}
 Restarting replication in 5 seconds.
 [Thu, 09 Apr 2015 11:03:30 GMT] [error] [0.25603.1698] ** Generic server 
 0.25603.1698 terminating
 ** Last message in was {'$gen_cast',checkpoint}
 ** When Server state == {rep_state,
 
 Replication is setup the following way:
 1.dc1 - 1.dc2
 2.dc1 - 2.dc2
 1.dc1 -2.dc1
 1.dc2 - 2.dc2
 
 We have 7 dbs replicating this way but only 5 of them are showing this 
 problem.
 Any help solving this would be greatly appreciated.

Do any of the databases have the same value for 
http://127.0.0.1:5984/_config/couchdb/uuid?

Best
Jan
-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: CouchDB 1.6.1 crashes when compacting 5GB database

[Jan Lehnardt]

 On 09 Apr 2015, at 10:39, Jerry w...@everymatrix.com wrote:
 
 Hi Jan
 
 in the previous email I pasted the full log.
 Actually that is strange to me as there is no error in the log and the 
 process gone.

The log you sent doesn’t include any call to _compact, so as far as the log is 
concerned, you are not actually compacting :)

Best
Jan
--

 
 
 Br
 
 On 4/9/2015 4:00 PM, Jan Lehnardt wrote:
 Heya Jerry,
 
 can you post the full log somewhere?
 
 Best
 Jan
 

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: [VOTE] CouchDB Logo - Round #3

[Jan Lehnardt]

Heya Aurélien,

we can’t consider votes that happen here, but you can just send it to 
d...@couchdb.apache.org, without subscribing. Your message will go through 
moderation, but no worries, we’ll let it through :D

Best
Jan
--

 On 08 Apr 2015, at 21:40, Aurélien Bénel aurelien.be...@utt.fr wrote:
 
 Hi folks,
 
 A third round to vote on the CouchDB logo was just started. (...)
 Please send your votes (+1 / 0 / -1) for each proposed candidate, so we can 
 get an impression of which one(s) the community members like and which they 
 don’t. The vote is open for non-committers as well.
 
 Do we have to join the dev list, to vote for the logo?
 
 If it is OK to do that on the user list too, here is my vote:
 
 - PD  -1
 - NP 1-1
 - NP 2-1
 - NP 3-1
 - BN  -1
 - SB 1-1
 - SB 2-1
 - SB 3-1
 - CA  +1
 - OL  +1
 
 
 Regards,
 
 Aurélien

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: Couchdb flush to disk

[Jan Lehnardt]

 On 19 Mar 2015, at 19:51, Michael Power michael.po...@elotouch.com wrote:
 
 Is there a way we can ask couchdb to flush to disk and block all writes.  We 
 would like to get the disk in a consistent state and do a disk based backup.  
 Is this possible?  The actual backup takes milliseconds we just need the disk 
 in a consistent state for those milliseconds.

That’s CouchDB’s default mode of operation.

If you need to make sure you have the absolute latest data, set 
`delayed_writes` to `false` in the configuration, but for backup reasons, it 
shouldn’t make a difference.

Best
Jan
--

 
 Michael Power

--
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: CouchDB SSL Issue

[Jan Lehnardt]

Heya Joel,

have you considered using a dedicated HTTP proxy in front of CouchDB to do the 
SSL handling for you?

It might be easier :)

Best
Jan
--
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

 On 19 Mar 2015, at 16:30, joel.brews...@mutualofomaha.com wrote:
 
 
 Hello all,
 
 We are really struggling with applying SSL to our CouchDB database.  We are 
 running CouchDB 1.6.1 on a Windows 7 machine.  We
 have tried to implement the solution suggested here 
 http://wiki.apache.org/couchdb/How_to_enable_SSL on multiple Windows 7
 machines with no luck.  One of my co-workers even tried on his home Linux 
 machine (not sure which distribution) and could not
 get it working.
 
 I am confident that the certificates are good.   Our local.ini file has been 
 configured as suggested in the link previously
 mentioned.  I have restarted the CouchDB service after each configuration 
 change.  Still no luck.
 
 Here are snippets from the local:ini file
 
 [daemons]
 httpsd = {couch_httpd, start_link, [https]}
 
 [ssl]
 cert_file = c:/Program\ Files\ (x86)/Apache\ Software\ 
 Foundation/CouchDB/etc/mycert.pem
 key_file = c:/Program\ Files\ (x86)/Apache\ Software\ 
 Foundation/CouchDB/etc/privkey.pem
 
 We repeatedly receive the following error when I run curl -k -v 
 https://127.0.0.1:6984/; : curl: (35) schannel: failed to
 receive handshake, SSL/TLS connection failed
 
 Has anyone else experienced this problem and found a solution?
 
 TIA,
 
 Joel Brewster
 
 
 This e-mail and any files transmitted with it are confidential and are solely 
 for the use of the addressee.  It may contain material that is legally 
 privileged, proprietary or subject to copyright belonging to Mutual of Omaha 
 Insurance Company and its affiliates, and it may be subject to protection 
 under federal or state law.  If you are not the intended recipient, you are 
 notified that any use of this material is strictly prohibited.  If you 
 received this transmission in error, please contact the sender immediately by 
 replying to this e-mail and delete the material from your system.  Mutual of 
 Omaha Insurance Company may archive e-mails, which may be accessed by 
 authorized persons and may be produced to other parties, including public 
 authorities, in compliance with applicable laws.




signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: connection handling in long running bulk doc uploads

[Jan Lehnardt]

 On 16 Mar 2015, at 13:57, Ingo Radatz thewh...@googlemail.com wrote:
 
 Thank you for the quick answer.
 
 I think new streaming interfaces are the way to go. We’ve been talking about 
 them in other contexts like replication before. We won’t likely get to this 
 before 2.0, though.
 
 Is that included in the developer preview?

*won’t* :)

 
 Or write the doc validation in Erlang.
 
 I wish i could do that but my skills stuck at the Erlang 101. I have seen an 
 Erlang implementation that supports json-schema Draft 3 - i will analyse 
 whether our schemas using Draft 4 related definitions or not.
 
 
 An other question came into my mind. Is the JS code of a design doc (here the 
 validate_doc_update.js) already evaluated (maybe after the doc save action) 
 or gets it evaluated for every single doc again? When it is the second i 
 would see some smaller optimisation potentials in my current implementation 
 (e.g. bigger docs are more performant to validate then the same amount split 
 into smaller pieces)

The validation function code is already evaluated. The expensive bit is 
serialising the docs from Erlang to JSON and back while talking to the query 
server. Just like with views.

Best
Jan
--

--
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: connection handling in long running bulk doc uploads

[Jan Lehnardt]

Heya Ingo,

I think new streaming interfaces are the way to go. We’ve been talking about 
them in other contexts like replication before. We won’t likely get to this 
before 2.0, though.

In the meantime, the best you can do is, as you write, smaller batches.

An alternative approach would be to disable doc validation on the target DB, do 
the _bulk_docs inserts and then replicate the database into a local copy that 
does have doc validation.

Or write the doc validation in Erlang.

Best
Jan
--
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/



 On 16 Mar 2015, at 11:50, Ingo Radatz thewh...@googlemail.com wrote:
 
 Hi!
 
 My use case is to upload around 100k JSON documents (500Mb) to the CouchDB 
 via a PUT to the _bulk_doc handler. Everything works well and because some 
 schema validation is involved the upload time of an hour was not surprising.
 
 Unfortunately it turns out that some proxy servers (e.g. squid) on the sender 
 site cancel the connection (a default timeout config param was reached) while 
 they should wait for the response of the _bulk_doc handler. Because to change 
 the config of foreign proxy servers is theoretically but not practically 
 possible i'm looking for a solution that lets such proxies know that the 
 connection is healthy and should not be canceled.
 
 - Maybe a streamed approach for the up- and download phases?
 - Maybe a heartbeat?
 - Any other idea?
 
 A quick and dirty solution is done - uploads are made now in smaller batch 
 sizes. That cannot be the final solution because it depends on so many 
 variables that it stays just a hope to finish until some timeout will hit.
 
 Here a little repetition of the failed upload process:
 
 1. send 10k docs as one JSON payload to the _bulk_doc
 2. wait a time longer then the default timeout in the proxy (e.g. 4 minutes) 
 for the response
 3. get disconnected by the proxy without a response from CouchDB
 
 Best, ingo
 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Compaction doesn't work on big db - unknown_error, reason: function_clause

[Jan Lehnardt]

tl;dir: this doesn’t look good, and shouldn’t happen from what CouchDB is doing 
to the file. Maybe something external messed with the DB file.


Could you share the fist megabyte of the file somewhere?

What filesystem is this on?

Did the machine reboot at any time between the creation of the database and now?

Did the CouchDB process terminate (intentionally or unintentionally at any time 
between the creation of the database and now?

Did you backup/restore the hard drive at any time between the creation of the 
database and now?

Did you update the CouchDB server at any time between the creation of the 
database and now? If yes, from which version?

What is your “delayed_commits” setting?

Best,
Jan
--
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

 On 14 Mar 2015, at 08:51, Frank Röhm francwal...@gmx.net wrote:
 
 Hello
 
 On my Ubuntu 12.04 LTS server I have installed couchdb version 1.6.1-0ubuntu1.
 $ couchdb -V
 Apache CouchDB 1.6.1
 
 I have “scraped” the german wiki into this couchdb and now I have a 
 uncompressed 44 GB database “de-m-wikipedia-org”, which I try to compact with:
 
 $ curl -H Content-Type: application/json -X POST 
 http://localhost:5984/de-m-wikipedia-org/_compact
 {ok:true}
 
 But this throws an error:
 
 Error in process 0.247.0 with exit value: 
 {function_clause,[{couch_compress,decompress,[0 
 bytes]},{couch_file,pread_term,2},{couch_btree,get_node,2},{couch_btree,stream_node,7},{couch_btree,stream_kp_node,7},{couch_btree,stream_kp_node,8},{couch_btree,fold,4},{couch_db_updater...
 
 complete error is attached in the end of this mail.
 
 and I get a file de-m-wikipedia-org.couch.compact which has only 79 Byte and 
 wouldn’t grow bigger.
 
 If I compact another db, which had 7.4 GB, the compaction works flawless, I 
 read in the log:
 
 [Sat, 14 Mar 2015 07:37:21 GMT] [info] [0.695.0] Starting compaction for db 
 de-m-wiktionary-org
 [Sat, 14 Mar 2015 07:37:21 GMT] [info] [0.139.0] 127.0.0.1 - - POST 
 /de-m-wiktionary-org/_compact 202
 [Sat, 14 Mar 2015 07:39:39 GMT] [info] [0.695.0] Compaction for db 
 de-m-wiktionary-org completed.
 
 and after compaction the file is 1.7 GB.
 
 So I guess it is related to this big database.
 Should I just delete it and restart to scrape again from scratch?
 
 
 Thank you
 
 frank
 
 
 
 
 
 
 PS.: this is the complete error message in the couch.log
 
 
 [Sat, 14 Mar 2015 07:28:35 GMT] [info] [0.31.0] Apache CouchDB has started 
 on http://127.0.0.1:5984/
 [Sat, 14 Mar 2015 07:30:05 GMT] [info] [0.242.0] Starting compaction for db 
 de-m-wikipedia-org
 [Sat, 14 Mar 2015 07:30:05 GMT] [info] [0.137.0] 127.0.0.1 - - POST 
 /de-m-wikipedia-org/_compact 202
 [Sat, 14 Mar 2015 07:30:05 GMT] [error] [emulator] Error in process 0.247.0 
 with exit value: {function_clause,[{couch_compress,decompress,[0 
 bytes]},{couch_file,pread_term,2},{couch_btree,get_node,2},{couch_btree,stream_node,7},{couch_btree,stream_kp_node,7},{couch_btree,stream_kp_node,8},{couch_btree,fold,4},{couch_db_updater...
 
 
 [Sat, 14 Mar 2015 07:30:05 GMT] [error] [0.242.0] ** Generic server 
 0.242.0 terminating
 ** Last message in was {'EXIT',0.247.0,
   {function_clause,
   [{couch_compress,decompress,[]},
{couch_file,pread_term,2},
{couch_btree,get_node,2},
{couch_btree,stream_node,7},
{couch_btree,stream_kp_node,7},
{couch_btree,stream_kp_node,8},
{couch_btree,fold,4},
{couch_db_updater,copy_compact,3}]}}
 ** When Server state == {db,0.241.0,0.242.0,0.247.0,
1426318205344581,0.243.0,0.239.0,
0.245.0,
{db_header,6,2624284,0,
{46859286084,
 {1775313,0,15923938389},
 159502872},
{46859291358,1775313,123250096},
nil,0,nil,nil,1000},
2624284,
{btree,0.239.0,
{46859286084,
 {1775313,0,15923938389},
 159502872},
#Funcouch_db_updater.10.55895019,
#Funcouch_db_updater.11.100913286,
#Funcouch_btree.5.296538,
#Funcouch_db_updater.12.39068440,snappy},
{btree,0.239.0,
{46859291358,1775313,123250096},
#Funcouch_db_updater.13.114276184,
#Funcouch_db_updater.14.2340873,
#Funcouch_btree.5.296538,
 

Professional Support for Apache CouchDB™

[Jan Lehnardt]

Dear CouchDB Community,

I’m proud to announce that my company Neighbourhoodie Software
is now offering professional support for Apache CouchDB at

http://www.neighbourhood.ie/couchdb-support/

Our professional support offering is meant to complement the
great community support that CouchDB users can already get
here. Sometimes you just need a quicker response time or you
have confidential data that you can’t share with the public,
and that’s where we come in.

In addition, we are not interested in moving people to a so
called “enterprise” edition of CouchDB. We are supporting
the genuine article: Stock Apache CouchDB.

We expect regular contributions in terms of documentation,
code and bug fixes coming out of our work that will directly
go into Apache CouchDB.

I hope you are as excited about this as I am!

If you are ever stuck with anything CouchDB, during
development or in production and you need professional
help, get in touch!

Best
Jan
-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/

Re: Documentation for 2.0, First Take

[Jan Lehnardt]

 On 06 Mar 2015, at 12:00, Alexander Shorin kxe...@gmail.com wrote:
 
 On Mon, Mar 2, 2015 at 2:30 PM, Maria Andersson ma...@dualpose.com wrote:
 What does changelog do in the database document?
 
 Change log is a ordered list of changes in the shards of this database.
 
 What I could see in the sources, CouchDB only writes to it when
 creating a database, and then only add.
 
 And it also lack of timestamps, which makes it almost useless.
 
 Does this mean that changelog is only for the admins?
 
 Admins and devops, yes. At least they may see that some shard on some
 node was added for some database.
 
 What are the other actions?
 delete? Is there a move? (Should not exist as it is add + delete.)
 
 Not implemented I believe.

Correct, I implemented it for 1.x once, but then we realised it is
two operations and we can’t guarantee consistency across them, so
we took it out. Clients can do COPY  DELETE to get the same effect
plus the suual consistency guarantees.


 
 Should views be moved together with the shard?
 
 IMHO, yes.
 
 Or should they be recreated by the new node after moving the shard?
 
 If we had replicate a view feature, I would say they should be
 replicated. But in current state of things, they have to be rebuild
 from scratch. What may annoy some people.
 
 --
 ,,,^..^,,,



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Documentation for 2.0, First Take

[Jan Lehnardt]

Hi Maria,

this is amazing, please keep going :) — Let us know if we can help in any way!

Best
Jan
--

 On 01 Mar 2015, at 17:11, Maria ma...@dualpose.com wrote:
 
 
 (This is the correct email to respond to, sorry for the noise.)
 
 This is the result of months hanging on IRC picking up one piece of
 information at a time from the devs and one day wrestling with CouchDB.
 I am sending this to the list, hoping that it will be useful to someone.
 Please tell if you see an error or you have a question that was not
 answered.
 
 https://github.com/mar-ia/couchdb-crazy
 
 If that name does not make you think twice about running the master of a
 database under development using the instructions from someone on the
 Internet, then I do not know what will ;)
 Expect things to go wrong in spectacular ways.
 Sounds like fun?
 Well then, get popcorn, some servers and let's go!
 
 
 Section 1:
 01-setup
 02-installation
 03-deploy
 
 How to build a cluster of physical firewalled machines, or running all
 the nodes on a single computer.
 
 Section 2:
 04-use
 05-sharding
 
 Takes the cluster for a spin. Add nodes, remove nodes, move shards.
 Explains what n,q,r and w are.
 
 
 TL;DR
 Wait for the final release.
 
 // Maria



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: couch wiki - utils page

[Jan Lehnardt]

Heya Mike,

sounds like plan, go ahead :)

Best
Jan
--

 On 25 Feb 2015, at 21:05, Mike m...@wolman.co.uk wrote:
 
 Hi All,
 
 I have added a couple of bits to the 
 https://cwiki.apache.org/confluence/display/COUCHDB/Useful+utilities
 page and realised it is a little jumbled.
 
 Would it be sensible to split it up into clearer sections eg:
 
 Couch Apps
   Futon Chrome Bookmark App
   couchdb-utils - A fast and portable CouchDB command line utility (Go).
   couchapp - popular command line utility for managing design documents 
 (Python).
  
 
 Search/External Applications
   Elasticasearch -River info + links to tutorial ?
   Lucene -CouchDB-Lucene https://github.com/rnewson/couchdb-lucene - A 
 full-text indexer...
   Postgres
 
 
 Statistics
munin-plugin-couchdb - Munin plugin for graphing CouchDB statistics.
statsd backend - Backend for StatsD to emit metrics to CouchDB.
 
 
 Utilities
   jq - A command-line JSON processor. You can use it to parse, filter and 
 manipulate CouchDB's JSON output.
   jgrep - A Ruby-based CLI tool and API for parsing and displaying JSON data 
 using a logical expression syntax.
   Kundera - An Object-Datastore Mapping Library for couchdb.
   couch-cli - Command line interface to facilitate CouchDB replication and 
 other administrative tasks (Groovy)....
 
 
 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Why are temp views so much faster than design docs?

[Jan Lehnardt]

 On 11 Feb 2015, at 11:30, Alexander Shorin kxe...@gmail.com wrote:
 
 On Wed, Feb 11, 2015 at 1:25 PM, Mike Marino mmar...@gmail.com wrote:
 On Wed, Feb 11, 2015 at 11:18 AM, Alexander Shorin kxe...@gmail.com wrote:
 
 As for btree, it allows to deduplicate map functions result. Say, you
 need to apply different reduce functions onto the same map function.
 You'll create two views with the same maps and different reduces. On
 disk, only one map result will be stored.
 
 
 This is interesting.  How does Couch know that they are the same map
 function if they are in separate views?  Is this simply a string comparison?
 
 By source code hash iirc.

That is correct.



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: What replaces e4x for importing XML into couchdb?

[Jan Lehnardt]

 On 05 Jan 2015, at 11:57 , Sayth Renshaw flebber.c...@gmail.com wrote:
 
 Hi I am Sayth from Newcastle Australia.

Heya Sayth! :)

 Starting to use couchdb looking


Welcome!

 around at articles for importing xml data into couchdb I noticed E4X
 mentioned. Following up this it seems now that e4x is deprecated
 
 There is an article which starts to showw JSX replacing e4x
 http://blog.vjeux.com/2013/javascript/jsx-e4x-the-good-parts.html but it
 appears to me this may be more speculation than anything?
 
 Would I assume correctly that I should use xquery to filter the xml(large
 and somewhat complex) with the data I want and then convert it into json
 and import from there into couchdb?
 
 Or is there a more standard way to work with XML in couchdb?

The most forward looking suggestion would be to do all XML parsing in pure 
JavaScript. I’m sure there are various projects available to do what you need.

Best
Jan
--

Re: Behavior in event of disk i/o errors?

[Jan Lehnardt]

 On 01 Jan 2015, at 12:30 , jumbo jim jumboji...@gmail.com wrote:
 
 Cluster of unreliable commodity hardware.
 
 Given the nature of master/master replication, I would imagine that you
 could run cluster nodes without any form of raid?

This is a calculation of odds: how likely is it that your node fails vs.
how likely is it that raid fails vs. how likely it is that a single disk
fails, vs. how expensive is it to run either of these. All of this depends
on your setup, data and budget, but your suggestions might fit your setup.

 As document commits are all or nothing, I would assume that auto
 replication would not occur if io errors were experienced.
 
 How might couchdb behave in the event of io errors during reads?

we checksum data on disk and throw errors if the underlying blocks have
changed.

E.g. if you have an i/o error (of the nature that a block on disk was
flipped behind the back of CouchDB), that error will not replicate to
any replication targets.

Best
Jan
--




signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Disable purge? Also, hide all docs?

[Jan Lehnardt]

 On 30 Dec 2014, at 09:01 , Alexander Shorin kxe...@gmail.com wrote:
 
 The easiest way is indeed to put CouchDB behind a proxy, but for
 _all_docs you can just remove this http endpoint from config by adding
 the following in your local.ini
 
 [httpd_db_handlers]
 _all_docs =
 
 Suddenly, for _purge you cannot do the same trick. However, you might
 want to open issue on JIRA about since this looks like is an unwelcome
 behaviour.

+1 to file an issue for this.


 --
 ,,,^..^,,,
 
 
 On Tue, Dec 30, 2014 at 6:38 AM, jumbo jim jumboji...@gmail.com wrote:
 Hi,
 
 I don't want a particular user to be able to delete any documents - only
 create/update. I have a design doc to prevent normal deletion -
 
 function(newDoc, oldDoc, userCtx, secObj) {
if ( (newDoc._deleted === true)  (userCtx.roles.indexOf('_admin') ===
 -1) ) {
 throw({forbidden: 'no way!'});
}
else{
 return;
}
 }
 
 However, the user can still purge! Is there a way to stop this?
 
 Also, is there a way to hide all docs? ie, block this -
 http://localhost:5984/DB/_all_docs
 
 I know I could put a proxy in between and block/rewrite _purge or
 _all_docs, but is there an easier way?
 
 Thanks



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Replicating through an nginx server?

[Jan Lehnardt]

 On 29 Dec 2014, at 21:04 , Stephan Wissel step...@wissel.net wrote:
 
 Hi there,
 
 I have a timeout problem with my replications. My replication request
 roughly looks like:
 (I started with https, but now test with http first, step by step)
 
 {
target: wisselblog,
source: {
url: http://someserver.com/couch/wisselblog;,
auth: {
oauth: {
consumer_secret: somesecret,
consumer_key: somekey,
token_secret: tokensecret,
token: sometoken
}
}
},
connection_timeout: 6,
user_ctx: {
name: stw,
roles: [_admin]
}
}
 
 CouchDB sits behind a nginx HTTP server. When I access the database via
 browser everything works nicely.
 However I get timeouts and no replication when I try to replicate.
 
 The Error in the log looks like this:
 
 [Mon, 29 Dec 2014 19:38:49 GMT] [info] [0.16786.0] Retrying *HEAD*
 request tohttp://someserver.com/couch/wisselblog in 0.25 seconds due to
 error {conn_failed,{error,nxdomain}}
 
 I poked around with CURL and found a strange behavior of nginx:
 
 when executing curl -X HEAD the connection won't close (times out), only
 with curl --head it works as expected.

Just for the record, curl -X HEAD is not valid curl, you mean curl -I (capital 
i).

Best
Jan
--


 
 I suspect (?) CouchDB does something similar as CURL?
 Is there a way to fix that. Server Couch is 1.6x client 1.7
 
 
 Create a nice day!
 Stephan H. Wissel
 
 Phone: +65 96673269
 Blog http://www.wissel.net/blog Twitter http://twitter.com/notessensei
 Google+ https://plus.google.com/114116118906495091782/about Facebook
 http://www.facebook.com/stephan.wissel LinkedIn
 http://sg.linkedin.com/in/notessensei Xing
 https://www.xing.com/profile/StephanH_Wissel Sametime
 http://www.wissel.net/blog/d6plinks/SHWL-84KE8U *Skype*



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [BLOG] The CouchDB Weekly News and Highlights in 2014 are out

[Jan Lehnardt]

 On 19 Dec 2014, at 13:14 , Andy Wenk andyw...@apache.org wrote:
 
 Dear Lena,
 
 I would like to thank you very very much for your work in 2015. With your
 extremely cool Weekly news, you have helped to spread the word about
 CouchDB very much. This is simply awesome. I am convinced that we have
 reached out to many more people as we have in 2013. And when looking to the
 stats, the weekly news are very well received.
 
 I am looking forward to 2015. We will definitely have some LOUDNESS and BIG
 BANGS and we will communicate them with the weekly news :D
 
 All the best for you and stay ... relaxed!

This, very much this!

Thank you SO MUCH Lena! 3

Best
Jan
--

 
 Very very kind regards
 
 Andy
 
 On 19 December 2014 at 12:22, Lena Reinhard l...@thehoodiefirm.com wrote:
 
 Hi everyone,
 
 this week’s CouchDB Weekly News is out, this time with many highlights from
 2014:
 
 
 http://blog.couchdb.org/2014/12/19/couchdb-weekly-news-december-19-2014-and-couchdb-highlights-in-2014/
 
 Thanks to Andy, Jan and Akshat for their contributions.
 
 We want to ask you to help us promote the News, this is also a way to
 contribute to the project –
 Twitter:  https://twitter.com/CouchDB/status/545897769391890432
 Reddit:
 
 http://www.reddit.com/r/CouchDB/comments/2ps1rr/couchdb_weekly_news_december_19_2014_and_couchdb/
 G+:
 https://plus.google.com/b/109226482722655790973/+CouchDB/posts/D52no8zKZet
 Facebook:
 
 https://www.facebook.com/permalink.php?story_fbid=604937966205098id=507603582605204
 Linkedin:
 
 https://www.linkedin.com/company/5242010/comments?topic=5951663846908248064type=Uscope=5242010stype=Ca=jZvygoback=.bzo_*1_*1_*1_*1_*1_*1_*1_apache*5couchdb
 
 This is the last issue of the Weekly News in 2014, we'll be back on January
 08, 2015. Thanks to everyone who supported and shared the News this year.
 
 Best,
 Lena
 
 
 
 -- 
 Andy Wenk
 Hamburg - Germany
 RockIt!
 
 GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
 
 https://people.apache.org/keys/committer/andywenk.asc

Re: CouchDB is crashing and corrupting databases

[Jan Lehnardt]

 On 11 Dec 2014, at 00:53 , Alejandro Santiago 
 alejandro.santi...@genesys.com wrote:
 
 Hi,
 
 I have a CouchDB instance v1.3.0 with about 6 databases and one of them is 
 used for historical reporting usually with 15 million records and about 30 GB 
 to 45 GB disk space.
 This Sunday it just crashed first rejecting http requests and then even 
 killing the couchDB process.
 
 It is running in a Windows Server 2008 with about 32 GB of ram and 8 CPU 
 cores.
 
 While deleting a bunch of old records from the historical reports database 
 this error message pops-up
 
 [Sun, 07 Dec 2014 18:41:43 GMT] [info] [0.11190.28] 15.128.50.71 - - DELETE 
 /outbound-record-history/6ebaaebd8f0bcffd18121a227a3654f0?rev=1-bf5e3f4fb9cc341b2c7be7599a4f7a8f
  200
 [Sun, 07 Dec 2014 18:41:43 GMT] [info] [0.10928.28] 15.128.50.71 - - DELETE 
 /outbound-record-history/6ebaaebd8f0bcffd18121a227a3651d8?rev=1-7aca019d84960522a12a8d2a7c224e90
  200
 [Sun, 07 Dec 2014 18:41:43 GMT] [info] [0.13583.28] 15.128.50.71 - - DELETE 
 /outbound-record-history/6ebaaebd8f0bcffd18121a227a3655d2?rev=1-467254a30672ad3c56d59ada50e81789
  200
 [Sun, 07 Dec 2014 18:41:43 GMT] [info] [0.195.0] Index shutdown by monitor 
 notice for db: outbound-record-history idx: _design/RecordHistory
 [Sun, 07 Dec 2014 18:41:43 GMT] [info] [0.202.0] Index shutdown by monitor 
 notice for db: outbound-record-history idx: _design/FileFormatter
 [Sun, 07 Dec 2014 18:41:43 GMT] [error] [0.13583.28] Uncaught error in HTTP 
 request: {exit,
{{badmatch,
  {error,eio}},


error,eio points to rather serious issues with your disks:


case ERROR_ARENA_TRASHED:
case ERROR_INVALID_BLOCK:
case ERROR_BAD_ENVIRONMENT:
case ERROR_BAD_COMMAND:
case ERROR_CRC:
case ERROR_OUT_OF_PAPER:
case ERROR_READ_FAULT:
case ERROR_WRITE_FAULT:
case ERROR_WRONG_DISK:
case ERROR_NET_WRITE_FAULT:
return EIO;
default: /* not to do with files I expect. */
return EIO;

https://github.com/erlang/otp/blob/maint/erts/emulator/drivers/win32/win_efile.c#L256-268

I don’t think CouchDB can do much in the face of disk errors like this (and out 
of paper! :) bur crash.

Best
Jan
--



 [{couch_btree,
   
 '-write_node/3-lc$^0/1-0-',
   5,
   [{file,
 
 c:/cygwin/relax/APACHE~1.0/src/couchdb/couch_btree.erl},
{line,358}]},
  {couch_btree,
   write_node,3,
   [{file,
 
 c:/cygwin/relax/APACHE~1.0/src/couchdb/couch_btree.erl},
{line,356}]},
  {couch_btree,
   modify_node,4,
   [{file,
 
 c:/cygwin/relax/APACHE~1.0/src/couchdb/couch_btree.erl},
{line,325}]},
  {couch_btree,
   modify_kpnode,6,
   [{file,
 
 c:/cygwin/relax/APACHE~1.0/src/couchdb/couch_btree.erl},
{line,394}]},
  {couch_btree,
   modify_node,4,
   [{file,
 
 c:/cygwin/relax/APACHE~1.0/src/couchdb/couch_btree.erl},
{line,315}]},
  {couch_btree,
   modify_kpnode,6,
   [{file,
 
 c:/cygwin/relax/APACHE~1.0/src/couchdb/couch_btree.erl},
{line,394}]},
  {couch_btree,
   modify_node,4,
   

Re: compaction repeated timeouts causes the server to shutdown temporary when replication is broken

[Jan Lehnardt]

Heya Ciprian,

this sounds like a bug, could you file an issue on 
https://issues.apache.org/jira/browse/COUCHDB

Best
Jan
--

 On 05 Dec 2014, at 08:51 , Ciprian Trusca ctru...@totalsoft.ro wrote:
 
 We have turned on debugging for this test and it looks like the cause of this 
 error is the _replicator database.  
 
 After the list of fragmented databases we see no evidence that the compaction 
 for this database is being started in the log ( although fragmentation is and 
 above the 70% threshold) and then we have the compaction loop dying  after 
 approximately 5 seconds.  So I am guessing CouchDB fails to spawn the 
 compaction process.  
 
 I forgot to mention in the first post that we are running CouchDB 1.6.1 on a 
 Centos 6.4 server.
 
 Thanks for your time, any help will be appreciated.
 
 -Original Message-
 From: Ciprian Trusca [mailto:ctru...@totalsoft.ro] 
 Sent: Thursday, November 27, 2014 10:17 AM
 To: user@couchdb.apache.org
 Subject: compaction repeated timeouts causes the server to shutdown temporary 
 when replication is broken
 
 Hello all,
 we have encountered the following situation during an overnight load test.
 
 We get the following message repeatedly in the couch logs:
 
 ** Reason for termination ==
 
 ** {compaction_loop_died,
 
   {timeout,{gen_server,call,[0.117.0,start_compact]}}}
 
 
 
 At one time, we are getting it three times in an interval of 5 seconds and I 
 am guessing this causes the supervisor to shutdown temporary:
 
 
 [Thu, 20 Nov 2014 05:58:33 GMT] [error] [0.93.0] {error_report,0.30.0,
   {0.93.0,supervisor_report,
[{supervisor,{local,couch_secondary_services}},
 {errorContext,shutdown},
 {reason,reached_max_restart_intensity},
 {offender,
 [{pid,0.10114.14},
  {name,compaction_daemon},
  {mfargs,{couch_compaction_daemon,start_link,[]}},
  {restart_type,permanent},
  {shutdown,brutal_kill},
  {child_type,worker}]}]}}
 
 
 
 In this particular component load test the CouchDB peer is shutdown so the 
 replication is broken, meaning that there are a lot of backgrounds processes 
 that try to replicate and die, and there is a thread that removes the failed 
 replication and re-enables them (probably this is not a good idea anymore 
 since CouchDB detects that the peer came back online on its own now).  I 
 suspect that this might be related.
 
 
 
 In the Zenoss graphs we see a very significant spike in the IO read /writes 
 at that moment.
 
 
 
 Thank you very much for your time, and any hint will be appreciated.

Re: PBDK2 implementation specs

[Jan Lehnardt]

 On 08 Dec 2014, at 04:30 , Alexander Shorin kxe...@gmail.com wrote:
 
 On Mon, Dec 8, 2014 at 6:08 AM, Jason Winshell (Bear River)
 jas...@bearriver.com wrote:
 Thanks. That's exactly what I needed. The relevant government specification 
 is sp800-131A (section 8, 9 10). It looks like CouchDB's PBDK2 is FIPs 
 compliant because it meets the requirements of all these 3 sections -- 
 together.
 
 Section 8: HMAC key derivation functions are acceptable
 
 Section 9: Though these days SHA-256/384/512 are preferrer, the spec says 
 that SHA-1 is acceptable for Non-digital signature generation applications
 
 Section 10: HMAC to with key length = 112 bits OK past the year 2013.
 
 Good to know, thanks for the information!
 
 Since CouchDB is SHA-1, 160 bit key length, 128 salt and for non-digital 
 signatures, it's FIPs compliant. I'd like to suggest that CouchDB switch to 
 SHA-256, 256 bit key length.
 
 That isn't possible without dropping support of some Erlang releases
 series (R14 and R15). But somewhere in future that will be.

Excellent analysis everyone!

I filed an issue so that we remember to do the upgrade: 
https://issues.apache.org/jira/browse/COUCHDB-2495

Alexander, should we add the PBKDF2 details to the docs?

Best
Jan
--

Re: [NEWS] Your links for the CouchDB Weekly News

[Jan Lehnardt]

Hey Lena,

this excellent post by Robert would make a great addition:

http://robert-kowalski.de/blog/lets-learn-erlang-and-fix-a-bug-on-a-couchdb-cluster/

Best
Jan
--

 On 02 Dec 2014, at 17:28 , Lena Reinhard l...@thehoodiefirm.com wrote:
 
 Hi everyone,
 
 if you want to submit a link for this week’s CouchDB Weekly News, please send 
 it to this thread until Tuesday, December 04, 1pm UTC+1.
 
 Regards,
 Lena

Re: A elegant way around purging documents?

[Jan Lehnardt]

Heya Jason,

not sure this works if you have two way replication, because then the
server gets the delete and there is nothing to bump.

Best
Jan
--

 On 26 Nov 2014, at 15:51 , Jason Smith jason.h.sm...@gmail.com wrote:
 
 Hi, Jan. I'm sure you know this already, but to me, the current solution is
 also the correct one.
 
 The user deleted some information. The server has the old version. But now
 the user needs it again.
 
 Each of the sentences above seems to me like a bit of useful information in
 the application. And so it's reasonable that each one is a revision to the
 document.
 
 In other words: bump or touch your document so it will re-replicate
 down. This may be a good time to keep a timestamp or something else. Yes
 you have to do some work yourself, but it kind of makes sense if you look
 at it a certain way :)
 
 On Tue, Nov 25, 2014 at 8:24 PM, Jan Lehnardt j...@apache.org wrote:
 
 Heya Stefan,
 
 just a quick note: Purge is definitely the wrong approach.
 
 I’ve been trying to solve something similar recently for a client
 and we didn’t come up with a conclusive solution. I’d love for CouchDB
 and PouchDB to natively support this use-case, but this is currently
 not possible as per the design of CouchDB.
 
 Would you be interested in opening a ticket so we can discuss this
 with the developers? https://issues.apache.org/jira/browse/COUCHDB
 
 The project ended up deleting client-side databases to free space
 followed by a re-sync of only the “current” parts. It worked for
 the setup, but that’s by far not generally applicable, let alone
 a satisfactory solution.
 
 Best
 Jan
 --
 
 
 
 On 25 Nov 2014, at 15:18 , Stefan Klein st.fankl...@gmail.com wrote:
 
 Hi couch users,
 
 I got a main database and a database per user.
 A users Db is replicated to his device (mobile phone) - idea is as much
 functionality should be available offline.
 
 To simplify a bit, let's say i got documents of type item and of type
 image, each item may reference multiple images and each image may
 be referenced by multiple items.
 When a user gets a new item from the main database, a daemon checks if
 all referenced images are available for that user and triggers
 replication
 of the images missing.
 If a user still got images which aren't needed on his device anymore, i
 want to delete them, why waste space on his device?
 
 Here comes my problem:
 Say he got the document image1 in revision: 1-abc and doesn't need it
 anymore i will delete it, which creates {_id: 'image1', rev: '2-cde',
 _deleted: true} if for some reason he needs image1 again because it is
 also referenced by a different item i will replicate {_id: 'image1',
 rev:
 '1-abc', ./*.. */} from the main database to the users database again.
 The
 users database says i know an ancestor of that document the revision
 2-cde in that example and the document image1 will not show up again.
 
 One way to solve that by attaching the images directly to the items -
 but
 we got ~7 million items sharing ~4000 images, so that would increase the
 db
 size by much.
 
 An other way is using purge to delete an image so it can be replicated
 again, but it seams to be wrong to use purge for general application
 logic,
 think it should be kind of last resort.
 
 Any other ideas?
 
 Thanks  regards,
 Stefan
 
 

Re: Offline replication/synchronisation

[Jan Lehnardt]

 On 26 Nov 2014, at 10:47 , Alexander Harm cont...@aharm.de wrote:
 
 Hello Jan,
 
 does CouchDB figure out the delta based on a previous sync or always 
 comparing the full database? Just to know if there is a difference in the 
 first online sync after an offline sync (assuming the database is not 
 returned to the remote location after the sync).

CouchDB does figure out the delta on subsequent replications. IN case of the 
first replication the delta is of course the full database.

Best
Jan
--

 
 Regards,
 
 Alexander
 
 On 24. Nov. 2014, at 18:31, Jan Lehnardt j...@apache.org wrote:
 
 Heya Alex,
 
 great cause! Multiple options:
 
 - connections are bad, but work sometimes and it is ok if data is only 
 synchronised sometimes: This works today.
 
 - connections are bad, period. Copy the .couch file (“the CouchDB database”) 
 to something that you can bring to a connected hub. Put the .couch file into 
 a place that can run CouchDB, start CouchDB, sync from there. CouchDB will 
 figure out the delta.
 
 In this, you will be carrying the full DB, but only sync the delta. Just 
 carrying the delta is currently not really an option. If that is a concern, 
 maybe you can hack around with 
 https://github.com/nolanlawson/pouchdb-replication-stream / 
 https://github.com/nolanlawson/pouchdb-dump-cli / 
 https://github.com/nolanlawson/pouchdb-load
 
 Good luck!
 Best
 Jan
 --
 
 
 
 
 On 23 Nov 2014, at 21:40 , Alexander Harm cont...@aharm.de wrote:
 
 Hello,
 
 I work at a large humanitarian organisation and I would like to create a 
 new app to manage our local staff (roughly 9.000 in more than 30 
 countries). I was looking into CouchDB as possible backend and it meets 
 many of my requirements. However, there is one little issue I could find 
 nothing on the web about. Although probably all of our projects have some 
 sort of Internet connection (DSL, V-Sat, Thuraya) it still happens that 
 online synchronisation is simply impossible due to miserable connection 
 quality and not just somewhere in the bush.
 
 I was wondering if it is possible to create some sort of file with the 
 delta-update that could be transported to a place with better Internet 
 access or to the next hub (e. g. country coordination) where it is imported 
 in the instance running there. Does anyone know if something like that 
 could be feasible?
 
 Regards,
 
 alh
 
 

Re: A elegant way around purging documents?

[Jan Lehnardt]

Heya Stefan,

just a quick note: Purge is definitely the wrong approach.

I’ve been trying to solve something similar recently for a client
and we didn’t come up with a conclusive solution. I’d love for CouchDB
and PouchDB to natively support this use-case, but this is currently
not possible as per the design of CouchDB.

Would you be interested in opening a ticket so we can discuss this
with the developers? https://issues.apache.org/jira/browse/COUCHDB

The project ended up deleting client-side databases to free space
followed by a re-sync of only the “current” parts. It worked for
the setup, but that’s by far not generally applicable, let alone
a satisfactory solution.

Best
Jan
--



 On 25 Nov 2014, at 15:18 , Stefan Klein st.fankl...@gmail.com wrote:
 
 Hi couch users,
 
 I got a main database and a database per user.
 A users Db is replicated to his device (mobile phone) - idea is as much
 functionality should be available offline.
 
 To simplify a bit, let's say i got documents of type item and of type
 image, each item may reference multiple images and each image may
 be referenced by multiple items.
 When a user gets a new item from the main database, a daemon checks if
 all referenced images are available for that user and triggers replication
 of the images missing.
 If a user still got images which aren't needed on his device anymore, i
 want to delete them, why waste space on his device?
 
 Here comes my problem:
 Say he got the document image1 in revision: 1-abc and doesn't need it
 anymore i will delete it, which creates {_id: 'image1', rev: '2-cde',
 _deleted: true} if for some reason he needs image1 again because it is
 also referenced by a different item i will replicate {_id: 'image1', rev:
 '1-abc', ./*.. */} from the main database to the users database again. The
 users database says i know an ancestor of that document the revision
 2-cde in that example and the document image1 will not show up again.
 
 One way to solve that by attaching the images directly to the items - but
 we got ~7 million items sharing ~4000 images, so that would increase the db
 size by much.
 
 An other way is using purge to delete an image so it can be replicated
 again, but it seams to be wrong to use purge for general application logic,
 think it should be kind of last resort.
 
 Any other ideas?
 
 Thanks  regards,
 Stefan

Re: Offline replication/synchronisation

[Jan Lehnardt]

Heya Alex,

great cause! Multiple options:

- connections are bad, but work sometimes and it is ok if data is only 
synchronised sometimes: This works today.

- connections are bad, period. Copy the .couch file (“the CouchDB database”) to 
something that you can bring to a connected hub. Put the .couch file into a 
place that can run CouchDB, start CouchDB, sync from there. CouchDB will figure 
out the delta.

In this, you will be carrying the full DB, but only sync the delta. Just 
carrying the delta is currently not really an option. If that is a concern, 
maybe you can hack around with 
https://github.com/nolanlawson/pouchdb-replication-stream / 
https://github.com/nolanlawson/pouchdb-dump-cli / 
https://github.com/nolanlawson/pouchdb-load

Good luck!
Best
Jan
--




 On 23 Nov 2014, at 21:40 , Alexander Harm cont...@aharm.de wrote:
 
 Hello,
 
 I work at a large humanitarian organisation and I would like to create a new 
 app to manage our local staff (roughly 9.000 in more than 30 countries). I 
 was looking into CouchDB as possible backend and it meets many of my 
 requirements. However, there is one little issue I could find nothing on the 
 web about. Although probably all of our projects have some sort of Internet 
 connection (DSL, V-Sat, Thuraya) it still happens that online synchronisation 
 is simply impossible due to miserable connection quality and not just 
 somewhere in the bush.
 
 I was wondering if it is possible to create some sort of file with the 
 delta-update that could be transported to a place with better Internet access 
 or to the next hub (e. g. country coordination) where it is imported in the 
 instance running there. Does anyone know if something like that could be 
 feasible?
 
 Regards,
 
 alh

Re: [DISCUSS] Improved CouchDB logo

[Jan Lehnardt]

+user@

Thanks Joan for bringing this up!

Thanks Lena for your perspective and courage to voice it here.

Thanks all for your input.

+1 on reworking the logo.
 - an animal on the couch could do the trick, or just no person, but see the 
next point, really. I’m not worried about logo confusion with Couchbase.
+1 on letting professionals do it.
-1 on a colour change. Maybe toning it towards the Fauxtony orange is an 
option, but see the previous point.
-1000 on renaming the project.
+1 on moving on from “Relax” as a theme. It’s a Katzism and out of date as far 
as the people who run the project now. Something around sync would be nice, any 
other completely unrelated whimsy would also be appreciated.

Best
Jan
-- 




 On 30 Oct 2014, at 17:04 , Andy Wenk andyw...@apache.org wrote:
 
 On 30 October 2014 16:47, Noah Slater nsla...@apache.org wrote:
 
 Clear up massive brand confusion with Couchbase, basically.
 
 
 I am not sure how big this special issue is because I clearly see the
 situation from CouchDB. If we have a look to the people on the ML who think
 it's Couchbase, that's very minimal ... I would prefer to keep the name and
 clarify the situation as often as possible
 
 
 
 
 
 On 30 October 2014 16:35, Andy Wenk andyw...@apache.org wrote:
 sure - IF we want to rename then with 2.0. But before I give my whatever
 vote, I would like to know which advantages you expect to get with a
 rename
  as of now - I can only think of negative aspects ...
 
 On 30 October 2014 16:08, Jason Smith jason.h.sm...@gmail.com wrote:
 
 Love it. Get away from the WhateverDB thing. Just stake out an arbitrary
 mark from which we build clear, unambiguous IP.
 
 My proposal (as long as we are brainstorming names):
 
Eight Ball
 
 Tagline: Whatever you do. Don't get more eight ball!
 
 More seriously, yes, a name change has lots of problems. But it is worth
 re-considering Noah's point that now is the best time to do it, if it
 is to
 be done.
 
 On Thu, Oct 30, 2014 at 10:02 PM, Noah Slater nsla...@apache.org
 wrote:
 
 I've been wanting to rename it SleepyDB for a while now. (cf. Max
 Ogden's SLEEP protocol.)
 
 On 30 October 2014 11:43, Kevin Coombes kevin.r.coom...@gmail.com
 wrote:
 Definitely -1 on changing the name. It's already cooked into
 everything
 associated with the project. There are links (and hard copy
 references)
 to
 the name and the web site(s) all over the place. How long do you
 want
 to
 be
 forced to maintain DNS or HTTP entries to forward users from the
 current
 names to whatever you might pick for new ones?
 
 I would be perfectly happy to see a new logo however. And the last
 time I
 looked, there were a lot more colors than just red or blue
 
  Kevin
 On Oct 30, 2014 6:11 AM, Brad Noble br...@us.ibm.com wrote:
 
 *SyncDB*
 The database that syncs.
 
 Logo would embody that idea.
 
 
 
 [image: Inactive hide details for Jason Smith ---10/30/2014
 04:40:26
 AM---On Thu, Oct 30, 2014 at 12:52 AM, Noah Slater nslater@apache
 ]Jason
 Smith ---10/30/2014 04:40:26 AM---On Thu, Oct 30, 2014 at 12:52 AM,
 Noah
 Slater nsla...@apache.org wrote:  Blue is nice. :) But the
 
 From: Jason Smith jason.h.sm...@gmail.com
 To: d...@couchdb.apache.org d...@couchdb.apache.org
 Date: 10/30/2014 04:40 AM
 Subject: Re: [DISCUSS] Improved CouchDB logo
 --
 
 
 
 On Thu, Oct 30, 2014 at 12:52 AM, Noah Slater nsla...@apache.org
 wrote:
 
 Blue is nice. :) But then that tempts me to change the name too.
 I
 mean, if we're ever going to do this, now is the time. It's
 important
 to note though that we have a very STRONG brand, and changing it
 in
 a
 big will be expensive.
 
 
 Change the name! Change the name!
 
 Like all transitions, it seems scary. But once it's done we will
 all
 wonder
 what we were so worried about.
 
 The brand is strong, but the question is, how much of that strength
 can
 we
 keep, and how much do we lose.
 
 I think we will keep nearly all of it, plus we will get tons of
 attention
 with a 2.0 release and name change. And as if that already isn't
 enough, we
 get the opportunity to walk away from a ton of branding baggage and
 trademark problems.
 
 Change the name!
 
 
 
 
 
 --
 Noah Slater
 https://twitter.com/nslater
 
 
 
 
 
 --
 Andy Wenk
 Hamburg - Germany
 RockIt!
 
 GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
 
 https://people.apache.org/keys/committer/andywenk.asc
 
 
 
 --
 Noah Slater
 https://twitter.com/nslater
 
 
 
 
 -- 
 Andy Wenk
 Hamburg - Germany
 RockIt!
 
 GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
 
 https://people.apache.org/keys/committer/andywenk.asc

Re: Question: COPY /{db}/_local/{docid}

[Jan Lehnardt]

Heya,

I implemented COPY and I was definitely not taking _local docs into
any special consideration, so any behaviour is purely accidental :)

As far as I can tell, COPY just does what it tells you to do: It copies
a document from one URL to another. It is up to you to ensure that the
destination URL includes the _local identifier. There isn’t anything
special to local docs other than the id. So instead of “local” being
a property of docs, local docs are just docs with an id that has a
_local/ prefix and docs with this ID are treated specially by CouchDB,
but they are just regular documents like all other ones and you can
COPY them to docs with a new id that doesn’t have a _local/ prefix.

The response only includes the new doc id, not the database. The
proper id for a doc with slashes are the unencoded slashes. Since in
JSON the slashes don’t need to be encoded, they aren’t encoded in
the response :) As there is no db info there, a simple replace will
do the trick.

It isn’t clear from your example, but you might be trying to copy a
doc across databases. That won’t work, COPY only copies docs within
a single database. Cross-database COPY could be implemented though,
if you want to open a feature request.

Hope this helps :)

Best
Jan
--



On 27 Aug 2014, at 08:18 , muji freeformsyst...@gmail.com wrote:

 Hi all,
 
 Just wondering if this is expected behaviour.
 
 When using COPY with _local documents and the Destination header contains a
 document identifier that does not start with _local then the document
 appears to be copied without the copied document identifier starting with
 _local/, ie, the copied document is no longer local, it is a regular
 document.
 
 If the destination id starts with _local then it is copied as a local
 document.
 
 As the API call is COPY /{db}_local/{docid} I would expect it to only copy
 to another local document.
 
 I imagine this is by design but I find it a little counter-intuitive and a
 possible way to end up replicating documents inadvertently.
 
 The full request/response log looks like:
 
 [COPY] http://localhost:5984/mock%2Fdatabase/_local/mock%2Fdocument
 {
  destination: mock/document/copy,
  accept: application/json,
  host: localhost:5984
 }
 [201] http://localhost:5984/mock%2Fdatabase/_local/mock%2Fdocument
 {
  server: CouchDB/1.6.0 (Erlang OTP/R15B03),
  location: http://localhost:5984/mock/database/mock/document/copy;,
  etag: \9-0c29f3ff80cdf5234680c1f670653388\,
  date: Wed, 27 Aug 2014 06:09:50 GMT,
  content-type: application/json,
  content-length: 81,
  cache-control: must-revalidate
 }
 {
  ok: true,
  id: mock/document/copy,
  rev: 9-0c29f3ff80cdf5234680c1f670653388
 }
 
 Also just looking at the response log again, the Location of the new
 document has not encoded the {db} {docid} parameters which makes the
 Location header unusable in this situation.
 
 Any help/clarification much appreciated.
 
 -- 
 mischa (aka muji).



signature.asc
Description: Message signed with OpenPGP using GPGMail

On Benoit’s Resignation from the CouchDB PMC

[Jan Lehnardt]

Dear CouchDB Community,

Benoit has resigned from the PMC. We regret that the current situation wasn't 
working out.

We thank Benoit for his work to date and look forward to his continued 
contributions as a committer.

We will continue the trademark dialogue in the appropriate forum.

Jan Lehnardt,
PMC Chair
-- 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Startup issue with CouchDB on Buildroot

[Jan Lehnardt]

On 13 Jun 2014, at 12:38 , Nicolas Peeters nicoli...@gmail.com wrote:

 We have been trying to get the CouchDB working on a Buildroot linux. After
 quite some headaches, we actually have been able to compile it on Buildroot
 (which as such is quite a pain), but we’re getting some issues with the
 startup. Maybe you or some friend of yours from the CouchDB community would
 know what to do…?
 
 We really have no idea where to look (it saysFile not found...)
 
 Any help is really appreciated.
 
 Apache CouchDB 1.6.0 (LogLevel=info) is starting.
 {init terminating in
 do_boot,{{badmatch,{error,{bad_return,{{couch_app,start,[normal,[/usr/local/etc/couchdb/default.ini,/usr/local/etc/couchdb/local.ini]]},{'EXIT',{{badmatch,{error,{shutdown,

 {failed_to_start_child,couch_primary_services,{shutdown,{failed_to_start_child,collation_driver,File
 not found}},

^ This is the key. Meaning: couch_icu_driver.so can’t be found. It is usually 
built as part of the regular CouchDB build. Its source is in 
src/couchdb/priv/icu_driver/

Best
Jan
--



 [{couch_server_sup,start_server,1,[{file,couch_server_sup.erl},{line,98}]},{application_master,start_it_old,4,[{file,application_master.erl},{line,272}]}]}},[{couch,start,0,[{file,couch.erl},{line,18}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
 
 Crash dump was written to: erl_crash.dump
 init terminating in do_boot ()
 
 Being total newbies in Erlang we don’t even know where to look. Any kind of
 help is appreciated.
 
 Thanks a lot.
 
 Nicolas



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Startup issue with CouchDB on Buildroot

[Jan Lehnardt]

Heya Nicolas  Team,

I replied on the list.

Best
Jan
--

On 13 Jun 2014, at 12:38 , Nicolas Peeters nicoli...@gmail.com wrote:

 We have been trying to get the CouchDB working on a Buildroot linux. After
 quite some headaches, we actually have been able to compile it on Buildroot
 (which as such is quite a pain), but we’re getting some issues with the
 startup. Maybe you or some friend of yours from the CouchDB community would
 know what to do…?
 
 We really have no idea where to look (it saysFile not found...)
 
 Any help is really appreciated.
 
 Apache CouchDB 1.6.0 (LogLevel=info) is starting.
 {init terminating in
 do_boot,{{badmatch,{error,{bad_return,{{couch_app,start,[normal,[/usr/local/etc/couchdb/default.ini,/usr/local/etc/couchdb/local.ini]]},{'EXIT',{{badmatch,{error,{shutdown,{failed_to_start_child,couch_primary_services,{shutdown,{failed_to_start_child,collation_driver,File
 not 
 found}},[{couch_server_sup,start_server,1,[{file,couch_server_sup.erl},{line,98}]},{application_master,start_it_old,4,[{file,application_master.erl},{line,272}]}]}},[{couch,start,0,[{file,couch.erl},{line,18}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
 
 Crash dump was written to: erl_crash.dump
 init terminating in do_boot ()
 
 Being total newbies in Erlang we don’t even know where to look. Any kind of
 help is appreciated.
 
 Thanks a lot.
 
 Nicolas

Jan Lehnardt
The Neighbourhoodie Software GmbH
Adalbertstr. 7-8, 10999 Berlin
neighbourhood.ie

Handelsregister HRB 157851 B Amtsgericht Charlottenburg
Geschäftsführung: Jan Lehnardt, Lena Reinhard



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Startup issue with CouchDB on Buildroot

[Jan Lehnardt]

On 13 Jun 2014, at 13:32 , Nicolas Peeters nicoli...@gmail.com wrote:

 @Jan: How would we know that it's the couch_icu_driver.so ?

Because it says “collation_driver,File not found”. CouchDB only does one type 
of collation: in view indexes, and it uses ICU for that :)

Best
Jan
--



 
 
 On Fri, Jun 13, 2014 at 1:26 PM, Jan Lehnardt j...@apache.org wrote:
 
 
 On 13 Jun 2014, at 12:38 , Nicolas Peeters nicoli...@gmail.com wrote:
 
 We have been trying to get the CouchDB working on a Buildroot linux.
 After
 quite some headaches, we actually have been able to compile it on
 Buildroot
 (which as such is quite a pain), but we’re getting some issues with the
 startup. Maybe you or some friend of yours from the CouchDB community
 would
 know what to do…?
 
 We really have no idea where to look (it saysFile not found...)
 
 Any help is really appreciated.
 
 Apache CouchDB 1.6.0 (LogLevel=info) is starting.
 {init terminating in
 
 do_boot,{{badmatch,{error,{bad_return,{{couch_app,start,[normal,[/usr/local/etc/couchdb/default.ini,/usr/local/etc/couchdb/local.ini]]},{'EXIT',{{badmatch,{error,{shutdown,
 
 
 {failed_to_start_child,couch_primary_services,{shutdown,{failed_to_start_child,collation_driver,File
 not found}},
 
 ^ This is the key. Meaning: couch_icu_driver.so can’t be found. It is
 usually built as part of the regular CouchDB build. Its source is in
 src/couchdb/priv/icu_driver/
 
 Best
 Jan
 --
 
 
 
 
 [{couch_server_sup,start_server,1,[{file,couch_server_sup.erl},{line,98}]},{application_master,start_it_old,4,[{file,application_master.erl},{line,272}]}]}},[{couch,start,0,[{file,couch.erl},{line,18}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
 
 Crash dump was written to: erl_crash.dump
 init terminating in do_boot ()
 
 Being total newbies in Erlang we don’t even know where to look. Any kind
 of
 help is appreciated.
 
 Thanks a lot.
 
 Nicolas
 
 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: ApacheCon CFP closes June 25

[Jan Lehnardt]

Heya,

thanks Andy for the heads-up!

In addition, there is an Apache Travel Grant programme, that could help you out 
getting  to the conference: http://www.apache.org/travel/

Best
Jan
--



On 11 Jun 2014, at 10:51 , Andy Wenk andyw...@apache.org wrote:

 Dear CouchDB enthusiast,
 
 As you may be aware, ApacheCon will be held this year in Budapest, on
 November 17-23. (See http://apachecon.eu for more info.)
 
 The Call For Papers for that conference is still open, but will be
 closing soon. We need you talk proposals, to represent CouchDB at
 ApacheCon. We need all kinds of talks - deep technical talks, hands-on
 tutorials, introductions for beginners, or case studies about the
 awesome stuff you're doing with CouchDB.
 
 Please consider submitting a proposal, at
 http://events.linuxfoundation.org//events/apachecon-europe/program/cfp
 
 Thanks!
 
 -- 
 Andy Wenk
 Hamburg - Germany
 RockIt!
 
 GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
 
 https://people.apache.org/keys/committer/andywenk.asc



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [ANNOUNCE] Apache CouchDB 1.6.0 released

[Jan Lehnardt]

Good catch Tito! Fixed.

On 10 Jun 2014, at 19:27 , Tito Ciuro tci...@mac.com wrote:

 The download section still shows 1.5.1 as the current version.
 
 -- Tito
 
 On Jun 10, 2014, at 10:23 AM, Dirkjan Ochtman d...@apache.org wrote:
 
 Dear Community,
 
 Apache CouchDB 1.6.0 has been released and is available for download.
 
 CouchDB is a database that completely embraces the web. Store your
 data with JSON documents. Access your documents with your web browser,
 via HTTP. Query, combine, and transform your documents with
 JavaScript. CouchDB works well with modern web and mobile apps. You
 can even serve web apps directly out of CouchDB. And you can
 distribute your data, or your apps, efficiently using CouchDB’s
 incremental replication. CouchDB supports master-master setups with
 automatic conflict detection.
 
 Grab your copy here:
 
http://couchdb.apache.org/
 
 Pre-built packages for Windows and OS X are available.
 
 CouchDB 1.6.0 is a feature release, and was originally published on 
 2014-06-10.
 
 These release notes are based on the changelog.
 
 Upgrade Notes
 
 The Proxy Authentication handler was renamed to
 proxy_authentication_handler to follow the *_authentication_handler
 form of all other handlers. The old proxy_authentification_handler
 name is marked as deprecated and will be removed in future releases.
 It’s strongly recommended to update the httpd/authentication_handlers
 option with the new value if you have this handler configured.
 
 What's New
 
 * COUCHDB-2200: support Erlang/OTP 17.0 #35e16032
 * Fauxton: many improvements in our experimental new user interface,
 including switching the code editor from CodeMirror to Ace as well as
 better support for various browsers.
 * Add the max_count option (UUIDs Configuration) to allow
 rate-limiting the amount of UUIDs that can be requested from the
 /_uuids handler in a single request (CVE 2014-2668).
 * COUCHDB-1986: increase socket buffer size to improve replication
 speed for large documents and attachments, and fix tests on BSD-like
 systems. #9a0e561b
 * COUCHDB-1953: improve performance of multipart/related requests. #ce3e89dc
 * COUCHDB-2221: verify that authentication-related configuration
 settings are well-formed. #dbe769c6
 * COUCHDB-1922: fix CORS exposed headers. #4f619833
 * Rename proxy_authentification_handler to
 proxy_authentication_handler. #c66ac4a8
 * COUCHDB-1795: ensure the startup script clears the pid file on
 termination. #818ef4f9
 * COUCHDB-1962: replication can now be performed without having write
 access to the source database (#1d5fe2aa), the replication checkpoint
 interval is now configurable (#0693f98e).
 * COUCHDB-2025: add support for SOCKS5 proxies for replication. #fcd76c9
 * COUCHDB-1930: redirect to the correct page after submitting a new
 document with a different ID than the one suggested by Futon.
 #4906b591
 * COUCHDB-1923: add support for attachments and att_encoding_info
 options (formerly only available on the documents API) to the view
 API. #ca41964b
 * COUCHDB-1647: for failed replications originating from a document in
 the _replicator database, store the failure reason in the document.
 #08cac68b
 * A number of improvements for the documentation.
 
 On behalf of the CouchDB PMC,
 
 Dirkjan
 

Jan Lehnardt
The Neighbourhoodie Software GmbH
Adalbertstr. 7-8, 10999 Berlin
neighbourhood.ie

Handelsregister HRB 157851 B Amtsgericht Charlottenburg
Geschäftsführung: Jan Lehnardt, Lena Reinhard



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: multipart/related attachments

[Jan Lehnardt]

What error message are you getting back from CouchDB?

On 14 May 2014, at 06:56 , Rian R. Maloney rian.malo...@yahoo.com wrote:

 I am trying to PUT a multipart document that contains several images using 
 PHP. I have managed to base64 encode the images and issue a PUT but I would 
 prefer to do it w/o base 64 encoding. The socket i/o is hanging as if it is 
 expecting a CRLF terminator - which I have tried many combinations. Here is 
 my log:
 
 Host: 127.0.0.1
 Authorization: Basic cmlhbjpwYXNzd29yZA==
 Content-Type: multipart/related;boundary=abc123
 
 --abc123
 content-type: application/json
 
 {body:This is a 
 body.,_attachments:{fgs:{content_type:text\/plain,follows:true,length:21},rgs:{content_type:text\/plain,follows:true,length:20
 
 --abc123
 
 this is 21 chars long
 --abc123
 
 this is 20 chars lon
 --abc123--
 
 
 The example in the doc has an extra comma in the json which I assume is an 
 error:
 PUT /test_suite_db/multipart HTTP/1.1 Content-Type: 
 multipart/related;boundary=abc123  --abc123 content-type: application/json  
 {body:This is a body., _attachments:{  foo.txt: {  follows:true,  
 content_type:text/plain,  length:21  },  bar.txt: {  follows:true,  
 content_type:text/plain,  length:20  }, 
- EXTRA COMMA  } }  --abc123  this is 21 chars long --abc123  
 this is 20 chars lon --abc123--
 Any ideas would be much appreciated.
 
 Rian



signature.asc
Description: Message signed with OpenPGP using GPGMail

CouchDB and the Heartbleed SSL/TLS Vulnerability

[Jan Lehnardt]

Dear CouchDB community,

You may or may not have heard about the Heartbleed SSL/TLS Vulnerability yet 
(http://heartbleed.com). Without much exaggeration, this is a big one.

What does this mean for CouchDB?

1. If you are using CouchDB with the built-in SSL support, you are at the whim 
of Erlang/OTP’s handling of SSL. Lucky for you, while they do use OpenSSL for 
the heavy lifting, they do the TLS/SSL handshake logic in Erlang 
(http://erlang.org/pipermail/erlang-questions/2014-April/078537.html). That 
means you are not affected by this issue.


2. If you are using CouchDB behind a third-party proxy server you are at the 
whim of the SSL library it uses. For the big three Apache, nginx and HAProxy 
it’s all OpenSSL. So if they are using OpenSSL 1.0.1-1.0.1f with heartbeat 
support (RFC6520) enabled (the default), you need to take action. As far as I 
can tell now:

0. Check if you are vulnerable[1]
1. Stop your service.
2. Upgrade to OpenSSL 1.0.1g or recompile OpenSSL without heartbeat support.
3. Request new cert from your SSL cert vendor.
4. Revoke your old cert.
5. Invalidate all existing sessions by changing the CouchDB 
couchdb_httpd_auth/secret configuration value to a new UUID.
6. Restart your service.
7. Invalidate all your user’s passwords and/or OAuth tokens.
8. Notify your users that any of their data and passwords are potentially 
compromised.

[1]: 
https://gist.githubusercontent.com/takeshixx/10107280/raw/8052d8479ad0c6150464748d639b0f5e877e8c37/hb-test.py

Stay safe! 3
Jan
--



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: CouchDB Weekly News, April 3

[Jan Lehnardt]

This is incredible, yet again. I can’t believe there is so much going on! :)

Best
Jan
-- 

On 03 Apr 2014, at 12:27 , Noah Slater nsla...@apache.org wrote:

 Hi folks,
 
 Here's the second edition of the CouchDB Weekly News:
 
 https://blogs.apache.org/couchdb/entry/couchdb_weekly_news_april_3
 
 Get it while it's hot!
 
 Want to help the project out? Do us a favour and spread the news:
 
 http://www.reddit.com/r/nosql/comments/223by3/couchdb_weekly_news_april_3/
 https://plus.google.com/u/2/109226482722655790973/posts/4Gk5peR2XCp
 https://plus.google.com/u/2/109226482722655790973/posts/XZDJazDGcMU
 https://www.facebook.com/permalink.php?story_fbid=524727654226130id=507603582605204stream_ref=10
 
 Thanks,
 
 -- 
 Noah Slater
 https://twitter.com/nslater



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Replication vs. Compaction

[Jan Lehnardt]

On 31 Jan 2014, at 20:08 , Jason Smith j...@apache.org wrote:

 Anyway I think the broader point is, compaction is for compacting databases
 (removing old document revisions), and replication is for making a copy of
 a database (or subset). If compaction is causing downtime then that is a
 different bug to talk about, but it should be totally transparent.
 
 Jens (incidentally it's nice to talk with you again): the compactor will
 notice that it has not caught up yet, and it will run again from the old
 end to the real end. Of course, there may be changes during that run too,
 so it will repeat. Usually each iteration has a much, much smaller window.
 In practice, you tend to see one not caught up message in the logs, and
 then it's done.



 However there is a pathological situation where you are
 updating faster than the compactor can run, and you will get an infinite
 loop (plus very heavy i/o and filesystem waste as the compactor is
 basically duplicating your .couch into a .couch.compact forever).

Just a little clarification on this point: CouchDB will try to catch up,
I think 10 times, before giving up and reporting the result in the logs.

Best
Jan
-- 



 
 
 On Sat, Feb 1, 2014 at 12:59 AM, Jens Alfke j...@couchbase.com wrote:
 
 
 On Jan 31, 2014, at 9:46 AM, Mark Hahn m...@hahnca.com wrote:
 
 It wouldn't matter if it did.  Within the same server linux
 short-circuits
 http to make it the same as unix sockets, i.e. very little overhead.
 
 I think you mean it short-circuits TCP :)
 There's extra work involved in HTTP generation  parsing no matter what
 transport you're sending it over. And then the replicator is doing a bunch
 of JSON and multipart generation/parsing.
 Whereas the compactor, I would imagine, is mostly just making raw
 read/write calls while walking the b-tree.
 
 Anyway; this makes me wonder what happens when changes are made to a
 database during compaction. The compaction processes working off of a
 snapshot of the database from the point that it started, so it's not going
 to copy over new changes. Does that mean they get lost, or does the
 compactor have extra smarts to run a second phase where it copies over all
 revs created since the snapshot?
 
 —Jens



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: CouchDB Talk at GeekToGeek Meetup Berlin

[Jan Lehnardt]

On 18 Feb 2014, at 16:19 , Andy Wenk a...@nms.de wrote:

 this will be great. I can't make it unfortunately. Have a lot of fun in
 good 'ol Berlin :)
 
 Cheers
 
 Andy
 
 P.S.: Video?

nope, sorry, the meetup doesn’t have funds for video.

Jan
--

 
 
 On 18 February 2014 16:03, I.R. thewh...@googlemail.com wrote:
 
 me too
 
 On 18 Feb 2014, at 15:42, Noah Slater nsla...@apache.org wrote:
 
 I'll be there. Looking forward to meeting some CouchDB folks! :)
 
 On 17 February 2014 11:49, Jan Lehnardt j...@apache.org wrote:
 Hey all,
 
 I'll be giving a talk about CouchDB and what makes it special at this
 week's GeekToGeek meetup in Berlin.
 
 You all likely already know all about it, but it'd be lovely to hang
 out with you there :)
 
 Plus, it's their NoSQL evening, so there is more database nerdery
 guaranteed!
 
 Date: Friday, February 21st
 Time: 19:00
 Location: http://co-up.de/
 Details: http://www.meetup.com/geek2geek/
 
 Best
 Jan
 --
 
 
 
 
 --
 Noah Slater
 https://twitter.com/nslater
 
 
 
 
 -- 
 Andy Wenk
 Hamburg - Germany
 RockIt!
 
 http://www.couchdb-buch.de
 http://www.pg-praxisbuch.de
 
 GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
 
 https://people.apache.org/keys/committer/andywenk.asc



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Replication vs. Compaction

[Jan Lehnardt]

On 18 Feb 2014, at 17:09 , Paul Davis paul.joseph.da...@gmail.com wrote:

 On Tue, Feb 18, 2014 at 9:30 AM, Adam Kocoloski kocol...@apache.org wrote:
 On Feb 18, 2014, at 11:38 AM, Jan Lehnardt j...@apache.org wrote:
 
 On 31 Jan 2014, at 20:08 , Jason Smith j...@apache.org wrote:
 
 However there is a pathological situation where you are
 updating faster than the compactor can run, and you will get an infinite
 loop (plus very heavy i/o and filesystem waste as the compactor is
 basically duplicating your .couch into a .couch.compact forever).
 
 Just a little clarification on this point: CouchDB will try to catch up,
 I think 10 times, before giving up and reporting the result in the logs.
 
 Best
 Jan
 --
 
 I'm not aware of any limit to the number of iterations executed by the 
 compactor.  Regards,
 
 Adam
 
 Confirmed after a quick skim that compaction doesn't stop retrying.

I stand corrected, but I could swear CouchDB used to do this in the old days.

Jan
-- 


signature.asc
Description: Message signed with OpenPGP using GPGMail

CouchDB Talk at GeekToGeek Meetup Berlin

[Jan Lehnardt]

Hey all,

I’ll be giving a talk about CouchDB and what makes it special at this week’s 
GeekToGeek meetup in Berlin.

You all likely already know all about it, but it’d be lovely to hang out with 
you there :)

Plus, it’s their NoSQL evening, so there is more database nerdery guaranteed!

Date: Friday, February 21st
Time: 19:00
Location: http://co-up.de/
Details: http://www.meetup.com/geek2geek/

Best
Jan
-- 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: A CouchDB/Cloudant Scale Authorization question

[Jan Lehnardt]

On 09 Jan 2014, at 17:25 , Alexander Gabriel a...@barbalex.ch wrote:

 have you looked at http://www.couchbase.com/mobile#sync-gateway ?
 You can sync with http://www.couchbase.com/download#cb-mobile, pouchdb and
 couchdb and it was developed to scale better than using a database per user

I’d say scaling database per user is easier because you have total HTTP-level 
freedom for routing requests for free, but that’s just me :)

Best
Jan
-- 






signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Disabling doc include

[Jan Lehnardt]

On 02 Jan 2014, at 20:49 , Jens Alfke j...@couchbase.com wrote:

 
 On Jan 2, 2014, at 4:20 AM, Robert Newson rnew...@apache.org wrote:
 
 It is relevant, the OP could use multiple databases to expose the subset of 
 documents to the appropriate subset of users.
 
 True. But FWIW, we found in 2012 that this setup is a hassle to configure 
 (i.e. provisioning user accounts on demand) and won’t scale to large numbers 
 of users (or subsets of users, if you partition that way.)

Out of curiosity, what scaling limit have you found? Is this documented 
somewhere?

Best
Jan
-- 


 
 Mentioning Couchbase is not relevant. :)
 
 The Sync Gateway is replication-compatible with CouchDB. I think it’s as 
 relevant to mention here as, say, PouchDB.
 
 —Jens



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [ANNOUNCE] Nick North elected as CouchDB committer

[Jan Lehnardt]

Welcome aboard, Nick! :)

Best
Jan
-- 

On 01 Jan 2014, at 20:24 , Dave Cottlehuber d...@jsonified.com wrote:

 Dear community,
 
 There's nothing like starting off the New Year with a New Committer!!
 
 I am pleased to announce that the CouchDB Project Management Committee
 has elected Nick North as a CouchDB committer.
 
Apache ID: nicknorth
 
IRC nick: NickN
 
Twitter: @_shastra
 
 By default, external contributions to the project follow the
 Review-Then-Commit model. Being a committer means you can follow the
 Commit-Then-Review model. In other words, Nick can now make changes at
 will.
 
 This election was made in recognition of Nick's existing contributions
 and commitment to the project.
 
 Please join me in extending a warm welcome to Nick!
 
 On behalf of the CouchDB PMC,
 
 Dave Cottlehuber



signature.asc
Description: Message signed with OpenPGP using GPGMail

Scaling db-per-user (Was: Re: Disabling doc include)

[Jan Lehnardt]

On 02 Jan 2014, at 21:08 , Jens Alfke j...@couchbase.com wrote:

 
 On Jan 2, 2014, at 11:56 AM, Jan Lehnardt j...@apache.org wrote:
 
 Out of curiosity, what scaling limit have you found? Is this documented 
 somewhere?
 
 By “we found” I should have said “we extrapolated”. We have customers that 
 will need hundreds of thousands of user accounts, and attaching that many 
 replications to a central master database wouldn’t be practical. Especially 
 since they’d be filtered replications — every time a document was added to 
 the master database, CouchDB would have to run n filter functions to decide 
 whether to push it to n different user databases. Another scaling factor is 
 that, if documents are accessible to many users, the storage space needed for 
 those documents will be greatly multiplied since many replicas of them will 
 exist.
 
 If someone only needs a few hundred user or user-subset databases, though, 
 this should be a feasible approach.

Thanks Jens. Sounds sensible to me.

We added /_db_updates in 1.4.0 that allows building the above with the 
difference that a replication only runs for active users, thus delaying most of 
the work until it is needed *and* avoiding having to run hundreds of thousands 
of replications at the same time. Managing all that however is still an 
exercise left to the user (I know of two implementations of this in node) and 
we should see if we can smart up the replicator accordingly.

Storage aside, do you thing that would worked for your scenario?


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Data usage and compaction status

[Jan Lehnardt]

On 27 Dec 2013, at 22:01 , Jens Rantil jens.ran...@gmail.com wrote:

 Hi,
 
 When I look at `/mydb`[1] I am given a lot of statistics. Among them is 
 `data_size`. However, when I look in the documentation[1] I can't find it. Is 
 it missing or has it been removed?

It has not been removed, it might not have been documented yet. You are welcome 
to send a patch :)

 What value does it show?

The amount of bytes your mydb.couch file uses on disk. E.g. the same value a 
`ls -l` would show.

Best
Jan
--

 I am running CouchDB 1.5.0.
 
 Two related questions:
 
 1. The `disk_size` measurement does not include views indexes size. Is there 
 any way for me to check the size of a view index (over HTTP, that is ;))?
 
 2. The `/mydb` shows when a compaction is going on of the database (but not 
 views, right?). Is the only way to check whether a view compaction is running 
 by looking in the `/_active_tasks` response?
 
 [1] 
 http://docs.couchdb.org/en/latest/api/database/common.html?highlight=disk_size#get--db
 
 Cheers,
 Jens



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Data usage and compaction status

[Jan Lehnardt]

On 27 Dec 2013, at 22:22 , Jens Rantil jens.ran...@gmail.com wrote:

 Gah, I should have seen that one. Thanks!
 
 I've submitted a pull request to add `data_size` to the documentation here:
 https://github.com/apache/couchdb/pull/123

Alex merged it already, thanks :)

Best
Jan
-- 

 
 /J
 
 
 On Fri, Dec 27, 2013 at 10:11 PM, Alexander Shorin kxe...@gmail.com wrote:
 
 On Sat, Dec 28, 2013 at 1:01 AM, Jens Rantil jens.ran...@gmail.com
 wrote:
 
 1. The `disk_size` measurement does not include views indexes size. Is
 there any way for me to check the size of a view index (over HTTP, that is
 ;))?
 
 2. The `/mydb` shows when a compaction is going on of the database (but
 not views, right?). Is the only way to check whether a view compaction is
 running by looking in the `/_active_tasks` response?
 
 For view indexes similar information is available at ddoc info resource:
 
 http://docs.couchdb.org/en/latest/api/ddoc/common.html#db-design-design-doc-info
 
 --
 ,,,^..^,,,
 



signature.asc
Description: Message signed with OpenPGP using GPGMail

The State of CouchDB 2013

[Jan Lehnardt]

Hi all,

I just published the transcript of my CouchDB Conf, Vancouver Keynote 
(https://blogs.apache.org/couchdb/entry/the_state_of_couchdb), and I’d like to 
share it with you as well.

My thanks to everyone who helped making this year the success it has been! 3

* * *

This is a rough transcript of the CouchDB Conf, Vancouver Keynote.


## Welcome

Good morning everyone. I thank you all for coming on this fine day in 
Vancouver. I’m very happy to be here. My name is Jan Lehnardt and I am the Vice 
President of Apache CouchDB at the Apache Software Foundation, but that’s just 
a fancy title that means I have to do a bunch of extra work behind the scenes. 
I’m also a core contributor to Apache CouchDB and I am the longest active 
committer to the project at this point.

I started helping out with CouchDB in 2006 and that feels like a lifetime ago. 
We’ve come a long way, we’ve shaped the database industry in a big way, we went 
though a phoenix from the ashes time and came out still inspiring future 
generations of developers to do great things.

So it is with great honour that I get to be here on stage before you to take a 
look at the state of CouchDB.


## Numbers

I’d like to start with some numbers:

- In 2013 we **added 15 committers** to the project, up to a total of 30. Thats 
2x the number of people regularly contributing to CouchDB!

- The year isn’t yet over, but these committers already created 3x the commits 
of 2012. And they have committed more than in any other year in CouchDB’s 
history.

- We have **shipped eight releases**: 1.0.4 1.1.2, 1.2.1, 1.2.2, 1.3.0, 1.3.1, 
1,4.0 and 1.5.0 just this year, that is up from one(!) last year.
  - thanks to our new release schedule we are getting more features to more 
people faster by focusing on small iterative changes forward.

- 20% more JIRA tickets and 50% more GitHub issues

We have made a lot of changes in 2012 to make 2013 a great year for CouchDB and 
it sure looks like we succeeded and that 2014 is only going to trump that.

I’d like to thank everyone on the team for their hard work.


## Currently

We’ve just shipped CouchDB 1.5.0 last week and it comes with a few exciting new 
things as previews, for you to try out and play with and report any issues with 
back to us. And that is on top of all the regular bug fixing and other 
improvements.


1. A completely new developed admin UI, nicknamed Fauxton, that is poised to 
replace the much-loved, but increasingly dated Futon. I’d like to personally 
thank the Fauxton team: Sue “Deathbear” Lockwood, Russell “Chewbranca” Branca, 
Garren Smith and many more volunteers for their work as well as the company 
Cloudant for sponsoring a good chunk of that work. Great job everyone! Fauxton 
is going to be replacing Futon in one of the next few releases and will give us 
the foundation for the next stage of CouchDB’s life.

2. Plugins. While it was always possible to write plugins for CouchDB, you kind 
of had to be an expert in CouchDB to get started. We believe that writing 
plugins is a great gateway drug to getting more people to hack on CouchDB 
proper, so we made it simpler to build plugins and to install plugins into a 
running instance of CouchDB. It is still very early days, we don’t even have a 
plugin registry yet, but we are surely excited about the prospects of 
installing GeoCouch with a single click of a button in Futon or Fauxton. We 
also included a template plugin that you can easily extend and make your own, 
along with a guide to get you started.

The plugins effort also supports a larger trend we are starting to follow with 
the CouchDB core codebase: decide on a well-defined core set of functionality 
and delegate more esoteric things to a rich plugin system That means we no 
longer have to decline the inclusion of useful code like we’ve done in the 
past, because it wasn’t applicable to the majority of CouchDB users. Now we can 
support fringe features and plugins that are only useful to a few of our users, 
but who really need them.

3. A Node.JS query server. CouchDB relies on JavaScript for a number of core 
features and we want to continue to do so. In order to keep up with the rapid 
improvements made to the JavaScript ecosystem we have tentative plans to switch 
from a Spidermonkey-driven query server to a V8-driven one. In addition, the 
Node.js project has a really good installation story, something that we had 
trouble with in the past, and includes a few utilities that make it very easy 
for us to switch the query server over.

All this however is not to blindly follow the latest trends, but to encourage 
the community to take on the query server and introduce much needed 
improvements. The current view server is a tricky mix of JS, Erlang and C and 
we are not seeing many people daring to jump into that. In a second step we 
expect these improvements to trickle down to the other query server 
implementations like Python or PHP and make things better for everyone

Re: How to add BrowserId plugin to OSX binary install of CouchDB

[Jan Lehnardt]

On 18 Dec 2013, at 23:08 , Sjoerd de Jong sjo...@weett.nl wrote:

 Dave,
 
 Thanks for the extensive description. It’s a little hacky indeed but I’ve 
 learned quite a bit from your approach. We’re getting closer!
 
 From a different perspective, what is the reason to not include Persona into 
 the regular build of CouchDB? Is it to prevent growth of the distribution?

There is some preliminary work on a plugin system for CouchDB that would allow 
anyone to publish and install plugins for CouchDB without the need for us to 
include and maintain that code as part of core CouchDB.

The Persona plugin can easily live in that infrastructure.

Best
Jan
-- 


 
 From my point of view, Persona is great way to have decent security in a 
 really simple setup. It enables couchapps to be fully secure, without the 
 need of installing any other services, which is a major feature imho. As 
 OAuth is already a bundled security plugin, I’d love too see Persona / 
 BrowserId to be bundled too!
 
 Cheers,
 Sjoerd
 
 On 18 Dec 2013, at 15:49, Dave Cottlehuber d...@jsonified.com wrote:
 
 On 18. Dezember 2013 at 11:09:56, Sjoerd de Jong (sjo...@weett.nl) wrote:
 
 Hello,
 
 For some time I am struggling with getting the BrowserId (Persona)  
 plugin running on my machine (OSX Mavericks).
 
 Previously I compiled https://github.com/iriscouch/browserid_couchdb  
 using `rake plugin=git://github.com/iriscouch/browserid_couchdb  
 origin/master”` on build-couchdb. For some reason this fails  
 for 1.5 for me, and instead of diving into the error I’d like to  
 just use the download for OSX from the couchdb.apache.org main  
 site.
 
 In Jason Smiths talk on Plugins 
 (http://www.youtube.com/watch?v=saNjXpE1h1Y)  
 he showed installing a plugin is as easy as copying a .beam file  
 into the CouchDB application. Is this also true for the downloadable  
 version from the main website? If so, is it possible to install  
 the BrowserId plugin this way?
 
 If not, what is the recommended way to get CouchDB running with  
 the BrowserId plugin installed?
 
 Thanks for the help!
 
 Sjoerd
 
 Hey Sjoerd,
 
 I’d love for Jan or Jason to chip in here, as it looks like we need
 some extra bits within the release, and I didn’t quite get this working
 either. Here’s what I tried, all very hacky:
 
 ```shell
 # download  unpack CouchDB 1.5.0 first
 # hack!
 cd ~/Applications/Apache\ CouchDB*.app/Contents/Resources/couchdbx-core/
 
 # set up path helpers
 export COUCH_PATH=`pwd -P`
 export PATH=$COUCH_PATH/lib/erlang/erts-5.10.2/bin:$COUCH_PATH/bin:$PATH
 
 # install erlang to update erlexec script with current location
 ./lib/erlang/Install -minimal `pwd -P`/lib/erlang
 # `erl +V` should return Erlang (SMP,ASYNC_THREADS) (BEAM) emulator version 
 5.10.2
 
 # we need erl_interface headers from a working R16B01 install as its missing
 cp -R /some/path/to/erl_interface-3.7.13 $COUCH_PATH/lib/erlang/lib/
 
 # get browserid_couchdb with my modified rebar.config
 cd $COUCH_PATH/lib/couchdb/erlang/lib/
 git clone git://github.com/jsonified/browserid_couchdb.git
 cd browserid_couchdb
 ./rebar compile
 # link in the browserid.ini so couchdb finds it
 ln -s 
 $COUCH_PATH/lib/couchdb/erlang/lib/browserid_couchdb/etc/couchdb/default.d/browserid.ini
  $COUCH_PATH/etc/couchdb/default.d/browserid.ini
 
 # have a go, I used the terminal so I can see what’s going on
 cd $COUCH_PATH
 ./bin/couchdb -i
 1 application:start(browserid_couchdb).
 
 # follow instructions in 
 https://github.com/iriscouch/browserid_couchdb#so-simple
 
 I pushed a new db and doc containing as attachment the .html snippet from
 Jason’s plugin, the logon worked fine but the account isn’t created. I’m
 missing something about how the extension should be activated too.
 
 A+
 dch@
 
 
 
 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Fauxton + Pouchdb

[Jan Lehnardt]

This is way cool! :D

Best
Jan
-- 

On 04 Nov 2013, at 15:07 , Garren Smith gar...@apache.org wrote:

 Hi All,
 
 We have have Fauxton working with Pouchdb on Node.js. Nick Thompson created a 
 Pouchdb express.js server. Fauxton was always designed to be a modular web 
 app which means we can include or exclude modules. In Pouchdb we removed all 
 the modules that were not needed and just left standard database 
 interactivity and replication. 
 
 I thought with would be worth mentioning. Check it out. 
 https://github.com/nick-thompson/express-pouchdb
 
 Cheers
 Garren
 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [ANNOUNCE] Apache CouchDB 1.4.0 released

[Jan Lehnardt]

Congrats and thank you Dirkjan for a splendid release! :)

Best
Jan
--

On 03.09.2013, at 19:44, Dirkjan Ochtman dirk...@ochtman.nl wrote:

 Dear community,
 
 Apache CouchDB 1.4.0 has been released and is available for download.
 
 CouchDB is a database that completely embraces the web. Store your
 data with JSON documents. Access your documents with your web browser,
 via HTTP. Query, combine, and transform your documents with
 JavaScript. CouchDB works well with modern web and mobile apps. You
 can even serve web apps directly out of CouchDB. And you can
 distribute your data, or your apps, efficiently using CouchDB’s
 incremental replication. CouchDB supports master-master setups with
 automatic conflict detection.
 
 Grab your copy here:
 
http://couchdb.apache.org/
 
 Pre-built packages for Windows and OS X are available.
 
 CouchDB 1.4.0 is a feature release, and was originally published on 
 2013-09-03.
 
 These release notes are based on the changelog.
 
 Upgrade Notes
 
 * We now support Erlang/OTP R16B and R16B01; the minimum required
 version is R14B.
 * User document role values must now be strings. Other types of values
 will be refused when saving the user document.
 
 Changes
 
 * COUCHDB-1684: Support for server-wide changes feed reporting on
 creation, updates and deletion of databases. #917d8988
 * COUCHDB-1139: it’s possible to apply list functions to _all_docs
 view. #54fd258e
 * Automatic loading of CouchDB plugins. #3fab6bb5
 * COUCHDB-1634: Reduce PBKDF2 work factor. #f726bc4d
 * Allow storing pre-hashed admin passwords via _config API. #c98ba561
 * COUCHDB-1772: Prevent invalid JSON output when using all_or_nothing
 _bulk_docs API. #dfd39d57
 * Add a configurable whitelist of user document properties. #8d7ab8b1
 * COUCHDB-1852: Support Last-Event-ID header in EventSource changes
 feeds. #dfd2199a
 * Much improved documentation, including an expanded description of
 validate_doc_update functions (commit:ef9ac469) and a description of
 how CouchDB handles JSON number values (#bbd93f77).
 * COUCHDB-1632: Ignore epilogues in multipart/related MIME
 attachments. #2b4ab67a
 * Split up replicator_db tests into multiple independent tests.
 
 On behalf of the CouchDB PMC,
 
 Dirkjan