[GitHub] abdasgupta commented on issue #68: Provide and publish ppc64le images

[GitBox]

abdasgupta commented on issue #68: Provide and publish ppc64le images
URL: https://github.com/apache/couchdb-docker/issues/68#issuecomment-391670042
 
 
   Hi @wohali  Currently I am pursuing this task. Can I get an update on the 
progress of this issue??


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] iilyak commented on issue #1338: Configuring node "flavors" in a cluster

[GitBox]

iilyak commented on issue #1338: Configuring node "flavors" in a cluster
URL: https://github.com/apache/couchdb/issues/1338#issuecomment-391691432
 
 
   I think that we need to specialize type of nodes for following reasons:
   
   1. we want to modernize CouchDB API and deprecate not scalable interfaces.
   2. we want to experiment with latest research on distributed databases
   3. we want to scale the parts independently
   4. it makes economic sense to run some types of nodes in containerized 
environment
   5. we might want to consider rich client architecture
   
   In order to achieve `#1` we need to introduce API versioning. Since it would 
take time to update all clients which use CouchDB. We would have to support 
multiple versions of API for a while. How we can do it? Option 1) restructure 
the codebase to have a dispatcher module and have separate versions of modules 
implementing http endpoints and fabric logic. 2) split the node to `api`  and 
`storage` and connect them using stable (non Erlang distribution based) 
protocol. I do believe that second approach is better because in such case the 
`api` node for old version of API could be implemented using `adapter` pattern. 
Which makes it possible to run two versions of `api` nodes at the same time.
   
   This architecture makes it possible to instantiate dedicated `api` nodes per 
user. In such case we can do:
- throttle requests on `api` nodes based on the current load of `storage` 
layer.
- place `api` nodes in VPC available to customer and tunnel traffic to 
storage backend
- implement application firewall on `api` node
- over time move authentication and authorization modules to `api` node so 
unauthorized requests do not reach `storage` layer at all 
   
   The same for `replicator`. We might want to support new versions of 
replication protocol which is based on modern fast set reconciliation 
algorithms. We need a way to run two versions of protocol side by side.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] ericdai opened a new issue #1339: if will support arm

[GitBox]

ericdai opened a new issue #1339: if will support arm
URL: https://github.com/apache/couchdb/issues/1339
 
 
   
   
   ## Expected Behavior
   
   
   
   ## Current Behavior
   
   
   
   ## Possible Solution
   
   
   
   ## Steps to Reproduce (for bugs)
   
   
   1.
   2.
   3.
   4.
   
   ## Context
   
   
   
   ## Your Environment
   
   * Version used:
   * Browser Name and version:
   * Operating System and version (desktop or mobile):
   * Link to your project:
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali closed issue #1339: if will support arm linux?

[GitBox]

wohali closed issue #1339: if will support arm linux?
URL: https://github.com/apache/couchdb/issues/1339
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali commented on issue #68: Provide and publish ppc64le images

[GitBox]

wohali commented on issue #68: Provide and publish ppc64le images
URL: https://github.com/apache/couchdb-docker/issues/68#issuecomment-391749321
 
 
   Still blocked by everything stated in my last update.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] abdasgupta commented on issue #68: Provide and publish ppc64le images

[GitBox]

abdasgupta commented on issue #68: Provide and publish ppc64le images
URL: https://github.com/apache/couchdb-docker/issues/68#issuecomment-391761181
 
 
   Hi @wohali , Can we work to solve the first problem of the list at first? 
Can I know what is to be done yet to close the first problem?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali commented on issue #68: Provide and publish ppc64le images

[GitBox]

wohali commented on issue #68: Provide and publish ppc64le images
URL: https://github.com/apache/couchdb-docker/issues/68#issuecomment-391770967
 
 
   Start with: If you have access to a ppc64le machine, run `make check` and 
confirm that all tests pass. If some fail, log them here:
   
   https://github.com/apache/couchdb/issues
   
   as ppc64le specific failures.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali commented on issue #1339: if will support arm linux?

[GitBox]

wohali commented on issue #1339: if will support arm linux?
URL: https://github.com/apache/couchdb/issues/1339#issuecomment-391749726
 
 
   Duplicate of #1103 and #892.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali commented on issue #1340: View creation failed on CentOS 7 - CouchDB 2.1.1

[GitBox]

wohali commented on issue #1340: View creation failed on CentOS 7 - CouchDB 
2.1.1
URL: https://github.com/apache/couchdb/issues/1340#issuecomment-391826998
 
 
   Duplicate of #1293.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali closed issue #1340: View creation failed on CentOS 7 - CouchDB 2.1.1

[GitBox]

wohali closed issue #1340: View creation failed on CentOS 7 - CouchDB 2.1.1
URL: https://github.com/apache/couchdb/issues/1340
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] plcollard opened a new issue #1340: View creation failed on CentOS 7 - CouchDB 2.1.1

[GitBox]

plcollard opened a new issue #1340: View creation failed on CentOS 7 - CouchDB 
2.1.1
URL: https://github.com/apache/couchdb/issues/1340
 
 
   CouchDB failed to create views on fresh and updated server with CentOS 7.
   
   ## Expected Behavior
   Verification of the setup should work .
   
   ## Current Behavior
   I've first seen the issue in a CentOS 7 server where the packages where 
updated.
   I reproduced the error in a fresh server.
   
   ## Steps to Reproduce (for bugs)
   1. Install CentOS 7 and update it.
   2. Setup repo as described in 
http://docs.couchdb.org/en/latest/install/unix.html#installation-using-the-apache-couchdb-convenience-binary-packages
   3. Install couchdb package and run the setup.
   4. Launch the verification in the web interface and the problem appears at 
the "Create view" steps. 
   
   ## Output of logs :
   
   [notice] 2018-05-24T18:05:41.461809Z couchdb@127.0.0.1 <0.1522.0> e3ce237828 
10.50.50.130:5984 10.50.50.70 undefined GET /verifytestdb 404 ok 3
   [notice] 2018-05-24T18:05:41.464446Z couchdb@127.0.0.1 <0.1521.0> 079947299c 
10.50.50.130:5984 10.50.50.70 undefined GET /verifytestdb_replicate 404 ok 5
   [notice] 2018-05-24T18:05:41.514311Z couchdb@127.0.0.1 <0.1521.0> f2f1464e89 
10.50.50.130:5984 10.50.50.70 admin PUT /verifytestdb 201 ok 41
   [notice] 2018-05-24T18:05:41.537452Z couchdb@127.0.0.1 <0.1522.0> 14d27f1fbf 
10.50.50.130:5984 10.50.50.70 admin PUT /verifytestdb/test_doc_1 201 ok 14
   [notice] 2018-05-24T18:05:41.554945Z couchdb@127.0.0.1 <0.1522.0> 3063dde463 
10.50.50.130:5984 10.50.50.70 admin PUT /verifytestdb/test_doc_1 201 ok 9
   [notice] 2018-05-24T18:05:41.571887Z couchdb@127.0.0.1 <0.1522.0> 7a159de3e1 
10.50.50.130:5984 10.50.50.70 admin DELETE 
/verifytestdb/test_doc_1?rev=2-d588d3e93ee155c5afffdf0247a2c5ef 200 ok 9
   [notice] 2018-05-24T18:05:41.602334Z couchdb@127.0.0.1 <0.1522.0> bf8be929af 
10.50.50.130:5984 10.50.50.70 admin PUT /verifytestdb/test_doc_10 201 ok 21
   [notice] 2018-05-24T18:05:41.604973Z couchdb@127.0.0.1 <0.1524.0> 1e62a52beb 
10.50.50.130:5984 10.50.50.70 admin PUT /verifytestdb/test_doc_20 201 ok 21
   [notice] 2018-05-24T18:05:41.612475Z couchdb@127.0.0.1 <0.2903.0> f116ca8328 
10.50.50.130:5984 10.50.50.70 admin PUT /verifytestdb/test_doc_30 201 ok 29
   [info] 2018-05-24T18:05:41.639492Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3043.0> died normal
   [error] 2018-05-24T18:05:41.639645Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3043.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.660287Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3046.0> died normal
   [error] 2018-05-24T18:05:41.660387Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3046.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.675095Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3049.0> died normal
   [error] 2018-05-24T18:05:41.675173Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3049.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.694356Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3052.0> died normal
   [error] 2018-05-24T18:05:41.694441Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3052.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.708698Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3055.0> died normal
   [error] 2018-05-24T18:05:41.708790Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3055.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.731796Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3058.0> died normal
   [error] 2018-05-24T18:05:41.731895Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3058.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.748254Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3077.0> died normal
   [error] 2018-05-24T18:05:41.748351Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3077.0> :: {os_process_error,{exit_status,1}}
   [info] 2018-05-24T18:05:41.762309Z couchdb@127.0.0.1 <0.212.0>  
couch_proc_manager <0.3080.0> died normal
   [error] 2018-05-24T18:05:41.762387Z couchdb@127.0.0.1 <0.3041.0>  OS 
Process Error <0.3080.0> :: {os_process_error,{exit_status,1}}
   ...
   
   
   
   ## Environment
   * Version used: 2.1.1 (installed from repo)
   * Operating System: CentOS 7 server
   * Kernel: 3.10.0-862.3.2.el7.x86_64
   
   
   If you need more informations let me know and if you have a solution I would 
be happy to test it.
   
   Thanks!
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:

[GitHub] pzduniak opened a new issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

pzduniak opened a new issue #1341: CouchDB degrades and starts timeouting on 
all index operations
URL: https://github.com/apache/couchdb/issues/1341
 
 
   ## Expected Behavior
   CouchDB should start up with ~100 records in its databases and ~14 views. 
2.1.1 in Docker. Doesn't work as both root and couchdb user.
   
   ## Current Behavior
   https://gist.github.com/pzduniak/4d4d9c148ee910bb053d9cfdd1b04216
   Indexing is never done.
   
   ## Steps to Reproduce (for bugs)
   We're casually running into it using our Dockerfile.
   [couchdb.zip](https://github.com/apache/couchdb/files/2036951/couchdb.zip)
   
   I honestly have absolutely no idea how to fix it. As you can see in the 
Dockerfile, timeout values are high.
   
   ## Context
   Our application waits until CouchDB is done starting up (using 
`/healthcheck.sh`) and then tries to create design documents in its database 
(`cubefiles`), eventually querying for a user by his username (where everything 
fails, because indexes can't be built).
   
   ## Your Environment
   Ubuntu 16.04 running Docker 18.03
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489870#comment-16489870
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit 5230cf01d847705171ab28d8219dbef1c2d6f2a6 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=5230cf0 ]

[01/N] Clustered Purge: Define new purge API

This is the first of a series of commits to implement the new clustered
purge API. Currently purge is a single-node only API that allows for
removing document revisions (and by extension entire documents)
completely from a database. However, given our anti-entropy measures
this API is extremely difficult to use in a cluster and requires
significant operator intervention.

Along with the operator intervention, this API is inherently unsafe with
regards to accidentally triggering the rebuild of secondary indices. As
such this patch set is aimed at creating a cluster aware API that is
both easier to use and less likely to cause application downtime while
secondary indices are rebuilt.

There are four major areas that will be covered by this patch set:

  1. Single node APIs and behavior changes
  2. Cluster aware APIs
  3. Anti-entropy updates
  4. Cluster HTTP implementation

This patch set is split up into a series of commits to aid in the review
by other commiters that will hopefully allow for a logical and intuitive
progression of implementation rather than landing as a single opaque
commit covering a huge swath of the code base.

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489877#comment-16489877
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit 3bc28506e56be93c4bc4f210968278a8d42e70e2 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=3bc2850 ]

[07/N] Clustered Purge: Internal replication

This commit implements the internal replication of purge requests. This
part of the anit-entropy process is important for ensuring that shard
copies continue to be eventually consistent even if updates happen to
shards independently due to a network split or other event that prevents
the successful purge request to a given copy.

The main addition to internal replication is that we both pull and push
purge requests between the source and target shards. The push direction
is obvious given that internal replication is in the push direction
already. Pull isn't quite as obvious but is required so that we don't
push an update that was already purged on the target.

Of note is that internal replication also has to maintain _local doc
checkpoints to prevent compaction removing old purge requests or else
shard copies could end up missing purge requests which would prevent the
shard copies from ever reaching a consistent state.

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489878#comment-16489878
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit 3ef43f885e510ae13c28fe3e9e9d9f1ca6de38cf in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=3ef43f8 ]

[08/N] Clustered Purge: Fabric API

This commit implements the clustered API for performing purge requests.
This change should be a fairly straightforward change for anyone already
familiar with the general implementation of a fabric coordinator given
that the purge API is fairly simple.

TODO: This includes pre-emptive changes for read-repair that need to go
into the next commit (and then probably swap this commit to before we
implement anti-entropy updates)

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] wohali commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

wohali commented on issue #1341: CouchDB degrades and starts timeouting on all 
index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-391861526
 
 
   How are you launching your container? What is your host's CPU/RAM/disk setup?
   
   I don't have time to look at your database, but assuming they're 
straightforward views, you probably don't have enough CPUs allocated. See #1301 
for instance.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali closed issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

wohali closed issue #1341: CouchDB degrades and starts timeouting on all index 
operations
URL: https://github.com/apache/couchdb/issues/1341
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489880#comment-16489880
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit d4afc1440660ffb7d994432787bc0ce333afb2fd in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=d4afc14 ]

[10/N] Clustered Purge: Clustered HTTP API

The HTTP API for clustered purge is fairly straightforward. It is
designed to match the general shape of the single node API. The only
major caveat here is that the purge sequence is now hardcoded as null
since the purge sequence would now otherwise be an opaque blob similar
to the update_seq blobs.

Its important to note that there is as yet no API invented for
traversing the history of purge requests in any shape or form as that
would mostly invalidate the entire purpose of using purge to remove any
trace of a document from a database at the HTTP level. Although there
will still be traces in individual shard files until all database
components have processed the purge and compaction has run (while
allowing for up to purge_infos_limit requests to remain available in
perpetuity).

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489871#comment-16489871
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit f822cdd3e087e41c93c577f1c1fa217fd0dbe7e6 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=f822cdd ]

[02/N] Clustered Purge: Update single node APIs

This patch updates the single node API implementations for use with the
new clustered purge API. At the single node level the major change is to
store a history of purge requests that can then be consumed by various
other parts of the database system.

The simpler of the major areas to use this new functionality will be any
secondary indices. Rather than checking that only a single purge request
has occurred each secondary index will store a _local document
referencing its oldest processed purge request. During index updates
each secondary index implementation will process any new purge requests
and update its local doc checkpoint. In this way secondary indexes will
no longer be sensitive to reset when multiple purge requests are issued
against the database.

The two other major areas that will make use of the newly stored purge
request history are both of the anit-entropy mechanisms: read-repair and
internal replication.

Read-repair will use the purge request history to know when a node
should discard updates that have come from a node that has not yet
processed a purge request during internal replication. Otherwise
read-repair would effectively undo any purge replication that happened
"recently".

Internal replication will use the purge request history to be able to
mend any differences between shards. For instance, if a shard is down
when a purge request is issue against a cluster this process will pull
the purge request and apply it during internal replication. And
similarly any local purge requests will be applied on the target before
normal internal replication.

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489872#comment-16489872
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit f72396a8e4bf8496f1ec8206fa780a9a379a41a3 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=f72396a ]

[03/N] Clustered Purge: Update couch_bt_engine

This commit updates the couch_bt_engine storage engine implementation to
satisfy the newly defined single-node purge APIs. This is accomplished
by storing two new database btrees.

The purge_seq_tree orders purge requests by their purge_seq. This tree
is used to satisfy the fold_purge_infos API for database components to
enumerate the list of purge requests in a defined order.

The second index is the purge_tree which orders purge requests by their
UUID to make for an efficient lookup when filtering replicated purge
requests.

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489875#comment-16489875
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit b28e5749aba28d1f97129d9bc8698532750c59ec in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=b28e574 ]

[06/N] Clustered Purge: Update mrview indexes

This commit updates the mrview secondary index to properly process the
new history of purge requests as well as to store the _local purge
checkpoint doc.

The importance of the _local checkpoint doc is to ensure that compaction
of a database does not remove any purge requests that have not yet been
processed by this secondary index.

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489873#comment-16489873
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit b1a68831610487c212ca730a86209753af625e57 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=b1a6883 ]

[04/N] Clustered Purge: Update eunit tests

This commit updates all of the various existing eunit tests to work with
the new single node purge APIs.

TODO: Split this into two commits: new tests and updated tests

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489874#comment-16489874
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit 1ab1665c0a572993a2c432aab3f29a1ed353fb26 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=1ab1665 ]

[05/N] Clustered Purge: EPI Hook for compaction

Blah. This probably needs to go into a pr3.5 as its own change. Also it
shouldn't be implemented in couch_bt_engine_compactor...

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (COUCHDB-3326) Implement clustered purge API: _purge

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/COUCHDB-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489879#comment-16489879
 ] 

ASF subversion and git services commented on COUCHDB-3326:
--

Commit d1400d775d713a0a5e7056c15f7246dd7289f654 in couchdb's branch 
refs/heads/COUCHDB-3326-clustered-purge-pr4-implementation from 
[~paul.joseph.davis]
[ https://gitbox.apache.org/repos/asf?p=couchdb.git;h=d1400d7 ]

[09/N] Clustered Purge: Update read-repair

Read-repair needs to know which nodes have requested an update to a
local doc so that it can determine if the update is applied. The basic
idea here is that we may have gotten an update from a remote node that
has yet to apply a purge request. If the local node were to apply this
update it would effectively undo a succesful purge request.

COUCHDB-3326

Co-authored-by: Mayya Sharipova 
Co-authored-by: jiangphcn 


> Implement clustered purge API: _purge
> -
>
> Key: COUCHDB-3326
> URL: https://issues.apache.org/jira/browse/COUCHDB-3326
> Project: CouchDB
>  Issue Type: New Feature
>  Components: Database Core, Documentation, HTTP Interface
>Reporter: Mayya Sharipova
>Priority: Major
>
> This implements the clustered purge API:
> {code:}
> curl -H 'Content-Type: application/json' -X POST 
> "http://adm:pass@127.0.0.1:5984/test1/_purge; -d 
> '{"d1":["3-410e46c04b51b4c3304ed232790a49da", 
> "3-420e46c04b51b4c3304ed232790a35db"],"d2":["2-a39d6d63f29a956ae39930f84dd71ec3"],
>  "d3":["1-bdca7a3ac9503bf6e46d7d7a782e8f03"]}'
> {code}
> Response: status_code 201 or 202
> {code:javascript}
> {
>   "purged": [
> {
>   "ok": true, //Quorum was reached, at least W nodes 
> successfully purged doc
>   "id": "d1",
>   "revs": [
> "3-410e46c04b51b4c3304ed232790a49da",
>"3-420e46c04b51b4c3304ed232790a35db"
>   ]
> },
> {
>   "accepted": true,  //Quorum was NOT reached, but request was 
> accepted
>   "id": "d2",   
>   "revs": [
> "2-a39d6d63f29a956ae39930f84dd71ec3"
>   ]
> },
> {
>   "ok": true,   
>   "id": "d3",
>   "revs": []//(DocId or Revs missing) OR (Revs are not leaf 
> revisions) 
> }  ],
>   "purge_seq": 
> "6-g1BMeJzLYWBgYMpgTmHgz8tPSTV2MDQy1zMAQsMckEQiQ5L8sxKZ4UoMcSrJAgC9PRRl"
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on 
all index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-39103
 
 
   Let me rephrase: the database doesn't start at all. The view functionality 
is dead. This should not happen. This is some obscure bug with Erlang failing 
to startup and I have not modified the scripts in the container enough to 
warrant any suspicion that it could happen in my code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on 
all index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-391889983
 
 
   Alright, what the hell, if CouchDB can't even start on a single core (it 
worked just fine on 1.x), then why is this behavior accepted. Is there 
absolutely no way to increase the timeouts on the initial handshake or whatever?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] wohali commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

wohali commented on issue #1341: CouchDB degrades and starts timeouting on all 
index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-391900971
 
 
   @pzduniak I'll remind you we have a [Code of 
Conduct](https://couchdb.apache.org/conduct.html) at this project. Please 
choose your words a bit more carefully. You're also getting all of this support 
for free. We are a volunteer-run project here.
   
   /cc @janl 
   
   "The database doesn't start at all" vs. "the view functionality is dead" are 
two different complaints. If you are able to `GET /` and get a response back, 
CouchDB is running.
   
   CouchDB runs inside of Erlang. The JavaScript view functionality is provided 
by 1 or more additional, forked, external `couchjs` processes. When CouchDB is 
busy, Erlang will gladly eat an entire core, meaning you may need to provide an 
additional core for `couchjs`.
   
   Further, in 2.x, the default is to split each database into 8 shards on 
disk. CouchDB 2.x forks a separate `couchjs` process for each of these shards, 
meaning your Docker container is running 8 additional, separate processes _per 
design document_ to build those views. More databases and more design documents 
mean multiplicatively more external `couchjs` processes.
   
   There are ways to alter [how many processes can run 
simultaneously](http://docs.couchdb.org/en/latest/config/query-servers.html#query_server_config/limit),
 and ways to [adjust the number of shards for each newly created 
database](http://docs.couchdb.org/en/latest/api/database/common.html?highlight=shards#put--db)
 as well as a [global default for newly created 
databases](http://docs.couchdb.org/en/latest/config/cluster.html#cluster-options).
 You may also wish to pipeline your view builds at startup so that not all of 
the views are attempted to be built at once (by querying them one at a time). 
Finally, if you simply want to wait longer, you can increase the [os process 
timeout](http://docs.couchdb.org/en/latest/config/couchdb.html?highlight=os_process_timeout#couchdb/os_process_timeout),
 and have your startup script try multiple times to `GET` the results of a view 
before it gives up.
   
   If you're looking to reduce CPU requirements, look into whether or not you 
can replace your views with declarative Mango secondary indexes. With the 
addition of [Mango-powered secondary 
indexes](http://docs.couchdb.org/en/latest/api/database/find.html) to CouchDB, 
JS views are becoming less necessary. As Mango runs inside of the Erlang 
process, it shares CPU with the main process more reasonably.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on 
all index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-391888317
 
 
   >How are you launching your container?
   
   Default Docker image params + my small script, basically the couchdb-docker 
repo.
   
   >What is your host's CPU/RAM/disk setup?
   Google Cloud, 1 vCPU, 3.75GB of RAM, their standard 300GB disk
   
   >you probably don't have enough CPUs allocated.
   Are you seriously suggesting that it should take more than 4 minutes to 
generate views for 15 rows?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on 
all index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-391888317
 
 
   >How are you launching your container?
   
   Default Docker image params + my small script, basically the couchdb-docker 
repo.
   
   >What is your host's CPU/RAM/disk setup?
   
   Google Cloud, 1 vCPU, 3.75GB of RAM, their standard 300GB disk
   
   >you probably don't have enough CPUs allocated.
   
   Are you seriously suggesting that it should take more than 4 minutes to 
generate views for 15 rows?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on all index operations

[GitBox]

pzduniak commented on issue #1341: CouchDB degrades and starts timeouting on 
all index operations
URL: https://github.com/apache/couchdb/issues/1341#issuecomment-391945581
 
 
   Thank you, limiting the process count seems to work. But:
   
   1) What's up with the sudden CoC mention? Are we supposed to be G-rated? Or 
did you think that some message was a personal attack? I just don't get it. I 
expressed my shock at how CouchDB has diverted from its roots. The same project 
that my colleague used on Raspberry Pis on farms in Africa suddenly started 
choking on relatively powerful hardware when it has like 10 views on barely any 
data, who would've thought? Maybe if 2.x breaks so easily, a dynamic default 
should be used for the process limit, probably one scaling with the CPU count? 
For example, Go doesn't spawn "up to 100 processes" when it starts up a hello 
world app.
   
   2) There were no docs to assist me. A guy on IRC had to tell me about the 
existence of something called "fabric" (that I still don't know what it is, I 
only know that it has a setting that I had to increase), which is nearly 
unmentioned in the docs (I only found some references in the changelog). 
Additionally, I feel like there should be a huge disclaimer that says "the 
software might break if you run it on perfectly fine 2 server cores, please 
change the following settings to prevent it" somewhere in the docs.
   
   3) Additionally, it feels like my issue was discarded nearly immediately 
without anyone looking into the actual configuration. Out of 3 interlinked 
issues, 1 was someone starving the database on purpose (50MB of RAM, come on), 
1 person with a non-insignificant amount of data and 1 person dealing with 
different behaviour (degradation vs startup issue). Most of the timeouts that 
you've suggested were already in place.
   
   I still believe that what I reported _is_ a valid issue. CouchDB somehow 
manages to break its startup process by spawning tens of compute-hungry 
processes, even if only 1-2 cores are available. Is 2.x only supposed to be run 
on beefy servers now? A product that I'm working on (where I was forced to use 
CouchDB to in fact limit resource usage) started out with 1.x running on mobile 
servers (think notebook CPUs in small boxes) that are used to deliver 
educational content to students in developing countries. It would be a shame if 
I had to rewrite the database interaction layer just because it's impossible to 
both use modern features of CouchDB and have any reliability on slower hardware.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services