Re: High CPU usage caused by clustering and/or replication?

[Joan Touzet]

Best I can suggest at this point is to send on your logfiles, 
and if you have any data that helps reproduce the problem you 
are experiencing that you can share, that'd be great, too. 

If you can provide this data, please file a new ticket at 
https://issues.apache.org/jira/secure/CreateIssue!default.jspa 
selecting CouchDB as the project. 

-Joan 

- Original Message -

> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 6:59:12 AM
> Subject: Re: High CPU usage caused by clustering and/or replication?

> Same thing is happening on a Linux cluster. :( It even causes the
> beam process to crash.
> I am using free tier instances on AWS, with 1 Core and 1GB RAM. Would
> that cause this issue?

> From: Joan Touzet <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 4:59 AM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?

> I would recommend that you try and reproduce the scenario you're
> running into with a Linux-based cluster, and if you find it there
> as well, file a bug with us so we can try and work your problem out.

> If the problem goes away, then yes, I recommend for now you consider
> a
> Linux cluster. ;)

> I also recommend you use a minimum of 3 nodes for a cluster of q=8
> (the
> default).

> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> > Sent: Wednesday, September 14, 2016 12:04:04 AM
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I actually stopped all replications and the high CPU occurs on the
> > 2nd node. I have only 2 nodes in the cluster.
> >
> > We really want to move to CouchDB 2.0, but we can't afford this
> > issue
> > I am encountering.
> >
> > Do you suggest we just use Linux-based cluster? :(
> >
> >
> >
> > 
> > From: Joan Touzet <woh...@apache.org>
> > Sent: Wednesday, September 14, 2016 3:47 AM
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > Joey,
> >
> > I'm sorry, I can't reproduce this problem. If you're able to
> > provide
> > sample data that causes this issue, I'd be pleased to test (on a
> > non-virtualized Windows machine) with that.
> >
> > I will note that I am only testing with a local single CouchDB
> > server,
> > not in a cluster. What if you try replicating to a non-clustered
> > Windows server? We have absolutely zero testing done on CouchDB
> > clustered on Windows, and in fact really should declare our support
> > for
> > that configuration as beta. I will state I have not set up a local
> > Windows-based cluster (on separate Windows machines; I have run a 3
> > node-on-1-server development cluster setup) so I can't guarantee
> > that
> > this isn't the problem.
> >
> > Please note this Known Issue in CouchDB 2.0:
> > https://issues.apache.org/jira/browse/COUCHDB-2980
> >
> > To quote our Known Issues document: "for now, we recommend always
> > using
> > the URL format for both source and target specifications."
> >
> > Finally, you should know it was my decision to continue Windows
> > support
> > for CouchDB, as no one else was able to step up. I did so primarily
> > because, in my experience, we have developers who develop against a
> > local CouchDB instance on their Windows laptops, then move to a
> > Linux-hosted cluster for server use. I didn't expect people would
> > be
> > trying to leverage the Windows port in a real server configuration.
> >
> > That's not to say we couldn't eventually support Windows in a
> > server
> > cluster configuration as "out of beta," but it'll take more
> > experience
> > and more repeatable bugs at the very least - and active CouchDB
> > Windows
> > developers with performance tuning experience at best.
> >
> > -Joan
> >
> > - Original Message -
> > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> > > Sent: Tuesday, September 13, 2016 10:53:18 PM
> > > Subject: Re: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > I have tried to stop all rep

Re: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

Same thing is happening on a Linux cluster. :( It even causes the beam process 
to crash.

I am using free tier instances on AWS, with 1 Core and 1GB RAM. Would that 
cause this issue?




From: Joan Touzet <woh...@apache.org>
Sent: Wednesday, September 14, 2016 4:59 AM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

I would recommend that you try and reproduce the scenario you're
running into with a Linux-based cluster, and if you find it there
as well, file a bug with us so we can try and work your problem out.

If the problem goes away, then yes, I recommend for now you consider a
Linux cluster. ;)

I also recommend you use a minimum of 3 nodes for a cluster of q=8 (the
default).

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 12:04:04 AM
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> I actually stopped all replications and the high CPU occurs on the
> 2nd node. I have only 2 nodes in the cluster.
>
> We really want to move to CouchDB 2.0, but we can't afford this issue
> I am encountering.
>
> Do you suggest we just use Linux-based cluster? :(
>
>
>
> 
> From: Joan Touzet <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 3:47 AM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> Joey,
>
> I'm sorry, I can't reproduce this problem. If you're able to provide
> sample data that causes this issue, I'd be pleased to test (on a
> non-virtualized Windows machine) with that.
>
> I will note that I am only testing with a local single CouchDB
> server,
> not in a cluster.  What if you try replicating to a non-clustered
> Windows server? We have absolutely zero testing done on CouchDB
> clustered on Windows, and in fact really should declare our support
> for
> that configuration as beta. I will state I have not set up a local
> Windows-based cluster (on separate Windows machines; I have run a 3
> node-on-1-server development cluster setup) so I can't guarantee that
> this isn't the problem.
>
> Please note this Known Issue in CouchDB 2.0:
>   https://issues.apache.org/jira/browse/COUCHDB-2980
>
> To quote our Known Issues document: "for now, we recommend always
> using
> the URL format for both source and target specifications."
>
> Finally, you should know it was my decision to continue Windows
> support
> for CouchDB, as no one else was able to step up. I did so primarily
> because, in my experience, we have developers who develop against a
> local CouchDB instance on their Windows laptops, then move to a
> Linux-hosted cluster for server use. I didn't expect people would be
> trying to leverage the Windows port in a real server configuration.
>
> That's not to say we couldn't eventually support Windows in a server
> cluster configuration as "out of beta," but it'll take more
> experience
> and more repeatable bugs at the very least - and active CouchDB
> Windows
> developers with performance tuning experience at best.
>
> -Joan
>
> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> > Sent: Tuesday, September 13, 2016 10:53:18 PM
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I have tried to stop all replications, then restarted CouchDB, but
> > still having almost 100% CPU with erl.exe
> >
> >
> > 
> > From: Joan Touzet <woh...@apache.org>
> > Sent: Friday, September 9, 2016 8:11 PM
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I don't know, a simple view with 100 documents should build quickly
> > unless your view code is unusual.
> >
> > I would recommend replicating the database to a UNIX-based 2.0
> > cluster
> > to see if the problem is Windows-specific.
> >
> > Sorry that I don't have any other ideas at the moment.
> >
> > -Joan
> >
> > - Original Message -
> > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > To: dev@couchdb.apache.org
> > > Sent: Friday, September 9, 2016 3:08:37 PM
> > > Subject: RE: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > Hi m

Re: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

I am not really very experienced with Linux. But what I am going to do first is 
use 3 nodes (Windows) with better specs. If the problem is still there, then I 
have no choice but to use Linux. :(




From: Joan Touzet <woh...@apache.org>
Sent: Wednesday, September 14, 2016 4:59 AM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

I would recommend that you try and reproduce the scenario you're
running into with a Linux-based cluster, and if you find it there
as well, file a bug with us so we can try and work your problem out.

If the problem goes away, then yes, I recommend for now you consider a
Linux cluster. ;)

I also recommend you use a minimum of 3 nodes for a cluster of q=8 (the
default).

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 12:04:04 AM
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> I actually stopped all replications and the high CPU occurs on the
> 2nd node. I have only 2 nodes in the cluster.
>
> We really want to move to CouchDB 2.0, but we can't afford this issue
> I am encountering.
>
> Do you suggest we just use Linux-based cluster? :(
>
>
>
> 
> From: Joan Touzet <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 3:47 AM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> Joey,
>
> I'm sorry, I can't reproduce this problem. If you're able to provide
> sample data that causes this issue, I'd be pleased to test (on a
> non-virtualized Windows machine) with that.
>
> I will note that I am only testing with a local single CouchDB
> server,
> not in a cluster.  What if you try replicating to a non-clustered
> Windows server? We have absolutely zero testing done on CouchDB
> clustered on Windows, and in fact really should declare our support
> for
> that configuration as beta. I will state I have not set up a local
> Windows-based cluster (on separate Windows machines; I have run a 3
> node-on-1-server development cluster setup) so I can't guarantee that
> this isn't the problem.
>
> Please note this Known Issue in CouchDB 2.0:
>   https://issues.apache.org/jira/browse/COUCHDB-2980
>
> To quote our Known Issues document: "for now, we recommend always
> using
> the URL format for both source and target specifications."
>
> Finally, you should know it was my decision to continue Windows
> support
> for CouchDB, as no one else was able to step up. I did so primarily
> because, in my experience, we have developers who develop against a
> local CouchDB instance on their Windows laptops, then move to a
> Linux-hosted cluster for server use. I didn't expect people would be
> trying to leverage the Windows port in a real server configuration.
>
> That's not to say we couldn't eventually support Windows in a server
> cluster configuration as "out of beta," but it'll take more
> experience
> and more repeatable bugs at the very least - and active CouchDB
> Windows
> developers with performance tuning experience at best.
>
> -Joan
>
> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> > Sent: Tuesday, September 13, 2016 10:53:18 PM
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I have tried to stop all replications, then restarted CouchDB, but
> > still having almost 100% CPU with erl.exe
> >
> >
> > 
> > From: Joan Touzet <woh...@apache.org>
> > Sent: Friday, September 9, 2016 8:11 PM
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I don't know, a simple view with 100 documents should build quickly
> > unless your view code is unusual.
> >
> > I would recommend replicating the database to a UNIX-based 2.0
> > cluster
> > to see if the problem is Windows-specific.
> >
> > Sorry that I don't have any other ideas at the moment.
> >
> > -Joan
> >
> > - Original Message -
> > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > To: dev@couchdb.apache.org
> > > Sent: Friday, September 9, 2016 3:08:37 PM
> > > Subject: RE: High CPU usage caused by clustering and/or
> > > replication?
>

Re: High CPU usage caused by clustering and/or replication?

[Joan Touzet]

I would recommend that you try and reproduce the scenario you're
running into with a Linux-based cluster, and if you find it there
as well, file a bug with us so we can try and work your problem out.

If the problem goes away, then yes, I recommend for now you consider a
Linux cluster. ;)

I also recommend you use a minimum of 3 nodes for a cluster of q=8 (the
default).

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 12:04:04 AM
> Subject: Re: High CPU usage caused by clustering and/or replication?
> 
> I actually stopped all replications and the high CPU occurs on the
> 2nd node. I have only 2 nodes in the cluster.
> 
> We really want to move to CouchDB 2.0, but we can't afford this issue
> I am encountering.
> 
> Do you suggest we just use Linux-based cluster? :(
> 
> 
> 
> 
> From: Joan Touzet <woh...@apache.org>
> Sent: Wednesday, September 14, 2016 3:47 AM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
> 
> Joey,
> 
> I'm sorry, I can't reproduce this problem. If you're able to provide
> sample data that causes this issue, I'd be pleased to test (on a
> non-virtualized Windows machine) with that.
> 
> I will note that I am only testing with a local single CouchDB
> server,
> not in a cluster.  What if you try replicating to a non-clustered
> Windows server? We have absolutely zero testing done on CouchDB
> clustered on Windows, and in fact really should declare our support
> for
> that configuration as beta. I will state I have not set up a local
> Windows-based cluster (on separate Windows machines; I have run a 3
> node-on-1-server development cluster setup) so I can't guarantee that
> this isn't the problem.
> 
> Please note this Known Issue in CouchDB 2.0:
>   https://issues.apache.org/jira/browse/COUCHDB-2980
> 
> To quote our Known Issues document: "for now, we recommend always
> using
> the URL format for both source and target specifications."
> 
> Finally, you should know it was my decision to continue Windows
> support
> for CouchDB, as no one else was able to step up. I did so primarily
> because, in my experience, we have developers who develop against a
> local CouchDB instance on their Windows laptops, then move to a
> Linux-hosted cluster for server use. I didn't expect people would be
> trying to leverage the Windows port in a real server configuration.
> 
> That's not to say we couldn't eventually support Windows in a server
> cluster configuration as "out of beta," but it'll take more
> experience
> and more repeatable bugs at the very least - and active CouchDB
> Windows
> developers with performance tuning experience at best.
> 
> -Joan
> 
> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> > Sent: Tuesday, September 13, 2016 10:53:18 PM
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I have tried to stop all replications, then restarted CouchDB, but
> > still having almost 100% CPU with erl.exe
> >
> >
> > 
> > From: Joan Touzet <woh...@apache.org>
> > Sent: Friday, September 9, 2016 8:11 PM
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > I don't know, a simple view with 100 documents should build quickly
> > unless your view code is unusual.
> >
> > I would recommend replicating the database to a UNIX-based 2.0
> > cluster
> > to see if the problem is Windows-specific.
> >
> > Sorry that I don't have any other ideas at the moment.
> >
> > -Joan
> >
> > - Original Message -
> > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > To: dev@couchdb.apache.org
> > > Sent: Friday, September 9, 2016 3:08:37 PM
> > > Subject: RE: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > Hi ma'am Joan,
> > >
> > > But if the cause is a view being rebuilt, why is it so high? I
> > > only
> > > have less than 100 documents on that database which I wanted to
> > > test
> > > initially.
> > >
> > > > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > > > From: woh...@apache.org
> > 

Re: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

I actually stopped all replications and the high CPU occurs on the 2nd node. I 
have only 2 nodes in the cluster.

We really want to move to CouchDB 2.0, but we can't afford this issue I am 
encountering.

Do you suggest we just use Linux-based cluster? :(




From: Joan Touzet <woh...@apache.org>
Sent: Wednesday, September 14, 2016 3:47 AM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

Joey,

I'm sorry, I can't reproduce this problem. If you're able to provide
sample data that causes this issue, I'd be pleased to test (on a
non-virtualized Windows machine) with that.

I will note that I am only testing with a local single CouchDB server,
not in a cluster.  What if you try replicating to a non-clustered
Windows server? We have absolutely zero testing done on CouchDB
clustered on Windows, and in fact really should declare our support for
that configuration as beta. I will state I have not set up a local
Windows-based cluster (on separate Windows machines; I have run a 3
node-on-1-server development cluster setup) so I can't guarantee that
this isn't the problem.

Please note this Known Issue in CouchDB 2.0:
  https://issues.apache.org/jira/browse/COUCHDB-2980

To quote our Known Issues document: "for now, we recommend always using
the URL format for both source and target specifications."

Finally, you should know it was my decision to continue Windows support
for CouchDB, as no one else was able to step up. I did so primarily
because, in my experience, we have developers who develop against a
local CouchDB instance on their Windows laptops, then move to a
Linux-hosted cluster for server use. I didn't expect people would be
trying to leverage the Windows port in a real server configuration.

That's not to say we couldn't eventually support Windows in a server
cluster configuration as "out of beta," but it'll take more experience
and more repeatable bugs at the very least - and active CouchDB Windows
developers with performance tuning experience at best.

-Joan

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> Sent: Tuesday, September 13, 2016 10:53:18 PM
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> I have tried to stop all replications, then restarted CouchDB, but
> still having almost 100% CPU with erl.exe
>
>
> 
> From: Joan Touzet <woh...@apache.org>
> Sent: Friday, September 9, 2016 8:11 PM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
>
> I don't know, a simple view with 100 documents should build quickly
> unless your view code is unusual.
>
> I would recommend replicating the database to a UNIX-based 2.0
> cluster
> to see if the problem is Windows-specific.
>
> Sorry that I don't have any other ideas at the moment.
>
> -Joan
>
> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org
> > Sent: Friday, September 9, 2016 3:08:37 PM
> > Subject: RE: High CPU usage caused by clustering and/or
> > replication?
> >
> > Hi ma'am Joan,
> >
> > But if the cause is a view being rebuilt, why is it so high? I only
> > have less than 100 documents on that database which I wanted to
> > test
> > initially.
> >
> > > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > > From: woh...@apache.org
> > > To: dev@couchdb.apache.org
> > > Subject: Re: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > Hi Joey,
> > >
> > > I'm going to oversimplify a bit here to make the explanation
> > > easier.
> > >
> > > If you check the process manager/top on each node, do you see
> > > couchjs
> > > responsible for that 100% CPU?
> > >
> > > Remember that views are not replicated, only documents. So if you
> > > have
> > > design documents that you replicated, those views may be being
> > > rebuilt.
> > >
> > > So-called "internal replication" will be placing segments of the
> > > database
> > > onto each node, and view building will be happening on each node
> > > as
> > > well.
> > > Since the view is also sharded, each shard of the database gets
> > > its
> > > own
> > > view shard; that has to be calculated independently on each node.
> > >
> > > Even if you're not building views, remember that a cluster places
> > > part
> > > of 

Re: High CPU usage caused by clustering and/or replication?

[Joan Touzet]

Joey,

I'm sorry, I can't reproduce this problem. If you're able to provide
sample data that causes this issue, I'd be pleased to test (on a
non-virtualized Windows machine) with that. 

I will note that I am only testing with a local single CouchDB server,
not in a cluster.  What if you try replicating to a non-clustered
Windows server? We have absolutely zero testing done on CouchDB
clustered on Windows, and in fact really should declare our support for
that configuration as beta. I will state I have not set up a local
Windows-based cluster (on separate Windows machines; I have run a 3
node-on-1-server development cluster setup) so I can't guarantee that
this isn't the problem. 

Please note this Known Issue in CouchDB 2.0:
  https://issues.apache.org/jira/browse/COUCHDB-2980 

To quote our Known Issues document: "for now, we recommend always using
the URL format for both source and target specifications."

Finally, you should know it was my decision to continue Windows support
for CouchDB, as no one else was able to step up. I did so primarily
because, in my experience, we have developers who develop against a
local CouchDB instance on their Windows laptops, then move to a
Linux-hosted cluster for server use. I didn't expect people would be
trying to leverage the Windows port in a real server configuration.

That's not to say we couldn't eventually support Windows in a server
cluster configuration as "out of beta," but it'll take more experience
and more repeatable bugs at the very least - and active CouchDB Windows
developers with performance tuning experience at best.

-Joan

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org, "Joan Touzet" <woh...@apache.org>
> Sent: Tuesday, September 13, 2016 10:53:18 PM
> Subject: Re: High CPU usage caused by clustering and/or replication?
> 
> I have tried to stop all replications, then restarted CouchDB, but
> still having almost 100% CPU with erl.exe
> 
> 
> 
> From: Joan Touzet <woh...@apache.org>
> Sent: Friday, September 9, 2016 8:11 PM
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
> 
> I don't know, a simple view with 100 documents should build quickly
> unless your view code is unusual.
> 
> I would recommend replicating the database to a UNIX-based 2.0
> cluster
> to see if the problem is Windows-specific.
> 
> Sorry that I don't have any other ideas at the moment.
> 
> -Joan
> 
> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org
> > Sent: Friday, September 9, 2016 3:08:37 PM
> > Subject: RE: High CPU usage caused by clustering and/or
> > replication?
> >
> > Hi ma'am Joan,
> >
> > But if the cause is a view being rebuilt, why is it so high? I only
> > have less than 100 documents on that database which I wanted to
> > test
> > initially.
> >
> > > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > > From: woh...@apache.org
> > > To: dev@couchdb.apache.org
> > > Subject: Re: High CPU usage caused by clustering and/or
> > > replication?
> > >
> > > Hi Joey,
> > >
> > > I'm going to oversimplify a bit here to make the explanation
> > > easier.
> > >
> > > If you check the process manager/top on each node, do you see
> > > couchjs
> > > responsible for that 100% CPU?
> > >
> > > Remember that views are not replicated, only documents. So if you
> > > have
> > > design documents that you replicated, those views may be being
> > > rebuilt.
> > >
> > > So-called "internal replication" will be placing segments of the
> > > database
> > > onto each node, and view building will be happening on each node
> > > as
> > > well.
> > > Since the view is also sharded, each shard of the database gets
> > > its
> > > own
> > > view shard; that has to be calculated independently on each node.
> > >
> > > Even if you're not building views, remember that a cluster places
> > > part
> > > of the database on each node, spreading the load out as equally
> > > as
> > > possible. In a standard 3-node cluster, each document write will
> > > be
> > > written redundantly to every node, meaning CPU load will be
> > > (mostly)
> > > equal across all 3 nodes as you write that document.
> > >
> > > Hope this helps,
> > > Joan
> > >
> > >
> > > - Original Message -
> > > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > > To: dev@couchdb.apache.org
> > > > Sent: Thursday, September 8, 2016 11:30:46 PM
> > > > Subject: High CPU usage caused by clustering and/or
> > > > replication?
> > > >
> > > > Good day,
> > > >
> > > > I have setup a cluster with two nodes, and I am replicating the
> > > > first
> > > > node to an existing database through a loadbalancer. But I am
> > > > getting a 100% CPU usage after a few minutes on both nodes. Is
> > > > this
> > > > an issue with RC4?
> > > >
> > > > Thanks.
> > > >
> >
> 

Re: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

I have tried to stop all replications, then restarted CouchDB, but still having 
almost 100% CPU with erl.exe



From: Joan Touzet <woh...@apache.org>
Sent: Friday, September 9, 2016 8:11 PM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

I don't know, a simple view with 100 documents should build quickly
unless your view code is unusual.

I would recommend replicating the database to a UNIX-based 2.0 cluster
to see if the problem is Windows-specific.

Sorry that I don't have any other ideas at the moment.

-Joan

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org
> Sent: Friday, September 9, 2016 3:08:37 PM
> Subject: RE: High CPU usage caused by clustering and/or replication?
>
> Hi ma'am Joan,
>
> But if the cause is a view being rebuilt, why is it so high? I only
> have less than 100 documents on that database which I wanted to test
> initially.
>
> > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > From: woh...@apache.org
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> >
> > Hi Joey,
> >
> > I'm going to oversimplify a bit here to make the explanation
> > easier.
> >
> > If you check the process manager/top on each node, do you see
> > couchjs
> > responsible for that 100% CPU?
> >
> > Remember that views are not replicated, only documents. So if you
> > have
> > design documents that you replicated, those views may be being
> > rebuilt.
> >
> > So-called "internal replication" will be placing segments of the
> > database
> > onto each node, and view building will be happening on each node as
> > well.
> > Since the view is also sharded, each shard of the database gets its
> > own
> > view shard; that has to be calculated independently on each node.
> >
> > Even if you're not building views, remember that a cluster places
> > part
> > of the database on each node, spreading the load out as equally as
> > possible. In a standard 3-node cluster, each document write will be
> > written redundantly to every node, meaning CPU load will be
> > (mostly)
> > equal across all 3 nodes as you write that document.
> >
> > Hope this helps,
> > Joan
> >
> >
> > - Original Message -
> > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > To: dev@couchdb.apache.org
> > > Sent: Thursday, September 8, 2016 11:30:46 PM
> > > Subject: High CPU usage caused by clustering and/or replication?
> > >
> > > Good day,
> > >
> > > I have setup a cluster with two nodes, and I am replicating the
> > > first
> > > node to an existing database through a loadbalancer. But I am
> > > getting a 100% CPU usage after a few minutes on both nodes. Is
> > > this
> > > an issue with RC4?
> > >
> > > Thanks.
> > >
>

Re: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

But it might be also because of me using a AWS free-tier server :(


Sent from Outlook<http://aka.ms/weboutlook>



From: Joey Samonte <csharpdevelo...@hotmail.com>
Sent: Wednesday, September 14, 2016 2:18 AM
To: dev@couchdb.apache.org; Joan Touzet
Subject: Re: High CPU usage caused by clustering and/or replication?

Seems that erl is the one having a 100% CPU usage, not couchjs, when running 
replication.


Sent from Outlook<http://aka.ms/weboutlook>



From: Joan Touzet <woh...@apache.org>
Sent: Friday, September 9, 2016 11:10 AM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

Hi Joey,

I'm going to oversimplify a bit here to make the explanation easier.

If you check the process manager/top on each node, do you see couchjs
responsible for that 100% CPU?

Remember that views are not replicated, only documents. So if you have
design documents that you replicated, those views may be being rebuilt.

So-called "internal replication" will be placing segments of the database
onto each node, and view building will be happening on each node as well.
Since the view is also sharded, each shard of the database gets its own
view shard; that has to be calculated independently on each node.

Even if you're not building views, remember that a cluster places part
of the database on each node, spreading the load out as equally as
possible. In a standard 3-node cluster, each document write will be
written redundantly to every node, meaning CPU load will be (mostly)
equal across all 3 nodes as you write that document.

Hope this helps,
Joan


- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org
> Sent: Thursday, September 8, 2016 11:30:46 PM
> Subject: High CPU usage caused by clustering and/or replication?
>
> Good day,
>
> I have setup a cluster with two nodes, and I am replicating the first
> node to an existing database through a loadbalancer. But I am
> getting a 100% CPU usage after a few minutes on both nodes. Is this
> an issue with RC4?
>
> Thanks.
>

Re: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

Seems that erl is the one having a 100% CPU usage, not couchjs, when running 
replication.


Sent from Outlook<http://aka.ms/weboutlook>



From: Joan Touzet <woh...@apache.org>
Sent: Friday, September 9, 2016 11:10 AM
To: dev@couchdb.apache.org
Subject: Re: High CPU usage caused by clustering and/or replication?

Hi Joey,

I'm going to oversimplify a bit here to make the explanation easier.

If you check the process manager/top on each node, do you see couchjs
responsible for that 100% CPU?

Remember that views are not replicated, only documents. So if you have
design documents that you replicated, those views may be being rebuilt.

So-called "internal replication" will be placing segments of the database
onto each node, and view building will be happening on each node as well.
Since the view is also sharded, each shard of the database gets its own
view shard; that has to be calculated independently on each node.

Even if you're not building views, remember that a cluster places part
of the database on each node, spreading the load out as equally as
possible. In a standard 3-node cluster, each document write will be
written redundantly to every node, meaning CPU load will be (mostly)
equal across all 3 nodes as you write that document.

Hope this helps,
Joan


- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org
> Sent: Thursday, September 8, 2016 11:30:46 PM
> Subject: High CPU usage caused by clustering and/or replication?
>
> Good day,
>
> I have setup a cluster with two nodes, and I am replicating the first
> node to an existing database through a loadbalancer. But I am
> getting a 100% CPU usage after a few minutes on both nodes. Is this
> an issue with RC4?
>
> Thanks.
>

Re: High CPU usage caused by clustering and/or replication?

[Joan Touzet]

I don't know, a simple view with 100 documents should build quickly
unless your view code is unusual.

I would recommend replicating the database to a UNIX-based 2.0 cluster
to see if the problem is Windows-specific.

Sorry that I don't have any other ideas at the moment.

-Joan

- Original Message -
> From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> To: dev@couchdb.apache.org
> Sent: Friday, September 9, 2016 3:08:37 PM
> Subject: RE: High CPU usage caused by clustering and/or replication?
> 
> Hi ma'am Joan,
>  
> But if the cause is a view being rebuilt, why is it so high? I only
> have less than 100 documents on that database which I wanted to test
> initially.
>  
> > Date: Fri, 9 Sep 2016 07:10:35 -0400
> > From: woh...@apache.org
> > To: dev@couchdb.apache.org
> > Subject: Re: High CPU usage caused by clustering and/or
> > replication?
> > 
> > Hi Joey,
> > 
> > I'm going to oversimplify a bit here to make the explanation
> > easier.
> > 
> > If you check the process manager/top on each node, do you see
> > couchjs
> > responsible for that 100% CPU?
> > 
> > Remember that views are not replicated, only documents. So if you
> > have
> > design documents that you replicated, those views may be being
> > rebuilt.
> > 
> > So-called "internal replication" will be placing segments of the
> > database
> > onto each node, and view building will be happening on each node as
> > well.
> > Since the view is also sharded, each shard of the database gets its
> > own
> > view shard; that has to be calculated independently on each node.
> > 
> > Even if you're not building views, remember that a cluster places
> > part
> > of the database on each node, spreading the load out as equally as
> > possible. In a standard 3-node cluster, each document write will be
> > written redundantly to every node, meaning CPU load will be
> > (mostly)
> > equal across all 3 nodes as you write that document.
> > 
> > Hope this helps,
> > Joan
> > 
> > 
> > - Original Message -
> > > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > > To: dev@couchdb.apache.org
> > > Sent: Thursday, September 8, 2016 11:30:46 PM
> > > Subject: High CPU usage caused by clustering and/or replication?
> > > 
> > > Good day,
> > >  
> > > I have setup a cluster with two nodes, and I am replicating the
> > > first
> > > node to an existing database through a loadbalancer. But I am
> > > getting a 100% CPU usage after a few minutes on both nodes. Is
> > > this
> > > an issue with RC4?
> > >  
> > > Thanks.
> > > 
> 

RE: High CPU usage caused by clustering and/or replication?

[Joey Samonte]

Hi ma'am Joan,
 
But if the cause is a view being rebuilt, why is it so high? I only have less 
than 100 documents on that database which I wanted to test initially.
 
> Date: Fri, 9 Sep 2016 07:10:35 -0400
> From: woh...@apache.org
> To: dev@couchdb.apache.org
> Subject: Re: High CPU usage caused by clustering and/or replication?
> 
> Hi Joey,
> 
> I'm going to oversimplify a bit here to make the explanation easier.
> 
> If you check the process manager/top on each node, do you see couchjs 
> responsible for that 100% CPU?
> 
> Remember that views are not replicated, only documents. So if you have
> design documents that you replicated, those views may be being rebuilt.
> 
> So-called "internal replication" will be placing segments of the database
> onto each node, and view building will be happening on each node as well.
> Since the view is also sharded, each shard of the database gets its own
> view shard; that has to be calculated independently on each node.
> 
> Even if you're not building views, remember that a cluster places part
> of the database on each node, spreading the load out as equally as
> possible. In a standard 3-node cluster, each document write will be
> written redundantly to every node, meaning CPU load will be (mostly)
> equal across all 3 nodes as you write that document.
> 
> Hope this helps,
> Joan
> 
> 
> - Original Message -
> > From: "Joey Samonte" <csharpdevelo...@hotmail.com>
> > To: dev@couchdb.apache.org
> > Sent: Thursday, September 8, 2016 11:30:46 PM
> > Subject: High CPU usage caused by clustering and/or replication?
> > 
> > Good day,
> >  
> > I have setup a cluster with two nodes, and I am replicating the first
> > node to an existing database through a loadbalancer. But I am
> > getting a 100% CPU usage after a few minutes on both nodes. Is this
> > an issue with RC4?
> >  
> > Thanks.
> > 
  

Re: High CPU usage caused by clustering and/or replication?

[Joan Touzet]

Hi Joey,

I'm going to oversimplify a bit here to make the explanation easier.

If you check the process manager/top on each node, do you see couchjs 
responsible for that 100% CPU?

Remember that views are not replicated, only documents. So if you have
design documents that you replicated, those views may be being rebuilt.

So-called "internal replication" will be placing segments of the database
onto each node, and view building will be happening on each node as well.
Since the view is also sharded, each shard of the database gets its own
view shard; that has to be calculated independently on each node.

Even if you're not building views, remember that a cluster places part
of the database on each node, spreading the load out as equally as
possible. In a standard 3-node cluster, each document write will be
written redundantly to every node, meaning CPU load will be (mostly)
equal across all 3 nodes as you write that document.

Hope this helps,
Joan


- Original Message -
> From: "Joey Samonte" 
> To: dev@couchdb.apache.org
> Sent: Thursday, September 8, 2016 11:30:46 PM
> Subject: High CPU usage caused by clustering and/or replication?
> 
> Good day,
>  
> I have setup a cluster with two nodes, and I am replicating the first
> node to an existing database through a loadbalancer. But I am
> getting a 100% CPU usage after a few minutes on both nodes. Is this
> an issue with RC4?
>  
> Thanks.
>