Joan, I only have measurements for the same database (v1.6.1) on a Linux with a different HW: 4 x Intel(R) Xeon(R) CPU E5-2683 v3 @ 2.00GHz 10M docs, 10-view design doc rebuilt in 35hrs. Currently I'm running a single-view rebuild, which is 1hr in, seems it could be finished within 8 hours.
For completeness, the view rebuild I initially wrote about (v2.0 on Windows7) completed with the indexers for the respective shards taking from 32 to 42 hours. Our of curiosity, I will go on and test the SSD disk usage. Thanks, Daniel >From Joan Touzet <woh...@apache.org> >Subject Re: view/index rebuild in v2.0 >Date Wed, 30 Nov 2016 18:57:08 GMT > >Please note that absolutely *zero* work has been done on our Windows >port in terms of performance characterization and optimization. > >I'd be interested in hearing your results on the exact same hardware >running the Linux version of CouchDB 2.0 - my hunch is that it will >perform significantly better. > >-Joan > >----- Original Message ----- >> From: "DA" <daniel.adam...@gmail.com> >> To: user@couchdb.apache.org >> Sent: Wednesday, November 30, 2016 1:50:30 PM >> Subject: Re: view/index rebuild in v2.0 >> >> Hi, >> thanks Peter, but I was thinking of a measures suitable for >> production usage. >> >> Meanwhile, few observations: >> - the 8 respective indexer tasks are not progressing evenly, some are >> at 50%, some at 90% >> - I noticed the system usage fell by almost 50%, both CPU and disk >> activity. >> So the system is now even more under-utilized, quite strange. >> >> When it's completed, I will try the same excercise on an SSD disk. >> >> Cheers, Daniel >> >> >> On Wed, Nov 30, 2016 at 10:03 AM Peyton Vaughn <pvau...@6fusion.com> >> wrote: >> >> > Hi Daniel, >> > Do you have 'delayed_commits' set to true? It's now defaulted to >> > false in >> > couch 2.0, but it really causes a pretty hefty performance hit when >> > disabled. >> > >> > Peyton >> > >> > On Wed, Nov 30, 2016 at 5:57 AM, Daniel Adam >> > <daniel.adam...@gmail.com> >> > wrote: >> > >> > > Hi, >> > > I'm running an index/view re-generation with 10M documents, >> > > 10views, >> > having >> > > 1 node and 8 shards on it. >> > > It seems it will take more than 1 day. What are limiting factors >> > > for >> > that? >> > > Looking at perfromance monitoring tools (Task Manager and Process >> > > Explorer), it seems neither I/O or CPU is saturated: >> > > CPU (8 cores) each core taking <30% its capacity >> > > I/O reads ~ 8MB/s >> > > I/O writes ~ 1.5MB/s >> > > >> > > Windows 7 64bit >> > > 16GB memory >> > > Intel i7-4800 @ 2.7GHz >> > > HDD Seagate ST500LM0 >> > > >> > > Any explanation why the system limits are not being saturated? >> > > Seems to >> > me >> > > the bottleneck is I/O, but the disk is capable of 100MB/s >> > > throughput in >> > > sequential reads/writes. If the limiting factor is indeed I/O, I >> > > assume >> > > changing number of shards would not gain much, possibly even 1 >> > > shard >> > would >> > > provide a similar speed. >> > > >> > > Any way to improve the speed of index rebuild? >> > > >> > > Thanks, >> > > Daniel >> > > >> > >> > >
