Re: High disk write usage

2017-07-11 Thread Antonio De Miguel
Thanks Shawn! I will try to change the values of those parameters 2017-07-10 14:57 GMT+02:00 Shawn Heisey : > On 7/10/2017 2:57 AM, Antonio De Miguel wrote: > > I continue deeping inside this problem... high writing rates continues. > > > > Searching in logs i see this:

Re: High disk write usage

2017-07-10 Thread Antonio De Miguel
=1.784 MB docs/MB=244.978 A flush happens each 10 seconds (my autosoftcommit time is 10 secs and hardcommit 5 minutes). ¿is the expected behaviour? I thought soft commits does not write into disk... 2017-07-06 0:02 GMT+02:00 Antonio De Miguel : > Hi erik. > > What i want to said is tha

Help with updateHandler commit stats

2017-07-07 Thread Antonio De Miguel
Hi, I'm taking a look to UpdateHandler stats... and i see when autosoftcommit occurs (every 10 secs) both metrics, "commits" and "soft autocommits" increments by one. ¿is this normal? My config is: autoCommit: 180 secs autoSoftCommit: 10 secs Thanks!

Re: High disk write usage

2017-07-05 Thread Antonio De Miguel
> How much physical memory does the machine have and how much memory is > allocated to _all_ of the JVMs running on that machine? > > see: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory- > on-64bit.html > > Best, > Erick > > > On Wed, Jul 5, 2017 at 9:

Re: High disk write usage

2017-07-05 Thread Antonio De Miguel
isking > your data. That might be a quick experiment you could run though, > disable tlogs and see what that changes. Of course I'd do this on my > test system ;). > > But yeah, Solr will use a lot of I/O in the scenario you are outlining > I'm afraid. > > Best, >

Re: High disk write usage

2017-07-05 Thread Antonio De Miguel
thanks Markus! We already have SSD. About changing topology we probed yesterday with 10 shards, but system goes more inconsistent than with the current topology (5x10). I dont know why... too many traffic perhaps? About merge factor.. we set default configuration for some days... but when a

Re: High disk write usage

2017-07-05 Thread Antonio De Miguel
Thnaks a lot alessandro! Yes, we have very big physical dedicated machines, with a topology of 5 shards and10 replicas each shard. 1. transaction log files are increasing but not with this rate 2. we 've probed with values between 300 and 2000 MB... without any visible results 3. We don't us

High disk write usage

2017-07-05 Thread Antonio De Miguel
Hi, We are implementing a solrcloud cluster (6.6 version) with NRT requisites. We are indexing 600 docs/sec with 1500 docs/sec peaks, and we are serving about 1500qps. Our documents has 300 fields with some doc values, about 4kb and we have 3 million of documents. HardCommit is set to 15 minutes