Thanks Shawn!
I will try to change the values of those parameters 2017-07-10 14:57 GMT+02:00 Shawn Heisey <apa...@elyograg.org>: > On 7/10/2017 2:57 AM, Antonio De Miguel wrote: > > I continue deeping inside this problem... high writing rates continues. > > > > Searching in logs i see this: > > > > 2017-07-10 08:46:18.888 INFO (commitScheduler-11-thread-1) [c:ads > s:shard2 > > r:core_node47 x:ads_shard2_replica3] o.a.s.u.LoggingInfoStream > > [DWPT][commitScheduler-11-thread-1]: flushed: segment=_mb7 > ramUsed=7.531 MB > > newFlushedSize=2.472 MB docs/MB=334.132 > > 2017-07-10 08:46:29.336 INFO (commitScheduler-11-thread-1) [c:ads > s:shard2 > > r:core_node47 x:ads_shard2_replica3] o.a.s.u.LoggingInfoStream > > [DWPT][commitScheduler-11-thread-1]: flushed: segment=_mba > ramUsed=8.079 MB > > newFlushedSize=1.784 MB docs/MB=244.978 > > > > > > A flush happens each 10 seconds (my autosoftcommit time is 10 secs and > > hardcommit 5 minutes). ¿is the expected behaviour? > > If you are indexing continuously, then the auto soft commit time of 10 > seconds means that this will be happening every ten seconds. > > > I thought soft commits does not write into disk... > > If you are using the correct DirectoryFactory type, a soft commit has > the *possibility* of not writing to disk, but the amount of memory > reserved is fairly small. > > Looking into the source code for NRTCachingDirectoryFactory, I see that > maxMergeSizeMB defaults to 4, and maxCachedMB defaults to 48. This is a > little bit different than what the javadoc states for > NRTCachingDirectory (5 and 60): > > http://lucene.apache.org/core/6_6_0/core/org/apache/lucene/ > store/NRTCachingDirectory.html > > The way I read this, assuming the amount of segment data created is > small, only the first few soft commits will be entirely handled in > memory. After that, older segments must be flushed to disk to make room > for new ones. > > If the indexing rate is high, there's not really much difference between > soft commits and hard commits. This also assumes that you have left the > directory at the default of NRTCachingDirectoryFactory. If this has > been changed, then there is no caching in RAM, and soft commit probably > behaves *exactly* the same as hard commit. > > Thanks, > Shawn > >
