[ https://issues.apache.org/jira/browse/SOLR-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115348#comment-16115348 ]
Amrit Sarkar edited comment on SOLR-11200 at 8/7/17 4:10 PM: ------------------------------------------------------------- Ah! I don't think this will serve our purpose in bulk indexing, logs :: {code} mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15, ioThrottle=false 2017-08-05 09:14:03.005 INFO (qtp1205044462-19) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-19]: updateMergeThreads ioThrottle=false targetMBPerSec=10240.0 MB/sec mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15, ioThrottle=false 2017-08-05 09:15:51.196 INFO (qtp1205044462-69) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:15:56.711 INFO (Lucene Merge Thread #0) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #0]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:10.752 INFO (qtp1205044462-17) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-17]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:18.229 INFO (Lucene Merge Thread #1) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #1]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:26.516 INFO (qtp1205044462-69) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:35.551 INFO (Lucene Merge Thread #2) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #2]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:38.580 INFO (qtp1205044462-18) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-18]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:49.397 INFO (Lucene Merge Thread #3) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #3]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:56.630 INFO (qtp1205044462-15) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-15]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec {code} See the {{targetMBPerSec}} is initialised to {{10gbps}}, but then it falls back to default {{20mbps}}, instead of maintaining at 10gbps. Maybe {{SolrIndexConfig#buildMergeSchedule}} is not the right place to do it. I will look more. was (Author: sarkaramr...@gmail.com): Ah! I don't this will serve our purpose in bulk indexing, logs :: {code} mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15, ioThrottle=false 2017-08-05 09:14:03.005 INFO (qtp1205044462-19) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-19]: updateMergeThreads ioThrottle=false targetMBPerSec=10240.0 MB/sec mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=5, maxMergeCount=15, ioThrottle=false 2017-08-05 09:15:51.196 INFO (qtp1205044462-69) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:15:56.711 INFO (Lucene Merge Thread #0) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #0]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:10.752 INFO (qtp1205044462-17) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-17]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:18.229 INFO (Lucene Merge Thread #1) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #1]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:26.516 INFO (qtp1205044462-69) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-69]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:35.551 INFO (Lucene Merge Thread #2) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #2]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:38.580 INFO (qtp1205044462-18) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-18]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:49.397 INFO (Lucene Merge Thread #3) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][Lucene Merge Thread #3]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec 2017-08-05 09:16:56.630 INFO (qtp1205044462-15) [c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.u.LoggingInfoStream [MergeScheduler][qtp1205044462-15]: updateMergeThreads ioThrottle=false targetMBPerSec=20.0 MB/sec {code} See the {{targetMBPerSec}} is initialised to {{10gbps}}, but then it falls back to default {{20mbps}}, instead of maintaining at 10gbps. Maybe {{SolrIndexConfig#buildMergeSchedule}} is not the right place to do it. I will look more. > provide a config to enable disable ConcurrentMergeSchedule.doAutoIOThrottle > --------------------------------------------------------------------------- > > Key: SOLR-11200 > URL: https://issues.apache.org/jira/browse/SOLR-11200 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Reporter: Nawab Zada Asad iqbal > Priority: Minor > Attachments: SOLR-11200.patch, SOLR-11200.patch > > > This config can be useful while bulk indexing. Lucene introduced it > https://issues.apache.org/jira/browse/LUCENE-6119 . -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org