[
https://issues.apache.org/jira/browse/IGNITE-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046835#comment-17046835
]
Maxim Muzafarov commented on IGNITE-12684:
------------------------------------------
[[email protected]], [~ascherbakov], [~zstan]
Folks,
I've just suddenly found that your fix would not work as you expected. The root
cause is related to this line of code:
https://github.com/apache/ignite/pull/7432/files#diff-e7d7d29811367d4d13a05c912b0ffbdaR1963
With this configuration, all index rebuild procedures will be processed
single-threaded. This is my explanation of why. Look here [1]: If there are
more than corePoolSize but less than maximumPoolSize threads running, a new
thread will be created only if the queue is full. This means that if you define
a thread pool with the following code (corePoolSize==0), it will never start
the 2nd thread because the LinkedBlockingQueue is unbounded.
[1]
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/java/util/concurrent/ThreadPoolExecutor.java#l83
> Optimize count of threads for rebuild indexes
> ---------------------------------------------
>
> Key: IGNITE-12684
> URL: https://issues.apache.org/jira/browse/IGNITE-12684
> Project: Ignite
> Issue Type: Improvement
> Reporter: Kirill Tkalenko
> Assignee: Kirill Tkalenko
> Priority: Major
> Fix For: 2.9
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We were recently implemented the new property
> {{INDEX_REBUILDING_PARALLELISM}} (by issue IGNITE-11979).
> That fix do possible to rebuild indexes, after restart without index.bin or
> restore from snapshot to another topology, in parallel threads.
> But now the process of rebuild can to consume a lot of threads, which limited
> on INDEX_REBUILDING_PARALLELISM * AVAILABLE_PROC_CNT (count of thread in
> public pool).
> For example in case of INDEX_REBUILDING_PARALLELISM = 28 and 64 CPU it will
> be more then 1.5К threads.
> Look at the method, it will create dedicate thread on each invocation
> `org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl#processPartitionsAsync`
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
