[jira] [Updated] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
[ https://issues.apache.org/jira/browse/IGNITE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-8481: -- Fix Version/s: (was: 2.9) 2.10 > VisorValidateIndexesJob works very slowly in case of many partitions/keys for > each partition. > - > > Key: IGNITE-8481 > URL: https://issues.apache.org/jira/browse/IGNITE-8481 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.5 >Reporter: Alexey Scherbakov >Priority: Major > Fix For: 2.10 > > Attachments: ignite.zip, thrdump-server.log > > > I tried to validate indexes using newly introduced VisorValidateIndexesTask > from control.sh and found what on large data set it works very slowly. > Process was not finished for 12 hours from start. > Looking through a thread dump I've noticed following problems: > 1. ValidateIndexesClosure works not in optimal way by doing btree lookup for > each index for each entry of each partition. It should be faster to validate > by scanning index tree. > 2. Thread dump shows contention on acquiring segment read lock by worker > pool-XXX threads, but no obvious reason for holding write lock (no load on > grid) > 3. > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration > generates garbage on each page access. > Check attachment for log and thread dump. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
[ https://issues.apache.org/jira/browse/IGNITE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8481: Fix Version/s: (was: 2.8) 2.9 > VisorValidateIndexesJob works very slowly in case of many partitions/keys for > each partition. > - > > Key: IGNITE-8481 > URL: https://issues.apache.org/jira/browse/IGNITE-8481 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.5 >Reporter: Alexei Scherbakov >Priority: Major > Fix For: 2.9 > > Attachments: ignite.zip, thrdump-server.log > > > I tried to validate indexes using newly introduced VisorValidateIndexesTask > from control.sh and found what on large data set it works very slowly. > Process was not finished for 12 hours from start. > Looking through a thread dump I've noticed following problems: > 1. ValidateIndexesClosure works not in optimal way by doing btree lookup for > each index for each entry of each partition. It should be faster to validate > by scanning index tree. > 2. Thread dump shows contention on acquiring segment read lock by worker > pool-XXX threads, but no obvious reason for holding write lock (no load on > grid) > 3. > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration > generates garbage on each page access. > Check attachment for log and thread dump. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
[ https://issues.apache.org/jira/browse/IGNITE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexei Scherbakov updated IGNITE-8481: -- Fix Version/s: (was: 2.7) 2.8 > VisorValidateIndexesJob works very slowly in case of many partitions/keys for > each partition. > - > > Key: IGNITE-8481 > URL: https://issues.apache.org/jira/browse/IGNITE-8481 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.5 >Reporter: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > Attachments: ignite.zip, thrdump-server.log > > > I tried to validate indexes using newly introduced VisorValidateIndexesTask > from control.sh and found what on large data set it works very slowly. > Process was not finished for 12 hours from start. > Looking through a thread dump I've noticed following problems: > 1. ValidateIndexesClosure works not in optimal way by doing btree lookup for > each index for each entry of each partition. It should be faster to validate > by scanning index tree. > 2. Thread dump shows contention on acquiring segment read lock by worker > pool-XXX threads, but no obvious reason for holding write lock (no load on > grid) > 3. > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration > generates garbage on each page access. > Check attachment for log and thread dump. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
[ https://issues.apache.org/jira/browse/IGNITE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-8481: Fix Version/s: (was: 2.8) 2.7 > VisorValidateIndexesJob works very slowly in case of many partitions/keys for > each partition. > - > > Key: IGNITE-8481 > URL: https://issues.apache.org/jira/browse/IGNITE-8481 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.5 >Reporter: Alexei Scherbakov >Priority: Major > Fix For: 2.7 > > Attachments: ignite.zip, thrdump-server.log > > > I tried to validate indexes using newly introduced VisorValidateIndexesTask > from control.sh and found what on large data set it works very slowly. > Process was not finished for 12 hours from start. > Looking through a thread dump I've noticed following problems: > 1. ValidateIndexesClosure works not in optimal way by doing btree lookup for > each index for each entry of each partition. It should be faster to validate > by scanning index tree. > 2. Thread dump shows contention on acquiring segment read lock by worker > pool-XXX threads, but no obvious reason for holding write lock (no load on > grid) > 3. > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration > generates garbage on each page access. > Check attachment for log and thread dump. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
[ https://issues.apache.org/jira/browse/IGNITE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexei Scherbakov updated IGNITE-8481: -- Fix Version/s: (was: 2.7) 2.8 > VisorValidateIndexesJob works very slowly in case of many partitions/keys for > each partition. > - > > Key: IGNITE-8481 > URL: https://issues.apache.org/jira/browse/IGNITE-8481 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.5 >Reporter: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > Attachments: ignite.zip, thrdump-server.log > > > I tried to validate indexes using newly introduced VisorValidateIndexesTask > from control.sh and found what on large data set it works very slowly. > Process was not finished for 12 hours from start. > Looking through a thread dump I've noticed following problems: > 1. ValidateIndexesClosure works not in optimal way by doing btree lookup for > each index for each entry of each partition. It should be faster to validate > by scanning index tree. > 2. Thread dump shows contention on acquiring segment read lock by worker > pool-XXX threads, but no obvious reason for holding write lock (no load on > grid) > 3. > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration > generates garbage on each page access. > Check attachment for log and thread dump. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-8481) VisorValidateIndexesJob works very slowly in case of many partitions/keys for each partition.
[ https://issues.apache.org/jira/browse/IGNITE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Pavlov updated IGNITE-8481: --- Fix Version/s: (was: 2.6) 2.7 > VisorValidateIndexesJob works very slowly in case of many partitions/keys for > each partition. > - > > Key: IGNITE-8481 > URL: https://issues.apache.org/jira/browse/IGNITE-8481 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.5 >Reporter: Alexei Scherbakov >Priority: Major > Fix For: 2.7 > > Attachments: ignite.zip, thrdump-server.log > > > I tried to validate indexes using newly introduced VisorValidateIndexesTask > from control.sh and found what on large data set it works very slowly. > Process was not finished for 12 hours from start. > Looking through a thread dump I've noticed following problems: > 1. ValidateIndexesClosure works not in optimal way by doing btree lookup for > each index for each entry of each partition. It should be faster to validate > by scanning index tree. > 2. Thread dump shows contention on acquiring segment read lock by worker > pool-XXX threads, but no obvious reason for holding write lock (no load on > grid) > 3. > org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.Segment#partGeneration > generates garbage on each page access. > Check attachment for log and thread dump. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
