[
https://issues.apache.org/jira/browse/IGNITE-13270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161780#comment-17161780
]
Tanmay Ambre edited comment on IGNITE-13270 at 7/21/20, 7:07 AM:
-----------------------------------------------------------------
Have taken multiple snapshots [for the same thread]
----
1
----------------------
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read
line: 5896
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown
line: 1365
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown
line: 1374
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doFind
line: 1332
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findOne
line: 1300
org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.find
line: 360
org.h2.index.BaseIndex.find line: 130
org.h2.index.IndexCursor.find line: 176
org.h2.table.TableFilter.next line: 471
org.h2.table.TableFilter.next line: 541
org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow line: 1452
org.h2.result.LazyResult.hasNext line: 79
org.h2.result.LazyResult.next line: 59
org.h2.command.dml.Select.queryFlat line: 527
org.h2.command.dml.Select.queryWithoutCache line: 633
org.h2.command.dml.Query.queryWithoutCacheLazyCheck line: 114
org.h2.command.dml.Query.query line: 352
org.h2.command.dml.Query.query line: 333
org.h2.command.CommandContainer.query line: 114
org.h2.command.Command.executeQuery line: 202
org.h2.jdbc.JdbcPreparedStatement.executeQuery line: 114
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery
line: 824
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer
line: 912
org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest0
line: 417
org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest
line: 242
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.onMessage
line: 2138
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.lambda$start$17
line: 2095
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$$Lambda$482/222427158.onMessage
line: not available
org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage
line: 3379
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener
line: 1843
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0
line: 1468
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200
line: 229
org.apache.ignite.internal.managers.communication.GridIoManager$9.run line:
1365
java.util.concurrent.ThreadPoolExecutor.runWorker line: 1149
java.util.concurrent.ThreadPoolExecutor$Worker.run line: 624
java.lang.Thread.run line: 748
was (Author: tambre):
Have taken multiple snapshots
----
1
----------------------
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read
line: 5896
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown
line: 1365
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findDown
line: 1374
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doFind
line: 1332
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findOne
line: 1300
org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.find
line: 360
org.h2.index.BaseIndex.find line: 130
org.h2.index.IndexCursor.find line: 176
org.h2.table.TableFilter.next line: 471
org.h2.table.TableFilter.next line: 541
org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow line: 1452
org.h2.result.LazyResult.hasNext line: 79
org.h2.result.LazyResult.next line: 59
org.h2.command.dml.Select.queryFlat line: 527
org.h2.command.dml.Select.queryWithoutCache line: 633
org.h2.command.dml.Query.queryWithoutCacheLazyCheck line: 114
org.h2.command.dml.Query.query line: 352
org.h2.command.dml.Query.query line: 333
org.h2.command.CommandContainer.query line: 114
org.h2.command.Command.executeQuery line: 202
org.h2.jdbc.JdbcPreparedStatement.executeQuery line: 114
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery
line: 824
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer
line: 912
org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest0
line: 417
org.apache.ignite.internal.processors.query.h2.twostep.GridMapQueryExecutor.onQueryRequest
line: 242
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.onMessage
line: 2138
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.lambda$start$17
line: 2095
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$$Lambda$482/222427158.onMessage
line: not available
org.apache.ignite.internal.managers.communication.GridIoManager$ArrayListener.onMessage
line: 3379
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener
line: 1843
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0
line: 1468
org.apache.ignite.internal.managers.communication.GridIoManager.access$5200
line: 229
org.apache.ignite.internal.managers.communication.GridIoManager$9.run line:
1365
java.util.concurrent.ThreadPoolExecutor.runWorker line: 1149
java.util.concurrent.ThreadPoolExecutor$Worker.run line: 624
java.lang.Thread.run line: 748
> Massive CPU burn in 2.8.0 and 2.8.1 after upgrading from 2.7.5
> --------------------------------------------------------------
>
> Key: IGNITE-13270
> URL: https://issues.apache.org/jira/browse/IGNITE-13270
> Project: Ignite
> Issue Type: Bug
> Components: sql
> Affects Versions: 2.8, 2.8.1
> Reporter: Tanmay Ambre
> Priority: Major
>
> After upgrading to ignite 2.8.0 from 2.7.5 the following query is causing
> massive CPU burn. We have a 8 node server cluster for Ignite (with 32 CPUs on
> each node). with 2.8.0, 2 nodes are maxed out on CPU. with 2.8.1 all nodes
> are above 80%. On 2.7.5 the same query was performing very nicely.
> The query is a join with 3 tables having same affinity key. however, we dont
> know the affinity key when we are querying and there this results in a merge
> scan.
> Query:
> select A.A_ID, A.AFFINITYKEY, B.B_ID from A, B, C
> where
> A.A_ID = B.A_ID AND
> B.B_ID = C.B_ID AND
> A.AFFINITYKEY = B.AFFINITYKEY AND
> B.AFFINITYKEY = C.AFFINITYKEY AND
> B.B_ID = ? AND
> C.C_ID = ?
>
> Performance:
> 2.7.5 ~ 2 to 5 ms
> 2.8.0 ~ 6 to 12 seconds
> 2.8.1 ~ within 3 seconds
>
> Volume (we have a EDA based system where messages come in burst. One burst
> has approx. 150K events - each event correspond to one query).
>
> throughput of our java based processor (that fires this query):
> 2.7.5 ~ 750 transactions per second
> 2.8.0 ~ 3 transactions per second
> 2.8.1 ~ 6 transactions per second
>
> CPU burn
> 2.7.5 ~ 10 to 15% on each node
> 2.8.0 ~ 100% on 2 nodes other nodes are idling
> 2.8.1 ~ 80 to 90% on each node
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
