[
https://issues.apache.org/jira/browse/IGNITE-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273943#comment-15273943
]
Alexei Scherbakov commented on IGNITE-2751:
-------------------------------------------
Measured performance of counting querires with and without fix.
Benchmark code in the attachment.
Benchmark output below:
Fix on, small value
[14:27:23,246] Cache size: 100000
[14:27:23,246] Average object size: 107.778 bytes.
[14:27:23,246] Testing "select count(*) from Person", details: warming up
[14:27:25,254] time(ms): 2007.512 time/query(ms):66.917 count:3000000
[14:27:25,255] Testing "select count(*) from Person", details: full scan
[14:27:28,373] time(ms): 3118.792 time/query(ms):62.376 count:5000000
[14:27:28,373] Testing "select count(*) from Person where _KEY < 100000",
details: id is indexed
[14:27:32,370] time(ms): 3997.077 time/query(ms):79.941 count:5000000
[14:27:32,371] Testing "select count(*) from Person where id < 100000",
details: id is indexed, has the same value as key
[14:27:45,693] time(ms): 13321.395 time/query(ms):266.428 count:5000000
[14:27:45,693] Testing "select count(*) from Person where orgId < 100000",
details: orgId is not indexed
[14:27:58,276] time(ms): 12582.177 time/query(ms):251.644 count:5000000
Fix off, small value
[14:41:37,932] Cache size: 100000
[14:41:37,932] Average object size: 107.778 bytes.
[14:41:37,932] Testing "select count(*) from Person", details: warming up
[14:41:44,154] time(ms): 6221.135 time/query(ms):207.371 count:3000000
[14:41:44,155] Testing "select count(*) from Person", details: full scan
[14:41:54,139] time(ms): 9984.377 time/query(ms):199.687 count:5000000
[14:41:54,139] Testing "select count(*) from Person where _KEY < 100000",
details: id is indexed
[14:42:05,330] time(ms): 11189.357 time/query(ms):223.787 count:5000000
[14:42:05,330] Testing "select count(*) from Person where id < 100000",
details: id is indexed, has the same value as key
[14:42:17,919] time(ms): 12588.818 time/query(ms):251.776 count:5000000
[14:42:17,919] Testing "select count(*) from Person where orgId < 100000",
details: orgId is not indexed
[14:42:30,081] time(ms): 12161.518 time/query(ms):243.23 count:5000000
Fix on, large value
[14:46:07,020] Cache size: 100000
[14:46:07,020] Average object size: 1121.778 bytes.
[14:46:07,020] Testing "select count(*) from Person", details: warming up
[14:46:08,953] time(ms): 1932.41 time/query(ms):64.414 count:3000000
[14:46:08,953] Testing "select count(*) from Person", details: full scan
[14:46:11,877] time(ms): 2924.013 time/query(ms):58.48 count:5000000
[14:46:11,877] Testing "select count(*) from Person where _KEY < 100000",
details: id is indexed
[14:46:15,523] time(ms): 3645.613 time/query(ms):72.912 count:5000000
[14:46:15,523] Testing "select count(*) from Person where id < 100000",
details: id is indexed, has the same value as key
[14:46:31,003] time(ms): 15479.398 time/query(ms):309.588 count:5000000
[14:46:31,003] Testing "select count(*) from Person where orgId < 100000",
details: orgId is not indexed
[14:46:46,443] time(ms): 15438.951 time/query(ms):308.779 count:5000000
Fix off, large value
[14:36:29,596] Cache size: 100000
[14:36:29,596] Average object size: 1121.778 bytes.
[14:36:29,596] Testing "select count(*) from Person", details: warming up
[14:36:37,935] time(ms): 8339.751 time/query(ms):277.992 count:3000000
[14:36:37,936] Testing "select count(*) from Person", details: full scan
[14:36:51,614] time(ms): 13677.008 time/query(ms):273.54 count:5000000
[14:36:51,614] Testing "select count(*) from Person where _KEY < 100000",
details: id is indexed
[14:37:07,064] time(ms): 15449.301 time/query(ms):308.986 count:5000000
[14:37:07,064] Testing "select count(*) from Person where id < 100000",
details: id is indexed, has the same value as key
[14:37:23,292] time(ms): 16226.891 time/query(ms):324.538 count:5000000
[14:37:23,292] Testing "select count(*) from Person where orgId < 100000",
details: orgId is not indexed
[14:37:39,063] time(ms): 15770.452 time/query(ms):315.409 count:5000000
Conclusion:
2 of 4 queries have ~3x time speedup with the fix.
The query "select count(*) from Person where id < 100000" is running
unexpectedly slow. Will investigate it further.
> GridH2TreeIndex.getRowCount unwraps objects' values that are not used later
> ---------------------------------------------------------------------------
>
> Key: IGNITE-2751
> URL: https://issues.apache.org/jira/browse/IGNITE-2751
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 1.5.0.final
> Reporter: Denis Magda
> Assignee: Alexei Scherbakov
> Priority: Critical
> Labels: community, important
> Fix For: 1.6
>
>
> If to run query like "SELECT COUNT(*) FROM table" then our implementation
> will perform a full scan filtering out entries that are not primary for a
> given node and will calculate only a number of primary ones.
> If entries are stored off-heap then both a key and value of an entry are
> unswapped, Unswapped values are not used by backup filters thus we have to
> omit unswapping of the values at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
