[
https://issues.apache.org/jira/browse/IGNITE-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16520282#comment-16520282
]
Taras Ledkov commented on IGNITE-7526:
--------------------------------------
Unfortunately H2 ResultTempTable isn't full appropriate for Ignite case.
H2 storage corrupts in simplest concurrent queries, e.g.: {{SELECT * FROM
<TABLENAME> }}
H2 Error (h2 version: 1.4.195):
{code}
java.lang.RuntimeException: old!=record pos:1037 old:page[1037] data leaf
table:30 TEMP_RESULT_SET_30 entries:0 parent:0 keys:null offsets:null
new:page[1037] data leaf table:29 TEMP_RESULT_SET_29 entries:28 parent:1051
keys:[309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322,
323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 350, 350]
offsets:[4004, 3912, 3820, 3728, 3636, 3544, 3452, 3360, 3268, 3176, 3084,
2992, 2900, 2808, 2716, 2624, 2532, 2440, 2348, 2256, 2164, 2072, 1980, 1888,
1796, 1704, 1612, 1520, 1428, 1336]
at org.h2.message.DbException.throwInternalError(DbException.java:242)
at org.h2.util.CacheLRU.update(CacheLRU.java:126)
at org.h2.store.PageStore.update(PageStore.java:1097)
at org.h2.index.PageDataNode.addRowTry(PageDataNode.java:139)
at org.h2.index.PageDataIndex.addTry(PageDataIndex.java:172)
at org.h2.index.PageDataIndex.add(PageDataIndex.java:135)
at org.h2.table.RegularTable.addRow(RegularTable.java:121)
{code}
I guess we have to implement our own implementation of the
{{org.h2.result.ResultExternal}} and patch the H2 to configure type of the
external result.
> SQL: Introduce memory region for reducer merge results with disk offload
> ------------------------------------------------------------------------
>
> Key: IGNITE-7526
> URL: https://issues.apache.org/jira/browse/IGNITE-7526
> Project: Ignite
> Issue Type: Task
> Components: sql
> Reporter: Vladimir Ozerov
> Assignee: Taras Ledkov
> Priority: Major
>
> Currently all results received from map nodes are stored inside reducer's
> heap memory. What is worse, in case of complex queries, such as having sorts
> or groupings, need to collect all results from mappers first before final
> processing could be applied. In case of big results set (or intermediate
> results) this could easily lead to OOME on reducer.
> To mitigate this we should introduce special memory area where intermediate
> results could be stored. All final processing should be stored in the same
> area as well. This area should be of limited size and should be able to
> offload results to disk in case of overflow.
> We could start with our B+Tree and free list and store results in some K-V
> form.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
