[ https://issues.apache.org/jira/browse/IGNITE-7526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511100#comment-16511100 ]
Vladimir Ozerov commented on IGNITE-7526: ----------------------------------------- Created a prototype with persistent database - was able to execute SQL query successfully. > SQL: Introduce memory region for reducer merge results with disk offload > ------------------------------------------------------------------------ > > Key: IGNITE-7526 > URL: https://issues.apache.org/jira/browse/IGNITE-7526 > Project: Ignite > Issue Type: Task > Components: sql > Reporter: Vladimir Ozerov > Assignee: Taras Ledkov > Priority: Major > > Currently all results received from map nodes are stored inside reducer's > heap memory. What is worse, in case of complex queries, such as having sorts > or groupings, need to collect all results from mappers first before final > processing could be applied. In case of big results set (or intermediate > results) this could easily lead to OOME on reducer. > To mitigate this we should introduce special memory area where intermediate > results could be stored. All final processing should be stored in the same > area as well. This area should be of limited size and should be able to > offload results to disk in case of overflow. > We could start with our B+Tree and free list and store results in some K-V > form. -- This message was sent by Atlassian JIRA (v7.6.3#76005)