[
https://issues.apache.org/jira/browse/HIVE-24311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qiang.Kang updated HIVE-24311:
------------------------------
Description:
Hi, We found that `Rowcontainer.clearRows()` only reset some index, such as:
`addCursor`, `itrCursor`, etc, without resetting read blocks.
As we all know, `currentReadBlock` and `currentWriteBlock` is the main memory
usage of a `Rowcontainer` and might be very large, depending on the data
pattern.
`currentReadBlock` and `currentWriteBlock` won't be the same object after
rowcontainer flushed data to disk.
Resetting `currentReadBlock` and `currentWriteBlock` while clearing rows will
prevent OOM.
> Rowcontainer should reset readBlocks when we clear rows to prevent OOM.
> -----------------------------------------------------------------------
>
> Key: HIVE-24311
> URL: https://issues.apache.org/jira/browse/HIVE-24311
> Project: Hive
> Issue Type: Bug
> Affects Versions: All Versions
> Reporter: Qiang.Kang
> Assignee: Qiang.Kang
> Priority: Major
>
> Hi, We found that `Rowcontainer.clearRows()` only reset some index, such as:
> `addCursor`, `itrCursor`, etc, without resetting read blocks.
> As we all know, `currentReadBlock` and `currentWriteBlock` is the main memory
> usage of a `Rowcontainer` and might be very large, depending on the data
> pattern.
> `currentReadBlock` and `currentWriteBlock` won't be the same object after
> rowcontainer flushed data to disk.
> Resetting `currentReadBlock` and `currentWriteBlock` while clearing rows will
> prevent OOM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
