[
https://issues.apache.org/jira/browse/HIVE-20247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-20247:
------------------------------------
Description:
LLAP IO creates unallocated buffer objects inside the read-related data
structures, then allocates them in bulk, then decompresses into them and
increfs them.
If allocate or decompress steps fail, it's hard for the higher-level cleanup to
tell what the state of the buffers in the read-related structures is - they may
be unallocated, allocated but not incref-ed, or incref-ed.
Some cleanup paths only deal with the latter case, resulting in bugs.
Moreover, currently allocator returns partial results on such error. The
allocation should be all-or-nothing.
This only happens on one paths, others allocate and use buffers in a single
place.
was:
LLAP IO creates unallocated buffer objects inside the read-related data
structures, then allocates them in bulk, then decompresses into them and
increfs them.
If allocate or decompress steps fail, it's hard for the higher-level cleanup to
tell what the state of the buffers in the read-related structures is - they may
be unallocated, allocated but not incref-ed, or incref-ed.
Some cleanup paths only deal with the latter case, resulting in bugs.
This only happens on one paths, others allocate and use buffers in a single
place.
> cleanup issues in LLAP IO after cache OOM
> -----------------------------------------
>
> Key: HIVE-20247
> URL: https://issues.apache.org/jira/browse/HIVE-20247
> Project: Hive
> Issue Type: Bug
> Reporter: Prasanth Jayachandran
> Assignee: Sergey Shelukhin
> Priority: Major
>
> LLAP IO creates unallocated buffer objects inside the read-related data
> structures, then allocates them in bulk, then decompresses into them and
> increfs them.
> If allocate or decompress steps fail, it's hard for the higher-level cleanup
> to tell what the state of the buffers in the read-related structures is -
> they may be unallocated, allocated but not incref-ed, or incref-ed.
> Some cleanup paths only deal with the latter case, resulting in bugs.
> Moreover, currently allocator returns partial results on such error. The
> allocation should be all-or-nothing.
> This only happens on one paths, others allocate and use buffers in a single
> place.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
