Sergey Shelukhin created HIVE-10068:
---------------------------------------
Summary: LLAP: adjust allocation after decompression
Key: HIVE-10068
URL: https://issues.apache.org/jira/browse/HIVE-10068
Project: Hive
Issue Type: Sub-task
Reporter: Sergey Shelukhin
We don't know decompressed size of a compression buffer in ORC, all we know is
the file-level compression buffer size. For many files, compression buffers can
be smaller than that because of compact encoding, or because compression block
ends for other reasons (different streams, etc. - "present" streams for example
are very small).
BuddyAllocator should be able to accept back parts of the allocated memory
(e.g. allocate 256Kb with minimum allocation of 32Kb, decompress 45Kb, return
the last 192Kb as 64+128Kb). For generality (this depends on implementation),
we can make an API like "offer", and allocator can decide to take back however
much it can.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
