[ https://issues.apache.org/jira/browse/HIVE-7250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034991#comment-14034991 ]
Prasanth J commented on HIVE-7250: ---------------------------------- Was able to load 15K columns of the test dataset similar to the qtest dataset in patch with default 1GB heap. 20K columns causes OOM. > Adaptive compression buffer size for wide tables in ORC > ------------------------------------------------------- > > Key: HIVE-7250 > URL: https://issues.apache.org/jira/browse/HIVE-7250 > Project: Hive > Issue Type: Improvement > Components: File Formats > Affects Versions: 0.14.0 > Reporter: Prasanth J > Assignee: Prasanth J > Labels: orcfile > Attachments: HIVE-7250.1.patch, HIVE-7250.2.patch > > > If the input table is wide (in the order of 1000s), ORC compression buffer > size overhead becomes significant causing OOM issues. To overcome this issue, > buffer size should be adaptively chosen based on the available memory and the > number of columns. -- This message was sent by Atlassian JIRA (v6.2#6252)