[
https://issues.apache.org/jira/browse/CARBONDATA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15499635#comment-15499635
]
ASF GitHub Bot commented on CARBONDATA-247:
-------------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/162
> Higher MAXCOLUMNS value in load DML options is leading to out of memory error
> -----------------------------------------------------------------------------
>
> Key: CARBONDATA-247
> URL: https://issues.apache.org/jira/browse/CARBONDATA-247
> Project: CarbonData
> Issue Type: Bug
> Reporter: dhatchayani
> Priority: Minor
>
> When a higher value lets say Integer max value is configured for maxcolumns
> option in load DML and executor memory is less, then in that case
> UnivocityCsvParser throws an out of memory error when it tries to create an
> array of size of maxColumns option value.
> java.lang.OutOfMemoryError: Java heap space
> at
> com.univocity.parsers.common.ParserOutput.<init>(ParserOutput.java:86)
> at
> com.univocity.parsers.common.AbstractParser.<init>(AbstractParser.java:66)
> at com.univocity.parsers.csv.CsvParser.<init>(CsvParser.java:50)
> at
> org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114)
> at
> org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427)
> at
> org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60)
> at
> org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
