[
https://issues.apache.org/jira/browse/CARBONDATA-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala resolved CARBONDATA-2927.
-----------------------------------------
Resolution: Fixed
Fix Version/s: 1.5.0
> Multiple issue fixes for varchar column and complex columns that grows more
> than 2MB
> ------------------------------------------------------------------------------------
>
> Key: CARBONDATA-2927
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2927
> Project: CarbonData
> Issue Type: Bug
> Reporter: Ajantha Bhat
> Assignee: Ajantha Bhat
> Priority: Major
> Fix For: 1.5.0
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> *Fixed:*
> *1. varchar data length is more than 2MB, buffer overflow exception (thread
> local row buffer)*
> *root* casue*: thread* loaclbuffer *was hardcoded with 2MB.*
> *solution: grow dynamically based on the row size.*
> *2. read data from carbon file having one row of varchar data with 150 MB
> length is very slow.*
> *root casue: At UnsafeDMStore, ensure memory is just incresing by 8KB each
> time and lot of time malloc and free happens before reaching 150MB. hence
> very slow performance.*
> *solution: directly check and allocate the required size.*
> *3. Jvm crash when data size is more than 128 MB in unsafe sort step.*
> *root cause: unsafeCarbonRowPage is of 128MB, so if data is more than 128MB
> for one row, we access block beyond allocated, leading to JVM crash.*
> *solution: validate the size before access and prompt user to increase unsafe
> memory. (by carbon property)*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
