Ajantha Bhat created CARBONDATA-2927:
----------------------------------------
Summary: Multiple issue fixes for varchar column and complex
columns that grows more than 2MB
Key: CARBONDATA-2927
URL: https://issues.apache.org/jira/browse/CARBONDATA-2927
Project: CarbonData
Issue Type: Bug
Reporter: Ajantha Bhat
Assignee: Ajantha Bhat
*Fixed:*
*1. varchar data length is more than 2MB, buffer overflow exception (thread
local row buffer)*
*root* casue*: thread* loaclbuffer *was hardcoded with 2MB.*
*solution: grow dynamically based on the row size.*
*2. read data from carbon file having one row of varchar data with 150 MB
length is very slow.*
*root casue: At UnsafeDMStore, ensure memory is just incresing by 8KB each
time and lot of time malloc and free happens before reaching 150MB. hence very
slow performance.*
*solution: directly check and allocate the required size.*
*3. Jvm crash when data size is more than 128 MB in unsafe sort step.*
*root cause: unsafeCarbonRowPage is of 128MB, so if data is more than 128MB
for one row, we access block beyond allocated, leading to JVM crash.*
*solution: validate the size before access and prompt user to increase unsafe
memory. (by carbon property)*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
