[
https://issues.apache.org/jira/browse/CARBONDATA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacky Li resolved CARBONDATA-3761.
----------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
> Remove redundant conversion for complex type insert
> ---------------------------------------------------
>
> Key: CARBONDATA-3761
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3761
> Project: CarbonData
> Issue Type: Improvement
> Reporter: Ajantha Bhat
> Assignee: Ajantha Bhat
> Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> h3. Problem
> # In {{PrimitiveDataType#writeByteArray}}
> {{DataTypeUtil.parseValue(**input.toString()**, carbonDimension)}}
> Here we convert every complex child element to string and then parse as an
> object to handle bad records. Which leads to heavy GC
> # {{DatatypeUtil#getBytesDataDataTypeForNoDictionaryColumn}} ->
> double,float, byte, decimal case is missing. so we convert them to string and
> then convert to bytes. which create more redundant objects
> h3. Solution
> # For new Insert into flow, no need to handle bad records for complex types
> as it is already validated in source table. So, use object directly. This can
> decrease the memory foot print for complex type insert
> # Add a case for double,float, byte, decimal data type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
