[ 
https://issues.apache.org/jira/browse/CARBONDATA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-3761.
----------------------------------
    Fix Version/s: 2.0.0
       Resolution: Fixed

> Remove redundant conversion for complex type insert
> ---------------------------------------------------
>
>                 Key: CARBONDATA-3761
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3761
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: Ajantha Bhat
>            Assignee: Ajantha Bhat
>            Priority: Major
>             Fix For: 2.0.0
>
>          Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> h3. Problem
>  # In {{PrimitiveDataType#writeByteArray}}
> {{DataTypeUtil.parseValue(**input.toString()**, carbonDimension)}}
> Here we convert every complex child element to string and then parse as an 
> object to handle bad records. Which leads to heavy GC
>  # {{DatatypeUtil#getBytesDataDataTypeForNoDictionaryColumn}} -> 
> double,float, byte, decimal case is missing. so we convert them to string and 
> then convert to bytes. which create more redundant objects
> h3. Solution
>  # For new Insert into flow, no need to handle bad records for complex types 
> as it is already validated in source table. So, use object directly. This can 
> decrease the memory foot print for complex type insert
>  # Add a case for double,float, byte, decimal data type.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to