Ajantha Bhat created CARBONDATA-3761:
----------------------------------------
Summary: Remove redundant conversion for complex type insert
Key: CARBONDATA-3761
URL: https://issues.apache.org/jira/browse/CARBONDATA-3761
Project: CarbonData
Issue Type: Improvement
Reporter: Ajantha Bhat
Assignee: Ajantha Bhat
h3. Problem
# In {{PrimitiveDataType#writeByteArray}}
{{DataTypeUtil.parseValue(**input.toString()**, carbonDimension)}}
Here we convert every complex child element to string and then parse as an
object to handle bad records. Which leads to heavy GC
# {{DatatypeUtil#getBytesDataDataTypeForNoDictionaryColumn}} -> double,float,
byte, decimal case is missing. so we convert them to string and then convert to
bytes. which create more redundant objects
h3. Solution
# For new Insert into flow, no need to handle bad records for complex types as
it is already validated in source table. So, use object directly. This can
decrease the memory foot print for complex type insert
# Add a case for double,float, byte, decimal data type.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
