[ https://issues.apache.org/jira/browse/CARBONDATA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jacky Li resolved CARBONDATA-3761. ---------------------------------- Fix Version/s: 2.0.0 Resolution: Fixed > Remove redundant conversion for complex type insert > --------------------------------------------------- > > Key: CARBONDATA-3761 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3761 > Project: CarbonData > Issue Type: Improvement > Reporter: Ajantha Bhat > Assignee: Ajantha Bhat > Priority: Major > Fix For: 2.0.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > h3. Problem > # In {{PrimitiveDataType#writeByteArray}} > {{DataTypeUtil.parseValue(**input.toString()**, carbonDimension)}} > Here we convert every complex child element to string and then parse as an > object to handle bad records. Which leads to heavy GC > # {{DatatypeUtil#getBytesDataDataTypeForNoDictionaryColumn}} -> > double,float, byte, decimal case is missing. so we convert them to string and > then convert to bytes. which create more redundant objects > h3. Solution > # For new Insert into flow, no need to handle bad records for complex types > as it is already validated in source table. So, use object directly. This can > decrease the memory foot print for complex type insert > # Add a case for double,float, byte, decimal data type. -- This message was sent by Atlassian Jira (v8.3.4#803005)