Venugopal Reddy K created CARBONDATA-3519:
---------------------------------------------
Summary: A new column page MemoryBlock is allocated at each row
addition to table page if having string column with local dictionary enabled.
Key: CARBONDATA-3519
URL: https://issues.apache.org/jira/browse/CARBONDATA-3519
Project: CarbonData
Issue Type: Improvement
Components: core
Reporter: Venugopal Reddy K
*Context:*
For a string column with local dictionary enabled, a column page of
`UnsafeFixLengthColumnPage` with datatype `DataTypes.BYTE_ARRAY` is created for
`encodedPage` along with regular `actualPage` of `UnsafeVarLengthColumnPage`.
We have `capacity` field in
the `UnsafeFixLengthColumnPage`. And this field indicates the capacity of
allocated
`memoryBlock` for the page. `ensureMemory()` method is being called while
adding rows to check if
`totalLength + requestSize > capacity` to allocate a new memoryBlock if there
is no room to add the next row, copy the old context(prev rows) and free the
old memoryBlock.
*Issue:*
While, UnsafeFixLengthColumnPage with with datatype `DataTypes.BYTE_ARRAY` is
created for `encodedPage`, we have not assigned the `capacity` field with
allocated memory block size. Hence, when we add a row to tablePage,
ensureMemory() check fails, allocates a new column page memoryBlock, copy the
old context(prev rows) and free the old memoryBlock. This allocation of new
memoryBlock and free of old memoryBlock happens at row addition for the string
columns with local dictionary enabled.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
