Ajantha Bhat created CARBONDATA-3515:
----------------------------------------
Summary: Limit local dictionary size to 10% of allowed blocklet
size
Key: CARBONDATA-3515
URL: https://issues.apache.org/jira/browse/CARBONDATA-3515
Project: CarbonData
Issue Type: Improvement
Reporter: Ajantha Bhat
problem: currently local dictionary max size is 2GB, because of this for
varchar columns or long string columns, local dictionary can be of 2GB size.
so, as local dictionary is stored in blocklet. blocklet size will exceed 2 GB,
even though configured maximum blocklet size is 64MB. some places inter
overflow happens during casting.
solution: limit the local dictionary size to 10% of maximum allowed blocklet
size
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
