Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/3046
@xuchuanyin The problem was that when using varchar column with email data
the key for the dictionary map is very huge. When fallback happens the same
data is kept in memory twice, which causes the application to throw OOM
exception.
As this is a minor version therefore we do not want to expose this property
to user, we can take in the next major version.
Plus parquet also has a size based limitation mechanism which ensures the
size does not grow more than what the system can handle.
This PR is just to add the size based limitation so that the map size can
be controlled.
---
