Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2252#discussion_r194603907
--- Diff:
core/src/main/java/org/apache/carbondata/core/datastore/chunk/store/impl/safe/SafeVariableLengthDimensionDataChunkStore.java
---
@@ -56,7 +57,8 @@ public SafeVariableLengthDimensionDataChunkStore(boolean
isInvertedIndex, int nu
* @param invertedIndexReverse inverted index reverse to be stored
* @param data data to be stored
*/
- @Override public void putArray(final int[] invertedIndex, final int[]
invertedIndexReverse,
+ @Override
+ public void putArray(final int[] invertedIndex, final int[]
invertedIndexReverse,
--- End diff --
@kumarvishal09 yeah, you are right. Considering that scenario will make the
code much more complex.
As we all know, a maximum of string/bytearray is about 2GB. I think it is
enough for current scenarios. 2GB can support average 67104 bytes per column
(2GB/32000) in one column page.
Having discussed with @jackylk , we have the following conclusions:
1. Add an error message to indicate that the maximum size of one column
page is 2GB.
2. We can reduce the row size in the page to support longer characters, for
example if the content is 67104*2, user can reduce the row size from 32000 to
16000.
1 will be included in this PR and 2 will be implemented in another PR.
Is that OK? @kumarvishal09
---
