Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2383#discussion_r196487039
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
---
@@ -371,8 +371,13 @@ private void setWritingConfiguration() throws
CarbonDataWriterException {
this.pageSize = Integer.parseInt(CarbonProperties.getInstance()
.getProperty(CarbonCommonConstants.BLOCKLET_SIZE,
CarbonCommonConstants.BLOCKLET_SIZE_DEFAULT_VAL));
+ // support less than 32000 rows in one page, because we support super
long string,
+ // if it is long enough, a clomun page with 32000 rows will exceed 2GB
if (version == ColumnarFormatVersion.V3) {
- this.pageSize =
CarbonV3DataFormatConstants.NUMBER_OF_ROWS_PER_BLOCKLET_COLUMN_PAGE_DEFAULT;
+ this.pageSize =
--- End diff --
how much is the default value for page size ?
---
