Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/189#discussion_r81279628
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/writer/AbstractFactDataWriter.java
---
@@ -196,11 +197,22 @@ public AbstractFactDataWriter(String storeLocation,
int measureCount, int mdKeyL
blockIndexInfoList = new ArrayList<>();
// get max file size;
CarbonProperties propInstance = CarbonProperties.getInstance();
- this.fileSizeInBytes = Long.parseLong(propInstance
- .getProperty(CarbonCommonConstants.MAX_FILE_SIZE,
- CarbonCommonConstants.MAX_FILE_SIZE_DEFAULT_VAL))
- * CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR
- * CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR * 1L;
+ // If blocksize is defined by create table ddl, then use this value
otherwise
+ // use system level property
+ if (blocksize >=
CarbonCommonConstants.MAX_FILE_SIZE_DEFAULT_VAL_MIN_VAL &&
+ blocksize <=
CarbonCommonConstants.MAX_FILE_SIZE_DEFAULT_VAL_MAX_VAL) {
+ LOGGER.audit("The max file size is set in table level. ");
+ this.fileSizeInBytes = blocksize *
CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR
+ * CarbonCommonConstants.BYTE_TO_KB_CONVERSION_FACTOR * 1L;
+ } else {
+ LOGGER.audit("The max file size in create table ddl is not set or is
invalid, " +
+ "so here use " + CarbonCommonConstants.MAX_FILE_SIZE + "to set.
");
--- End diff --
@Zhangshunyu ...I am thinking instead of validating block size here
specified by the user, we can validate at the time of creating the table and in
error message we can display the range of allowed block size so that user is
not confused.
If user does not specify the block size the consider the default value and
update in the schema
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
