Github user jackylk commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1808#discussion_r166936712 --- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonLoadOptionConstants.java --- @@ -114,4 +114,14 @@ */ public static final int MAX_EXTERNAL_DICTIONARY_SIZE = 10000000; + /** + * enable block size based block allocation while loading data. By default, carbondata assigns + * blocks to node based on block number. If this option is set to `true`, carbondata will + * consider block size first and make sure that all the nodes will process almost equal size of + * data. This option is especially useful when you encounter skewed data. + */ + @CarbonProperty + public static final String ENABLE_CARBON_LOAD_SKEWED_DATA_OPTIMIZATION + = "carbon.load.skewed.data.optimization"; --- End diff -- change to `carbon.load.skewedDataOptimization.enabled`
---