condition for single pass
Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/bbcc487a Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/bbcc487a Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/bbcc487a Branch: refs/heads/branch-1.1 Commit: bbcc487a423d1ba42efd1926457fe27d763c00af Parents: 917152a Author: sgururajshetty <sgururajshe...@gmail.com> Authored: Tue May 30 16:42:32 2017 +0530 Committer: ravipesala <ravi.pes...@gmail.com> Committed: Thu Jun 15 13:17:34 2017 +0530 ---------------------------------------------------------------------- docs/configuration-parameters.md | 2 +- docs/dml-operation-on-carbondata.md | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/carbondata/blob/bbcc487a/docs/configuration-parameters.md ---------------------------------------------------------------------- diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md index c63f73d..b71cdbc 100644 --- a/docs/configuration-parameters.md +++ b/docs/configuration-parameters.md @@ -130,7 +130,7 @@ This section provides the details of all the configurations required for CarbonD | Parameter | Default Value | Description | |---------------------------------------|---------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality columns of the dictionary code are automatically recognized and these columns will not be used as global dictionary encoding. If the parameter is false, all dictionary encoding columns are used as dictionary encoding. The high cardinality column must meet the following requirements: value of cardinality > configured value of high.cardinality. Equally, the value of cardinality is higher than the threshold.value of cardinality/ row number x 100 > configured value of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data row number is higher than the configured percentage. | +| high.cardinality.identify.enable | true | If the parameter is true, the high cardinality columns of the dictionary code are automatically recognized and these columns will not be used as global dictionary encoding. If the parameter is false, all dictionary encoding columns are used as dictionary encoding. The high cardinality column must meet the following requirements: value of cardinality > configured value of high.cardinality. Equally, the value of cardinality is higher than the threshold.value of cardinality/ row number x 100 > configured value of high.cardinality.row.count.percentage. Equally, the ratio of the cardinality value to data row number is higher than the configured percentage. Note: If SINGLE_PASS is used during data load, then this property will be disabled.| | high.cardinality.threshold | 1000000 | It is a threshold to identify high cardinality of the columns.If the value of columns' cardinality > the configured value, then the columns are excluded from dictionary encoding. | | high.cardinality.row.count.percentage | 80 | Percentage to identify whether column cardinality is more than configured percent of total row count.Configuration value formula:Value of cardinality/ row number x 100 > configured value of high.cardinality.row.count.percentage. The value of the parameter must be larger than 0. | | carbon.cutOffTimestamp | 1970-01-01 05:30:00 | Sets the start date for calculating the timestamp. Java counts the number of milliseconds from start of "1970-01-01 00:00:00". This property is used to customize the start of position. For example "2000-01-01 00:00:00". The date must be in the form "carbon.timestamp.format". NOTE: The CarbonData supports data store up to 68 years from the cut-off time defined. For example, if the cut-off time is 1970-01-01 05:30:00, then the data can be stored up to 2038-01-01 05:30:00. | http://git-wip-us.apache.org/repos/asf/carbondata/blob/bbcc487a/docs/dml-operation-on-carbondata.md ---------------------------------------------------------------------- diff --git a/docs/dml-operation-on-carbondata.md b/docs/dml-operation-on-carbondata.md index e315468..45f7464 100644 --- a/docs/dml-operation-on-carbondata.md +++ b/docs/dml-operation-on-carbondata.md @@ -146,6 +146,9 @@ You can use the following options to load data: * If this option is set to TRUE then data loading will take less time. * If this option is set to some invalid value other than TRUE or FALSE then it uses the default value. + + * If this option is set to TRUE, then high.cardinality.identify.enable property will be disabled during data load. + ### Example: ```
