[09/45] carbondata git commit: [CARBONDATA-2973] Added documentation for fallback condition for complex columns in local Dictionary
[CARBONDATA-2973] Added documentation for fallback condition for complex columns in local Dictionary 1. Added documentation for fallback condition for complex columns in local Dictionary 2. Added documentation for system level property" carbon.local.dictionary.decoder.fallback" This closes #2766 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/3f99e9b7 Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/3f99e9b7 Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/3f99e9b7 Branch: refs/heads/branch-1.5 Commit: 3f99e9b7f87b387f03cb5bece2b2a8c5a50b Parents: a9ddfbd Author: praveenmeenakshi56 Authored: Wed Sep 26 12:40:37 2018 +0530 Committer: manishgupta88 Committed: Wed Sep 26 18:14:44 2018 +0530 -- docs/configuration-parameters.md | 2 +- docs/ddl-of-carbondata.md| 16 +++- docs/faq.md | 2 +- 3 files changed, 13 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/carbondata/blob/3f99e9b7/docs/configuration-parameters.md -- diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md index 7edae47..662525b 100644 --- a/docs/configuration-parameters.md +++ b/docs/configuration-parameters.md @@ -119,7 +119,7 @@ This section provides the details of all the configurations required for the Car | Parameter | Default Value | Description | |--|---|---| -| carbon.max.driver.lru.cache.size | -1 | Maximum memory **(in MB)** upto which the driver process can cache the data (BTree and dictionary values). Beyond this, least recently used data will be removed from cache before loading new set of values.Default value of -1 means there is no memory limit for caching. Only integer values greater than 0 are accepted.**NOTE:** Minimum number of entries that needs to be removed from cache in order to load the new set of data is determined and unloaded.ie.,for example if 3 cache entries qualify for pre-emption, out of these, those entries that free up more cache memory is removed prior to others. Please refer [FAQs](./faq.md#how-to-check-LRU-cache-memory-footprint) for checking LRU cache memory footprint. | +| carbon.max.driver.lru.cache.size | -1 | Maximum memory **(in MB)** upto which the driver process can cache the data (BTree and dictionary values). Beyond this, least recently used data will be removed from cache before loading new set of values.Default value of -1 means there is no memory limit for caching. Only integer values greater than 0 are accepted.**NOTE:** Minimum number of entries that needs to be removed from cache in order to load the new set of data is determined and unloaded.ie.,for example if 3 cache entries qualify for pre-emption, out of these, those entries that free up more cache memory is removed prior to others. Please refer [FAQs](./faq.md#how-to-check-lru-cache-memory-footprint) for checking LRU cache memory footprint. | | carbon.max.executor.lru.cache.size | -1 | Maximum memory **(in MB)** upto which the executor process can cache the data (BTree and reverse dictionary values).Default value of -1 means there is no memory limit for caching. Only integer values greater than 0 are accepted.**NOTE:** If this parameter is not configured, then the value of ***carbon.max.driver.lru.cache.size*** will be used. | | max.query.execution.time | 60 | Maximum time allowed for one query to be executed. The value is in minutes. | | carbon.enableMinMax | true | CarbonData maintains the metadata which enables to prune unnecessary files from being scanned as per the query conditions.To achieve pruning, Min,Max of each column is maintined.Based on the filter condition in the query, certain data can be skipped from scanning by matching the filter value against the min,max values of the column(s) present in that carbondata file.This pruing enhances query performance significantly. | http://git-wip-us.apache.org/repos/asf/carbondata/blob/3f99e9b7/docs/ddl-of-carbondata.md -- diff --git a/docs/ddl-of-carbondata.md b/docs/ddl-of-carbondata.md index 2a467a2..5eeba86 100644 --- a/docs/ddl-of-carbondata.md +++ b/docs/ddl-of-carbondata.md @@ -231,7 +231,13 @@ CarbonData DDL statements are documented here,which includes: * In case of multi-level complex dataType columns, primitive string/varchar/char columns are considered for local dictionary generation. - Local dictionary will have to be enabled explicitly during create table or by enabling the **system property** ***carbon.local.dictionary.enable***. By default,
carbondata git commit: [CARBONDATA-2973] Added documentation for fallback condition for complex columns in local Dictionary
Repository: carbondata Updated Branches: refs/heads/master a9ddfbd7b -> 3f99e9b7f [CARBONDATA-2973] Added documentation for fallback condition for complex columns in local Dictionary 1. Added documentation for fallback condition for complex columns in local Dictionary 2. Added documentation for system level property" carbon.local.dictionary.decoder.fallback" This closes #2766 Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/3f99e9b7 Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/3f99e9b7 Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/3f99e9b7 Branch: refs/heads/master Commit: 3f99e9b7f87b387f03cb5bece2b2a8c5a50b Parents: a9ddfbd Author: praveenmeenakshi56 Authored: Wed Sep 26 12:40:37 2018 +0530 Committer: manishgupta88 Committed: Wed Sep 26 18:14:44 2018 +0530 -- docs/configuration-parameters.md | 2 +- docs/ddl-of-carbondata.md| 16 +++- docs/faq.md | 2 +- 3 files changed, 13 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/carbondata/blob/3f99e9b7/docs/configuration-parameters.md -- diff --git a/docs/configuration-parameters.md b/docs/configuration-parameters.md index 7edae47..662525b 100644 --- a/docs/configuration-parameters.md +++ b/docs/configuration-parameters.md @@ -119,7 +119,7 @@ This section provides the details of all the configurations required for the Car | Parameter | Default Value | Description | |--|---|---| -| carbon.max.driver.lru.cache.size | -1 | Maximum memory **(in MB)** upto which the driver process can cache the data (BTree and dictionary values). Beyond this, least recently used data will be removed from cache before loading new set of values.Default value of -1 means there is no memory limit for caching. Only integer values greater than 0 are accepted.**NOTE:** Minimum number of entries that needs to be removed from cache in order to load the new set of data is determined and unloaded.ie.,for example if 3 cache entries qualify for pre-emption, out of these, those entries that free up more cache memory is removed prior to others. Please refer [FAQs](./faq.md#how-to-check-LRU-cache-memory-footprint) for checking LRU cache memory footprint. | +| carbon.max.driver.lru.cache.size | -1 | Maximum memory **(in MB)** upto which the driver process can cache the data (BTree and dictionary values). Beyond this, least recently used data will be removed from cache before loading new set of values.Default value of -1 means there is no memory limit for caching. Only integer values greater than 0 are accepted.**NOTE:** Minimum number of entries that needs to be removed from cache in order to load the new set of data is determined and unloaded.ie.,for example if 3 cache entries qualify for pre-emption, out of these, those entries that free up more cache memory is removed prior to others. Please refer [FAQs](./faq.md#how-to-check-lru-cache-memory-footprint) for checking LRU cache memory footprint. | | carbon.max.executor.lru.cache.size | -1 | Maximum memory **(in MB)** upto which the executor process can cache the data (BTree and reverse dictionary values).Default value of -1 means there is no memory limit for caching. Only integer values greater than 0 are accepted.**NOTE:** If this parameter is not configured, then the value of ***carbon.max.driver.lru.cache.size*** will be used. | | max.query.execution.time | 60 | Maximum time allowed for one query to be executed. The value is in minutes. | | carbon.enableMinMax | true | CarbonData maintains the metadata which enables to prune unnecessary files from being scanned as per the query conditions.To achieve pruning, Min,Max of each column is maintined.Based on the filter condition in the query, certain data can be skipped from scanning by matching the filter value against the min,max values of the column(s) present in that carbondata file.This pruing enhances query performance significantly. | http://git-wip-us.apache.org/repos/asf/carbondata/blob/3f99e9b7/docs/ddl-of-carbondata.md -- diff --git a/docs/ddl-of-carbondata.md b/docs/ddl-of-carbondata.md index 2a467a2..5eeba86 100644 --- a/docs/ddl-of-carbondata.md +++ b/docs/ddl-of-carbondata.md @@ -231,7 +231,13 @@ CarbonData DDL statements are documented here,which includes: * In case of multi-level complex dataType columns, primitive string/varchar/char columns are considered for local dictionary generation. - Local dictionary will have to be enabled explicitly during create table or by
