Indhumathi27 commented on a change in pull request #3744:
URL: https://github.com/apache/carbondata/pull/3744#discussion_r419885797
##########
File path: docs/configuration-parameters.md
##########
@@ -63,7 +63,7 @@ This section provides the details of all the configurations
required for the Car
| carbon.number.of.cores.while.loading | 2 | Number of cores to be used while
loading data. This also determines the number of threads to be used to read the
input files (csv) in parallel.**NOTE:** This configured value is used in every
data loading step to parallelize the operations. Configuring a higher value can
lead to increased early thread pre-emption by OS and there by reduce the
overall performance. |
| enable.unsafe.sort | true | CarbonData supports unsafe operations of Java to
avoid GC overhead for certain operations. This configuration enables to use
unsafe functions in CarbonData. **NOTE:** For operations like data loading,
which generates more short lived Java objects, Java GC can be a bottle neck.
Using unsafe can overcome the GC overhead and improve the overall performance. |
| enable.offheap.sort | true | CarbonData supports storing data in off-heap
memory for certain operations during data loading and query. This helps to
avoid the Java GC and thereby improve the overall performance. This
configuration enables using off-heap memory for sorting of data during data
loading.**NOTE:** ***enable.unsafe.sort*** configuration needs to be
configured to true for using off-heap |
-| carbon.load.sort.scope | LOCAL_SORT | CarbonData can support various sorting
options to match the balance between load and query performance. LOCAL_SORT:All
the data given to an executor in the single load is fully sorted and written to
carbondata files. Data loading performance is reduced a little as the entire
data needs to be sorted in the executor. GLOBAL SORT:Entire data in the data
load is fully sorted and written to carbondata files. Data loading performance
would get reduced as the entire data needs to be sorted. But the query
performance increases significantly due to very less false positives and
concurrency is also improved. **NOTE 1:** This property will be taken into
account only when SORT COLUMNS are specified explicitly while creating table,
otherwise it is always NO SORT |
+| carbon.load.sort.scope | NO_SORT [If sort columns are not specified while
creating table] and LOCAL_SORT [If sort columns are specified] | CarbonData can
support various sorting options to match the balance between load and query
performance. LOCAL_SORT:All the data given to an executor in the single load is
fully sorted and written to carbondata files. Data loading performance is
reduced a little as the entire data needs to be sorted in the executor. GLOBAL
SORT:Entire data in the data load is fully sorted and written to carbondata
files. Data loading performance would get reduced as the entire data needs to
be sorted. But the query performance increases significantly due to very less
false positives and concurrency is also improved. **NOTE 1:** This property
will be taken into account only when SORT COLUMNS are specified explicitly
while creating table, otherwise it is always NO SORT |
Review comment:
```suggestion
| carbon.load.sort.scope | NO_SORT [If sort columns are not specified while
creating table] and LOCAL_SORT [If sort columns are specified] | CarbonData can
support various sorting options to match the balance between load and query
performance. LOCAL_SORT: All the data given to an executor in the single load
is fully sorted and written to carbondata files. Data loading performance is
reduced a little as the entire data needs to be sorted in the executor. GLOBAL
SORT: Entire data in the data load is fully sorted and written to carbondata
files. Data loading performance would get reduced as the entire data needs to
be sorted. But the query performance increases significantly due to very less
false positives and concurrency is also improved. **NOTE 1:** This property
will be taken into account only when SORT COLUMNS are specified explicitly
while creating table, otherwise it is always NO SORT |
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
