[jira] [Created] (CARBONDATA-1624) If SORT_SCOPE is non-GLOBAL_SORT with Spark, set 'carbon.number.of.cores.while.loading' dynamically as per the available executor cores

Thu, 26 Oct 2017 02:51:11 -0700 

Zhichao  Zhang created CARBONDATA-1624:
------------------------------------------

             Summary: If SORT_SCOPE is non-GLOBAL_SORT with Spark, set 
'carbon.number.of.cores.while.loading' dynamically as per the available 
executor cores 
                 Key: CARBONDATA-1624
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1624
             Project: CarbonData
          Issue Type: Improvement
          Components: data-load, spark-integration
    Affects Versions: 1.3.0
            Reporter: Zhichao  Zhang
            Priority: Minor


If we are using carbondata + spark to load data, we can set 
carbon.number.of.cores.while.loading to the  number of executor cores. 

For example, when set the number of executor cores to 6, it shows that there 
are at 
least 6 cores per node for loading data, so we can set 
carbon.number.of.cores.while.loading to 6 automatically. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to