[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3806: Update build status to spark 2.4

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3806:
URL: https://github.com/apache/carbondata/pull/3806#issuecomment-649235654


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1489/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3806: Update build status to spark 2.4

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3806:
URL: https://github.com/apache/carbondata/pull/3806#issuecomment-649233717


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3216/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] asfgit merged pull request #3806: Update build status to spark 2.4

2020-06-24 Thread GitBox


asfgit merged pull request #3806:
URL: https://github.com/apache/carbondata/pull/3806


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] asfgit closed pull request #3806: Update build status to spark 2.4

2020-06-24 Thread GitBox


asfgit closed pull request #3806:
URL: https://github.com/apache/carbondata/pull/3806


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-06-24 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3855:
-
Attachment: CarbonData SDK support load from file.pdf

> Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
> --
>
> Key: CARBONDATA-3855
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
> Project: CarbonData
>  Issue Type: New Feature
>Reporter: Nihal kumar ojha
>Priority: Major
> Attachments: CarbonData SDK support load from file.pdf
>
>
> Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.

2020-06-24 Thread Nihal kumar ojha (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nihal kumar ojha updated CARBONDATA-3855:
-
Attachment: (was: CarbonData SDK support load from file .pdf)

> Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
> --
>
> Key: CARBONDATA-3855
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3855
> Project: CarbonData
>  Issue Type: New Feature
>Reporter: Nihal kumar ojha
>Priority: Major
>
> Please find the solution document attached.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] chenliang613 opened a new pull request #3806: Update build status to spark 2.4

2020-06-24 Thread GitBox


chenliang613 opened a new pull request #3806:
URL: https://github.com/apache/carbondata/pull/3806


### Why is this PR needed?


### What changes were proposed in this PR?
   
   
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
   
### Is any new testcase added?
- No
- Yes
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3804:
URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648949038


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1486/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3784: [CARBONDATA-3841] Remove useless string in create and alter command

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3784:
URL: https://github.com/apache/carbondata/pull/3784#issuecomment-648947398


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1487/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3804:
URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648946730


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3214/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3784: [CARBONDATA-3841] Remove useless string in create and alter command

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3784:
URL: https://github.com/apache/carbondata/pull/3784#issuecomment-648945566


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3213/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648912506


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1485/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648909599


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3212/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3771:
URL: https://github.com/apache/carbondata/pull/3771#issuecomment-648838314


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3211/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3771:
URL: https://github.com/apache/carbondata/pull/3771#issuecomment-648837271


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1484/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3770:
URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648835883


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3210/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3770:
URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648830470


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1483/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3785: [CARBONDATA-3843] Fix merge index issue in streaming table

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3785:
URL: https://github.com/apache/carbondata/pull/3785#issuecomment-648829424


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1482/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Indhumathi27 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


Indhumathi27 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648828392


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648826642


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3207/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648826450


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1481/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3785: [CARBONDATA-3843] Fix merge index issue in streaming table

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3785:
URL: https://github.com/apache/carbondata/pull/3785#issuecomment-648826422


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3209/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3805: [CARBONDATA-3870] Optimize global lock to object lock for CarbonEnv.init

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3805:
URL: https://github.com/apache/carbondata/pull/3805#issuecomment-648772216


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3205/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3805: [CARBONDATA-3870] Optimize global lock to object lock for CarbonEnv.init

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3805:
URL: https://github.com/apache/carbondata/pull/3805#issuecomment-648771222


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1479/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3804:
URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648768762


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1478/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3804:
URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648767888


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3204/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types

2020-06-24 Thread GitBox


ajantha-bhat commented on a change in pull request #3771:
URL: https://github.com/apache/carbondata/pull/3771#discussion_r44482



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala
##
@@ -679,18 +681,20 @@ private[sql] class CarbonLateDecodeStrategy extends 
SparkStrategy {
 // In case of ComplexType dataTypes no filters should be pushed down. 
IsNotNull is being
 // explicitly added by spark and pushed. That also has to be handled and 
pushed back to
 // Spark for handling.
-val predicatesWithoutComplex = predicates.filter(predicate =>
+// allow array_contains() push down
+val filteredPredicates = predicates.filter(predicate =>

Review comment:
   ok





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types

2020-06-24 Thread GitBox


ajantha-bhat commented on pull request #3771:
URL: https://github.com/apache/carbondata/pull/3771#issuecomment-648763485


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types

2020-06-24 Thread GitBox


ajantha-bhat commented on a change in pull request #3771:
URL: https://github.com/apache/carbondata/pull/3771#discussion_r444824720



##
File path: 
core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecuterImpl.java
##
@@ -222,49 +228,103 @@ public BitSetGroup applyFilter(RawBlockletColumnChunks 
rawBlockletColumnChunks,
   }
 }
 BitSetGroup bitSetGroup = new BitSetGroup(pageNumbers);
-for (int i = 0; i < pageNumbers; i++) {
-  BitSet set = new BitSet(numberOfRows[i]);
-  RowIntf row = new RowImpl();
-  BitSet prvBitset = null;
-  // if bitset pipe line is enabled then use rowid from previous bitset
-  // otherwise use older flow
-  if (!useBitsetPipeLine ||
-  null == rawBlockletColumnChunks.getBitSetGroup() ||
-  null == bitSetGroup.getBitSet(i) ||
-  rawBlockletColumnChunks.getBitSetGroup().getBitSet(i).isEmpty()) {
+if (isDimensionPresentInCurrentBlock.length == 1 && 
isDimensionPresentInCurrentBlock[0]

Review comment:
   @QiangCai : can you please tell me, why new expression is required ? why 
equalTo is not enough ?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader

2020-06-24 Thread GitBox


ajantha-bhat commented on pull request #3770:
URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648760782


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader

2020-06-24 Thread GitBox


ajantha-bhat commented on pull request #3770:
URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648760696


   @akashrn5 , @xubo245 , @jack86596 : reminder !
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3801: adding comments with issue numbers.

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3801:
URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648760641


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3206/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3785: [CARBONDATA-3843] Fix merge index issue in streaming table

2020-06-24 Thread GitBox


ajantha-bhat commented on pull request #3785:
URL: https://github.com/apache/carbondata/pull/3785#issuecomment-648760375







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3801: adding comments with issue numbers.

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3801:
URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648760295


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1480/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3801: adding comments with issue numbers.

2020-06-24 Thread GitBox


ajantha-bhat commented on pull request #3801:
URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648759885


   @ackelcn : Its a good tool that you have.
   But inside IDE, simple annotate can tell the changes are from which PR.
   I personally feel, adding the issue ID is not good and these numbers will 
distract code reading. So, I think it is not required.
   
   @jackylk , @QiangCai , @kunal642 can tell you opinion on this.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3801: adding comments with issue numbers.

2020-06-24 Thread GitBox


ajantha-bhat commented on pull request #3801:
URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648758304


   add to whitelist



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648730477


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1477/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648729525


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3203/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (CARBONDATA-3872) IndexOutOfBoundsException in ResizableArray

2020-06-24 Thread SeaAndHill (Jira)
SeaAndHill created CARBONDATA-3872:
--

 Summary: IndexOutOfBoundsException in ResizableArray
 Key: CARBONDATA-3872
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3872
 Project: CarbonData
  Issue Type: Bug
  Components: sql
Affects Versions: 1.5.1
Reporter: SeaAndHill
 Attachments: carbondata.png

carbondata 在通过in 查询时,in 是子查询,运行报数组越界, 对应spark 版本是2.2.1, hadoop 版本是 2.7.2



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] QiangCai opened a new pull request #3805: [CARBONDATA-3870] Optimize global lock to object lock for CarbonEnv.init

2020-06-24 Thread GitBox


QiangCai opened a new pull request #3805:
URL: https://github.com/apache/carbondata/pull/3805


### Why is this PR needed?
global lock of CarbonEnv.init impacts the performance of the concurrent 
query.

### What changes were proposed in this PR?
   1. Optimize global lock to object lock for CarbonEnv.init
   2. The global lock is only used to create CarbonEnv(not include init).
   
### Does this PR introduce any user interface change?
- No
   
### Is any new testcase added?
- No
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (CARBONDATA-3871) Optimize performance when getting row from heap

2020-06-24 Thread Manhua Jiang (Jira)
Manhua Jiang created CARBONDATA-3871:


 Summary: Optimize performance when getting row from heap
 Key: CARBONDATA-3871
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3871
 Project: CarbonData
  Issue Type: Improvement
Reporter: Manhua Jiang






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] kevinjmh opened a new pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap

2020-06-24 Thread GitBox


kevinjmh opened a new pull request #3804:
URL: https://github.com/apache/carbondata/pull/3804


### Why is this PR needed?
Currently carbon uses priority queue to sort holders of sorted rows. 
It first polls a holder from the heap, and adds it back if holder is not 
empty.
This will cause two times heap maintainance. We can reduce half of that. 

### What changes were proposed in this PR?
What will be done when poll item from priority queue is:
   1. remove first item.
   2. move the last item to the position of first item, siftDown the new first 
item.
   
   In the case in carbon, we can peek(without removing from heap) the first 
item and get a row, 
   and siftDown the holder to a proper position if the holder is not empty.
   
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
   
### Is any new testcase added?
- No
- Yes
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3803: During Concurrent loads in main table with SI table with isSITableEnabled = false, one of the concurrent load fails

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3803:
URL: https://github.com/apache/carbondata/pull/3803#issuecomment-648699905


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3202/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3803: During Concurrent loads in main table with SI table with isSITableEnabled = false, one of the concurrent load fails

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3803:
URL: https://github.com/apache/carbondata/pull/3803#issuecomment-648698402


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1476/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3802: Delete Stale Segment files from Metadata folders when SI segments are deleted and added case for compaction in SILoadEventListenerF

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3802:
URL: https://github.com/apache/carbondata/pull/3802#issuecomment-648697533


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1475/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3802: Delete Stale Segment files from Metadata folders when SI segments are deleted and added case for compaction in SILoadEventListenerF

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3802:
URL: https://github.com/apache/carbondata/pull/3802#issuecomment-648696512


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3201/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] asfgit closed pull request #3799: [CARBONDATA-3862] Insert stage performance optimazation

2020-06-24 Thread GitBox


asfgit closed pull request #3799:
URL: https://github.com/apache/carbondata/pull/3799


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (CARBONDATA-3870) Global lock impact the performance of the concurrent query

2020-06-24 Thread David Cai (Jira)
David Cai created CARBONDATA-3870:
-

 Summary: Global lock impact  the performance of the concurrent 
query
 Key: CARBONDATA-3870
 URL: https://issues.apache.org/jira/browse/CARBONDATA-3870
 Project: CarbonData
  Issue Type: Improvement
Reporter: David Cai






--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3803: During Concurrent loads in main table with SI table with isSITableEnabled = false, one of the concurrent load fails

2020-06-24 Thread GitBox


vikramahuja1001 opened a new pull request #3803:
URL: https://github.com/apache/carbondata/pull/3803


### Why is this PR needed?
   Concurrent load failure in main table with SI table with isSITableEnabled  = 
false

### What changes were proposed in this PR?
   Check if the load name is present in the mainTableDetails array
   Adding compaction case in SILoadEventListenerForFailedSegments
   
### Does this PR introduce any user interface change?
- No
   
### Is any new testcase added?
- No
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3802: Delete Stale Segment files from Metadata folders when SI segments are deleted and added case for compaction in SILoadEventListene

2020-06-24 Thread GitBox


vikramahuja1001 opened a new pull request #3802:
URL: https://github.com/apache/carbondata/pull/3802


### Why is this PR needed?
   In the case when in the SI table a segment is deleted and it's entry is 
deleted from the tablestatus file, during next load command, load into SI 
fails. 

### What changes were proposed in this PR?
   Deleting the .segment file from the metadata folder as it contains the 
mergefilename which does not exist anymore cause the segments are deleted.
   Adding compaction case in SILoadEventListenerForFailedSegments
   
### Does this PR introduce any user interface change?
- No
   
### Is any new testcase added?
- No
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648623470


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3199/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization

2020-06-24 Thread GitBox


CarbonDataQA1 commented on pull request #3789:
URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648622890


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1473/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] niuge01 commented on pull request #3799: [CARBONDATA-3862] Insert stage performance optimazation

2020-06-24 Thread GitBox


niuge01 commented on pull request #3799:
URL: https://github.com/apache/carbondata/pull/3799#issuecomment-648622942


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org