[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3806: Update build status to spark 2.4
CarbonDataQA1 commented on pull request #3806: URL: https://github.com/apache/carbondata/pull/3806#issuecomment-649235654 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1489/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3806: Update build status to spark 2.4
CarbonDataQA1 commented on pull request #3806: URL: https://github.com/apache/carbondata/pull/3806#issuecomment-649233717 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3216/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit merged pull request #3806: Update build status to spark 2.4
asfgit merged pull request #3806: URL: https://github.com/apache/carbondata/pull/3806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3806: Update build status to spark 2.4
asfgit closed pull request #3806: URL: https://github.com/apache/carbondata/pull/3806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
[ https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal kumar ojha updated CARBONDATA-3855: - Attachment: CarbonData SDK support load from file.pdf > Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON. > -- > > Key: CARBONDATA-3855 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3855 > Project: CarbonData > Issue Type: New Feature >Reporter: Nihal kumar ojha >Priority: Major > Attachments: CarbonData SDK support load from file.pdf > > > Please find the solution document attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CARBONDATA-3855) Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON.
[ https://issues.apache.org/jira/browse/CARBONDATA-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal kumar ojha updated CARBONDATA-3855: - Attachment: (was: CarbonData SDK support load from file .pdf) > Support Carbondata SDK to load data from parquet, ORC, CSV, Avro and JSON. > -- > > Key: CARBONDATA-3855 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3855 > Project: CarbonData > Issue Type: New Feature >Reporter: Nihal kumar ojha >Priority: Major > > Please find the solution document attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] chenliang613 opened a new pull request #3806: Update build status to spark 2.4
chenliang613 opened a new pull request #3806: URL: https://github.com/apache/carbondata/pull/3806 ### Why is this PR needed? ### What changes were proposed in this PR? ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap
CarbonDataQA1 commented on pull request #3804: URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648949038 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1486/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3784: [CARBONDATA-3841] Remove useless string in create and alter command
CarbonDataQA1 commented on pull request #3784: URL: https://github.com/apache/carbondata/pull/3784#issuecomment-648947398 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1487/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap
CarbonDataQA1 commented on pull request #3804: URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648946730 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3214/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3784: [CARBONDATA-3841] Remove useless string in create and alter command
CarbonDataQA1 commented on pull request #3784: URL: https://github.com/apache/carbondata/pull/3784#issuecomment-648945566 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3213/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648912506 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1485/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648909599 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3212/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types
CarbonDataQA1 commented on pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#issuecomment-648838314 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3211/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types
CarbonDataQA1 commented on pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#issuecomment-648837271 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1484/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
CarbonDataQA1 commented on pull request #3770: URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648835883 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3210/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
CarbonDataQA1 commented on pull request #3770: URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648830470 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1483/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3785: [CARBONDATA-3843] Fix merge index issue in streaming table
CarbonDataQA1 commented on pull request #3785: URL: https://github.com/apache/carbondata/pull/3785#issuecomment-648829424 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1482/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
Indhumathi27 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648828392 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648826642 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3207/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648826450 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1481/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3785: [CARBONDATA-3843] Fix merge index issue in streaming table
CarbonDataQA1 commented on pull request #3785: URL: https://github.com/apache/carbondata/pull/3785#issuecomment-648826422 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3209/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3805: [CARBONDATA-3870] Optimize global lock to object lock for CarbonEnv.init
CarbonDataQA1 commented on pull request #3805: URL: https://github.com/apache/carbondata/pull/3805#issuecomment-648772216 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3205/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3805: [CARBONDATA-3870] Optimize global lock to object lock for CarbonEnv.init
CarbonDataQA1 commented on pull request #3805: URL: https://github.com/apache/carbondata/pull/3805#issuecomment-648771222 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1479/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap
CarbonDataQA1 commented on pull request #3804: URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648768762 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1478/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap
CarbonDataQA1 commented on pull request #3804: URL: https://github.com/apache/carbondata/pull/3804#issuecomment-648767888 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3204/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types
ajantha-bhat commented on a change in pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#discussion_r44482 ## File path: integration/spark/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala ## @@ -679,18 +681,20 @@ private[sql] class CarbonLateDecodeStrategy extends SparkStrategy { // In case of ComplexType dataTypes no filters should be pushed down. IsNotNull is being // explicitly added by spark and pushed. That also has to be handled and pushed back to // Spark for handling. -val predicatesWithoutComplex = predicates.filter(predicate => +// allow array_contains() push down +val filteredPredicates = predicates.filter(predicate => Review comment: ok This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types
ajantha-bhat commented on pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#issuecomment-648763485 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3771: [CARBONDATA-3849] pushdown array_contains filter to carbon for array of primitive types
ajantha-bhat commented on a change in pull request #3771: URL: https://github.com/apache/carbondata/pull/3771#discussion_r444824720 ## File path: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/RowLevelFilterExecuterImpl.java ## @@ -222,49 +228,103 @@ public BitSetGroup applyFilter(RawBlockletColumnChunks rawBlockletColumnChunks, } } BitSetGroup bitSetGroup = new BitSetGroup(pageNumbers); -for (int i = 0; i < pageNumbers; i++) { - BitSet set = new BitSet(numberOfRows[i]); - RowIntf row = new RowImpl(); - BitSet prvBitset = null; - // if bitset pipe line is enabled then use rowid from previous bitset - // otherwise use older flow - if (!useBitsetPipeLine || - null == rawBlockletColumnChunks.getBitSetGroup() || - null == bitSetGroup.getBitSet(i) || - rawBlockletColumnChunks.getBitSetGroup().getBitSet(i).isEmpty()) { +if (isDimensionPresentInCurrentBlock.length == 1 && isDimensionPresentInCurrentBlock[0] Review comment: @QiangCai : can you please tell me, why new expression is required ? why equalTo is not enough ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
ajantha-bhat commented on pull request #3770: URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648760782 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3770: [CARBONDATA-3829] Support pagination in SDK reader
ajantha-bhat commented on pull request #3770: URL: https://github.com/apache/carbondata/pull/3770#issuecomment-648760696 @akashrn5 , @xubo245 , @jack86596 : reminder ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3801: adding comments with issue numbers.
CarbonDataQA1 commented on pull request #3801: URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648760641 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3206/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3785: [CARBONDATA-3843] Fix merge index issue in streaming table
ajantha-bhat commented on pull request #3785: URL: https://github.com/apache/carbondata/pull/3785#issuecomment-648760375 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3801: adding comments with issue numbers.
CarbonDataQA1 commented on pull request #3801: URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648760295 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1480/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3801: adding comments with issue numbers.
ajantha-bhat commented on pull request #3801: URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648759885 @ackelcn : Its a good tool that you have. But inside IDE, simple annotate can tell the changes are from which PR. I personally feel, adding the issue ID is not good and these numbers will distract code reading. So, I think it is not required. @jackylk , @QiangCai , @kunal642 can tell you opinion on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] ajantha-bhat commented on pull request #3801: adding comments with issue numbers.
ajantha-bhat commented on pull request #3801: URL: https://github.com/apache/carbondata/pull/3801#issuecomment-648758304 add to whitelist This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648730477 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1477/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648729525 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3203/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-3872) IndexOutOfBoundsException in ResizableArray
SeaAndHill created CARBONDATA-3872: -- Summary: IndexOutOfBoundsException in ResizableArray Key: CARBONDATA-3872 URL: https://issues.apache.org/jira/browse/CARBONDATA-3872 Project: CarbonData Issue Type: Bug Components: sql Affects Versions: 1.5.1 Reporter: SeaAndHill Attachments: carbondata.png carbondata 在通过in 查询时,in 是子查询,运行报数组越界, 对应spark 版本是2.2.1, hadoop 版本是 2.7.2 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] QiangCai opened a new pull request #3805: [CARBONDATA-3870] Optimize global lock to object lock for CarbonEnv.init
QiangCai opened a new pull request #3805: URL: https://github.com/apache/carbondata/pull/3805 ### Why is this PR needed? global lock of CarbonEnv.init impacts the performance of the concurrent query. ### What changes were proposed in this PR? 1. Optimize global lock to object lock for CarbonEnv.init 2. The global lock is only used to create CarbonEnv(not include init). ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-3871) Optimize performance when getting row from heap
Manhua Jiang created CARBONDATA-3871: Summary: Optimize performance when getting row from heap Key: CARBONDATA-3871 URL: https://issues.apache.org/jira/browse/CARBONDATA-3871 Project: CarbonData Issue Type: Improvement Reporter: Manhua Jiang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] kevinjmh opened a new pull request #3804: [CARBONDATA-3871] Optimize performance when getting row from heap
kevinjmh opened a new pull request #3804: URL: https://github.com/apache/carbondata/pull/3804 ### Why is this PR needed? Currently carbon uses priority queue to sort holders of sorted rows. It first polls a holder from the heap, and adds it back if holder is not empty. This will cause two times heap maintainance. We can reduce half of that. ### What changes were proposed in this PR? What will be done when poll item from priority queue is: 1. remove first item. 2. move the last item to the position of first item, siftDown the new first item. In the case in carbon, we can peek(without removing from heap) the first item and get a row, and siftDown the holder to a proper position if the holder is not empty. ### Does this PR introduce any user interface change? - No - Yes. (please explain the change and update document) ### Is any new testcase added? - No - Yes This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3803: During Concurrent loads in main table with SI table with isSITableEnabled = false, one of the concurrent load fails
CarbonDataQA1 commented on pull request #3803: URL: https://github.com/apache/carbondata/pull/3803#issuecomment-648699905 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3202/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3803: During Concurrent loads in main table with SI table with isSITableEnabled = false, one of the concurrent load fails
CarbonDataQA1 commented on pull request #3803: URL: https://github.com/apache/carbondata/pull/3803#issuecomment-648698402 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1476/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3802: Delete Stale Segment files from Metadata folders when SI segments are deleted and added case for compaction in SILoadEventListenerF
CarbonDataQA1 commented on pull request #3802: URL: https://github.com/apache/carbondata/pull/3802#issuecomment-648697533 Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1475/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3802: Delete Stale Segment files from Metadata folders when SI segments are deleted and added case for compaction in SILoadEventListenerF
CarbonDataQA1 commented on pull request #3802: URL: https://github.com/apache/carbondata/pull/3802#issuecomment-648696512 Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3201/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #3799: [CARBONDATA-3862] Insert stage performance optimazation
asfgit closed pull request #3799: URL: https://github.com/apache/carbondata/pull/3799 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (CARBONDATA-3870) Global lock impact the performance of the concurrent query
David Cai created CARBONDATA-3870: - Summary: Global lock impact the performance of the concurrent query Key: CARBONDATA-3870 URL: https://issues.apache.org/jira/browse/CARBONDATA-3870 Project: CarbonData Issue Type: Improvement Reporter: David Cai -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3803: During Concurrent loads in main table with SI table with isSITableEnabled = false, one of the concurrent load fails
vikramahuja1001 opened a new pull request #3803: URL: https://github.com/apache/carbondata/pull/3803 ### Why is this PR needed? Concurrent load failure in main table with SI table with isSITableEnabled = false ### What changes were proposed in this PR? Check if the load name is present in the mainTableDetails array Adding compaction case in SILoadEventListenerForFailedSegments ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] vikramahuja1001 opened a new pull request #3802: Delete Stale Segment files from Metadata folders when SI segments are deleted and added case for compaction in SILoadEventListene
vikramahuja1001 opened a new pull request #3802: URL: https://github.com/apache/carbondata/pull/3802 ### Why is this PR needed? In the case when in the SI table a segment is deleted and it's entry is deleted from the tablestatus file, during next load command, load into SI fails. ### What changes were proposed in this PR? Deleting the .segment file from the metadata folder as it contains the mergefilename which does not exist anymore cause the segments are deleted. Adding compaction case in SILoadEventListenerForFailedSegments ### Does this PR introduce any user interface change? - No ### Is any new testcase added? - No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648623470 Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3199/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3789: [WIP] [CARBONDATA-3864] Store Size Optimization
CarbonDataQA1 commented on pull request #3789: URL: https://github.com/apache/carbondata/pull/3789#issuecomment-648622890 Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1473/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [carbondata] niuge01 commented on pull request #3799: [CARBONDATA-3862] Insert stage performance optimazation
niuge01 commented on pull request #3799: URL: https://github.com/apache/carbondata/pull/3799#issuecomment-648622942 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org