[jira] [Resolved] (CARBONDATA-2798) Fix Dictionary_Include for ComplexDataType
[ https://issues.apache.org/jira/browse/CARBONDATA-2798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Kapoor resolved CARBONDATA-2798. -- Resolution: Fixed Fix Version/s: 1.4.1 > Fix Dictionary_Include for ComplexDataType > -- > > Key: CARBONDATA-2798 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2798 > Project: CarbonData > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Minor > Fix For: 1.4.1 > > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Github user kunal642 commented on the issue: https://github.com/apache/carbondata/pull/2578 LGTM ---
[GitHub] carbondata pull request #2578: [CARBONDATA-2798] Fix Dictionary_Include for ...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2578 ---
[GitHub] carbondata issue #2587: [CARBONDATA-2806] Delete delete delta files upon cle...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2587 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6090/ ---
[jira] [Resolved] (CARBONDATA-2788) Fix bugs in incorrect query result with bloom datamap
[ https://issues.apache.org/jira/browse/CARBONDATA-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Gupta resolved CARBONDATA-2788. -- Resolution: Fixed Fix Version/s: 1.4.1 > Fix bugs in incorrect query result with bloom datamap > - > > Key: CARBONDATA-2788 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2788 > Project: CarbonData > Issue Type: Sub-task >Reporter: xuchuanyin >Assignee: xuchuanyin >Priority: Major > Fix For: 1.4.1 > > Time Spent: 13h 20m > Remaining Estimate: 0h > > revert modification in PR2539 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2565: [HotFix][CARBONDATA-2788][BloomDataMap] Fix b...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2565 ---
[GitHub] carbondata issue #2587: [CARBONDATA-2806] Delete delete delta files upon cle...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2587 retest sdv please ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2589 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6089/ ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2589 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6088/ ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2589 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7695/ ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2589 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6421/ ---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2591 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6087/ ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2589 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7694/ ---
[GitHub] carbondata pull request #2544: [CARBONDATA-2776][CarbonStore] Support ingest...
Github user QiangCai closed the pull request at: https://github.com/apache/carbondata/pull/2544 ---
[GitHub] carbondata pull request #2591: [WIP][CARBONDATA-2808] Insert into select is ...
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2591#discussion_r206733033
--- Diff:
integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonScanRDD.scala
---
@@ -457,13 +458,25 @@ class CarbonScanRDD[T: ClassTag](
}
}
+ // TODO: rewrite this logic to call free memory in FailureListener
on failures. On success,
+ // no memory leak should be there, resources should be freed on
success completion.
+ // TODO: If CarbonRecordReader and VectorizedCarbonReader are called
directly
+ // from any other callers, then in the caller, resource clearing
should be taken care
+ val listeners =
CarbonReflectionUtils.getField("onCompleteCallbacks", context)
+.asInstanceOf[ArrayBuffer[TaskCompletionListener]]
+ val isAdded = listeners.exists(p =>
p.isInstanceOf[InsertTaskCompletionListener])
// add task completion before calling initialize as initialize
method will internally call
// for usage of unsafe method for processing of one blocklet and if
there is any exception
// while doing that the unsafe memory occupied for that task will
not get cleared
- context.addTaskCompletionListener { _ =>
-closeReader.apply()
-close()
-logStatistics(executionId, taskId, queryStartTime,
model.getStatisticsRecorder, split)
+ context.addTaskCompletionListener { new
QueryTaskCompletionListener(!isAdded,
--- End diff --
Remove the duplicated code which is copied to `QueryTaskCompletionListener`
from this class
---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2591 retest sdv please ---
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2572 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6086/ ---
[GitHub] carbondata issue #2567: [CARBONDATA-2790][BloomDataMap]Optimize default para...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2567 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7693/ ---
[GitHub] carbondata issue #2567: [CARBONDATA-2790][BloomDataMap]Optimize default para...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2567 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6419/ ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2571 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6418/ ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2571 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7692/ ---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2578 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6085/ ---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2591 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6417/ ---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2591 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7691/ ---
[GitHub] carbondata issue #2568: [Presto-integration-Technical-note] created document...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2568 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6413/ ---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2591 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6414/ ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2571 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6084/ ---
[GitHub] carbondata issue #2581: [CARBONDATA-2800][Doc] Add useful tips about bloomfi...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2581 retest sdv please ---
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2572 retest sdv please ---
[GitHub] carbondata issue #2565: [HotFix][CARBONDATA-2788][BloomDataMap] Fix bugs in ...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2565 retest sdv please ---
[GitHub] carbondata issue #2567: [CARBONDATA-2790][BloomDataMap]Optimize default para...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2567 retest this please ---
[GitHub] carbondata issue #2441: [CARBONDATA-2625] optimize CarbonReader performance
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2441 retest sdv please ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2571 retest sdv please ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2571 retest this please ---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2578 retest sdv please ---
[GitHub] carbondata issue #2579: [HOTFIX][PR 2575] Fixed modular plan creation only i...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2579 retest sdv please ---
[GitHub] carbondata issue #2585: [CARBONDATA-2805]fix the ordering mismatch of segmen...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2585 retest sdv please ---
[GitHub] carbondata issue #2583: [CARBONDATA-2803]fix wrong datasize calculation and ...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2583 retest sdv please ---
[GitHub] carbondata issue #2587: [CARBONDATA-2806] Delete delete delta files upon cle...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2587 retest sdv please ---
[GitHub] carbondata issue #2580: [CARBONDATA-2799][BloomDataMap] Fix bugs in querying...
Github user brijoobopanna commented on the issue: https://github.com/apache/carbondata/pull/2580 retest sdv please ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2589 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6416/ ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2589 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7690/ ---
[GitHub] carbondata issue #2590: [CARBONDATA-2750] Updated documentation on Local Dic...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2590 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6083/ ---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user dhatchayani commented on the issue: https://github.com/apache/carbondata/pull/2591 Retest this please ---
[GitHub] carbondata issue #2591: [WIP][CARBONDATA-2808] Insert into select is crashin...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2591 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7688/ ---
[GitHub] carbondata pull request #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
GitHub user jackylk reopened a pull request: https://github.com/apache/carbondata/pull/2589 [WIP][CARBONSTORE] Refactor CarbonStore API Interface of carbon prepared to integrate with Spark DatasourceV2 Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jackylk/incubator-carbondata store-parallelscan Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2589.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2589 commit 6d08a5261037c762ea7b5297a22840c00689e890 Author: Jacky Li Date: 2018-07-31T18:16:26Z modify CarbonStore API commit 63e2c2cba4221f70c788325277b47b93bbc94de3 Author: Jacky Li Date: 2018-07-31T18:39:45Z correct Horizon ---
[GitHub] carbondata pull request #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user jackylk closed the pull request at: https://github.com/apache/carbondata/pull/2589 ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2589 CTable is removed, now I have added new API in CarbonStore interface only ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2571 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6412/ ---
[GitHub] carbondata issue #2589: [WIP][CARBONSTORE] Refactor CarbonStore API
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2589 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7689/ ---
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2572 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6411/ ---
[GitHub] carbondata issue #2576: [CARBONDATA-2795] Add documentation for S3
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2576 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6410/ ---
[GitHub] carbondata issue #2577: [CARBONDATA-2796][32K]Fix data loading problem when ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2577 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6409/ ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2571 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6082/ ---
[GitHub] carbondata pull request #2591: [WIP][CARBONDATA-2808] Insert into select is ...
GitHub user dhatchayani opened a pull request: https://github.com/apache/carbondata/pull/2591 [WIP][CARBONDATA-2808] Insert into select is crashing as both are sharing the same task context Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dhatchayani/carbondata CARBONDATA-2808 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2591.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2591 commit 73254ac6de65bcf6c8ec9811bf3cf23512b2eb72 Author: dhatchayani Date: 2018-07-31T17:41:49Z [CARBONDATA-2808] Insert into select is crashing as both are sharing the same task context ---
[jira] [Created] (CARBONDATA-2808) Insert into select is crashing as both are sharing the same task context
dhatchayani created CARBONDATA-2808: --- Summary: Insert into select is crashing as both are sharing the same task context Key: CARBONDATA-2808 URL: https://issues.apache.org/jira/browse/CARBONDATA-2808 Project: CarbonData Issue Type: Bug Reporter: dhatchayani Assignee: dhatchayani Insert into select is failing as both are running as the same task and both are sharing the same taskcontext and resources are cleared once any one of the RDD's task is completed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2572 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7687/ ---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2578 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6408/ ---
[GitHub] carbondata issue #2590: [CARBONDATA-2750] Updated documentation on Local Dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2590 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6407/ ---
[GitHub] carbondata issue #2568: [Presto-integration-Technical-note] created document...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2568 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6081/ ---
[GitHub] carbondata issue #2567: [CARBONDATA-2790][BloomDataMap]Optimize default para...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2567 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6406/ ---
[GitHub] carbondata issue #2577: [CARBONDATA-2796][32K]Fix data loading problem when ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2577 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7685/ ---
[GitHub] carbondata issue #2544: [CARBONDATA-2776][CarbonStore] Support ingesting dat...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2544 merging into carbonstore branch ---
[GitHub] carbondata issue #2544: [CARBONDATA-2776][CarbonStore] Support ingesting dat...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2544 LGTM ---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2578 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7684/ ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2571 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7686/ ---
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2572 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6405/ ---
[GitHub] carbondata issue #2576: [CARBONDATA-2795] Add documentation for S3
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2576 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6080/ ---
[GitHub] carbondata pull request #2570: [CARBONDATA-2585]disable local dictionary by ...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2570 ---
[GitHub] carbondata issue #2576: [CARBONDATA-2795] Add documentation for S3
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2576 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6404/ ---
[GitHub] carbondata issue #2568: [Presto-integration-Technical-note] created document...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2568 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7682/ ---
[GitHub] carbondata pull request #2559: [CARBONDATA-2606][Complex DataType Enhancemen...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2559 ---
[GitHub] carbondata issue #2590: [CARBONDATA-2750] Updated documentation on Local Dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2590 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7683/ ---
[GitHub] carbondata issue #2577: [CARBONDATA-2796][32K]Fix data loading problem when ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2577 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6403/ ---
[GitHub] carbondata issue #2578: [CARBONDATA-2798] Fix Dictionary_Include for Complex...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2578 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6402/ ---
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2572 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7680/ ---
[GitHub] carbondata issue #2580: [CARBONDATA-2799][BloomDataMap] Fix bugs in querying...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2580 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6401/ ---
[GitHub] carbondata issue #2576: [CARBONDATA-2795] Add documentation for S3
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2576 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7679/ ---
[GitHub] carbondata issue #2571: [CARBONDATA-2792][schema restructure] Create externa...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2571 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7681/ ---
[GitHub] carbondata pull request #2590: [CARBONDATA-2750] Updated documentation on Lo...
Github user chenliang613 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2590#discussion_r206553068 --- Diff: docs/data-management-on-carbondata.md --- @@ -126,20 +126,33 @@ This tutorial is going to introduce all commands and data operations on CarbonDa - **Local Dictionary Configuration** - Local Dictionary is generated only for no-dictionary string/varchar datatype columns. It helps in: + Local Dictionary is generated only for string/varchar datatype columns which are not included in dictionary include. It helps in: 1. Getting more compression on dimension columns with less cardinality. 2. Filter queries and full scan queries on No-dictionary columns with local dictionary will be faster as filter will be done on encoded data. 3. Reducing the store size and memory footprint as only unique values will be stored as part of local dictionary and corresponding data will be stored as encoded data. - - By default, Local Dictionary will be enabled and generated for all no-dictionary string/varchar datatype columns. + + **Bottleneck for Local Dictionary:** The memory size will increase when local dictionary is enabled. --- End diff -- Please change "bottleneck" to "The cost" ---
[GitHub] carbondata issue #2582: [CARBONDATA-2801]Added documentation for flat folder
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2582 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6079/ ---
[GitHub] carbondata issue #2582: [CARBONDATA-2801]Added documentation for flat folder
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2582 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7678/ ---
[GitHub] carbondata pull request #2552: [CARBONDATA-2781] Added fix for Null Pointer ...
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2552#discussion_r206546864
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/datamap/CarbonCreateDataMapCommand.scala
---
@@ -181,7 +181,18 @@ case class CarbonCreateDataMapCommand(
override def undoMetadata(sparkSession: SparkSession, exception:
Exception): Seq[Row] = {
if (dataMapProvider != null) {
- dataMapProvider.cleanMeta()
+ if (dataMapSchema.getProviderName
--- End diff --
My suggestion is `CarbonDropDataMapCommand` should be called for all
datamaps not just for pre-aggregate
---
[jira] [Resolved] (CARBONDATA-2801) Add documentation for flat folder
[ https://issues.apache.org/jira/browse/CARBONDATA-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen resolved CARBONDATA-2801. Resolution: Fixed Fix Version/s: 1.4.1 1.5.0 > Add documentation for flat folder > - > > Key: CARBONDATA-2801 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2801 > Project: CarbonData > Issue Type: Improvement > Components: docs >Reporter: Ravindra Pesala >Assignee: Ravindra Pesala >Priority: Major > Fix For: 1.5.0, 1.4.1 > > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2581: [CARBONDATA-2800][Doc] Add useful tips about bloomfi...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2581 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6400/ ---
[jira] [Assigned] (CARBONDATA-2801) Add documentation for flat folder
[ https://issues.apache.org/jira/browse/CARBONDATA-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen reassigned CARBONDATA-2801: -- Assignee: Ravindra Pesala > Add documentation for flat folder > - > > Key: CARBONDATA-2801 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2801 > Project: CarbonData > Issue Type: Improvement > Components: docs >Reporter: Ravindra Pesala >Assignee: Ravindra Pesala >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2582: [CARBONDATA-2801]Added documentation for flat...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2582 ---
[GitHub] carbondata issue #2585: [CARBONDATA-2805]fix the ordering mismatch of segmen...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2585 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6398/ ---
[GitHub] carbondata issue #2582: [CARBONDATA-2801]Added documentation for flat folder
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2582 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6399/ ---
[GitHub] carbondata issue #2590: [CARBONDATA-2750] Updated documentation on Local Dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2590 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7675/ ---
[GitHub] carbondata issue #2572: [CARBONDATA-2793][32k][Doc] Add 32k support in docum...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2572 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7677/ ---
[GitHub] carbondata pull request #2578: [CARBONDATA-2798] Fix Dictionary_Include for ...
Github user kunal642 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2578#discussion_r206531160
--- Diff:
integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/createTable/TestNonTransactionalCarbonTable.scala
---
@@ -52,10 +52,11 @@ import
org.apache.carbondata.core.datastore.page.encoding.DefaultEncodingFactory
import org.apache.carbondata.core.metadata.ColumnarFormatVersion
import org.apache.carbondata.core.metadata.datatype.DataTypes
import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil,
DataFileFooterConverterV3}
+import
org.apache.carbondata.processing.loading.exception.CarbonDataLoadingException
import org.apache.carbondata.sdk.file._
-class TestNonTransactionalCarbonTable extends QueryTest with
BeforeAndAfterAll {
+class TestNonTransactiogitnalCarbonTable extends QueryTest with
BeforeAndAfterAll {
--- End diff --
Please revert this change
---
[GitHub] carbondata pull request #2571: [CARBONDATA-2792][schema restructure] Create ...
Github user mohammadshahidkhan commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2571#discussion_r206525134
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/util/CarbonSparkUtil.scala
---
@@ -87,18 +88,20 @@ object CarbonSparkUtil {
val fields = new Array[String](
carbonRelation.dimensionsAttr.size + carbonRelation.measureAttr.size)
val carbonTable = carbonRelation.carbonTable
+val columnSchemas: mutable.Buffer[ColumnSchema] =
carbonTable.getTableInfo.getFactTable.
+ getListOfColumns.asScala.filter(_.getSchemaOrdinal !=
-1).filter(!_.isInvisible).
--- End diff --
@kunal642 Changed the filter condition from && to ||, the earlier filters
were serving the OR.
---
[GitHub] carbondata pull request #2552: [CARBONDATA-2781] Added fix for Null Pointer ...
Github user praveenmeenakshi56 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2552#discussion_r206521693
--- Diff:
integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/datamap/CarbonCreateDataMapCommand.scala
---
@@ -181,7 +181,18 @@ case class CarbonCreateDataMapCommand(
override def undoMetadata(sparkSession: SparkSession, exception:
Exception): Seq[Row] = {
if (dataMapProvider != null) {
- dataMapProvider.cleanMeta()
+ if (dataMapSchema.getProviderName
--- End diff --
We need this check because we want the implementation of cleanMeta() method
in PreAggregregateDataMapProvider. For others, It will take from
IndexDataMapProvider
---
[GitHub] carbondata issue #2567: [CARBONDATA-2790][BloomDataMap]Optimize default para...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2567 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7674/ ---
[GitHub] carbondata issue #2590: [CARBONDATA-2750] Updated documentation on Local Dic...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2590 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6396/ ---
[GitHub] carbondata issue #2577: [CARBONDATA-2796][32K]Fix data loading problem when ...
Github user kevinjmh commented on the issue: https://github.com/apache/carbondata/pull/2577 retest this please ---
[GitHub] carbondata issue #2590: [CARBONDATA-2750] Updated documentation on Local Dic...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2590 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6078/ ---
[GitHub] carbondata pull request #2576: [CARBONDATA-2795] Add documentation for S3
Github user KanakaKumar commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2576#discussion_r206515601 --- Diff: docs/data-management-on-carbondata.md --- @@ -730,6 +736,8 @@ Users can specify which columns to include and exclude for local dictionary gene * If the IGNORE option is used, then bad records are neither loaded nor written to the separate CSV file. * In loaded data, if all records are bad records, the BAD_RECORDS_ACTION is invalid and the load operation fails. * The maximum number of characters per column is 32000. If there are more than 32000 characters in a column, data loading will fail. + * Since Bad Records Path can be specified in both create, load and carbon properties. --- End diff -- "both" does not suite in this statement. Please rewrite. ---
[GitHub] carbondata issue #2582: [CARBONDATA-2801]Added documentation for flat folder
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2582 LGTM ---
