[GitHub] carbondata issue #2163: [CARBONDATA-2311][Streaming] Fix bug to avoid to app...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2163 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4956/ ---
[GitHub] carbondata issue #2163: [CARBONDATA-2311][Streaming] Fix bug to avoid to app...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2163 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4413/ ---
[GitHub] carbondata issue #2163: [CARBONDATA-2311][Streaming] Fix bug to avoid to app...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2163 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4955/ ---
[GitHub] carbondata issue #2163: [CARBONDATA-2311][Streaming] Fix bug to avoid to app...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2163 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4412/ ---
[GitHub] carbondata pull request #2163: [CARBONDATA-2311][Streaming] Fix bug to avoid...
GitHub user QiangCai opened a pull request: https://github.com/apache/carbondata/pull/2163 [CARBONDATA-2311][Streaming] Fix bug to avoid to append data to strea⦠At the begin of each micro batch, check the status of current segment.if the status is streaming, continue to use this segment if the status is streaming finish, open new streaming segment to accept new streaming data This closes #2135 Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/QiangCai/carbondata pr2135_branch1_3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2163.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2163 commit 2f2bb3e9ef2b28004b5f245517d1af91f56da944 Author: QiangCaiDate: 2018-04-03T06:32:59Z [CARBONDATA-2311][Streaming] Fix bug to avoid to append data to streaming finish segment At the begin of each micro batch, check the status of current segment.if the status is streaming, continue to use this segment if the status is streaming finish, open new streaming segment to accept new streaming data This closes #2135 ---
[GitHub] carbondata issue #2162: [CARBONDATA-2337] Fix duplicately acquiring 'streami...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2162 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4411/ ---
[GitHub] carbondata issue #2162: [CARBONDATA-2337] Fix duplicately acquiring 'streami...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2162 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4954/ ---
[GitHub] carbondata pull request #2162: [CARBONDATA-2337] Fix duplicately acquiring '...
GitHub user zzcclp opened a pull request: https://github.com/apache/carbondata/pull/2162 [CARBONDATA-2337] Fix duplicately acquiring 'streaming.lock' error when integrating with spark-streaming. After merged [PR2135](https://github.com/apache/carbondata/pull/2135), it will acquire 'streaming.lock' duplicately when integrating with spark-streaming. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? No - [ ] Any backward compatibility impacted? No - [ ] Document update required? No - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zzcclp/carbondata CARBONDATA-2337 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2162.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2162 commit c491e4e927840016f0279e2c82959c9d2e2d8b19 Author: Zhang Zhichao <441586683@...> Date: 2018-04-12T04:24:45Z [CARBONDATA-2337] Fix duplicately acquiring 'streaming.lock' error when integrating with spark-streaming. After merged [PR2135|https://github.com/apache/carbondata/pull/2135], it will acquire 'streaming.lock' duplicately when integrating with spark-streaming. ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2161 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4410/ ---
[GitHub] carbondata pull request #2127: [CARBONDATA-2301][SDK] CarbonStore interface ...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2127 ---
[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2127 LGTM ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2161 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4953/ ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2161 @chandrasaripaka Can you provide more detail about this pull request. ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2161 add to whitelist ---
[GitHub] carbondata issue #2152: [CARBONDATA-2155][CARBONDATA-2152] [Presto] Fixed IS...
Github user anubhav100 commented on the issue: https://github.com/apache/carbondata/pull/2152 @chenliang613 please review ---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2141 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4409/ ---
[GitHub] carbondata issue #2141: [CARBONDATA-2313] Fixed SDK writer issues and added ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2141 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4952/ ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2161 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2161 Can one of the admins verify this patch? ---
[GitHub] carbondata issue #2161: Adding Alluxio Supporty
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2161 Can one of the admins verify this patch? ---
[GitHub] carbondata pull request #2161: Adding Alluxio Supporty
GitHub user chandrasaripaka opened a pull request: https://github.com/apache/carbondata/pull/2161 Adding Alluxio Supporty Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [No ] Any interfaces changed? - [No ] Any backward compatibility impacted? - [Yes ] Document update required? - [Partially ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? It is the support for alluxio that has been fixed. - How it is tested? Please attach test report. It is tested with external alluxio. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. Added Alluxio Support , as it is not reflecting the correct FileSystem. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/DataHeaps/carbondata alluxio-support Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2161.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2161 commit 35ce7550954fbc8d9211352b79876c1ee148b609 Author: pourDate: 2018-03-22T08:00:38Z Adding Alluxio Supporty ---
[GitHub] carbondata issue #2160: [CARBONDATA-2335] Fix auto handoff for preaggregate ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2160 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4408/ ---
[GitHub] carbondata issue #2160: [CARBONDATA-2335] Fix auto handoff for preaggregate ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2160 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4951/ ---
[GitHub] carbondata issue #2160: [CARBONDATA-2335] Fix auto handoff for preaggregate ...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2160 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4407/ ---
[jira] [Updated] (CARBONDATA-2337) Fix duplicately acquiring 'streaming.lock' error when integrating with spark-streaming
[ https://issues.apache.org/jira/browse/CARBONDATA-2337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhichao Zhang updated CARBONDATA-2337: --- Description: After merged [PR2135|[https://github.com/apache/carbondata/pull/2135]] it will acquire 'streaming.lock' duplicately when integrating with spark-streaming. (was: After merged [PR2135|[https://github.com/apache/carbondata/pull/2135],] it will acquire 'streaming.lock' duplicately when integrating with spark-streaming.) > Fix duplicately acquiring 'streaming.lock' error when integrating with > spark-streaming > -- > > Key: CARBONDATA-2337 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2337 > Project: CarbonData > Issue Type: Bug > Components: spark-integration >Affects Versions: 1.4.0 >Reporter: Zhichao Zhang >Assignee: Zhichao Zhang >Priority: Minor > Fix For: 1.4.0 > > > After merged [PR2135|[https://github.com/apache/carbondata/pull/2135]] it > will acquire 'streaming.lock' duplicately when integrating with > spark-streaming. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CARBONDATA-2337) Fix duplicately acquiring 'streaming.lock' error when integrating with spark-streaming
Zhichao Zhang created CARBONDATA-2337: -- Summary: Fix duplicately acquiring 'streaming.lock' error when integrating with spark-streaming Key: CARBONDATA-2337 URL: https://issues.apache.org/jira/browse/CARBONDATA-2337 Project: CarbonData Issue Type: Bug Components: spark-integration Affects Versions: 1.4.0 Reporter: Zhichao Zhang Assignee: Zhichao Zhang Fix For: 1.4.0 After merged [PR2135|[https://github.com/apache/carbondata/pull/2135],] it will acquire 'streaming.lock' duplicately when integrating with spark-streaming. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2159 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4406/ ---
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2159 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4405/ ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2128 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4404/ ---
[jira] [Resolved] (CARBONDATA-2267) Implement Reading Of Carbon Partition From Presto
[ https://issues.apache.org/jira/browse/CARBONDATA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang Chen resolved CARBONDATA-2267. Resolution: Fixed Fix Version/s: 1.4.0 > Implement Reading Of Carbon Partition From Presto > - > > Key: CARBONDATA-2267 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2267 > Project: CarbonData > Issue Type: New Feature >Reporter: anubhav tarar >Assignee: anubhav tarar >Priority: Trivial > Fix For: 1.4.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (CARBONDATA-2336) SecureDictionaryServer#shutdown should cover TransportServer
Ted Yu created CARBONDATA-2336: -- Summary: SecureDictionaryServer#shutdown should cover TransportServer Key: CARBONDATA-2336 URL: https://issues.apache.org/jira/browse/CARBONDATA-2336 Project: CarbonData Issue Type: Bug Reporter: Ted Yu In bindToPort : {code} TransportServer transportServer = context .createServer(host, port, Lists.newArrayList(bootstrap)); {code} The transportServer should be covered in {{shutdown}} as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2160: [CARBONDATA-2335] Fix auto handoff for preaggregate ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2160 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3733/ ---
[GitHub] carbondata pull request #2139: [CARBONDATA-2267] [Presto] Support Reading Ca...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2139 ---
[GitHub] carbondata issue #2139: [CARBONDATA-2267] [Presto] Support Reading CarbonDat...
Github user chenliang613 commented on the issue: https://github.com/apache/carbondata/pull/2139 LGTM ---
[GitHub] carbondata pull request #2139: [CARBONDATA-2267] [Presto] Support Reading Ca...
Github user chenliang613 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2139#discussion_r180789005 --- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/PrestoFilterUtil.java --- @@ -69,6 +72,13 @@ private static MapfilterMap = new HashMap<>(); + private final static String HIVE_DEFAULT_DYNAMIC_PARTITION = "__HIVE_DEFAULT_PARTITION__"; --- End diff -- ok ---
[GitHub] carbondata issue #2160: [CARBONDATA-2335] Fix auto handoff for preaggregate ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2160 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3732/ ---
[GitHub] carbondata pull request #2160: [CARBONDATA-2335] Fix auto handoff for preagg...
GitHub user kunal642 opened a pull request: https://github.com/apache/carbondata/pull/2160 [CARBONDATA-2335] Fix auto handoff for preaggregate table with streaming Solution: MetaListener was not called for auto handoff case therefore child load commands were not being created thus throwing NullPointerException. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kunal642/carbondata fix_auto_handoff_aggregate_load Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2160.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2160 commit f6bc57708f8d3da77d547640ea373e4e5c9b970a Author: kunal642Date: 2018-04-11T14:56:01Z fix auto handoff for preaggregate table with streaming ---
[jira] [Created] (CARBONDATA-2335) Autohandoff is failing when preaggregate is created on streaming table
Kunal Kapoor created CARBONDATA-2335: Summary: Autohandoff is failing when preaggregate is created on streaming table Key: CARBONDATA-2335 URL: https://issues.apache.org/jira/browse/CARBONDATA-2335 Project: CarbonData Issue Type: Bug Reporter: Kunal Kapoor Assignee: Kunal Kapoor Auto hand off is failing with NullPointerException when preaggregate table is present in the streaming table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2128 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4403/ ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user gvramana commented on the issue: https://github.com/apache/carbondata/pull/2128 LGTM ---
[jira] [Resolved] (CARBONDATA-2303) If dataload is failed for parition table then cleanup is not working.
[ https://issues.apache.org/jira/browse/CARBONDATA-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Gupta resolved CARBONDATA-2303. -- Resolution: Fixed Assignee: Rahul Kumar Fix Version/s: 1.3.2 1.4.0 > If dataload is failed for parition table then cleanup is not working. > - > > Key: CARBONDATA-2303 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2303 > Project: CarbonData > Issue Type: Bug >Reporter: Rahul Kumar >Assignee: Rahul Kumar >Priority: Major > Fix For: 1.4.0, 1.3.2 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > Test Step : > 1. create table > 2. load data (make sure data load is failed either manually or other) > 3. clean files for table > *Expected Output*: after clean files data from HDFS should be delete for > segments which is Marked for delete. > *Actual Output:* Alter cleanup ,data are not deleted from HDFS > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2159: [CARBONDATA-2303] clean files issue resolved ...
Github user rahulforallp closed the pull request at: https://github.com/apache/carbondata/pull/2159 ---
[GitHub] carbondata pull request #2158: [CARBONDATA-2316] Executor task is failed but...
Github user rahulforallp closed the pull request at: https://github.com/apache/carbondata/pull/2158 ---
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2159 LGTM ---
[GitHub] carbondata issue #2158: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2158 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4402/ ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2128 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4401/ ---
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2159 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3731/ ---
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2159 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4950/ ---
[jira] [Resolved] (CARBONDATA-2316) Even though one of the Compaction task failed at executor. All the executor task is showing success in UI and Job fails from driver.
[ https://issues.apache.org/jira/browse/CARBONDATA-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Gupta resolved CARBONDATA-2316. -- Resolution: Fixed Fix Version/s: 1.3.2 1.4.0 > Even though one of the Compaction task failed at executor. All the executor > task is showing success in UI and Job fails from driver. > > > Key: CARBONDATA-2316 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2316 > Project: CarbonData > Issue Type: Improvement >Reporter: Rahul Kumar >Assignee: Rahul Kumar >Priority: Major > Fix For: 1.4.0, 1.3.2 > > Time Spent: 4h > Remaining Estimate: 0h > > *At UI:* > *!http://dts.huawei.com/net/dts/fckeditor/download.ashx?Path=9wrYLnPvC4cVkbqrwtlHFbeC2%2fnVXKrZXwY7%2bglgnCQfywSBPcodImAcj29S40n0M8Rx%2bHUEMVQ%3d!* > *At Executor :* > !http://dts.huawei.com/net/dts/fckeditor/download.ashx?Path=9wrYLnPvC4cVkbqrwtlHFbeC2%2fnVXKrZXwY7%2bglgnCQfywSBPcodInHgG%2fMfoX2avMU6b0kZ9Jo%3d! > > *At driver* > !http://dts.huawei.com/net/dts/fckeditor/download.ashx?Path=9wrYLnPvC4cVkbqrwtlHFbeC2%2fnVXKrZXwY7%2bglgnCQfywSBPcodIruNtISaQlvSdxWgQJLSrNA%3d! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2158: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2158 LGTM ---
[GitHub] carbondata pull request #2142: [CARBONDATA-2316] Executor task is failed but...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2142 ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2128 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4949/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2142 LGTM ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2128 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3730/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2142 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4400/ ---
[GitHub] carbondata pull request #2128: [CARBONDATA-2303] If dataload is failed for p...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2128 ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2128 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3728/ ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user rahulforallp commented on the issue: https://github.com/apache/carbondata/pull/2128 retest this please ---
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2159 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3729/ ---
[GitHub] carbondata issue #2158: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2158 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4945/ ---
[GitHub] carbondata issue #2158: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2158 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3726/ ---
[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2113 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4399/ ---
[GitHub] carbondata issue #2159: [CARBONDATA-2303] clean files issue resolved for par...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2159 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4948/ ---
[GitHub] carbondata pull request #2159: [CARBONDATA-2303] clean files issue resolved ...
GitHub user rahulforallp opened a pull request: https://github.com/apache/carbondata/pull/2159 [CARBONDATA-2303] clean files issue resolved for partition folder Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? No - [ ] Any backward compatibility impacted? No - [ ] Document update required? No - [ ] Testing done NA - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/rahulforallp/incubator-carbondata synch_part_tab_clean Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2159.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2159 commit 506f274e57bf0c57b65ccd869e960e88c6824db0 Author: rahulforallpDate: 2018-04-01T12:08:51Z [CARBONDATA-2303] clean files issue resolved for partition folder ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2128 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4947/ ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2128 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4946/ ---
[GitHub] carbondata issue #2128: [CARBONDATA-2303] If dataload is failed for parition...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2128 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3727/ ---
[GitHub] carbondata issue #2073: [CARBONDATA-2260] CarbonThriftServer should support ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2073 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4943/ ---
[GitHub] carbondata issue #2073: [CARBONDATA-2260] CarbonThriftServer should support ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2073 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3724/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2142 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3723/ ---
[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2127 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4398/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2142 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4942/ ---
[jira] [Commented] (CARBONDATA-2279) presto handle empty carbondata file
[ https://issues.apache.org/jira/browse/CARBONDATA-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433679#comment-16433679 ] Geetika Gupta commented on CARBONDATA-2279: --- @zhangwei please provide more details to replicate your scenario as I tried it on my machine, it works there. > presto handle empty carbondata file > > > Key: CARBONDATA-2279 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2279 > Project: CarbonData > Issue Type: Bug > Components: core >Affects Versions: 1.3.0 >Reporter: zhangwei >Assignee: Geetika Gupta >Priority: Major > Fix For: 1.3.0 > > Attachments: outofindex.patch > > > Caused by: java.lang.RuntimeException: Error creating Splits from > CarbonTableInputFormat > at > org.apache.carbondata.presto.impl.CarbonTableReader.getInputSplits2(CarbonTableReader.java:433) > at > org.apache.carbondata.presto.CarbondataSplitManager.getSplits(CarbondataSplitManager.java:75) > ... 22 more > Caused by: java.io.IOException: Problem in loading segment blocks. > at > org.apache.carbondata.core.indexstore.BlockletDataMapIndexStore.getAll(BlockletDataMapIndexStore.java:153) > at > org.apache.carbondata.core.indexstore.blockletindex.BlockletDataMapFactory.getDataMaps(BlockletDataMapFactory.java:76) > at > org.apache.carbondata.core.datamap.TableDataMap.prune(TableDataMap.java:95) > at > org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getDataBlocksOfSegment(CarbonTableInputFormat.java:748) > at > org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:679) > at > org.apache.carbondata.hadoop.api.CarbonTableInputFormat.getSplits(CarbonTableInputFormat.java:439) > at > org.apache.carbondata.presto.impl.CarbonTableReader.getInputSplits2(CarbonTableReader.java:417) > ... 23 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: 0 > at > org.apache.carbondata.core.datastore.filesystem.AbstractDFSCarbonFile.getLocations(AbstractDFSCarbonFile.java:514) > at > org.apache.carbondata.core.indexstore.BlockletDataMapIndexStore.getAll(BlockletDataMapIndexStore.java:142) > ... 29 more -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata issue #2157: [CARBONDATA-2334] Added Property enabling user to bl...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2157 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4941/ ---
[GitHub] carbondata issue #2157: [CARBONDATA-2334] Added Property enabling user to bl...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2157 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3722/ ---
[GitHub] carbondata pull request #2073: [CARBONDATA-2260] CarbonThriftServer should s...
GitHub user xubo245 reopened a pull request: https://github.com/apache/carbondata/pull/2073 [CARBONDATA-2260] CarbonThriftServer should support store carbon table on S3 CarbonThriftServer should support store carbon table on S3 Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [x] Any interfaces changed? - [x] Any backward compatibility impacted? - [x] Document update required? - [x] Testing done Step 1: start carbonThriftServer by org.apache.carbondata.examples.CarbonThriftServerExample User should use themselves storePath, AK, SK and endPoint Step 2: start beeline like: beeline -u jdbc:hive2://IP:1 IP and port should provide Now it support running with local, HDFS and S3 mode Test example: ``` root@ecs-909c:~# beeline -u jdbc:hive2://192.168.0.206:1 Connecting to jdbc:hive2://192.168.0.206:1 2018-03-17 17:44:29 INFO Utils:310 - Supplied authorities: 192.168.0.206:1 2018-03-17 17:44:29 INFO Utils:397 - Resolved authority: 192.168.0.206:1 2018-03-17 17:44:29 INFO HiveConnection:203 - Will try to open client transport with JDBC Uri: jdbc:hive2://192.168.0.206:1 Connected to: Spark SQL (version 2.2.1) Driver: Hive JDBC (version 1.2.1.spark2) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 1.2.1.spark2 by Apache Hive 0: jdbc:hive2://192.168.0.206:1> Drop table if exists local_table2; +-+--+ | Result | +-+--+ +-+--+ No rows selected (21.205 seconds) 0: jdbc:hive2://192.168.0.206:1> 0: jdbc:hive2://192.168.0.206:1> CREATE TABLE if not exists local_table2( shortField SHORT, intField INT, bigintField LONG, doubleField DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT ) STORED BY 'carbondata' LOCATION 's3a://carbonstore/local_table2' TBLPROPERTIES('SORT_COLUMNS'=''); +-+--+ | Result | +-+--+ +-+--+ No rows selected (1.296 seconds) 0: jdbc:hive2://192.168.0.206:1> 0: jdbc:hive2://192.168.0.206:1> 0: jdbc:hive2://192.168.0.206:1> LOAD DATA LOCAL INPATH '/huawei/xubo/git/carbondata2/examples/spark2/src/main/resources/data1.csv' INTO TABLE local_table2 OPTIONS('HEADER'='true'); +-+--+ | Result | +-+--+ +-+--+ No rows selected (3.339 seconds) 0: jdbc:hive2://192.168.0.206:1> 0: jdbc:hive2://192.168.0.206:1> select * from local_table2; +-+---+--+--+--++---+-++-+--+ | shortfield | intfield | bigintfield | doublefield | stringfield | timestampfield | decimalfield | datefield | charfield | floatfield | +-+---+--+--+--++---+-++-+--+ | 1 | 10| 1100 | 48.4 | spark| 2015-04-23 12:01:01.0 | 1.23 | 2015-04-23 | aaa| 2.5 | | 5 | 17| 1140 | 43.4 | spark| 2015-07-27 12:01:02.0 | 3.45 | 2015-07-27 | bbb| 2.5 | | 1 | 11| 1100 | 44.4 | flink| 2015-05-23 12:01:03.0 | 23.23 | 2015-05-23 | ccc| 2.5 | | 1 | 10| 1150 | 43.4 | spark| 2015-07-24 12:01:04.0 | 254.12| 2015-07-24 | ddd| 2.5 | | 1 | 10| 1100 | 47.4 | spark| 2015-07-23 12:01:05.0 | 876.14| 2015-07-23 | | 3.5 | | 3 | 14| 1160 | 43.4 | hive | 2015-07-26 12:01:06.0 | 3454.32 | 2015-07-26 | ff | 2.5 | | 2 | 10| 1100 | 43.4 | impala | 2015-07-23 12:01:07.0 | 456.98| 2015-07-23 | ggg| 2.5 | | 1 | 10| 1100 | 43.4 | spark| 2015-05-23 12:01:08.0 | 32.53 | 2015-05-23 | hhh| 2.5 | | 4 | 16| 1130 | 42.4 | impala | 2015-07-23 12:01:09.0 | 67.23 | 2015-07-23 | iii| 2.5 | | 1 | 10| 1100 | 43.4 | spark| 2015-07-23 12:01:10.0 | 832.23| 2015-07-23 | jjj| 2.5 |
[GitHub] carbondata issue #2139: [CARBONDATA-2267] [Presto] Support Reading CarbonDat...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2139 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3721/ ---
[GitHub] carbondata issue #2139: [CARBONDATA-2267] [Presto] Support Reading CarbonDat...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2139 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4940/ ---
[GitHub] carbondata issue #2154: [CARBONDATA-2329] Non Serializable extra info in ses...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2154 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4397/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2142 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4939/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2142 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3720/ ---
[jira] [Resolved] (CARBONDATA-2329) Non Serializable extra info in session is overwritten by values from thread
[ https://issues.apache.org/jira/browse/CARBONDATA-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Gupta resolved CARBONDATA-2329. -- Resolution: Fixed Fix Version/s: 1.3.2 1.4.0 > Non Serializable extra info in session is overwritten by values from thread > --- > > Key: CARBONDATA-2329 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2329 > Project: CarbonData > Issue Type: Bug > Components: core, spark-integration >Affects Versions: 1.3.0 >Reporter: Kanaka Kumar Avvaru >Assignee: Kanaka Kumar Avvaru >Priority: Major > Fix For: 1.4.0, 1.3.2 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > 1. Non Serializable extra info is copied from thread which causes stale data > from old session when the thread is reused by spark. > 2. CarboSessionInfo clone is not copying Non serializable info to new object > which can damage session level values if local query thread updates values. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2154: [CARBONDATA-2329] Non Serializable extra info...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2154 ---
[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2113 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3719/ ---
[GitHub] carbondata issue #2113: [WIP][LUCENE_DATAMAP]load issue in lucene datamap, m...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2113 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4938/ ---
[GitHub] carbondata issue #2154: [CARBONDATA-2329] Non Serializable extra info in ses...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2154 LGTM ---
[GitHub] carbondata pull request #2157: [CARBONDATA-2334] Added Property enabling use...
GitHub user praveenmeenakshi56 opened a pull request: https://github.com/apache/carbondata/pull/2157 [CARBONDATA-2334] Added Property enabling user to block partitioning of Pre-Aggregate t⦠While creating PreAgg table, in DMPROPERTIES, 'partitioning'='false' will disable partitioning of the Pre-Aggregate table - [ ] Any interfaces changed? NA - [ ] Any backward compatibility impacted? NA - [ ] Document update required? NA - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. NA - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. NA You can merge this pull request into a Git repository by running: $ git pull https://github.com/praveenmeenakshi56/carbondata part_block Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2157.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2157 commit ac8e4db6fd1371ac0f28d0a9dc0b964de6a4e1d9 Author: praveenmeenakshi56Date: 2018-04-11T08:43:14Z Added Property enabling user to block partitioning of Pre-Aggregate table ---
[jira] [Created] (CARBONDATA-2334) Add property to enable users to block partitioning of Pre-Aggregate table
Praveen M P created CARBONDATA-2334: --- Summary: Add property to enable users to block partitioning of Pre-Aggregate table Key: CARBONDATA-2334 URL: https://issues.apache.org/jira/browse/CARBONDATA-2334 Project: CarbonData Issue Type: Improvement Reporter: Praveen M P Assignee: Praveen M P -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (CARBONDATA-2321) Selecton after a Concurrent Load Failing for Partition columns
[ https://issues.apache.org/jira/browse/CARBONDATA-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manish Gupta resolved CARBONDATA-2321. -- Resolution: Fixed Fix Version/s: 1.3.2 > Selecton after a Concurrent Load Failing for Partition columns > -- > > Key: CARBONDATA-2321 > URL: https://issues.apache.org/jira/browse/CARBONDATA-2321 > Project: CarbonData > Issue Type: Bug > Components: core >Affects Versions: 1.4.0 > Environment: Spark-2.1 >Reporter: Jatin >Assignee: Jatin >Priority: Minor > Fix For: 1.4.0, 1.3.2 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > selection after a Concurrent load fails randomly for partition column. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2146: [CARBONDATA-2321] Fix for selection of partio...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2146 ---
[GitHub] carbondata issue #2146: [CARBONDATA-2321] Fix for selection of partion colum...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2146 LGTM ---
[GitHub] carbondata pull request #2155: [CARBONDATA-2321] Fix selection of partition ...
Github user jatin9896 closed the pull request at: https://github.com/apache/carbondata/pull/2155 ---
[GitHub] carbondata issue #2155: [CARBONDATA-2321] Fix selection of partition column ...
Github user manishgupta88 commented on the issue: https://github.com/apache/carbondata/pull/2155 LGTM ---
[GitHub] carbondata pull request #2139: [CARBONDATA-2267] [Presto] Support Reading Ca...
Github user geetikagupta16 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2139#discussion_r180672102 --- Diff: integration/presto/src/main/java/org/apache/carbondata/presto/PrestoFilterUtil.java --- @@ -69,6 +72,13 @@ private static MapfilterMap = new HashMap<>(); + private final static String HIVE_DEFAULT_DYNAMIC_PARTITION = "__HIVE_DEFAULT_PARTITION__"; + private final static String PARTITION_VALUE_WILDCARD = ""; --- End diff -- In hive PARTITION_VALUE_WILDCARD is used to select all the partitions from the table when we don't have any filter conditions on the partition column in our query, but in our code we are already quering all the partitions for the above case. So we can remove this part. ---
[GitHub] carbondata pull request #2156: [CARBONDATA-2317] Concurrent datamap with sam...
Github user rahulforallp closed the pull request at: https://github.com/apache/carbondata/pull/2156 ---
[GitHub] carbondata issue #2154: [CARBONDATA-2329] Non Serializable extra info in ses...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2154 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3717/ ---
[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and two...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2127 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3718/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2142 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4396/ ---
[GitHub] carbondata issue #2156: [CARBONDATA-2317] Concurrent datamap with same name ...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2156 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3715/ ---
[GitHub] carbondata issue #2142: [CARBONDATA-2316] Executor task is failed but UI sho...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2142 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3716/ ---