[jira] [Resolved] (CARBONDATA-463) Extract spark-common module

2016-11-29 Thread Ravindra Pesala (JIRA)
[ https://issues.apache.org/jira/browse/CARBONDATA-463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala resolved CARBONDATA-463. Resolution: Fixed > Extract spark-common module > --- >

[GitHub] incubator-carbondata issue #365: [CARBONDATA-463] Extract code to spark-comm...

2016-11-29 Thread ravipesala
Github user ravipesala commented on the issue: https://github.com/apache/incubator-carbondata/pull/365 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[jira] [Commented] (CARBONDATA-361) SMALL INT data showing unsupported datatype

2016-11-29 Thread SWATI RAO (JIRA)
[ https://issues.apache.org/jira/browse/CARBONDATA-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707814#comment-15707814 ] SWATI RAO commented on CARBONDATA-361: -- [~ravi.pesala] : As mentioned on your wiki page which is

[GitHub] incubator-carbondata pull request #371: [CARBONDATA-472] Added unit test for...

2016-11-29 Thread kunal642
GitHub user kunal642 opened a pull request: https://github.com/apache/incubator-carbondata/pull/371 [CARBONDATA-472] Added unit test for cache package Added unit test for CarbonLRUCacheTest Added unit test for ColumnDictionaryChunkIteratorTest Added unit test for

[jira] [Created] (CARBONDATA-472) Improve code coverage for core.cache package.

2016-11-29 Thread SWATI RAO (JIRA)
SWATI RAO created CARBONDATA-472: Summary: Improve code coverage for core.cache package. Key: CARBONDATA-472 URL: https://issues.apache.org/jira/browse/CARBONDATA-472 Project: CarbonData

[GitHub] incubator-carbondata pull request #370: [CARBONDATA-457] Added Unit Test cas...

2016-11-29 Thread PallaviSingh1992
GitHub user PallaviSingh1992 opened a pull request: https://github.com/apache/incubator-carbondata/pull/370 [CARBONDATA-457] Added Unit Test cases for core.writer package You can merge this pull request into a Git repository by running: $ git pull

[GitHub] incubator-carbondata issue #369: [CARBONDATA-470][WIP]Add unsafe offheap and...

2016-11-29 Thread ravipesala
Github user ravipesala commented on the issue: https://github.com/apache/incubator-carbondata/pull/369 @allwefantasy Thanks for your suggestion. Yes, we can use the Spark's memory manager instead of our own. I will open the interface and provide implementation to use

[GitHub] incubator-carbondata issue #369: [CARBONDATA-470][WIP]Add unsafe offheap and...

2016-11-29 Thread allwefantasy
Github user allwefantasy commented on the issue: https://github.com/apache/incubator-carbondata/pull/369 Does this PR have considered allocating memory from TaskMemoryManager? Many Spark application runs on Yarn,if you use off-heap,it's easy to trigger behavior of yarn's killing

[GitHub] incubator-carbondata pull request #369: [CARBONDATA-470][WIP]Add unsafe offh...

2016-11-29 Thread ravipesala
GitHub user ravipesala opened a pull request: https://github.com/apache/incubator-carbondata/pull/369 [CARBONDATA-470][WIP]Add unsafe offheap and on-heap sort in carbodata loading In the current carbondata system loading performance is not so encouraging since we need to sort the

[GitHub] incubator-carbondata issue #333: [CARBONDATA-471]Optimized no kettle flow an...

2016-11-29 Thread ravipesala
Github user ravipesala commented on the issue: https://github.com/apache/incubator-carbondata/pull/333 Updated PR, please review it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[jira] [Created] (CARBONDATA-471) Optimize no kettle flow and fix issues in cluster

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-471: -- Summary: Optimize no kettle flow and fix issues in cluster Key: CARBONDATA-471 URL: https://issues.apache.org/jira/browse/CARBONDATA-471 Project:

[GitHub] incubator-carbondata issue #265: [CARBONDATA-458]Improving First time query ...

2016-11-29 Thread kumarvishal09
Github user kumarvishal09 commented on the issue: https://github.com/apache/incubator-carbondata/pull/265 http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/711/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[jira] [Created] (CARBONDATA-470) Add unsafe offheap and on-heap sort in carbodata loading

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-470: -- Summary: Add unsafe offheap and on-heap sort in carbodata loading Key: CARBONDATA-470 URL: https://issues.apache.org/jira/browse/CARBONDATA-470 Project:

[GitHub] incubator-carbondata pull request #354: [WIP]Removed Decimal And Utf8String ...

2016-11-29 Thread kumarvishal09
Github user kumarvishal09 closed the pull request at: https://github.com/apache/incubator-carbondata/pull/354 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] incubator-carbondata issue #365: [CARBONDATA-463] Extract code to spark-comm...

2016-11-29 Thread ravipesala
Github user ravipesala commented on the issue: https://github.com/apache/incubator-carbondata/pull/365 LGTM except few comments. It seems there is an issue in pom files while using maven default build and using format. I will fix them in new PR. --- If your project is set up for

[GitHub] incubator-carbondata issue #358: [CARBONDATA-467] Adding bucketing to carbon...

2016-11-29 Thread ravipesala
Github user ravipesala commented on the issue: https://github.com/apache/incubator-carbondata/pull/358 CI cannot pass right now as there are thrift changes. But I have verified in local and all tests are passing. --- If your project is set up for it, you can reply to this email and

[GitHub] incubator-carbondata pull request #365: [CARBONDATA-463] Extract code to spa...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/365#discussion_r90044286 --- Diff: integration/spark/src/main/scala/org/apache/carbondata/spark/rdd/NewCarbonDataLoadRDD.scala --- @@ -51,7 +51,6 @@ class

[GitHub] incubator-carbondata pull request #365: [CARBONDATA-463] Extract code to spa...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/365#discussion_r90044359 --- Diff: integration/spark-common/src/main/java/org/apache/carbondata/integration/spark/merger/CarbonCompactionUtil.java --- @@ -38,7 +38,6

[GitHub] incubator-carbondata pull request #365: [CARBONDATA-463] Extract code to spa...

2016-11-29 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/365#discussion_r90040864 --- Diff: integration/spark-common/src/main/java/org/apache/carbondata/integration/spark/merger/CarbonCompactionUtil.java --- @@ -38,7

[GitHub] incubator-carbondata issue #367: [CARBONDATA-465] Spark streaming dataframe ...

2016-11-29 Thread allwefantasy
Github user allwefantasy commented on the issue: https://github.com/apache/incubator-carbondata/pull/367 Ok, i will remove the PR333 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] incubator-carbondata pull request #367: [CARBONDATA-465] Spark streaming dat...

2016-11-29 Thread allwefantasy
Github user allwefantasy closed the pull request at: https://github.com/apache/incubator-carbondata/pull/367 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[jira] [Created] (CARBONDATA-469) Optimize join in spark using bucketing information

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-469: -- Summary: Optimize join in spark using bucketing information Key: CARBONDATA-469 URL: https://issues.apache.org/jira/browse/CARBONDATA-469 Project:

[jira] [Updated] (CARBONDATA-467) Add bucketing information while creating table and update in thrift format.And load the data using bucketing information.

2016-11-29 Thread Ravindra Pesala (JIRA)
[ https://issues.apache.org/jira/browse/CARBONDATA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindra Pesala updated CARBONDATA-467: --- Summary: Add bucketing information while creating table and update in thrift

[jira] [Created] (CARBONDATA-467) Add bucketing information while creating table and update in thrift format.

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-467: -- Summary: Add bucketing information while creating table and update in thrift format. Key: CARBONDATA-467 URL: https://issues.apache.org/jira/browse/CARBONDATA-467

[GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/362#discussion_r90033298 --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/hive/DistributionUtil.scala --- @@ -101,47 +101,107 @@ object DistributionUtil

[jira] [Created] (CARBONDATA-466) Implement bucketing table in carbondata

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-466: -- Summary: Implement bucketing table in carbondata Key: CARBONDATA-466 URL: https://issues.apache.org/jira/browse/CARBONDATA-466 Project: CarbonData

[GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/362#discussion_r90032032 --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/hive/DistributionUtil.scala --- @@ -101,47 +101,107 @@ object DistributionUtil

[GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/362#discussion_r90032072 --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/hive/DistributionUtil.scala --- @@ -101,47 +101,107 @@ object DistributionUtil

[GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/362#discussion_r90031584 --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/CarbonContext.scala --- @@ -208,4 +208,36 @@ object CarbonContext { }

[GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/362#discussion_r90031916 --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/hive/DistributionUtil.scala --- @@ -101,47 +101,107 @@ object DistributionUtil

[GitHub] incubator-carbondata pull request #362: [CARBONDATA-459] Block distribution ...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/362#discussion_r90031795 --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/hive/DistributionUtil.scala --- @@ -101,47 +101,107 @@ object DistributionUtil

[GitHub] incubator-carbondata issue #366: [CARBONDATA-368]Insert into carbon table fe...

2016-11-29 Thread ashokblend
Github user ashokblend commented on the issue: https://github.com/apache/incubator-carbondata/pull/366 i have removed changes in pom.xml, but before merging we need to deploy carbon-format as there is change in schema.thrift. --- If your project is set up for it, you can reply to

[GitHub] incubator-carbondata issue #367: [CARBONDATA-465] Spark streaming dataframe ...

2016-11-29 Thread jackylk
Github user jackylk commented on the issue: https://github.com/apache/incubator-carbondata/pull/367 I think @ravipesala is still working on #333 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] incubator-carbondata pull request #366: [CARBONDATA-368]Insert into carbon t...

2016-11-29 Thread jackylk
Github user jackylk commented on a diff in the pull request: https://github.com/apache/incubator-carbondata/pull/366#discussion_r89998317 --- Diff: pom.xml --- @@ -93,6 +93,7 @@ common + format --- End diff -- should not add format

[GitHub] incubator-carbondata issue #367: [CARBONDATA-465] Spark streaming dataframe ...

2016-11-29 Thread allwefantasy
Github user allwefantasy commented on the issue: https://github.com/apache/incubator-carbondata/pull/367 Can we merge PR333 first then merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] incubator-carbondata issue #366: [WIP][CARBONDATA-368]Insert into carbon tab...

2016-11-29 Thread QiangCai
Github user QiangCai commented on the issue: https://github.com/apache/incubator-carbondata/pull/366 please rebase --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] incubator-carbondata issue #367: [CARBONDATA-465] Spark streaming dataframe ...

2016-11-29 Thread allwefantasy
Github user allwefantasy commented on the issue: https://github.com/apache/incubator-carbondata/pull/367 I am not sure whether it depends on PR333。I try to resolve issues when carbondata working with Spark Streaming by Merging PR333 however it does not work. Maybe I should

[jira] [Resolved] (CARBONDATA-368) Should improve performance of DataFrame loading

2016-11-29 Thread Jacky Li (JIRA)
[ https://issues.apache.org/jira/browse/CARBONDATA-368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacky Li resolved CARBONDATA-368. - Resolution: Fixed Fix Version/s: 0.3.0-incubating > Should improve performance of

[GitHub] incubator-carbondata issue #278: [CARBONDATA-368]Imporve performance of data...

2016-11-29 Thread jackylk
Github user jackylk commented on the issue: https://github.com/apache/incubator-carbondata/pull/278 LGTM CI passed http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/700/ --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] incubator-carbondata issue #367: [CARBONDATA-465] Spark streaming dataframe ...

2016-11-29 Thread jackylk
Github user jackylk commented on the issue: https://github.com/apache/incubator-carbondata/pull/367 Why include PR333 in this PR? Does it depends on PR333? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project