[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2683 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/449/ ---
[GitHub] carbondata issue #2718: [CARBONDATA-2933] Fix errors in spelling
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2718 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/448/ ---
[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2683 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8519/ ---
[GitHub] carbondata issue #2718: [CARBONDATA-2933] Fix errors in spelling
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2718 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8518/ ---
[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2683 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/280/ ---
[GitHub] carbondata issue #2718: [CARBONDATA-2933] Fix errors in spelling
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2718 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/279/ ---
[GitHub] carbondata pull request #2718: [CARBONDATA-2933] Fix errors in spelling
GitHub user xuchuanyin opened a pull request: https://github.com/apache/carbondata/pull/2718 [CARBONDATA-2933] Fix errors in spelling fix errors in spelling Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/xuchuanyin/carbondata error_spelling Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2718.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2718 commit 066415a006bc3a9df81ed93a4b1a18ec1bd7669d Author: xuchuanyin Date: 2018-09-14T03:22:23Z Fix errors in spelling fix errors in spelling ---
[jira] [Created] (CARBONDATA-2933) Fix errors in spelling
xuchuanyin created CARBONDATA-2933: -- Summary: Fix errors in spelling Key: CARBONDATA-2933 URL: https://issues.apache.org/jira/browse/CARBONDATA-2933 Project: CarbonData Issue Type: Bug Reporter: xuchuanyin Assignee: xuchuanyin -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] carbondata pull request #2683: [CARBONDATA-2916] Add CarbonCli tool for data...
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2683#discussion_r217589388
--- Diff:
core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java ---
@@ -168,6 +168,65 @@ public static Object
getMeasureObjectBasedOnDataType(ColumnPage measurePage, int
}
}
+ /**
+ * Calculate data percentage in [min, max] scope based on data type
+ * @param data data to calculate the percentage
+ * @param min min value
+ * @param max max value
+ * @param column column schema including data type
+ * @return result
+ */
+ public static double computePercentage(byte[] data, byte[] min, byte[]
max, ColumnSchema column) {
--- End diff --
ok
---
[GitHub] carbondata pull request #2683: [CARBONDATA-2916] Add CarbonCli tool for data...
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2683#discussion_r217589323
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/TestUtil.java ---
@@ -36,38 +36,34 @@
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.generic.GenericDatumWriter;
import org.apache.avro.io.DecoderFactory;
-import org.apache.avro.io.Encoder;
import org.apache.avro.io.JsonDecoder;
import org.apache.hadoop.conf.Configuration;
-import org.junit.Assert;
+@InterfaceAudience.Developer("Test")
public class TestUtil {
- public static Configuration configuration = new Configuration();
+ public static final Configuration configuration = new Configuration();
--- End diff --
Because now CLI is a separate module and testcase in CLI need to use
TestUtil to write carbonfiles using SDK, so I have to move the TestUtil to src
so that CLI testcase can use it. Is there other way?
---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/447/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8517/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/278/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/446/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8516/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/277/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/445/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8515/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/276/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/444/ ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2717 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/443/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8514/ ---
[GitHub] carbondata issue #2691: [CARBONDATA-2912] Support CSV table load csv data wi...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2691 Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/442/ ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2717 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8513/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/275/ ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2717 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/274/ ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user xubo245 commented on the issue: https://github.com/apache/carbondata/pull/2717 retest this please ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2717 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8511/ ---
[GitHub] carbondata issue #2691: [CARBONDATA-2912] Support CSV table load csv data wi...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2691 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/273/ ---
[GitHub] carbondata issue #2691: [CARBONDATA-2912] Support CSV table load csv data wi...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2691 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8512/ ---
[GitHub] carbondata pull request #2691: [CARBONDATA-2912] Support CSV table load csv ...
Github user xubo245 commented on a diff in the pull request: https://github.com/apache/carbondata/pull/2691#discussion_r217351302 --- Diff: integration/spark-common-test/src/test/resources/cars.csv --- @@ -0,0 +1,4 @@ +name,age --- End diff -- ok, done ---
[GitHub] carbondata issue #2717: [CARBONDATA-2932] FIx CarbonReader Projection cann't...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2717 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/272/ ---
[GitHub] carbondata pull request #2717: [CARBONDATA-2932] FIx CarbonReader Projection...
GitHub user xubo245 opened a pull request: https://github.com/apache/carbondata/pull/2717 [CARBONDATA-2932] FIx CarbonReader Projection cann't be empty exception run org.apache.carbondata.examples.sdk.CarbonReaderExample and then some exception. This PR fixd it. conclude: 1.close carbonReader after finished read, which solved the exception. 2.add example code for varchar and array, including read and write Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ X] Any interfaces changed? No - [ X] Any backward compatibility impacted? No - [X ] Document update required? No - [X] Testing done fix example - [ X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. No You can merge this pull request into a Git repository by running: $ git pull https://github.com/xubo245/carbondata CARBONDATA-2932-CarbonReaderProjection Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2717.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2717 commit d98cbe8f0464d4bacec025d6d8fe10223cd3eac1 Author: xubo245 Date: 2018-09-13T11:09:30Z [CARBONDATA-2932] FIx CarbonReader Projection cann't be empty exception ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/440/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8510/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/271/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user ravipesala commented on the issue: https://github.com/apache/carbondata/pull/2716 retest this please ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8509/ ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/439/ ---
[GitHub] carbondata pull request #2683: [CARBONDATA-2916] Add CarbonCli tool for data...
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2683#discussion_r217323443
--- Diff:
store/sdk/src/main/java/org/apache/carbondata/sdk/file/TestUtil.java ---
@@ -36,38 +36,34 @@
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.generic.GenericDatumWriter;
import org.apache.avro.io.DecoderFactory;
-import org.apache.avro.io.Encoder;
import org.apache.avro.io.JsonDecoder;
import org.apache.hadoop.conf.Configuration;
-import org.junit.Assert;
+@InterfaceAudience.Developer("Test")
public class TestUtil {
- public static Configuration configuration = new Configuration();
+ public static final Configuration configuration = new Configuration();
--- End diff --
Why is this test class changed? How it is related to CLI tool
---
[GitHub] carbondata pull request #2683: [CARBONDATA-2916] Add CarbonCli tool for data...
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2683#discussion_r217322845
--- Diff:
core/src/main/java/org/apache/carbondata/core/util/DataTypeUtil.java ---
@@ -168,6 +168,65 @@ public static Object
getMeasureObjectBasedOnDataType(ColumnPage measurePage, int
}
}
+ /**
+ * Calculate data percentage in [min, max] scope based on data type
+ * @param data data to calculate the percentage
+ * @param min min value
+ * @param max max value
+ * @param column column schema including data type
+ * @return result
+ */
+ public static double computePercentage(byte[] data, byte[] min, byte[]
max, ColumnSchema column) {
--- End diff --
Bette move this method to CI tool only as it is used only there
---
[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user xuchuanyin commented on the issue: https://github.com/apache/carbondata/pull/2683 LGTM ---
[GitHub] carbondata pull request #2695: [CARBONDATA-2919] Support ingest from Kafka i...
Github user asfgit closed the pull request at: https://github.com/apache/carbondata/pull/2695 ---
[GitHub] carbondata issue #2716: [WIP] Removed warn log level as CI console becomes v...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2716 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/270/ ---
[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2683 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/438/ ---
[GitHub] carbondata pull request #2716: [WIP] Removed warn log level as CI console be...
GitHub user ravipesala opened a pull request: https://github.com/apache/carbondata/pull/2716 [WIP] Removed warn log level as CI console becomes very big. Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? - [ ] Any backward compatibility impacted? - [ ] Document update required? - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravipesala/incubator-carbondata fixcitest2.3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2716.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2716 commit 7aca9feddea217770d71870bf68de64b6e3ecba8 Author: ravipesala Date: 2018-09-13T08:31:04Z Removed warn log level as CI console becomes very big. ---
[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2683 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8508/ ---
[GitHub] carbondata issue #2695: [CARBONDATA-2919] Support ingest from Kafka in Strea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2695 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/437/ ---
[jira] [Created] (CARBONDATA-2932) CarbonReaderExample throw some exception: Projection can't be empty
xubo245 created CARBONDATA-2932:
---
Summary: CarbonReaderExample throw some exception: Projection
can't be empty
Key: CARBONDATA-2932
URL: https://issues.apache.org/jira/browse/CARBONDATA-2932
Project: CarbonData
Issue Type: Bug
Reporter: xubo245
Assignee: xubo245
run org.apache.carbondata.examples.sdk.CarbonReaderExample and then some
exception:
{code:java}
Finished
java.lang.RuntimeException: Projection can't be empty
at
org.apache.carbondata.hadoop.api.CarbonInputFormat.setColumnProjection(CarbonInputFormat.java:219)
at
org.apache.carbondata.hadoop.api.CarbonFileInputFormat.getSplits(CarbonFileInputFormat.java:155)
at
org.apache.carbondata.sdk.file.CarbonReaderBuilder.build(CarbonReaderBuilder.java:213)
at
org.apache.carbondata.examples.sdk.CarbonReaderExample.main(CarbonReaderExample.java:121)
Projection can't be empty
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] carbondata issue #2683: [CARBONDATA-2916] Add CarbonCli tool for data summar...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2683 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/269/ ---
[GitHub] carbondata issue #2695: [CARBONDATA-2919] Support ingest from Kafka in Strea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2695 Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8507/ ---
[GitHub] carbondata issue #2695: [CARBONDATA-2919] Support ingest from Kafka in Strea...
Github user CarbonDataQA commented on the issue: https://github.com/apache/carbondata/pull/2695 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/268/ ---
[GitHub] carbondata issue #2695: [CARBONDATA-2919] Support ingest from Kafka in Strea...
Github user jackylk commented on the issue: https://github.com/apache/carbondata/pull/2695 retest this please ---
