date:20180620

Jenkins build is unstable: carbondata-master-spark-2.1 #2521

2018-06-20 Thread Apache Jenkins Server

See

Jenkins build is unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #2521

2018-06-20 Thread Apache Jenkins Server

See

Jenkins build is back to normal : carbondata-master-spark-2.1 » Apache CarbonData :: Core #2521

2018-06-20 Thread Apache Jenkins Server

See

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Parent #563

2018-06-20 Thread Apache Jenkins Server

See 


--
Established TCP socket on 43690
maven33-agent.jar already up to date
maven33-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
<===[JENKINS REMOTING CAPACITY]===>   channel started
Executing Maven:  -B -f 

 -Dmaven.repo.local=/home/jenkins/jenkins-slave/maven-repositories/1 -B -e 
clean install -U -Pspark-2.2 coveralls:report
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for 
org.apache.carbondata:carbondata-presto:presto-plugin:1.5.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for 
com.ning.maven.plugins:maven-duplicate-finder-plugin is missing. @ 
org.apache.carbondata:carbondata-presto:[unknown-version], 

 line 621, column 15
[WARNING] 'build.plugins.plugin.version' for 
pl.project13.maven:git-commit-id-plugin is missing. @ 
org.apache.carbondata:carbondata-presto:[unknown-version], 

 line 634, column 15
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for 
org.apache.carbondata:carbondata-examples-spark2:jar:1.5.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for 
com.ning.maven.plugins:maven-duplicate-finder-plugin is missing. @ 
org.apache.carbondata:carbondata-examples-spark2:[unknown-version], 

 line 163, column 15
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten 
the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support 
building such malformed projects.
[WARNING] 
[INFO] 
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache CarbonData :: Parent
[INFO] Apache CarbonData :: Common
[INFO] Apache CarbonData :: Core
[INFO] Apache CarbonData :: Processing
[INFO] Apache CarbonData :: Hadoop
[INFO] Apache CarbonData :: Streaming
[INFO] Apache CarbonData :: Spark Common
[INFO] Apache CarbonData :: Store SDK
[INFO] Apache CarbonData :: Search
[INFO] Apache CarbonData :: Lucene Index DataMap
[INFO] Apache CarbonData :: Bloom Index DataMap
[INFO] Apache CarbonData :: Spark2
[INFO] Apache CarbonData :: Spark Common Test
[INFO] Apache CarbonData :: DataMap Examples
[INFO] Apache CarbonData :: Assembly
[INFO] Apache CarbonData :: Hive
[INFO] Apache CarbonData :: presto
[INFO] Apache CarbonData :: Spark2 Examples
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Parent 1.5.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-parent ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-parent ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-parent ---
[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-parent >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-parent 
---
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-parent <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-parent ---
[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-parent ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-parent ---
[INFO] argLine set to

Build failed in Jenkins: carbondata-master-spark-2.2 #563

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[ravipesala] [CARBONDATA-2504][STREAM] Support StreamSQL for streaming job

--
[...truncated 53.50 MB...]
+-+-+
|   id| name|
+-+-+
|which test1 good4|who and name4|
|which test1 good4|who and name4|
|which test1 good8|who and name0|
|which test1 good8|who and name0|
|which test1 good9|who and name3|
|which test1 good9|who and name3|
+-+-+

+-+-+
|   id| name|
+-+-+
|which test1 good4|who and name4|
|which test1 good4|who and name4|
|which test1 good8|who and name0|
|which test1 good8|who and name0|
|which test1 good9|who and name3|
|which test1 good9|who and name3|
+-+-+

18/06/20 22:53:31 AUDIT CarbonDropTableCommand: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Deleting table [persontable] under 
database [default]
18/06/20 22:53:31 ERROR DataMapStoreManager: ScalaTest-main-running-RunExamples 
failed to get carbon table from table Path
18/06/20 22:53:31 AUDIT CarbonDropTableCommand: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Deleted table [persontable] under 
database [default]
[32m- LuceneDataMapExample[0m
18/06/20 22:53:31 AUDIT CarbonCreateTableCommand: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Creating Table with Database name 
[default] and Table name [origin_table]
18/06/20 22:53:31 ERROR DataMapStoreManager: ScalaTest-main-running-RunExamples 
failed to get carbon table from table Path
18/06/20 22:53:31 AUDIT CarbonCreateTableCommand: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Table created with Database name 
[default] and Table name [origin_table]
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load request has been 
received for table default.origin_table
18/06/20 22:53:31 ERROR DataLoadExecutor: [Executor task launch worker for task 
2297][partitionID:table;queryID:52112762594665043] Data Load is partially 
success for table origin_table
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load is successful for 
default.origin_table
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load request has been 
received for table default.origin_table
18/06/20 22:53:31 ERROR DataLoadExecutor: [Executor task launch worker for task 
2298][partitionID:table;queryID:52112762812220714] Data Load is partially 
success for table origin_table
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load is successful for 
default.origin_table
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load request has been 
received for table default.origin_table
18/06/20 22:53:31 ERROR DataLoadExecutor: [Executor task launch worker for task 
2299][partitionID:table;queryID:52112762989220843] Data Load is partially 
success for table origin_table
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load is successful for 
default.origin_table
18/06/20 22:53:31 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load request has been 
received for table default.origin_table
18/06/20 22:53:32 ERROR DataLoadExecutor: [Executor task launch worker for task 
2300][partitionID:table;queryID:52112763157177946] Data Load is partially 
success for table origin_table
18/06/20 22:53:32 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load is successful for 
default.origin_table
++
|count(1)|
++
|  40|
++

18/06/20 22:53:32 AUDIT CarbonCreateTableCommand: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Creating Table with Database name 
[default] and Table name [external_table]
18/06/20 22:53:32 AUDIT CarbonCreateTableCommand: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Table created with Database name 
[default] and Table name [external_table]
++
|count(1)|
++
|  40|
++

18/06/20 22:53:32 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load request has been 
received for table default.origin_table
18/06/20 22:53:32 ERROR DataLoadExecutor: [Executor task launch worker for task 
2305][partitionID:table;queryID:52112763484975296] Data Load is partially 
success for table origin_table
18/06/20 22:53:32 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load is successful for 
default.origin_table
18/06/20 22:53:32 AUDIT CarbonDataRDDFactory$: 
[asf920.gq1.ygridcore.net][jenkins][Thread-1]Data load request has been 
received for table default.origin_table
18/06/20 22:53:32 ERROR DataLoadExecutor: [Executor task launch

Jenkins build is unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #563

2018-06-20 Thread Apache Jenkins Server

See

carbondata git commit: [CARBONDATA-2615][32K] Support page size less than 32000 in CarbondataV3

2018-06-20 Thread kumarvishal09

Repository: carbondata
Updated Branches:
  refs/heads/master 2ea3b2dc5 -> 091a28bf8


[CARBONDATA-2615][32K] Support page size less than 32000 in CarbondataV3

Since we support super long string, if it is long enough, a column page
with 32000 rows will exceed 2GB, so we support a page less than 32000
rows.

This closes #2383


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/091a28bf
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/091a28bf
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/091a28bf

Branch: refs/heads/master
Commit: 091a28bf833a5296dd3878ddb11b243f7f37a8fc
Parents: 2ea3b2d
Author: xuchuanyin 
Authored: Wed Jun 20 19:07:03 2018 +0800
Committer: kumarvishal09 
Committed: Thu Jun 21 11:00:02 2018 +0530

--
 .../testsuite/dataload/TestLoadDataGeneral.scala| 16 
 .../store/CarbonFactDataHandlerColumnar.java|  7 ++-
 2 files changed, 22 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/091a28bf/integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/dataload/TestLoadDataGeneral.scala
--
diff --git 
a/integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/dataload/TestLoadDataGeneral.scala
 
b/integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/dataload/TestLoadDataGeneral.scala
index 688928f..8b51090 100644
--- 
a/integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/dataload/TestLoadDataGeneral.scala
+++ 
b/integration/spark-common-test/src/test/scala/org/apache/carbondata/integration/spark/testsuite/dataload/TestLoadDataGeneral.scala
@@ -259,6 +259,22 @@ class TestLoadDataGeneral extends QueryTest with 
BeforeAndAfterEach {
   CarbonLoadOptionConstants.ENABLE_CARBON_LOAD_DIRECT_WRITE_HDFS,
   originStatus)
   }
+
+  test("test data loading with page size less than 32000") {
+CarbonProperties.getInstance().addProperty(
+  CarbonCommonConstants.BLOCKLET_SIZE, "16000")
+
+val testData = s"$resourcesPath/sample.csv"
+sql(s"LOAD DATA LOCAL INPATH '$testData' into table loadtest")
+checkAnswer(
+  sql("SELECT COUNT(*) FROM loadtest"),
+  Seq(Row(6))
+)
+
+
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.BLOCKLET_SIZE,
+  CarbonCommonConstants.BLOCKLET_SIZE_DEFAULT_VAL)
+  }
+
   override def afterEach {
 sql("DROP TABLE if exists loadtest")
 sql("drop table if exists invalidMeasures")

http://git-wip-us.apache.org/repos/asf/carbondata/blob/091a28bf/processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
--
diff --git 
a/processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
 
b/processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
index c0acadd..5fe3261 100644
--- 
a/processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
+++ 
b/processing/src/main/java/org/apache/carbondata/processing/store/CarbonFactDataHandlerColumnar.java
@@ -371,8 +371,13 @@ public class CarbonFactDataHandlerColumnar implements 
CarbonFactHandler {
 this.pageSize = Integer.parseInt(CarbonProperties.getInstance()
 .getProperty(CarbonCommonConstants.BLOCKLET_SIZE,
 CarbonCommonConstants.BLOCKLET_SIZE_DEFAULT_VAL));
+// support less than 32000 rows in one page, because we support super long 
string,
+// if it is long enough, a clomun page with 32000 rows will exceed 2GB
 if (version == ColumnarFormatVersion.V3) {
-  this.pageSize = 
CarbonV3DataFormatConstants.NUMBER_OF_ROWS_PER_BLOCKLET_COLUMN_PAGE_DEFAULT;
+  this.pageSize =
+  pageSize < 
CarbonV3DataFormatConstants.NUMBER_OF_ROWS_PER_BLOCKLET_COLUMN_PAGE_DEFAULT ?
+  pageSize :
+  
CarbonV3DataFormatConstants.NUMBER_OF_ROWS_PER_BLOCKLET_COLUMN_PAGE_DEFAULT;
 }
 LOGGER.info("Number of rows per column blocklet " + pageSize);
 dataRows = new ArrayList<>(this.pageSize);

[1/2] carbondata git commit: [CARBONDATA-2504][STREAM] Support StreamSQL for streaming job

2018-06-20 Thread ravipesala

Repository: carbondata
Updated Branches:
  refs/heads/master 6eb360e1f -> 2ea3b2dc5


http://git-wip-us.apache.org/repos/asf/carbondata/blob/2ea3b2dc/store/sdk/src/main/java/org/apache/carbondata/store/LocalCarbonStore.java
--
diff --git 
a/store/sdk/src/main/java/org/apache/carbondata/store/LocalCarbonStore.java 
b/store/sdk/src/main/java/org/apache/carbondata/store/LocalCarbonStore.java
index 9e338e7..daa1447 100644
--- a/store/sdk/src/main/java/org/apache/carbondata/store/LocalCarbonStore.java
+++ b/store/sdk/src/main/java/org/apache/carbondata/store/LocalCarbonStore.java
@@ -65,7 +65,7 @@ class LocalCarbonStore extends MetaCachedCarbonStore {
 Objects.requireNonNull(projectColumns);
 
 CarbonTable table = getTable(path);
-if (table.isStreamingTable() || table.isHivePartitionTable()) {
+if (table.isStreamingSink() || table.isHivePartitionTable()) {
   throw new UnsupportedOperationException("streaming and partition table 
is not supported");
 }
 // TODO: use InputFormat to prune data and read data

http://git-wip-us.apache.org/repos/asf/carbondata/blob/2ea3b2dc/streaming/src/main/scala/org/apache/carbondata/streaming/parser/RowStreamParserImp.scala
--
diff --git 
a/streaming/src/main/scala/org/apache/carbondata/streaming/parser/RowStreamParserImp.scala
 
b/streaming/src/main/scala/org/apache/carbondata/streaming/parser/RowStreamParserImp.scala
index 1696fdc..5a888ef 100644
--- 
a/streaming/src/main/scala/org/apache/carbondata/streaming/parser/RowStreamParserImp.scala
+++ 
b/streaming/src/main/scala/org/apache/carbondata/streaming/parser/RowStreamParserImp.scala
@@ -17,7 +17,6 @@
 
 package org.apache.carbondata.streaming.parser
 
-import java.nio.charset.Charset
 import java.text.SimpleDateFormat
 
 import org.apache.hadoop.conf.Configuration

[2/2] carbondata git commit: [CARBONDATA-2504][STREAM] Support StreamSQL for streaming job

2018-06-20 Thread ravipesala

[CARBONDATA-2504][STREAM] Support StreamSQL for streaming job

Currently, user need to write Spark Streaming APP to use carbon streaming 
ingest feature, which is not so easy for some users. By providing StreamSQL, 
user can manage the streaming job more easily.

This closes #2328


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/2ea3b2dc
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/2ea3b2dc
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/2ea3b2dc

Branch: refs/heads/master
Commit: 2ea3b2dc5841c604c5fb44a6e7f18c7d2db8c543
Parents: 6eb360e
Author: Jacky Li 
Authored: Mon May 21 21:49:33 2018 +0800
Committer: ravipesala 
Committed: Thu Jun 21 09:44:22 2018 +0530

--
 .../exceptions/NoSuchStreamException.java   |  35 ++
 .../core/metadata/schema/table/CarbonTable.java |  15 +-
 .../spark/util/SparkDataTypeConverterImpl.java  |  45 +++
 .../apache/carbondata/spark/CarbonOption.scala  |   9 +-
 .../carbondata/spark/StreamingOption.scala  |  67 
 .../carbondata/spark/util/CarbonScalaUtil.scala |   5 +-
 .../spark/sql/catalyst/CarbonDDLSqlParser.scala |   3 +
 .../spark/util/CarbonMetastoreTypes.scala   | 104 ++
 .../apache/spark/util/SparkTypeConverter.scala  | 134 +++
 .../carbondata/stream/StreamJobManager.scala| 198 ++
 .../org/apache/spark/sql/CarbonSource.scala |   2 +-
 .../spark/sql/CarbonSparkStreamingFactory.scala |   2 +-
 .../datamap/CarbonCreateDataMapCommand.scala|   2 +-
 .../stream/CarbonCreateStreamCommand.scala  | 130 +++
 .../stream/CarbonDropStreamCommand.scala|  36 ++
 .../stream/CarbonShowStreamsCommand.scala   |  76 
 .../command/table/CarbonDropTableCommand.scala  |   2 +-
 .../strategy/CarbonLateDecodeStrategy.scala |   2 +-
 .../sql/execution/strategy/DDLStrategy.scala|   2 +-
 .../strategy/StreamingTableStrategy.scala   |   2 +-
 .../sql/hive/CarbonPreAggregateRules.scala  |   6 +-
 .../apache/spark/sql/hive/CarbonRelation.scala  |  89 +
 .../sql/parser/CarbonSpark2SqlParser.scala  |  44 ++-
 .../spark/sql/parser/CarbonSparkSqlParser.scala |   4 +-
 .../apache/spark/util/SparkTypeConverter.scala  | 135 ---
 .../TestStreamingTableOperation.scala   | 359 ++-
 .../carbondata/store/LocalCarbonStore.java  |   2 +-
 .../streaming/parser/RowStreamParserImp.scala   |   1 -
 28 files changed, 1254 insertions(+), 257 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/2ea3b2dc/common/src/main/java/org/apache/carbondata/common/exceptions/NoSuchStreamException.java
--
diff --git 
a/common/src/main/java/org/apache/carbondata/common/exceptions/NoSuchStreamException.java
 
b/common/src/main/java/org/apache/carbondata/common/exceptions/NoSuchStreamException.java
new file mode 100644
index 000..77fa7fb
--- /dev/null
+++ 
b/common/src/main/java/org/apache/carbondata/common/exceptions/NoSuchStreamException.java
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.common.exceptions;
+
+import org.apache.carbondata.common.annotations.InterfaceAudience;
+import org.apache.carbondata.common.annotations.InterfaceStability;
+
+@InterfaceAudience.User
+@InterfaceStability.Stable
+public class NoSuchStreamException extends Exception {
+  /**
+   * default serial version ID.
+   */
+  private static final long serialVersionUID = 133241232L;
+
+  public NoSuchStreamException(String streamName) {
+super("stream '" + streamName + "' not found");
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/carbondata/blob/2ea3b2dc/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java
 
b/core/src/main/java/org/apache/carbondata/core/metadata/schema/table/CarbonTable.java

carbondata git commit: [CARBONDATA-2616][BloomDataMap] Fix bugs in querying bloom datamap with two index columns

2018-06-20 Thread jackylk

Repository: carbondata
Updated Branches:
  refs/heads/master 01b48fc36 -> 6eb360e1f


[CARBONDATA-2616][BloomDataMap] Fix bugs in querying bloom datamap with two 
index columns

During pruning in bloomfilter datamap, the same blocklets has been added
to result more than once, thus causing explaining and querying returning
incorrect result.

This closes #2386


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/6eb360e1
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/6eb360e1
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/6eb360e1

Branch: refs/heads/master
Commit: 6eb360e1f5f577a576d185efb7dcbf1cc6a302e8
Parents: 01b48fc
Author: xuchuanyin 
Authored: Wed Jun 20 16:31:28 2018 +0800
Committer: Jacky Li 
Committed: Thu Jun 21 12:03:48 2018 +0800

--
 .../datamap/bloom/BloomCoarseGrainDataMap.java|  5 +++--
 .../bloom/BloomCoarseGrainDataMapSuite.scala  | 18 +-
 2 files changed, 20 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/6eb360e1/datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java
--
diff --git 
a/datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java
 
b/datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java
index e9af0ff..ed03256 100644
--- 
a/datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java
+++ 
b/datamap/bloom/src/main/java/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMap.java
@@ -21,6 +21,7 @@ import java.io.File;
 import java.io.IOException;
 import java.io.UnsupportedEncodingException;
 import java.util.ArrayList;
+import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
 
@@ -85,7 +86,7 @@ public class BloomCoarseGrainDataMap extends 
CoarseGrainDataMap {
   @Override
   public List prune(FilterResolverIntf filterExp, SegmentProperties 
segmentProperties,
   List partitions) {
-List hitBlocklets = new ArrayList();
+Set hitBlocklets = new HashSet<>();
 if (filterExp == null) {
   // null is different from empty here. Empty means after pruning, no 
blocklet need to scan.
   return null;
@@ -111,7 +112,7 @@ public class BloomCoarseGrainDataMap extends 
CoarseGrainDataMap {
 }
   }
 }
-return hitBlocklets;
+return new ArrayList<>(hitBlocklets);
   }
 
   private byte[] convertValueToBytes(DataType dataType, Object value) {

http://git-wip-us.apache.org/repos/asf/carbondata/blob/6eb360e1/integration/spark2/src/test/scala/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapSuite.scala
--
diff --git 
a/integration/spark2/src/test/scala/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapSuite.scala
 
b/integration/spark2/src/test/scala/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapSuite.scala
index 7df3901..c9a4097 100644
--- 
a/integration/spark2/src/test/scala/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapSuite.scala
+++ 
b/integration/spark2/src/test/scala/org/apache/carbondata/datamap/bloom/BloomCoarseGrainDataMapSuite.scala
@@ -71,7 +71,23 @@ class BloomCoarseGrainDataMapSuite extends QueryTest with 
BeforeAndAfterAll with
 checkAnswer(
   checkSqlHitDataMap(s"select * from $bloomDMSampleTable where city = 
'city_999'", dataMapName, shouldHit),
   sql(s"select * from $normalTable where city = 'city_999'"))
- checkAnswer(
+// query with two index_columns
+checkAnswer(
+  checkSqlHitDataMap(s"select * from $bloomDMSampleTable where id = 1 and 
city='city_1'", dataMapName, shouldHit),
+  sql(s"select * from $normalTable where id = 1 and city='city_1'"))
+checkAnswer(
+  checkSqlHitDataMap(s"select * from $bloomDMSampleTable where id = 999 
and city='city_999'", dataMapName, shouldHit),
+  sql(s"select * from $normalTable where id = 999 and city='city_999'"))
+checkAnswer(
+  checkSqlHitDataMap(s"select * from $bloomDMSampleTable where city = 
'city_1' and id = 0", dataMapName, shouldHit),
+  sql(s"select * from $normalTable where city = 'city_1' and id = 0"))
+checkAnswer(
+  checkSqlHitDataMap(s"select * from $bloomDMSampleTable where city = 
'city_999' and name='n999'", dataMapName, shouldHit),
+  sql(s"select * from $normalTable where city = 'city_999' and 
name='n999'"))
+checkAnswer(
+  checkSqlHitDataMap(s"select * from $bloomDMSampleTable where city = 
'city_999' and name='n1'", dataMapName, shouldHit),
+  sql(s"select * from $normalTable where city = 'city_999' and name='n1'"))
+checkAnswer(

Build failed in Jenkins: carbondata-master-spark-2.1 #2520

2018-06-20 Thread Apache Jenkins Server

See 


--
[...truncated 15.66 KB...]
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] ---

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2520

2018-06-20 Thread Apache Jenkins Server

See 


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.2 #561

2018-06-20 Thread Apache Jenkins Server

See 


--
[...truncated 15.79 KB...]
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 2 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.7 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] ---

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #561

2018-06-20 Thread Apache Jenkins Server

See 


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.7 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.1 #2519

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[chenliang613] [HOTFIX] Added Performance Optimization for Presto by using

--
[...truncated 15.39 KB...]
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 8 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2519

2018-06-20 Thread Apache Jenkins Server

See 


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.2 #560

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[chenliang613] [HOTFIX] Added Performance Optimization for Presto by using

--
[...truncated 15.80 KB...]
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #560

2018-06-20 Thread Apache Jenkins Server

See 


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO] -
[ERROR] COMPILATION ERROR : 
[INFO]

carbondata git commit: [HOTFIX] Added Performance Optimization for Presto by using MultiBlockSplit

2018-06-20 Thread chenliang613

Repository: carbondata
Updated Branches:
  refs/heads/master 0e1d550e8 -> 01b48fc36


[HOTFIX] Added Performance Optimization for Presto by using MultiBlockSplit

Added Performance Optimization for Presto by using MultiBlockSplit

This closes #2265


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/01b48fc3
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/01b48fc3
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/01b48fc3

Branch: refs/heads/master
Commit: 01b48fc36a93b74827edff4d3330cbf5546a5d38
Parents: 0e1d550
Author: Bhavya 
Authored: Mon Apr 16 11:54:17 2018 +0530
Committer: chenliang613 
Committed: Wed Jun 20 22:53:38 2018 +0800

--
 integration/presto/README.md|   9 +-
 integration/presto/pom.xml  |  14 +-
 .../carbondata/presto/CarbonVectorBatch.java|   2 +-
 .../carbondata/presto/CarbondataMetadata.java   |  87 +++-
 .../carbondata/presto/CarbondataPageSource.java |  11 +-
 .../presto/CarbondataPageSourceProvider.java|  27 ++--
 .../carbondata/presto/CarbondataSplit.java  |  24 +++-
 .../presto/CarbondataSplitManager.java  |  34 -
 .../PrestoCarbonVectorizedRecordReader.java |  39 +-
 .../presto/impl/CarbonLocalInputSplit.java  |  19 ++-
 .../presto/impl/CarbonLocalMultiBlockSplit.java |  86 
 .../presto/impl/CarbonTableConfig.java  |  40 +-
 .../presto/impl/CarbonTableReader.java  | 133 ---
 .../readers/DecimalSliceStreamReader.java   | 105 ---
 .../presto/src/main/resources/log4j.properties  |  11 ++
 15 files changed, 462 insertions(+), 179 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/01b48fc3/integration/presto/README.md
--
diff --git a/integration/presto/README.md b/integration/presto/README.md
index 53884a2..5a44f5c 100644
--- a/integration/presto/README.md
+++ b/integration/presto/README.md
@@ -77,6 +77,9 @@ Please follow the below steps to query carbondata in presto
   carbondata-store={schema-store-path}
   enable.unsafe.in.query.processing=false
   carbon.unsafe.working.memory.in.mb={value}
+  enable.unsafe.columnpage=false
+  enable.unsafe.sort=false
+
   ```
   Replace the schema-store-path with the absolute path of the parent directory 
of the schema.
   For example, if you have a schema named 'default' stored in 
hdfs://namenode:9000/test/carbondata/,
@@ -112,7 +115,11 @@ Please follow the below steps to query carbondata in presto
   Unsafe Properties
   enable.unsafe.in.query.processing property by default is true in CarbonData 
system, the carbon.unsafe.working.memory.in.mb 
   property defines the limit for Unsafe Memory usage in Mega Bytes, the 
default value is 512 MB.
-  If your tables are big you can increase the unsafe memory, or disable unsafe 
via setting enable.unsafe.in.query.processing=false.
+  Currently Presto does not support Unsafe Memory so we have to disable the 
unsafe feature by setting below properties to false.
+
+  enable.unsafe.in.query.processing=false.
+  enable.unsafe.columnpage=false
+  enable.unsafe.sort=false
 
   If you updated the jar balls or configuration files, make sure you have 
dispatched them
to all the presto nodes and restarted the presto servers on the nodes. The 
updates will not take effect before restarting.

http://git-wip-us.apache.org/repos/asf/carbondata/blob/01b48fc3/integration/presto/pom.xml
--
diff --git a/integration/presto/pom.xml b/integration/presto/pom.xml
index b91f070..c61023a 100644
--- a/integration/presto/pom.xml
+++ b/integration/presto/pom.xml
@@ -463,12 +463,6 @@
   provided
 
 
-  com.google.code.findbugs
-  jsr305
-  3.0.2
-  provided
-
-
   org.glassfish.hk2
   hk2-api
   2.5.0-b42
@@ -552,13 +546,7 @@
 src/test/scala
 
   
-src/resources
-  
-  
-.
-
-  CARBON_SPARK_INTERFACELogResource.properties
-
+src/main/resources
   
 
 

http://git-wip-us.apache.org/repos/asf/carbondata/blob/01b48fc3/integration/presto/src/main/java/org/apache/carbondata/presto/CarbonVectorBatch.java
--
diff --git 
a/integration/presto/src/main/java/org/apache/carbondata/presto/CarbonVectorBatch.java
 
b/integration/presto/src/main/java/org/apache/carbondata/presto/CarbonVectorBatch.java
index b230d6a..b6caaa3 100644
--- 
a/integration/presto/src/main/java/org/apache/carbondata/presto/CarbonVectorBatch.java
+++

Build failed in Jenkins: carbondata-master-spark-2.1 #2518

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[manishgupta88] [CARBONDATA-2617] Invalid tuple-id and block id getting formed 
for Non

--
[...truncated 15.93 KB...]
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 2 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/1/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.9 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2518

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[manishgupta88] [CARBONDATA-2617] Invalid tuple-id and block id getting formed 
for Non

--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.9 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.2 #559

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[manishgupta88] [CARBONDATA-2617] Invalid tuple-id and block id getting formed 
for Non

--
[...truncated 16.34 KB...]
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/0/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/0/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.6 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #559

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[manishgupta88] [CARBONDATA-2617] Invalid tuple-id and block id getting formed 
for Non

--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.6 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO]

carbondata git commit: [CARBONDATA-2617] Invalid tuple-id and block id getting formed for Non partition table

2018-06-20 Thread manishgupta88

Repository: carbondata
Updated Branches:
  refs/heads/master dc53dee24 -> 0e1d550e8


[CARBONDATA-2617] Invalid tuple-id and block id getting formed for Non 
partition table

Problem
Invalid tuple and block id getting formed for non partition table

Analysis
While creating a partition table a segment file was written in the Metadata 
folder under table structure. This was introduced during
development of partition table feature. At that time segment file was written 
only for partition table and it was used to distinguish between
parition and non partition table in the code. But later the code was modified 
to write the segment file for both parititon and non partition
table and the code to distinguish partition and non partition table was not 
modified which is causing this incorrect formation of block and tuple id.

Fix
Modify the logic to distinguish partitioned and non partitioned table and the 
same has been handled in this PR.

This closes #2385


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/0e1d550e
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/0e1d550e
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/0e1d550e

Branch: refs/heads/master
Commit: 0e1d550e8dacba798e9ffbdda25c4388e8933632
Parents: dc53dee
Author: rahul 
Authored: Tue Jun 19 19:23:26 2018 +0530
Committer: manishgupta88 
Committed: Wed Jun 20 16:37:23 2018 +0530

--
 .../core/mutate/CarbonUpdateUtil.java   |  4 +-
 .../executor/impl/AbstractQueryExecutor.java|  4 +-
 .../SegmentUpdateStatusManager.java | 20 ++---
 .../apache/carbondata/core/util/CarbonUtil.java |  4 +-
 .../iud/DeleteCarbonTableTestCase.scala | 83 
 .../command/mutation/DeleteExecution.scala  |  6 +-
 6 files changed, 100 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/0e1d550e/core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java 
b/core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
index 40d498c..8627bdb 100644
--- a/core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
+++ b/core/src/main/java/org/apache/carbondata/core/mutate/CarbonUpdateUtil.java
@@ -81,10 +81,10 @@ public class CarbonUpdateUtil {
   /**
* Returns block path from tuple id
*/
-  public static String getTableBlockPath(String tid, String tablePath, boolean 
isSegmentFile) {
+  public static String getTableBlockPath(String tid, String tablePath, boolean 
isPartitionTable) {
 String partField = getRequiredFieldFromTID(tid, TupleIdEnum.PART_ID);
 // If it has segment file then partfield can be appended directly to table 
path
-if (isSegmentFile) {
+if (isPartitionTable) {
   return tablePath + CarbonCommonConstants.FILE_SEPARATOR + 
partField.replace("#", "/");
 }
 String part = CarbonTablePath.addPartPrefix(partField);

http://git-wip-us.apache.org/repos/asf/carbondata/blob/0e1d550e/core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java
 
b/core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java
index 2bbe75c..f365045 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/scan/executor/impl/AbstractQueryExecutor.java
@@ -296,8 +296,8 @@ public abstract class AbstractQueryExecutor implements 
QueryExecutor {
 String blockId = CarbonUtil
 .getBlockId(queryModel.getAbsoluteTableIdentifier(), filePath, 
segment.getSegmentNo(),
 queryModel.getTable().getTableInfo().isTransactionalTable(),
-segment.getSegmentFileName() != null);
-if (segment.getSegmentFileName() != null) {
+queryModel.getTable().isHivePartitionTable());
+if (queryModel.getTable().isHivePartitionTable()) {
   
blockExecutionInfo.setBlockId(CarbonTablePath.getShortBlockIdForPartitionTable(blockId));
 } else {
   blockExecutionInfo.setBlockId(CarbonTablePath.getShortBlockId(blockId));

http://git-wip-us.apache.org/repos/asf/carbondata/blob/0e1d550e/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/statusmanager/SegmentUpdateStatusManager.java

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2517

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[kumarvishal09] [CARBONDATA-2420][32K] Support string longer than 32000 
characters

--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.5 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.1 #2517

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[kumarvishal09] [CARBONDATA-2420][32K] Support string longer than 32000 
characters

--
[...truncated 15.91 KB...]
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/0/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/0/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.5 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #558

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[kumarvishal09] [CARBONDATA-2420][32K] Support string longer than 32000 
characters

--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core ---
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default) @ carbondata-core ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 635 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] -
[WARNING] COMPILATION WARNING : 
[INFO] -
[WARNING] 
:[23,16]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[43,18]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[47,21]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[49,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[WARNING] 
:[70,17]
 sun.misc.Unsafe is internal proprietary API and may be removed in a future 
release
[INFO] 5 warnings 
[INFO] -
[INFO]

Build failed in Jenkins: carbondata-master-spark-2.2 #558

2018-06-20 Thread Apache Jenkins Server

See 


Changes:

[kumarvishal09] [CARBONDATA-2420][32K] Support string longer than 32000 
characters

--
[...truncated 16.32 KB...]
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 7 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 2 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 
/home/jenkins/jenkins-slave/maven-repositories/0/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.jar
[INFO] Installing 
 
to 
/home/jenkins/jenkins-slave/maven-repositories/0/org/apache/carbondata/carbondata-common/1.5.0-SNAPSHOT/carbondata-common-1.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 1.5.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloaded: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/1.5.0-SNAPSHOT/maven-metadata.xml
 (788 B at 1.8 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ carbondata-core ---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-core ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-core

[1/2] carbondata git commit: [CARBONDATA-2420][32K] Support string longer than 32000 characters

2018-06-20 Thread kumarvishal09

Repository: carbondata
Updated Branches:
  refs/heads/master c5a4ec07a -> dc53dee24


http://git-wip-us.apache.org/repos/asf/carbondata/blob/dc53dee2/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
--
diff --git 
a/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
 
b/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
new file mode 100644
index 000..419b306
--- /dev/null
+++ 
b/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
@@ -0,0 +1,279 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements.  See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License.  You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+package org.apache.carbondata.spark.testsuite.longstring
+
+import java.io.{File, PrintWriter}
+
+import org.apache.commons.lang3.RandomStringUtils
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import org.apache.carbondata.core.constants.CarbonCommonConstants
+import org.apache.carbondata.core.util.CarbonProperties
+
+class VarcharDataTypesBasicTestCase extends QueryTest with BeforeAndAfterEach 
with BeforeAndAfterAll {
+  private val longStringTable = "long_string_table"
+  private val inputDir = 
s"$resourcesPath${File.separator}varchartype${File.separator}"
+  private val fileName = s"longStringData.csv"
+  private val inputFile = s"$inputDir$fileName"
+  private val fileName_2g_column_page = 
s"longStringData_exceed_2gb_column_page.csv"
+  private val inputFile_2g_column_page = s"$inputDir$fileName_2g_column_page"
+  private val lineNum = 1000
+  private var content: Content = _
+  private var originMemorySize = CarbonProperties.getInstance().getProperty(
+CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB,
+CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB_DEFAULT)
+
+  case class Content(head: Int, desc_line_head: String, note_line_head: String,
+  mid: Int, desc_line_mid: String, note_line_mid: String,
+  tail: Int, desc_line_tail: String, note_line_tail: String)
+
+  override def beforeAll(): Unit = {
+// for one 32000 lines * 32000 characters column page, it use about 1GB 
memory, but here we have only 1000 lines
+CarbonProperties.getInstance().addProperty(
+  CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB,
+  CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB_DEFAULT)
+deleteFile(inputFile)
+if (!new File(inputDir).exists()) {
+  new File(inputDir).mkdir()
+}
+content = createFile(inputFile, line = lineNum)
+  }
+
+  override def afterAll(): Unit = {
+CarbonProperties.getInstance().addProperty(
+  CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB, originMemorySize)
+deleteFile(inputFile)
+deleteFile(inputFile_2g_column_page)
+if (new File(inputDir).exists()) {
+  new File(inputDir).delete()
+}
+  }
+
+  override def beforeEach(): Unit = {
+sql(s"drop table if exists $longStringTable")
+  }
+
+  override def afterEach(): Unit = {
+sql(s"drop table if exists $longStringTable")
+  }
+
+  private def prepareTable(): Unit = {
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED BY 'carbondata'
+ | TBLPROPERTIES('LONG_STRING_COLUMNS'='description, note', 
'SORT_COLUMNS'='name')
+ |""".stripMargin)
+sql(
+  s"""
+ | LOAD DATA LOCAL INPATH '$inputFile' INTO TABLE $longStringTable
+ | OPTIONS('header'='false')
+   """.stripMargin)
+  }
+
+  private def checkQuery(): Unit = {
+// query without long_string_column
+checkAnswer(sql(s"SELECT id, name, address FROM $longStringTable where id 
= ${content.tail}"),
+  Row(content.tail, s"name_${content.tail}", s"address_${content.tail}"))
+// query return long_string_column in the middle position
+checkAnswer(sql(s"SELECT id, name, description, address FROM

[2/2] carbondata git commit: [CARBONDATA-2420][32K] Support string longer than 32000 characters

2018-06-20 Thread kumarvishal09

[CARBONDATA-2420][32K] Support string longer than 32000 characters

Add a property in creating table 'long_string_columns' to support string 
columns that will contains more than 32000 characters.
Inside carbondata, it use an integer instead of short to store the length of 
bytes content.

Internally in Carbondata,

add a new datatype called varchar to represent the long string column
add a new encoding called DIRECT_COMPRESS_VARCHAR to the varcher column page 
meta
use an integer (previously short) to store the length of bytes content.
add 2GB constraint for one column page

This closes #2379


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/dc53dee2
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/dc53dee2
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/dc53dee2

Branch: refs/heads/master
Commit: dc53dee2448f366319764021d77c4be75d43b9e3
Parents: c5a4ec0
Author: xuchuanyin 
Authored: Sat Jun 2 15:17:04 2018 +0800
Committer: kumarvishal09 
Committed: Wed Jun 20 15:24:22 2018 +0530

--
 .../core/constants/CarbonCommonConstants.java   |   3 +
 .../impl/FixedLengthDimensionColumnPage.java|   2 +-
 .../impl/VariableLengthDimensionColumnPage.java |  11 +-
 ...mpressedDimensionChunkFileBasedReaderV1.java |   3 +-
 ...mpressedDimensionChunkFileBasedReaderV2.java |   3 +-
 ...mpressedDimensionChunkFileBasedReaderV3.java |   7 +-
 .../chunk/store/DimensionChunkStoreFactory.java |  22 +-
 ...ariableIntLengthDimensionDataChunkStore.java |  43 +++
 ...feVariableLengthDimensionDataChunkStore.java |  45 +--
 ...iableShortLengthDimensionDataChunkStore.java |  41 +++
 ...ariableIntLengthDimensionDataChunkStore.java |  44 +++
 ...feVariableLengthDimensionDataChunkStore.java |  54 ++--
 ...iableShortLengthDimensionDataChunkStore.java |  44 +++
 .../core/datastore/page/ColumnPage.java |  16 +-
 .../datastore/page/VarLengthColumnPageBase.java |   6 +
 .../page/encoding/DefaultEncodingFactory.java   |   1 +
 .../page/encoding/EncodingFactory.java  |   3 +-
 .../encoding/compress/DirectCompressCodec.java  |   6 +-
 .../legacy/HighCardDictDimensionIndexCodec.java |  13 +-
 .../statistics/LVLongStringStatsCollector.java  |  51 
 .../statistics/LVShortStringStatsCollector.java |  50 
 .../page/statistics/LVStringStatsCollector.java |  27 +-
 .../core/indexstore/UnsafeMemoryDMStore.java|  11 +-
 .../blockletindex/BlockletDataMap.java  |   8 +-
 .../core/indexstore/row/DataMapRow.java |   4 +-
 .../core/indexstore/row/UnsafeDataMapRow.java   |  60 ++--
 .../core/indexstore/schema/CarbonRowSchema.java |  10 +-
 .../core/metadata/blocklet/BlockletInfo.java|   2 +-
 .../ThriftWrapperSchemaConverterImpl.java   |   8 +
 .../core/metadata/datatype/DataType.java|   3 +
 .../core/metadata/datatype/DataTypes.java   |   5 +
 .../core/metadata/datatype/VarcharType.java |  34 +++
 .../core/metadata/encoder/Encoding.java |   5 +-
 .../schema/table/TableSchemaBuilder.java|   1 +
 .../util/AbstractDataFileFooterConverter.java   |   2 +
 .../apache/carbondata/core/util/CarbonUtil.java |   8 +-
 .../carbondata/core/util/DataTypeUtil.java  |   4 +
 .../ThriftWrapperSchemaConverterImplTest.java   |   2 +-
 format/src/main/thrift/schema.thrift|   3 +
 .../VarcharDataTypesBasicTestCase.scala | 279 +++
 .../carbondata/spark/util/CarbonScalaUtil.scala |   1 +
 .../spark/util/DataTypeConverterUtil.scala  |   1 +
 .../spark/sql/catalyst/CarbonDDLSqlParser.scala |  36 ++-
 .../command/carbonTableSchemaCommon.scala   |  43 ++-
 .../apache/spark/sql/hive/CarbonRelation.scala  |   1 +
 .../impl/NonDictionaryFieldConverterImpl.java   |  12 +-
 .../loading/csvinput/CSVInputFormat.java|   4 +-
 .../loading/row/IntermediateSortTempRow.java|  19 +-
 .../loading/sort/SortStepRowHandler.java|  26 +-
 .../merger/CompactionResultSortProcessor.java   |  11 +-
 .../sort/sortdata/SortParameters.java   |  21 +-
 .../sort/sortdata/TableFieldStat.java   |  37 ++-
 .../carbondata/processing/store/TablePage.java  |  37 ++-
 .../util/CarbonDataProcessorUtil.java   |  20 ++
 54 files changed, 1049 insertions(+), 164 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/dc53dee2/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
--
diff --git 
a/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 
b/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
index 5f06d08..118ff28 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
+++

Jenkins build is back to stable : carbondata-master-spark-2.1 #2516

2018-06-20 Thread Apache Jenkins Server

See

Jenkins build is back to stable : carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #2516

2018-06-20 Thread Apache Jenkins Server

See

[2/3] carbondata git commit: [CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create Table comand

2018-06-20 Thread kumarvishal09

http://git-wip-us.apache.org/repos/asf/carbondata/blob/c5a4ec07/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/localdictionary/LocalDictionarySupportCreateTableTest.scala
--
diff --git 
a/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/localdictionary/LocalDictionarySupportCreateTableTest.scala
 
b/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/localdictionary/LocalDictionarySupportCreateTableTest.scala
new file mode 100644
index 000..5f9af69
--- /dev/null
+++ 
b/integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/localdictionary/LocalDictionarySupportCreateTableTest.scala
@@ -0,0 +1,2102 @@
+package org.apache.carbondata.spark.testsuite.localdictionary
+
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+import 
org.apache.carbondata.common.exceptions.sql.MalformedCarbonCommandException
+
+class LocalDictionarySupportCreateTableTest extends QueryTest with 
BeforeAndAfterAll {
+
+  override protected def beforeAll(): Unit = {
+sql("DROP TABLE IF EXISTS LOCAL1")
+  }
+
+  test("test local dictionary default configuration") {
+sql("drop table if exists local1")
+sql(
+  """
+| CREATE TABLE local1(id int, name string, city string, age int)
+| STORED BY 'org.apache.carbondata.format'
+  """.stripMargin)
+
+val desc_result = sql("describe formatted local1")
+
+val descLoc = sql("describe formatted local1").collect
+descLoc.find(_.get(0).toString.contains("Local Dictionary Enabled")) match 
{
+  case Some(row) => assert(row.get(1).toString.contains("true"))
+}
+descLoc.find(_.get(0).toString.contains("Local Dictionary Threshold")) 
match {
+  case Some(row) => assert(row.get(1).toString.contains("1000"))
+}
+  }
+
+  test("test local dictionary custom configurations for local dict columns 
_001") {
+sql("drop table if exists local1")
+sql(
+  """
+| CREATE TABLE local1(id int, name string, city string, age int)
+| STORED BY 'org.apache.carbondata.format'
+| tblproperties('local_dictionary_include'='name')
+  """.
+stripMargin)
+val descFormatted1 = sql("describe formatted local1").collect
+descFormatted1.find(_.get(0).toString.contains("Local Dictionary 
Enabled")) match {
+  case Some(row) => assert(row.get(1).toString.contains("true"))
+}
+descFormatted1.find(_.get(0).toString.contains("Local Dictionary 
Include")) match {
+  case Some(row) => assert(row.get(1).toString.contains("name"))
+}
+  }
+
+  test(
+"test local dictionary custom configurations for local dict columns _002")
+  {
+sql("drop table if exists local1")
+
+intercept[MalformedCarbonCommandException] {
+  sql(
+"""
+  | CREATE TABLE local1(id int, name string, city string, age int)
+  | STORED BY 'org.apache.carbondata.format'
+  | tblproperties('local_dictionary_include'='name,name')
+""".stripMargin)
+}
+  }
+
+  test("test local dictionary custom configurations for local dict columns 
_003") {
+sql("drop table if exists local1")
+val exception = intercept[MalformedCarbonCommandException] {
+  sql(
+"""
+  | CREATE TABLE local1(id int, name string, city string, age int)
+  | STORED BY 'org.apache.carbondata.format'
+  | tblproperties('local_dictionary_include'='')
+""".
+  stripMargin)
+}
+assert(exception.getMessage
+  .contains(
+"LOCAL_DICTIONARY_INCLUDE/LOCAL_DICTIONARY_EXCLUDE column:  does not 
exist in table. Please check " +
+"create table statement"))
+  }
+
+  test("test local dictionary custom configurations for local dict columns 
_004") {
+sql("drop table if exists local1")
+val exception1 = intercept[MalformedCarbonCommandException] {
+  sql(
+"""
+  | CREATE TABLE local1(id int, name string, city string, age int)
+  | STORED BY 'org.apache.carbondata.format'
+  | tblproperties('local_dictionary_include'='abc')
+""".
+  stripMargin)
+}
+assert(exception1.getMessage
+  .contains(
+"LOCAL_DICTIONARY_INCLUDE/LOCAL_DICTIONARY_EXCLUDE column: abc does 
not exist in table. Please check " +
+"create table " +
+"statement"))
+  }
+
+  test("test local dictionary custom configurations for local dict columns 
_005") {
+sql("drop table if exists local1")
+val exception = intercept[MalformedCarbonCommandException] {
+  sql(
+"""
+  | CREATE TABLE local1(id int, name string, city string, age int)
+  | STORED BY 'org.apache.carbondata.format'
+  | tblproperties('local_dictionary_include'='id')
+""".
+  stripMargin)
+}
+assert(exception.getMessage
+

[3/3] carbondata git commit: [CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create Table comand

2018-06-20 Thread kumarvishal09

[CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create 
Table comand

What changes were proposed in this pull request?
In this PR, UTs and SDV test cases are added for local dictionary support for 
create table command and describe formatted command.

changed the error message for validation of local dictionary table properties 
and
fixed column name display without extra space in describe formatted command

This closes#2377


Project: http://git-wip-us.apache.org/repos/asf/carbondata/repo
Commit: http://git-wip-us.apache.org/repos/asf/carbondata/commit/c5a4ec07
Tree: http://git-wip-us.apache.org/repos/asf/carbondata/tree/c5a4ec07
Diff: http://git-wip-us.apache.org/repos/asf/carbondata/diff/c5a4ec07

Branch: refs/heads/master
Commit: c5a4ec07a8c03600741d6a2ae324a4fe4ab61659
Parents: be20fef
Author: praveenmeenakshi56 
Authored: Thu Jun 14 23:36:59 2018 +0530
Committer: kumarvishal09 
Committed: Wed Jun 20 12:13:09 2018 +0530

--
 ...CreateTableWithLocalDictionaryTestCase.scala | 2102 ++
 .../cluster/sdv/suite/SDVSuites.scala   |6 +-
 .../LocalDictionarySupportCreateTableTest.scala | 2102 ++
 .../spark/sql/catalyst/CarbonDDLSqlParser.scala |   31 +-
 .../table/CarbonDescribeFormattedCommand.scala  |2 +-
 5 files changed, 4227 insertions(+), 16 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/carbondata/blob/c5a4ec07/integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/CreateTableWithLocalDictionaryTestCase.scala
--
diff --git 
a/integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/CreateTableWithLocalDictionaryTestCase.scala
 
b/integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/CreateTableWithLocalDictionaryTestCase.scala
new file mode 100644
index 000..48a31b7
--- /dev/null
+++ 
b/integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/CreateTableWithLocalDictionaryTestCase.scala
@@ -0,0 +1,2102 @@
+package org.apache.carbondata.cluster.sdv.generated
+
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.BeforeAndAfterAll
+
+import 
org.apache.carbondata.common.exceptions.sql.MalformedCarbonCommandException
+
+class CreateTableWithLocalDictionaryTestCase extends QueryTest with 
BeforeAndAfterAll {
+
+  override protected def beforeAll(): Unit = {
+sql("DROP TABLE IF EXISTS LOCAL1")
+  }
+
+  test("test local dictionary default configuration") {
+sql("drop table if exists local1")
+sql(
+  """
+| CREATE TABLE local1(id int, name string, city string, age int)
+| STORED BY 'org.apache.carbondata.format'
+  """.stripMargin)
+
+val desc_result = sql("describe formatted local1")
+
+val descLoc = sql("describe formatted local1").collect
+descLoc.find(_.get(0).toString.contains("Local Dictionary Enabled")) match 
{
+  case Some(row) => assert(row.get(1).toString.contains("true"))
+}
+descLoc.find(_.get(0).toString.contains("Local Dictionary Threshold")) 
match {
+  case Some(row) => assert(row.get(1).toString.contains("1000"))
+}
+  }
+
+  test("test local dictionary custom configurations for local dict columns 
_001") {
+sql("drop table if exists local1")
+sql(
+  """
+| CREATE TABLE local1(id int, name string, city string, age int)
+| STORED BY 'org.apache.carbondata.format'
+| tblproperties('local_dictionary_include'='name')
+  """.
+stripMargin)
+val descFormatted1 = sql("describe formatted local1").collect
+descFormatted1.find(_.get(0).toString.contains("Local Dictionary 
Enabled")) match {
+  case Some(row) => assert(row.get(1).toString.contains("true"))
+}
+descFormatted1.find(_.get(0).toString.contains("Local Dictionary 
Include")) match {
+  case Some(row) => assert(row.get(1).toString.contains("name"))
+}
+  }
+
+  test(
+"test local dictionary custom configurations for local dict columns _002")
+  {
+sql("drop table if exists local1")
+
+intercept[MalformedCarbonCommandException] {
+  sql(
+"""
+  | CREATE TABLE local1(id int, name string, city string, age int)
+  | STORED BY 'org.apache.carbondata.format'
+  | tblproperties('local_dictionary_include'='name,name')
+""".stripMargin)
+}
+  }
+
+  test("test local dictionary custom configurations for local dict columns 
_003") {
+sql("drop table if exists local1")
+val exception = intercept[MalformedCarbonCommandException] {
+  sql(
+"""
+  | CREATE TABLE local1(id int, name string, city string, age int)
+  | STORED BY 'org.apache.carbondata.format'
+

[1/3] carbondata git commit: [CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create Table comand

2018-06-20 Thread kumarvishal09

Repository: carbondata
Updated Branches:
  refs/heads/master be20fefbe -> c5a4ec07a


http://git-wip-us.apache.org/repos/asf/carbondata/blob/c5a4ec07/integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
--
diff --git 
a/integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
 
b/integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
index 65ff76d..9af8817 100644
--- 
a/integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
+++ 
b/integration/spark-common/src/main/scala/org/apache/spark/sql/catalyst/CarbonDDLSqlParser.scala
@@ -321,9 +321,8 @@ abstract class CarbonDDLSqlParser extends 
AbstractCarbonSparkSQLParser {
 case scala.util.Failure(ex) =>
   LOGGER
 .debug(
-  "invalid value is configured for local_dictionary_threshold, 
considering the defaut" +
-  " " +
-  "value")
+  "invalid value is configured for local_dictionary_threshold, 
considering the " +
+  "default value")
   tableProperties.put(CarbonCommonConstants.LOCAL_DICTIONARY_THRESHOLD,
 CarbonCommonConstants.LOCAL_DICTIONARY_THRESHOLD_DEFAULT)
   }
@@ -362,9 +361,11 @@ abstract class CarbonDDLSqlParser extends 
AbstractCarbonSparkSQLParser {
   .distinct.length !=
 List(localDictIncludeCols, 
localDictExcludeCols).mkString(",").split(",")
   .length) {
+  val duplicateColumns = 
localDictIncludeCols.diff(localDictExcludeCols.distinct).distinct
   val errMsg =
-"Column ambiguity as duplicate columns present in 
LOCAL_DICTIONARY_INCLUDE and " +
-"LOCAL_DICTIONARY_INCLUDE.Duplicate columns are not allowed."
+"Column ambiguity as duplicate column(s):  " +
+duplicateColumns.mkString("(", ",", ")") + "are present in 
LOCAL_DICTIONARY_INCLUDE " +
+"and LOCAL_DICTIONARY_EXCLUDE. Duplicate columns are not allowed."
   throw new MalformedCarbonCommandException(errMsg)
 }
   }
@@ -411,10 +412,11 @@ abstract class CarbonDDLSqlParser extends 
AbstractCarbonSparkSQLParser {
 
 // check if the duplicate columns are specified in table schema
 if (localDictColumns.distinct.lengthCompare(localDictColumns.size) != 0) {
-  val a = localDictColumns.diff(localDictColumns.distinct).distinct
-  val errMsg = "LOCAL_DICTIONARY_INCLUDE/LOCAL_DICTIONARY_EXCLUDE contains 
Duplicate Columns " +
-   a.mkString("(", ",", ")") +
-   ". Please check create table statement."
+  val duplicateColumns = 
localDictColumns.diff(localDictColumns.distinct).distinct
+  val errMsg =
+"LOCAL_DICTIONARY_INCLUDE/LOCAL_DICTIONARY_EXCLUDE contains Duplicate 
Columns: " +
+duplicateColumns.mkString("(", ",", ")") +
+". Please check create table statement."
   throw new MalformedCarbonCommandException(errMsg)
 }
 
@@ -436,8 +438,9 @@ abstract class CarbonDDLSqlParser extends 
AbstractCarbonSparkSQLParser {
  !x.dataType.get.equalsIgnoreCase("ARRAY"))) {
 val errormsg = "LOCAL_DICTIONARY_INCLUDE/LOCAL_DICTIONARY_EXCLUDE 
column: " +
dictColm.trim +
-   " is not a String datatype column. 
LOCAL_DICTIONARY_COLUMN should be no " +
-   "dictionary string datatype column.Please check create 
table statement."
+   " is not a String/complex datatype column. 
LOCAL_DICTIONARY_COLUMN should " +
+   "be no dictionary string/complex datatype column.Please 
check create table" +
+   " statement."
 throw new MalformedCarbonCommandException(errormsg)
   }
 }
@@ -448,10 +451,12 @@ abstract class CarbonDDLSqlParser extends 
AbstractCarbonSparkSQLParser {
 
tableProperties(CarbonCommonConstants.DICTIONARY_INCLUDE).split(",").map(_.trim)
   localDictColumns.foreach { distCol =>
 if (dictIncludeColumns.exists(x => x.equalsIgnoreCase(distCol.trim))) {
+  val duplicateColumns = 
dictIncludeColumns.diff(localDictColumns.distinct).distinct
   val errormsg = "LOCAL_DICTIONARY_INCLUDE/LOCAL_DICTIONARY_EXCLUDE 
column: " +
- distCol.trim +
+ duplicateColumns.mkString("(", ",", ")") +
  " specified in Dictionary include. Local Dictionary 
will not be " +
- "generated for Dictionary include. Please check 
create table statement."
+ "generated for Dictionary include columns. Please 
check create table " +
+ "statement."
   throw new MalformedCarbonCommandException(errormsg)
 }
   }

Jenkins build is unstable: carbondata-master-spark-2.1 #2521

Jenkins build is unstable: carbondata-master-spark-2.1 » Apache CarbonData :: Spark Common Test #2521

Jenkins build is back to normal : carbondata-master-spark-2.1 » Apache CarbonData :: Core #2521

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Parent #563

Build failed in Jenkins: carbondata-master-spark-2.2 #563

Jenkins build is unstable: carbondata-master-spark-2.2 » Apache CarbonData :: Spark Common Test #563

carbondata git commit: [CARBONDATA-2615][32K] Support page size less than 32000 in CarbondataV3

[1/2] carbondata git commit: [CARBONDATA-2504][STREAM] Support StreamSQL for streaming job

[2/2] carbondata git commit: [CARBONDATA-2504][STREAM] Support StreamSQL for streaming job

carbondata git commit: [CARBONDATA-2616][BloomDataMap] Fix bugs in querying bloom datamap with two index columns

Build failed in Jenkins: carbondata-master-spark-2.1 #2520

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2520

Build failed in Jenkins: carbondata-master-spark-2.2 #561

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #561

Build failed in Jenkins: carbondata-master-spark-2.1 #2519

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2519

Build failed in Jenkins: carbondata-master-spark-2.2 #560

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #560

carbondata git commit: [HOTFIX] Added Performance Optimization for Presto by using MultiBlockSplit

Build failed in Jenkins: carbondata-master-spark-2.1 #2518

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2518

Build failed in Jenkins: carbondata-master-spark-2.2 #559

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #559

carbondata git commit: [CARBONDATA-2617] Invalid tuple-id and block id getting formed for Non partition table

Build failed in Jenkins: carbondata-master-spark-2.1 » Apache CarbonData :: Core #2517

Build failed in Jenkins: carbondata-master-spark-2.1 #2517

Build failed in Jenkins: carbondata-master-spark-2.2 » Apache CarbonData :: Core #558

Build failed in Jenkins: carbondata-master-spark-2.2 #558

[1/2] carbondata git commit: [CARBONDATA-2420][32K] Support string longer than 32000 characters

[2/2] carbondata git commit: [CARBONDATA-2420][32K] Support string longer than 32000 characters

Jenkins build is back to stable : carbondata-master-spark-2.1 #2516

Jenkins build is back to stable : carbondata-master-spark-2.1 » Apache CarbonData :: Spark2 #2516

[2/3] carbondata git commit: [CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create Table comand

[3/3] carbondata git commit: [CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create Table comand

[1/3] carbondata git commit: [CARBONDATA-2611] Added Test Cases for Local Dictionary Support for Create Table comand

35 matches

Site Navigation

Mail list logo

Footer information