svn commit: r39874 - /release/carbondata/2.0.1/

2020-06-01 Thread kunalkapoor
Author: kunalkapoor
Date: Mon Jun  1 16:05:41 2020
New Revision: 39874

Log:
Upload 2.0.1 release

Added:
release/carbondata/2.0.1/

release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar 
  (with props)

release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.asc

release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.sha512

release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar 
  (with props)

release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.asc

release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.sha512
release/carbondata/2.0.1/apache-carbondata-2.0.1-source-release.zip   (with 
props)
release/carbondata/2.0.1/apache-carbondata-2.0.1-source-release.zip.asc
release/carbondata/2.0.1/apache-carbondata-2.0.1-source-release.zip.md5
release/carbondata/2.0.1/apache-carbondata-2.0.1-source-release.zip.sha512

Added: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar
==
Binary file - no diff available.

Propchange: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar
--
svn:mime-type = application/octet-stream

Added: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.asc
==
--- 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.asc
 (added)
+++ 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.asc
 Mon Jun  1 16:05:41 2020
@@ -0,0 +1,14 @@
+-BEGIN PGP SIGNATURE-
+
+iQGzBAEBCgAdFiEE6JOJYvx7SBs+aFs9TIZi0mEeW3UFAl7U8BsACgkQTIZi0mEe
+W3U6jQwAptAV+iE+d82AkYs+pQPvzdJUlTzr7LzaGgTeILq6awj3ixYlGfscMYfs
+7VYzjmLqJ0/25V+tnwvCPYeaw4eEjesCWSsoiFbO29jrTNXhFi77scCiZMYaOo2T
+qBtGP/WqlHWKC5UEpe3bn3Sur7Zpl066MsE7WbCZsssIisufFwkjB/sEXNcbgiso
+JHEAGm0xjPCeTXlmzaeiEF1dwzI0nkVWNMkgK8GbPPp8RWvNqdG7Wfi+kuDGegFm
+uZCw3AVpsV3bmYHSHPr8OQldjd0ojIIncdCFgCLzRXDmNUX3jKqW3kE/g9mNuj3O
+QeP8pPjxf38vVrn/Jxvv4fnVDnL9KyTcRJhgOp0DWzlmDvMi+9l9Mb3htjOgYbyw
+tSpeQoPL4AO5fjFYr6bEqtdGg8FzLl+oR1SXPhAIGNaLRF5USAaK/0vXkwQZSDWV
+C1c0s4MIQNnYE1VijshwZukiJy7K+dJvjedcPflQANB5/gv67g+cP4+VwHyXxc6r
+zDYQ+Exo
+=kR/A
+-END PGP SIGNATURE-

Added: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.sha512
==
--- 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.sha512
 (added)
+++ 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar.sha512
 Mon Jun  1 16:05:41 2020
@@ -0,0 +1 @@
+9b714b7d9d930760ce574310e920dfc35db5f3aa2398b5695fa861afa7131d5de5d905ed955575f84ff7c5a66237c671209ea53b9173922def8dae19988d4b1c
  apache-carbondata-2.0.1-bin-spark2.3.4-hadoop2.7.2.jar

Added: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar
==
Binary file - no diff available.

Propchange: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar
--
svn:mime-type = application/octet-stream

Added: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.asc
==
--- 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.asc
 (added)
+++ 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.asc
 Mon Jun  1 16:05:41 2020
@@ -0,0 +1,14 @@
+-BEGIN PGP SIGNATURE-
+
+iQGzBAEBCgAdFiEE6JOJYvx7SBs+aFs9TIZi0mEeW3UFAl7U8IkACgkQTIZi0mEe
+W3U6ZwwAoCvQkKxRstU3PJ13GTQyCfSQF+/ytl268mf7HXzUObOc2q8Zgo2tNKQW
+bIKhODfoiO4pAEx2OwCGgPFHlxTlYs5+HZQSIto2m09nRwFt/s6aV5mFoOL+wy/X
+8uxF2gX2TAfBuP9sM3zaO0+EBb03+oKoUHYCqD16H+4K1v/4WRGI8SeNYSDO18Ys
+BZw4h5yIy9q2BrhObjkjVpck8EjUEAhEBr47prKBG/EpQC9HWuu4x7wt4tUM4Sb6
+M993xX/16KuVjQP+vJt2DILPu0+IxqikzcLUDjQ2ZtYVFGt4kyG+qCHmtPeR+Ii9
+0J7yVWUYXRR7nwl2rZA32cj5pT8h/MBu7U8kicVYSbXwWuqj65PdXsxJaeJO1z0/
+9KhhEKNchp03sCzSF5EPZiQ25KtrMPKllOyGqUp6AD6nrxIRTcYg7PeSt2VGANTY
+Scys2lC9AZ8IvCEp0d9o9k3rzBw0Ca1WisvOxBhUPYmfQgdL6mYAdtYN/t4PuhVY
+kGh7/8Dg
+=YfPE
+-END PGP SIGNATURE-

Added: 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.sha512
==
--- 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.sha512
 (added)
+++ 
release/carbondata/2.0.1/apache-carbondata-2.0.1-bin-spark2.4.5-hadoop2.7.2.jar.sha512
 Mon Jun  1 16:05:41 2020
@@ 

[carbondata] branch branch-2.0 updated: [maven-release-plugin] prepare for next development iteration

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
 new 05c3369  [maven-release-plugin] prepare for next development iteration
05c3369 is described below

commit 05c33697f0f4717927682768a6ac50f0529c2fea
Author: kunal642 
AuthorDate: Mon Jun 1 16:12:58 2020 +0530

[maven-release-plugin] prepare for next development iteration
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index bae9c4c..97afb90 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index e6f86c3..c4e62f7 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index eb237f9..8367df8 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index cd7a3d6..23eb398 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 6bda3d6..e60250f 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 61cd65f..7f8599a 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index e1a0ff1..248936b 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index b178e7a..5dc0380 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 28210d0..1114065 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index 10978ac..ab285ea 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 1aa242f..a2ab3d6 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index a024262..3cffbff 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index cf867ac..53f52a8 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml

[carbondata] 04/08: [CARBONDATA-3837] Fallback to the original plan when mv rewrite throw exception

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 86a6e88916eeb5995e65787021d600fa38cd17da
Author: QiangCai 
AuthorDate: Mon Jun 1 00:01:47 2020 +0800

[CARBONDATA-3837] Fallback to the original plan when mv rewrite throw 
exception

Why is this PR needed?
All plans are checking MVRewriteRule,
if MVRewriteRule throw an exception, it will lead to query failure

What changes were proposed in this PR?
Only the query should check MVRewriteRule, other plans should skip it 
quickly.

catch all exceptions of MVRewriteRule, and fallback to original plan.

Does this PR introduce any user interface change?

No
Is any new testcase added?

Yes

This closes #3777
---
 .../apache/carbondata/core/view/MVProvider.java| 20 ++---
 .../apache/carbondata/view/MVCatalogInSpark.scala  | 25 +++---
 .../apache/spark/sql/optimizer/MVRewriteRule.scala | 21 +++---
 3 files changed, 48 insertions(+), 18 deletions(-)

diff --git a/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java 
b/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
index 429f274..1259f91 100644
--- a/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
+++ b/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
@@ -149,14 +149,28 @@ public class MVProvider {
 
   private String getStatusFileName(MVManager viewManager, String databaseName) 
{
 String databaseLocation = viewManager.getDatabaseLocation(databaseName);
-return FileFactory.getCarbonFile(databaseLocation).getCanonicalPath() +
-CarbonCommonConstants.FILE_SEPARATOR + "_system" +
-CarbonCommonConstants.FILE_SEPARATOR + STATUS_FILE_NAME;
+try {
+  if (FileFactory.isFileExist(databaseLocation)) {
+return FileFactory.getCarbonFile(databaseLocation).getCanonicalPath()
++ CarbonCommonConstants.FILE_SEPARATOR + "_system"
++ CarbonCommonConstants.FILE_SEPARATOR + STATUS_FILE_NAME;
+  } else {
+// this database folder is not exists
+return null;
+  }
+} catch (IOException e) {
+  // avoid to impact other query on all databases because of mv failure on 
this database
+  LOG.warn("Failed to get mv status file for database " + databaseName, e);
+  return null;
+}
   }
 
   public List getStatusDetails(MVManager viewManager, String 
databaseName)
   throws IOException {
 String statusPath = this.getStatusFileName(viewManager, databaseName);
+if (statusPath == null) {
+  return Collections.emptyList();
+}
 Gson gsonObjectToRead = new Gson();
 DataInputStream dataInputStream = null;
 BufferedReader buffReader = null;
diff --git 
a/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
 
b/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
index b75a88b..c56b7e1 100644
--- 
a/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
+++ 
b/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
@@ -90,18 +90,19 @@ case class MVCatalogInSpark(session: SparkSession)
   def registerSchema(mvSchema: MVSchema): Unit = {
 withWriteLock {
   val currentDatabase = session.catalog.currentDatabase
-
-  // This is required because mv schemas are across databases, so while 
loading the
-  // catalog, if the mv is in database other than 
sparkSession.currentDataBase(), then it
-  // fails to register, so set the database present in the mvSchema Object
-  
session.catalog.setCurrentDatabase(mvSchema.getIdentifier.getDatabaseName)
-  val logicalPlan = MVHelper.dropDummyFunction(
-MVQueryParser.getQueryPlan(mvSchema.getQuery, session))
-  // here setting back to current database of current session, because if 
the actual query
-  // contains db name in query like, select db1.column1 from table and 
current database is
-  // default and if we drop the db1, still the session has current db as 
db1.
-  // So setting back to current database.
-  session.catalog.setCurrentDatabase(currentDatabase)
+  val logicalPlan = try {
+// This is required because mv schemas are across databases, so while 
loading the
+// catalog, if the mv is in database other than 
sparkSession.currentDataBase(), then it
+// fails to register, so set the database present in the mvSchema 
Object
+
session.catalog.setCurrentDatabase(mvSchema.getIdentifier.getDatabaseName)
+
MVHelper.dropDummyFunction(MVQueryParser.getQueryPlan(mvSchema.getQuery, 
session))
+  } finally {
+// here setting back to current database of current session, because 
if the actual query
+// contains db name in 

[carbondata] branch branch-2.0 created (now 2b03b0d)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at 2b03b0d  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1

This branch includes the following new commits:

 new 613d458  [maven-release-plugin] prepare for next development iteration
 new aefa10a  [HOTFIX] changed development version to 2.0.1
 new a3a27d2  [CARBONDATA-3835] Fix global sort issues
 new 86a6e88  [CARBONDATA-3837] Fallback to the original plan when mv 
rewrite throw exception
 new be50362  [CARBONDATA-3836] Fix metadata folder FileNotFoundException 
while creating new carbon table
 new 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
 new 8947655  [CARBONDATA-3840] Mark features as experimental
 new 2b03b0d  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1

The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




[carbondata] 07/08: [CARBONDATA-3840] Mark features as experimental

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 8947655194db7c76987e97ebb5d01bbadb587b8a
Author: kunal642 
AuthorDate: Mon Jun 1 13:07:24 2020 +0530

[CARBONDATA-3840] Mark features as experimental

Why is this PR needed?
Mark features as experimental because they are subject to change in future.

What changes were proposed in this PR?
Mark features as experimental because they are subject to change in future.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3783
---
 README.md | 8 
 1 file changed, 8 insertions(+)

diff --git a/README.md b/README.md
index 2d9fcc7..0293c4c 100644
--- a/README.md
+++ b/README.md
@@ -69,6 +69,14 @@ CarbonData is built using Apache Maven, to [build 
CarbonData](https://github.com
 * [Carbon as Spark's 
Datasource](https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md)
 
 * [FAQs](https://github.com/apache/carbondata/blob/master/docs/faq.md) 
 
+## Experimental Features
+
+Some features are marked as experimental because the syntax/implementation 
might change in the future.
+1. Hybrid format table using Add Segment.
+2. Accelerating performance using MV on parquet/orc.
+3. Merge API for Spark DataFrame.
+4. Hive write for non-transactional table.
+
 ##  Integration
 * [Hive](https://github.com/apache/carbondata/blob/master/docs/hive-guide.md)
 * 
[Presto](https://github.com/apache/carbondata/blob/master/docs/prestodb-guide.md)



[carbondata] 08/08: [maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 2b03b0d4d26fb8fad20bceed81fb14ad4fa82dae
Author: kunal642 
AuthorDate: Mon Jun 1 16:11:50 2020 +0530

[maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 25eac29..bae9c4c 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index c376e6e..e6f86c3 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 4eed3da..eb237f9 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index bb1fe3a..cd7a3d6 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 8753d92..6bda3d6 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index a0ca71d..61cd65f 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index d184910..e1a0ff1 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 0b7a57a..b178e7a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 1948c47..28210d0 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index b6f338b..10978ac 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 2d14e7b..1aa242f 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index c05b334..a024262 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index 647a523..cf867ac 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 3d375f7..1a6e7fa 100644
--- a/integration/flink-proxy/pom.xml
+++ 

[carbondata] 02/08: [HOTFIX] changed development version to 2.0.1

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit aefa10aee89c78101262b27f191c5758bd8532b8
Author: kunal642 
AuthorDate: Sat May 30 11:31:36 2020 +0530

[HOTFIX] changed development version to 2.0.1
---
 assembly/pom.xml  | 2 +-
 common/pom.xml| 2 +-
 core/pom.xml  | 2 +-
 examples/flink/pom.xml| 2 +-
 examples/spark/pom.xml| 2 +-
 format/pom.xml| 2 +-
 geo/pom.xml   | 2 +-
 hadoop/pom.xml| 2 +-
 index/bloom/pom.xml   | 2 +-
 index/examples/pom.xml| 2 +-
 index/lucene/pom.xml  | 2 +-
 index/secondary-index/pom.xml | 2 +-
 integration/flink-build/pom.xml   | 2 +-
 integration/flink-proxy/pom.xml   | 2 +-
 integration/flink/pom.xml | 2 +-
 integration/hive/pom.xml  | 2 +-
 integration/presto/pom.xml| 2 +-
 integration/spark-common-cluster-test/pom.xml | 2 +-
 integration/spark/pom.xml | 2 +-
 mv/plan/pom.xml   | 2 +-
 pom.xml   | 2 +-
 processing/pom.xml| 2 +-
 sdk/sdk/pom.xml   | 2 +-
 streaming/pom.xml | 2 +-
 tools/cli/pom.xml | 2 +-
 25 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 425e3e7..25eac29 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index bbf1593..c376e6e 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 5cf11a7..4eed3da 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index 8f4f371..bb1fe3a 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 715732c..8753d92 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 9017f4b..a0ca71d 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index 606089f..d184910 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 69cc55a..0b7a57a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index e58ac25..1948c47 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index aa2030f..b6f338b 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 3dff9e7..2d14e7b 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index c363c65..c05b334 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git 

[carbondata] annotated tag apache-carbondata-2.0.1-rc1 created (now b116752)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at b116752  (tag)
 tagging 2b03b0d4d26fb8fad20bceed81fb14ad4fa82dae (commit)
 replaces apache-carbondata-2.0.0-rc3
  by kunal642
  on Mon Jun 1 16:12:41 2020 +0530

- Log -
[maven-release-plugin] copy for tag apache-carbondata-2.0.1-rc1
---

No new revisions were added by this update.



[carbondata] 03/08: [CARBONDATA-3835] Fix global sort issues

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit a3a27d219d020ae3b0ba6a4565b88758b613c6c4
Author: ajantha-bhat 
AuthorDate: Sat May 30 08:24:39 2020 +0530

[CARBONDATA-3835] Fix global sort issues

Why is this PR needed?
For global sort without partition, string comes as byte[], if we use string 
comparator (StringSerializableComparator) it will convert byte[] to toString 
which gives address and comparison goes wrong.
For global sort with partition, when sort column is partition column, it 
was sorting on first column instead of partition columns.

What changes were proposed in this PR?
change data type to byte before choosing a comparator.
get the sorted column based on index, don't just take from first

Does this PR introduce any user interface change?
No

Is any new testcase added?
Yes

This closes #3779
---
 .../spark/load/DataLoadProcessBuilderOnSpark.scala | 15 --
 .../carbondata/spark/load/GlobalSortHelper.scala   |  8 ++-
 .../command/management/CommonLoadUtils.scala   |  7 ++-
 .../org/apache/spark/sql/test/util/QueryTest.scala | 17 +-
 .../StandardPartitionGlobalSortTestCase.scala  | 60 ++
 5 files changed, 94 insertions(+), 13 deletions(-)

diff --git 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
index 55eee11..e7e1baf 100644
--- 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
+++ 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
@@ -32,6 +32,7 @@ import org.apache.spark.sql.{DataFrame, SparkSession}
 import org.apache.spark.sql.catalyst.expressions.GenericInternalRow
 import org.apache.spark.sql.catalyst.InternalRow
 import org.apache.spark.sql.execution.command.ExecutionErrors
+import org.apache.spark.sql.types.{ByteType, DateType, LongType, StringType, 
TimestampType}
 import org.apache.spark.sql.util.{SparkSQLUtil, SparkTypeConverter}
 import org.apache.spark.storage.StorageLevel
 import org.apache.spark.unsafe.types.UTF8String
@@ -237,10 +238,15 @@ object DataLoadProcessBuilderOnSpark {
 CarbonProperties.getInstance().getGlobalSortRddStorageLevel()))
 }
 val sortColumnsLength = 
model.getCarbonDataLoadSchema.getCarbonTable.getSortColumns.size()
-val sortColumnDataTypes = dataTypes.take(sortColumnsLength)
-val rowComparator = 
GlobalSortHelper.generateRowComparator(sortColumnDataTypes)
+var sortColumnDataTypes = dataTypes.take(sortColumnsLength)
+sortColumnDataTypes = sortColumnDataTypes.map {
+  case StringType => ByteType
+  case TimestampType | DateType => LongType
+  case datatype => datatype
+}
+val rowComparator = 
GlobalSortHelper.generateRowComparator(sortColumnDataTypes.zipWithIndex)
 val sortRDD = rdd.sortBy(row =>
-  getKey(row, sortColumnsLength, sortColumnDataTypes),
+  getKey(row, sortColumnsLength),
   true,
   numPartitions)(
   rowComparator, classTag[Array[AnyRef]])
@@ -273,8 +279,7 @@ object DataLoadProcessBuilderOnSpark {
   }
 
   def getKey(row: Array[AnyRef],
-  sortColumnsLength: Int,
-  dataTypes: Seq[org.apache.spark.sql.types.DataType]): Array[AnyRef] = {
+  sortColumnsLength: Int): Array[AnyRef] = {
 val key: Array[AnyRef] = new Array[AnyRef](sortColumnsLength)
 System.arraycopy(row, 0, key, 0, sortColumnsLength)
 key
diff --git 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
index 91ed27e..00891b9 100644
--- 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
+++ 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
@@ -50,7 +50,7 @@ object GlobalSortHelper {
 
   def sortBy(updatedRdd: RDD[InternalRow],
   numPartitions: Int,
-  dataTypes: Seq[DataType]
+  dataTypes: Seq[(DataType, Int)]
   ): RDD[InternalRow] = {
 val keyExtractors = generateKeyExtractor(dataTypes)
 val rowComparator = generateRowComparator(dataTypes)
@@ -68,9 +68,8 @@ object GlobalSortHelper {
 key
   }
 
-  def generateKeyExtractor(dataTypes: Seq[DataType]): Array[KeyExtractor] = {
+  def generateKeyExtractor(dataTypes: Seq[(DataType, Int)]): 
Array[KeyExtractor] = {
 dataTypes
-  .zipWithIndex
   .map { attr =>
 attr._1 match {
   case StringType => UTF8StringKeyExtractor(attr._2)
@@ -91,9 +90,8 @@ object GlobalSortHelper {
   .toArray
   }
 
-  def generateRowComparator(dataTypes: Seq[DataType]): 

[carbondata] 01/08: [maven-release-plugin] prepare for next development iteration

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 613d458e85fe392d7bc2ce579285cae1be2e42a7
Author: kunal642 
AuthorDate: Sun May 17 13:28:30 2020 +0530

[maven-release-plugin] prepare for next development iteration
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index ca3f10c..425e3e7 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index f055fc3..bbf1593 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 8414b00..5cf11a7 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index 57e8398..8f4f371 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index b96afbd..715732c 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 9aaa631..9017f4b 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index f614997..606089f 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 24be836..69cc55a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 78fd642..e58ac25 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index f32e010..aa2030f 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 4d00226..3dff9e7 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index 626985f..c363c65 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index 26f0872..70aa992 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 22e305d..dc91066 100644
--- a/integration/flink-proxy/pom.xml
+++ 

[carbondata] 05/08: [CARBONDATA-3836] Fix metadata folder FileNotFoundException while creating new carbon table

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit be50362f14511e014a45b2eff7ecffd579a3b877
Author: Manhua 
AuthorDate: Sat May 30 10:47:10 2020 +0800

[CARBONDATA-3836] Fix metadata folder FileNotFoundException while creating 
new carbon table

Why is this PR needed?
1. In the case of using carbon only setting carbon.storelocation, carbon 
will use local spark warehouse path instead of the configured value.
2. FileNotFoundException is thrown when creating schema file for a brand 
new table. Because current implementation gets the schema file path by listing 
the Metadata directory which has not been created.

What changes were proposed in this PR?
1. spark.sql.warehouse.dir has its own default value in Spark, remove using 
carbonStorePath as default value, which will make 
hiveStorePath.equals(carbonStorePath) TRUE when user not set 
spark.sql.warehouse.dir.
2. create the Metadata directory before getting the schema file path.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3780
---
 integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala | 2 +-
 .../main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala| 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git 
a/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala 
b/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
index 9df5809..5062a43 100644
--- a/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
+++ b/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
@@ -328,7 +328,7 @@ object CarbonEnv {
 if ((!EnvHelper.isLegacy(sparkSession)) &&
 (dbName.equals("default") || databaseLocation.endsWith(".db"))) {
   val carbonStorePath = CarbonProperties.getStorePath()
-  val hiveStorePath = sparkSession.conf.get("spark.sql.warehouse.dir", 
carbonStorePath)
+  val hiveStorePath = sparkSession.conf.get("spark.sql.warehouse.dir")
   // if carbon.store does not point to spark.sql.warehouse.dir then follow 
the old table path
   // format
   if (carbonStorePath != null && !hiveStorePath.equals(carbonStorePath)) {
diff --git 
a/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
 
b/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
index 5156798..4c5f16d 100644
--- 
a/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
+++ 
b/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
@@ -402,8 +402,7 @@ class CarbonFileMetastore extends CarbonMetaStore {
   private def createSchemaThriftFile(
   identifier: AbsoluteTableIdentifier,
   thriftTableInfo: TableInfo): String = {
-val schemaFilePath = 
CarbonTablePath.getSchemaFilePath(identifier.getTablePath)
-val schemaMetadataPath = 
CarbonTablePath.getFolderContainingFile(schemaFilePath)
+val schemaMetadataPath = 
CarbonTablePath.getMetadataPath(identifier.getTablePath)
 if (!FileFactory.isFileExist(schemaMetadataPath)) {
   val isDirCreated = FileFactory
 .mkdirs(schemaMetadataPath, 
SparkSession.getActiveSession.get.sessionState.newHadoopConf())
@@ -411,6 +410,7 @@ class CarbonFileMetastore extends CarbonMetaStore {
 throw new IOException(s"Failed to create the metadata directory 
$schemaMetadataPath")
   }
 }
+val schemaFilePath = 
CarbonTablePath.getSchemaFilePath(identifier.getTablePath)
 val thriftWriter = new ThriftWriter(schemaFilePath, false)
 thriftWriter.open(FileWriteOperation.OVERWRITE)
 thriftWriter.write(thriftTableInfo)



[carbondata] 06/08: [CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 316ea18a8e3825090f1e9083b67dee278bb9a98c
Author: akashrn5 
AuthorDate: Mon Jun 1 09:58:31 2020 +0530

[CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

Why is this PR needed?
Rename file fails in HDFS when the FS object is of FilterFileSystem,
(which basically can contain any filesystem to use as basic filesystem)

What changes were proposed in this PR?
While rename force, have a check for this FS object

This closes #3781
---
 .../apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java | 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
index ac508c3..7d36f0e 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
@@ -23,6 +23,7 @@ import org.apache.carbondata.common.logging.LogServiceFactory;
 
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FilterFileSystem;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hdfs.DistributedFileSystem;
 import org.apache.log4j.Logger;
@@ -77,6 +78,11 @@ public class HDFSCarbonFile extends AbstractDFSCarbonFile {
 ((DistributedFileSystem) fileSystem).rename(path, new 
Path(changetoName),
 org.apache.hadoop.fs.Options.Rename.OVERWRITE);
 return true;
+  } else if ((fileSystem instanceof FilterFileSystem) && 
(((FilterFileSystem) fileSystem)
+  .getRawFileSystem() instanceof DistributedFileSystem)) {
+((DistributedFileSystem) ((FilterFileSystem) 
fileSystem).getRawFileSystem())
+.rename(path, new Path(changetoName), 
org.apache.hadoop.fs.Options.Rename.OVERWRITE);
+return true;
   } else {
 return fileSystem.rename(path, new Path(changetoName));
   }



[carbondata] annotated tag apache-carbondata-2.0.1-rc1 deleted (was 7b06f10)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


*** WARNING: tag apache-carbondata-2.0.1-rc1 was deleted! ***

   tag was  7b06f10

This change permanently discards the following revisions:

 discard 0821c7a  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1
 discard 8947655  [CARBONDATA-3840] Mark features as experimental
 discard 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
 discard be50362  [CARBONDATA-3836] Fix metadata folder FileNotFoundException 
while creating new carbon table
 discard 86a6e88  [CARBONDATA-3837] Fallback to the original plan when mv 
rewrite throw exception
 discard a3a27d2  [CARBONDATA-3835] Fix global sort issues
 discard aefa10a  [HOTFIX] changed development version to 2.0.1
 discard 613d458  [maven-release-plugin] prepare for next development iteration



[carbondata] branch branch-2.0 deleted (was eb98c21)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 was eb98c21  [maven-release-plugin] prepare for next development iteration

This change permanently discards the following revisions:

 discard eb98c21  [maven-release-plugin] prepare for next development iteration
 discard c2889fe  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1



[carbondata] 01/01: [maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 0821c7a857342ccd8af6d1114736443d07c5197f
Author: kunal642 
AuthorDate: Mon Jun 1 15:53:16 2020 +0530

[maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 25eac29..bae9c4c 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index c376e6e..e6f86c3 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 4eed3da..eb237f9 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index bb1fe3a..cd7a3d6 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 8753d92..6bda3d6 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index a0ca71d..61cd65f 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index d184910..e1a0ff1 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 0b7a57a..b178e7a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 1948c47..28210d0 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index b6f338b..10978ac 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 2d14e7b..1aa242f 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index c05b334..a024262 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index 647a523..cf867ac 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 3d375f7..1a6e7fa 100644
--- a/integration/flink-proxy/pom.xml

[carbondata] annotated tag apache-carbondata-2.0.1-rc1 deleted (was 8569d80)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


*** WARNING: tag apache-carbondata-2.0.1-rc1 was deleted! ***

   tag was  8569d80

The revisions that were on this annotated tag are still contained in
other references; therefore, this change does not discard any commits
from the repository.



[carbondata] annotated tag apache-carbondata-2.0.1-rc1 created (now 7b06f10)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at 7b06f10  (tag)
 tagging 0821c7a857342ccd8af6d1114736443d07c5197f (commit)
 replaces apache-carbondata-2.0.0-rc3
  by kunal642
  on Mon Jun 1 15:59:37 2020 +0530

- Log -
[maven-release-plugin] copy for tag apache-carbondata-2.0.1-rc1
---

This annotated tag includes the following new commits:

 new 0821c7a  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




svn commit: r39872 - in /dev/carbondata/2.0.1-rc1: ./ apache-carbondata-2.0.1-source-release.zip apache-carbondata-2.0.1-source-release.zip.asc apache-carbondata-2.0.1-source-release.zip.md5 apache-ca

2020-06-01 Thread kunalkapoor
Author: kunalkapoor
Date: Mon Jun  1 09:54:27 2020
New Revision: 39872

Log:
Upload 2.0.1-rc1

Added:
dev/carbondata/2.0.1-rc1/
dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip   (with 
props)
dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.asc
dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.md5
dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.sha512

Added: dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip
==
Binary file - no diff available.

Propchange: dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip
--
svn:mime-type = application/octet-stream

Added: dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.asc
==
--- dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.asc 
(added)
+++ dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.asc Mon 
Jun  1 09:54:27 2020
@@ -0,0 +1,14 @@
+-BEGIN PGP SIGNATURE-
+
+iQGzBAABCgAdFiEE6JOJYvx7SBs+aFs9TIZi0mEeW3UFAl7Uy8YACgkQTIZi0mEe
+W3WvNwv/aJPCXh8rp69aBfv8shM9Zd/HqzPeqsmEDORH8EvUALcAZtYb7RITThm0
+J1uOuKrZrob/Z2Ib8YYkku+ysyatis8OZ43MSxZcwiUr8Ylxtn9kDEaEpxKdYCNh
+MswbHFko03AhrCmuNrUuqHD8veiLxElnOaeohcdsRD/+jLs6zbVBB6Jb/wDiHCFs
+cOGeP3Xp4Cq36UHGzLGHe6eU1FG9B50qcX/ewHN27MoQUMG6KobZtnAaruL9t30Y
+/WyycI9AL61b2PMzO5BA+gnRQgttyhhBtwuQmt3olNRYOGgd0FOt68Qp4gxNZz5k
+8tDtQaAceuWJHQYwbVn+Z5R9aaZ2gtgDE7PiKqe5N3lZe0asaALBA0Zmwf7kqsd1
+dRHNpl03Zkx6jbjMK3RoE6QXnXaBl0XWfmaCwsYWv8oJtNnHV5zBy7Rbx8LZB/2k
+Y7geUHx1VDWFX/oBbHkR3xzEP/38dCg1/lCO+I/Ng5LTNTXQxE4svX2YxOw64ck3
++OoTL+bd
+=s7Fz
+-END PGP SIGNATURE-

Added: dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.md5
==
--- dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.md5 
(added)
+++ dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.md5 Mon 
Jun  1 09:54:27 2020
@@ -0,0 +1 @@
+1132c973f4fa2899559d624a64b374a9  apache-carbondata-2.0.1-source-release.zip

Added: 
dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.sha512
==
--- dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.sha512 
(added)
+++ dev/carbondata/2.0.1-rc1/apache-carbondata-2.0.1-source-release.zip.sha512 
Mon Jun  1 09:54:27 2020
@@ -0,0 +1 @@
+05951f45cf4316898fb184ccb82ab447f479e20c6ce1ccc5ed5ec146b1322de055fbf410c582b13dc8a88d42a533534d719e9de35145df8d943322c1bdae7bc3
  apache-carbondata-2.0.1-source-release.zip




[carbondata] 01/01: [maven-release-plugin] prepare for next development iteration

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit eb98c219c471ca1df83ce98d9e45fb308db71d6d
Author: kunal642 
AuthorDate: Mon Jun 1 15:14:10 2020 +0530

[maven-release-plugin] prepare for next development iteration
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index bae9c4c..97afb90 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index e6f86c3..c4e62f7 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index eb237f9..8367df8 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index cd7a3d6..23eb398 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 6bda3d6..e60250f 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 61cd65f..7f8599a 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index e1a0ff1..248936b 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index b178e7a..5dc0380 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 28210d0..1114065 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index 10978ac..ab285ea 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 1aa242f..a2ab3d6 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index a024262..3cffbff 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index cf867ac..53f52a8 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 1a6e7fa..c000999 100644
--- a/integration/flink-proxy/pom.xml
+++ 

[carbondata] 04/08: [CARBONDATA-3837] Fallback to the original plan when mv rewrite throw exception

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 86a6e88916eeb5995e65787021d600fa38cd17da
Author: QiangCai 
AuthorDate: Mon Jun 1 00:01:47 2020 +0800

[CARBONDATA-3837] Fallback to the original plan when mv rewrite throw 
exception

Why is this PR needed?
All plans are checking MVRewriteRule,
if MVRewriteRule throw an exception, it will lead to query failure

What changes were proposed in this PR?
Only the query should check MVRewriteRule, other plans should skip it 
quickly.

catch all exceptions of MVRewriteRule, and fallback to original plan.

Does this PR introduce any user interface change?

No
Is any new testcase added?

Yes

This closes #3777
---
 .../apache/carbondata/core/view/MVProvider.java| 20 ++---
 .../apache/carbondata/view/MVCatalogInSpark.scala  | 25 +++---
 .../apache/spark/sql/optimizer/MVRewriteRule.scala | 21 +++---
 3 files changed, 48 insertions(+), 18 deletions(-)

diff --git a/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java 
b/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
index 429f274..1259f91 100644
--- a/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
+++ b/core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
@@ -149,14 +149,28 @@ public class MVProvider {
 
   private String getStatusFileName(MVManager viewManager, String databaseName) 
{
 String databaseLocation = viewManager.getDatabaseLocation(databaseName);
-return FileFactory.getCarbonFile(databaseLocation).getCanonicalPath() +
-CarbonCommonConstants.FILE_SEPARATOR + "_system" +
-CarbonCommonConstants.FILE_SEPARATOR + STATUS_FILE_NAME;
+try {
+  if (FileFactory.isFileExist(databaseLocation)) {
+return FileFactory.getCarbonFile(databaseLocation).getCanonicalPath()
++ CarbonCommonConstants.FILE_SEPARATOR + "_system"
++ CarbonCommonConstants.FILE_SEPARATOR + STATUS_FILE_NAME;
+  } else {
+// this database folder is not exists
+return null;
+  }
+} catch (IOException e) {
+  // avoid to impact other query on all databases because of mv failure on 
this database
+  LOG.warn("Failed to get mv status file for database " + databaseName, e);
+  return null;
+}
   }
 
   public List getStatusDetails(MVManager viewManager, String 
databaseName)
   throws IOException {
 String statusPath = this.getStatusFileName(viewManager, databaseName);
+if (statusPath == null) {
+  return Collections.emptyList();
+}
 Gson gsonObjectToRead = new Gson();
 DataInputStream dataInputStream = null;
 BufferedReader buffReader = null;
diff --git 
a/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
 
b/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
index b75a88b..c56b7e1 100644
--- 
a/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
+++ 
b/integration/spark/src/main/scala/org/apache/carbondata/view/MVCatalogInSpark.scala
@@ -90,18 +90,19 @@ case class MVCatalogInSpark(session: SparkSession)
   def registerSchema(mvSchema: MVSchema): Unit = {
 withWriteLock {
   val currentDatabase = session.catalog.currentDatabase
-
-  // This is required because mv schemas are across databases, so while 
loading the
-  // catalog, if the mv is in database other than 
sparkSession.currentDataBase(), then it
-  // fails to register, so set the database present in the mvSchema Object
-  
session.catalog.setCurrentDatabase(mvSchema.getIdentifier.getDatabaseName)
-  val logicalPlan = MVHelper.dropDummyFunction(
-MVQueryParser.getQueryPlan(mvSchema.getQuery, session))
-  // here setting back to current database of current session, because if 
the actual query
-  // contains db name in query like, select db1.column1 from table and 
current database is
-  // default and if we drop the db1, still the session has current db as 
db1.
-  // So setting back to current database.
-  session.catalog.setCurrentDatabase(currentDatabase)
+  val logicalPlan = try {
+// This is required because mv schemas are across databases, so while 
loading the
+// catalog, if the mv is in database other than 
sparkSession.currentDataBase(), then it
+// fails to register, so set the database present in the mvSchema 
Object
+
session.catalog.setCurrentDatabase(mvSchema.getIdentifier.getDatabaseName)
+
MVHelper.dropDummyFunction(MVQueryParser.getQueryPlan(mvSchema.getQuery, 
session))
+  } finally {
+// here setting back to current database of current session, because 
if the actual query
+

[carbondata] 05/08: [CARBONDATA-3836] Fix metadata folder FileNotFoundException while creating new carbon table

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit be50362f14511e014a45b2eff7ecffd579a3b877
Author: Manhua 
AuthorDate: Sat May 30 10:47:10 2020 +0800

[CARBONDATA-3836] Fix metadata folder FileNotFoundException while creating 
new carbon table

Why is this PR needed?
1. In the case of using carbon only setting carbon.storelocation, carbon 
will use local spark warehouse path instead of the configured value.
2. FileNotFoundException is thrown when creating schema file for a brand 
new table. Because current implementation gets the schema file path by listing 
the Metadata directory which has not been created.

What changes were proposed in this PR?
1. spark.sql.warehouse.dir has its own default value in Spark, remove using 
carbonStorePath as default value, which will make 
hiveStorePath.equals(carbonStorePath) TRUE when user not set 
spark.sql.warehouse.dir.
2. create the Metadata directory before getting the schema file path.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3780
---
 integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala | 2 +-
 .../main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala| 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git 
a/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala 
b/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
index 9df5809..5062a43 100644
--- a/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
+++ b/integration/spark/src/main/scala/org/apache/spark/sql/CarbonEnv.scala
@@ -328,7 +328,7 @@ object CarbonEnv {
 if ((!EnvHelper.isLegacy(sparkSession)) &&
 (dbName.equals("default") || databaseLocation.endsWith(".db"))) {
   val carbonStorePath = CarbonProperties.getStorePath()
-  val hiveStorePath = sparkSession.conf.get("spark.sql.warehouse.dir", 
carbonStorePath)
+  val hiveStorePath = sparkSession.conf.get("spark.sql.warehouse.dir")
   // if carbon.store does not point to spark.sql.warehouse.dir then follow 
the old table path
   // format
   if (carbonStorePath != null && !hiveStorePath.equals(carbonStorePath)) {
diff --git 
a/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
 
b/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
index 5156798..4c5f16d 100644
--- 
a/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
+++ 
b/integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonFileMetastore.scala
@@ -402,8 +402,7 @@ class CarbonFileMetastore extends CarbonMetaStore {
   private def createSchemaThriftFile(
   identifier: AbsoluteTableIdentifier,
   thriftTableInfo: TableInfo): String = {
-val schemaFilePath = 
CarbonTablePath.getSchemaFilePath(identifier.getTablePath)
-val schemaMetadataPath = 
CarbonTablePath.getFolderContainingFile(schemaFilePath)
+val schemaMetadataPath = 
CarbonTablePath.getMetadataPath(identifier.getTablePath)
 if (!FileFactory.isFileExist(schemaMetadataPath)) {
   val isDirCreated = FileFactory
 .mkdirs(schemaMetadataPath, 
SparkSession.getActiveSession.get.sessionState.newHadoopConf())
@@ -411,6 +410,7 @@ class CarbonFileMetastore extends CarbonMetaStore {
 throw new IOException(s"Failed to create the metadata directory 
$schemaMetadataPath")
   }
 }
+val schemaFilePath = 
CarbonTablePath.getSchemaFilePath(identifier.getTablePath)
 val thriftWriter = new ThriftWriter(schemaFilePath, false)
 thriftWriter.open(FileWriteOperation.OVERWRITE)
 thriftWriter.write(thriftTableInfo)



[carbondata] 02/08: [HOTFIX] changed development version to 2.0.1

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit aefa10aee89c78101262b27f191c5758bd8532b8
Author: kunal642 
AuthorDate: Sat May 30 11:31:36 2020 +0530

[HOTFIX] changed development version to 2.0.1
---
 assembly/pom.xml  | 2 +-
 common/pom.xml| 2 +-
 core/pom.xml  | 2 +-
 examples/flink/pom.xml| 2 +-
 examples/spark/pom.xml| 2 +-
 format/pom.xml| 2 +-
 geo/pom.xml   | 2 +-
 hadoop/pom.xml| 2 +-
 index/bloom/pom.xml   | 2 +-
 index/examples/pom.xml| 2 +-
 index/lucene/pom.xml  | 2 +-
 index/secondary-index/pom.xml | 2 +-
 integration/flink-build/pom.xml   | 2 +-
 integration/flink-proxy/pom.xml   | 2 +-
 integration/flink/pom.xml | 2 +-
 integration/hive/pom.xml  | 2 +-
 integration/presto/pom.xml| 2 +-
 integration/spark-common-cluster-test/pom.xml | 2 +-
 integration/spark/pom.xml | 2 +-
 mv/plan/pom.xml   | 2 +-
 pom.xml   | 2 +-
 processing/pom.xml| 2 +-
 sdk/sdk/pom.xml   | 2 +-
 streaming/pom.xml | 2 +-
 tools/cli/pom.xml | 2 +-
 25 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 425e3e7..25eac29 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index bbf1593..c376e6e 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 5cf11a7..4eed3da 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index 8f4f371..bb1fe3a 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 715732c..8753d92 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 9017f4b..a0ca71d 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index 606089f..d184910 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 69cc55a..0b7a57a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index e58ac25..1948c47 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index aa2030f..b6f338b 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 3dff9e7..2d14e7b 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index c363c65..c05b334 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.1.0-SNAPSHOT
+2.0.1-SNAPSHOT
 ../../pom.xml
   
 
diff 

[carbondata] 07/08: [CARBONDATA-3840] Mark features as experimental

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 8947655194db7c76987e97ebb5d01bbadb587b8a
Author: kunal642 
AuthorDate: Mon Jun 1 13:07:24 2020 +0530

[CARBONDATA-3840] Mark features as experimental

Why is this PR needed?
Mark features as experimental because they are subject to change in future.

What changes were proposed in this PR?
Mark features as experimental because they are subject to change in future.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3783
---
 README.md | 8 
 1 file changed, 8 insertions(+)

diff --git a/README.md b/README.md
index 2d9fcc7..0293c4c 100644
--- a/README.md
+++ b/README.md
@@ -69,6 +69,14 @@ CarbonData is built using Apache Maven, to [build 
CarbonData](https://github.com
 * [Carbon as Spark's 
Datasource](https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md)
 
 * [FAQs](https://github.com/apache/carbondata/blob/master/docs/faq.md) 
 
+## Experimental Features
+
+Some features are marked as experimental because the syntax/implementation 
might change in the future.
+1. Hybrid format table using Add Segment.
+2. Accelerating performance using MV on parquet/orc.
+3. Merge API for Spark DataFrame.
+4. Hive write for non-transactional table.
+
 ##  Integration
 * [Hive](https://github.com/apache/carbondata/blob/master/docs/hive-guide.md)
 * 
[Presto](https://github.com/apache/carbondata/blob/master/docs/prestodb-guide.md)



[carbondata] 03/08: [CARBONDATA-3835] Fix global sort issues

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit a3a27d219d020ae3b0ba6a4565b88758b613c6c4
Author: ajantha-bhat 
AuthorDate: Sat May 30 08:24:39 2020 +0530

[CARBONDATA-3835] Fix global sort issues

Why is this PR needed?
For global sort without partition, string comes as byte[], if we use string 
comparator (StringSerializableComparator) it will convert byte[] to toString 
which gives address and comparison goes wrong.
For global sort with partition, when sort column is partition column, it 
was sorting on first column instead of partition columns.

What changes were proposed in this PR?
change data type to byte before choosing a comparator.
get the sorted column based on index, don't just take from first

Does this PR introduce any user interface change?
No

Is any new testcase added?
Yes

This closes #3779
---
 .../spark/load/DataLoadProcessBuilderOnSpark.scala | 15 --
 .../carbondata/spark/load/GlobalSortHelper.scala   |  8 ++-
 .../command/management/CommonLoadUtils.scala   |  7 ++-
 .../org/apache/spark/sql/test/util/QueryTest.scala | 17 +-
 .../StandardPartitionGlobalSortTestCase.scala  | 60 ++
 5 files changed, 94 insertions(+), 13 deletions(-)

diff --git 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
index 55eee11..e7e1baf 100644
--- 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
+++ 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/DataLoadProcessBuilderOnSpark.scala
@@ -32,6 +32,7 @@ import org.apache.spark.sql.{DataFrame, SparkSession}
 import org.apache.spark.sql.catalyst.expressions.GenericInternalRow
 import org.apache.spark.sql.catalyst.InternalRow
 import org.apache.spark.sql.execution.command.ExecutionErrors
+import org.apache.spark.sql.types.{ByteType, DateType, LongType, StringType, 
TimestampType}
 import org.apache.spark.sql.util.{SparkSQLUtil, SparkTypeConverter}
 import org.apache.spark.storage.StorageLevel
 import org.apache.spark.unsafe.types.UTF8String
@@ -237,10 +238,15 @@ object DataLoadProcessBuilderOnSpark {
 CarbonProperties.getInstance().getGlobalSortRddStorageLevel()))
 }
 val sortColumnsLength = 
model.getCarbonDataLoadSchema.getCarbonTable.getSortColumns.size()
-val sortColumnDataTypes = dataTypes.take(sortColumnsLength)
-val rowComparator = 
GlobalSortHelper.generateRowComparator(sortColumnDataTypes)
+var sortColumnDataTypes = dataTypes.take(sortColumnsLength)
+sortColumnDataTypes = sortColumnDataTypes.map {
+  case StringType => ByteType
+  case TimestampType | DateType => LongType
+  case datatype => datatype
+}
+val rowComparator = 
GlobalSortHelper.generateRowComparator(sortColumnDataTypes.zipWithIndex)
 val sortRDD = rdd.sortBy(row =>
-  getKey(row, sortColumnsLength, sortColumnDataTypes),
+  getKey(row, sortColumnsLength),
   true,
   numPartitions)(
   rowComparator, classTag[Array[AnyRef]])
@@ -273,8 +279,7 @@ object DataLoadProcessBuilderOnSpark {
   }
 
   def getKey(row: Array[AnyRef],
-  sortColumnsLength: Int,
-  dataTypes: Seq[org.apache.spark.sql.types.DataType]): Array[AnyRef] = {
+  sortColumnsLength: Int): Array[AnyRef] = {
 val key: Array[AnyRef] = new Array[AnyRef](sortColumnsLength)
 System.arraycopy(row, 0, key, 0, sortColumnsLength)
 key
diff --git 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
index 91ed27e..00891b9 100644
--- 
a/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
+++ 
b/integration/spark/src/main/scala/org/apache/carbondata/spark/load/GlobalSortHelper.scala
@@ -50,7 +50,7 @@ object GlobalSortHelper {
 
   def sortBy(updatedRdd: RDD[InternalRow],
   numPartitions: Int,
-  dataTypes: Seq[DataType]
+  dataTypes: Seq[(DataType, Int)]
   ): RDD[InternalRow] = {
 val keyExtractors = generateKeyExtractor(dataTypes)
 val rowComparator = generateRowComparator(dataTypes)
@@ -68,9 +68,8 @@ object GlobalSortHelper {
 key
   }
 
-  def generateKeyExtractor(dataTypes: Seq[DataType]): Array[KeyExtractor] = {
+  def generateKeyExtractor(dataTypes: Seq[(DataType, Int)]): 
Array[KeyExtractor] = {
 dataTypes
-  .zipWithIndex
   .map { attr =>
 attr._1 match {
   case StringType => UTF8StringKeyExtractor(attr._2)
@@ -91,9 +90,8 @@ object GlobalSortHelper {
   .toArray
   }
 
-  def generateRowComparator(dataTypes: 

[carbondata] annotated tag apache-carbondata-2.0.1-rc1 created (now 8569d80)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at 8569d80  (tag)
 tagging c2889fea484aea381ccc0e9e0c6c9cc14c22cfdc (commit)
 replaces apache-carbondata-2.0.0-rc3
  by kunal642
  on Mon Jun 1 15:13:51 2020 +0530

- Log -
[maven-release-plugin] copy for tag apache-carbondata-2.0.1-rc1
---

This annotated tag includes the following new commits:

 new 613d458  [maven-release-plugin] prepare for next development iteration
 new aefa10a  [HOTFIX] changed development version to 2.0.1
 new a3a27d2  [CARBONDATA-3835] Fix global sort issues
 new 86a6e88  [CARBONDATA-3837] Fallback to the original plan when mv 
rewrite throw exception
 new be50362  [CARBONDATA-3836] Fix metadata folder FileNotFoundException 
while creating new carbon table
 new 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
 new 8947655  [CARBONDATA-3840] Mark features as experimental
 new c2889fe  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1

The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




[carbondata] 01/08: [maven-release-plugin] prepare for next development iteration

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 613d458e85fe392d7bc2ce579285cae1be2e42a7
Author: kunal642 
AuthorDate: Sun May 17 13:28:30 2020 +0530

[maven-release-plugin] prepare for next development iteration
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index ca3f10c..425e3e7 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index f055fc3..bbf1593 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 8414b00..5cf11a7 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index 57e8398..8f4f371 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index b96afbd..715732c 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 9aaa631..9017f4b 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index f614997..606089f 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 24be836..69cc55a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 78fd642..e58ac25 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index f32e010..aa2030f 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 4d00226..3dff9e7 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index 626985f..c363c65 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index 26f0872..70aa992 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.0
+2.1.0-SNAPSHOT
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 22e305d..dc91066 100644
--- a/integration/flink-proxy/pom.xml
+++ 

[carbondata] 06/08: [CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 316ea18a8e3825090f1e9083b67dee278bb9a98c
Author: akashrn5 
AuthorDate: Mon Jun 1 09:58:31 2020 +0530

[CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

Why is this PR needed?
Rename file fails in HDFS when the FS object is of FilterFileSystem,
(which basically can contain any filesystem to use as basic filesystem)

What changes were proposed in this PR?
While rename force, have a check for this FS object

This closes #3781
---
 .../apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java | 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
index ac508c3..7d36f0e 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
@@ -23,6 +23,7 @@ import org.apache.carbondata.common.logging.LogServiceFactory;
 
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FilterFileSystem;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hdfs.DistributedFileSystem;
 import org.apache.log4j.Logger;
@@ -77,6 +78,11 @@ public class HDFSCarbonFile extends AbstractDFSCarbonFile {
 ((DistributedFileSystem) fileSystem).rename(path, new 
Path(changetoName),
 org.apache.hadoop.fs.Options.Rename.OVERWRITE);
 return true;
+  } else if ((fileSystem instanceof FilterFileSystem) && 
(((FilterFileSystem) fileSystem)
+  .getRawFileSystem() instanceof DistributedFileSystem)) {
+((DistributedFileSystem) ((FilterFileSystem) 
fileSystem).getRawFileSystem())
+.rename(path, new Path(changetoName), 
org.apache.hadoop.fs.Options.Rename.OVERWRITE);
+return true;
   } else {
 return fileSystem.rename(path, new Path(changetoName));
   }



[carbondata] branch branch-2.0 created (now eb98c21)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at eb98c21  [maven-release-plugin] prepare for next development iteration

This branch includes the following new commits:

 new eb98c21  [maven-release-plugin] prepare for next development iteration

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




[carbondata] 08/08: [maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit c2889fea484aea381ccc0e9e0c6c9cc14c22cfdc
Author: kunal642 
AuthorDate: Mon Jun 1 15:12:49 2020 +0530

[maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 25eac29..bae9c4c 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index c376e6e..e6f86c3 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 4eed3da..eb237f9 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index bb1fe3a..cd7a3d6 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 8753d92..6bda3d6 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index a0ca71d..61cd65f 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index d184910..e1a0ff1 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 0b7a57a..b178e7a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 1948c47..28210d0 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index b6f338b..10978ac 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 2d14e7b..1aa242f 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index c05b334..a024262 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index 647a523..cf867ac 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 3d375f7..1a6e7fa 100644
--- a/integration/flink-proxy/pom.xml

[carbondata] branch branch-2.0 deleted (was a4b62f2)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 was a4b62f2  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1

This change permanently discards the following revisions:

 discard a4b62f2  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1
 discard 8947655  [CARBONDATA-3840] Mark features as experimental
 discard 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
 discard be50362  [CARBONDATA-3836] Fix metadata folder FileNotFoundException 
while creating new carbon table
 discard 86a6e88  [CARBONDATA-3837] Fallback to the original plan when mv 
rewrite throw exception
 discard a3a27d2  [CARBONDATA-3835] Fix global sort issues
 discard aefa10a  [HOTFIX] changed development version to 2.0.1
 discard 613d458  [maven-release-plugin] prepare for next development iteration



[carbondata] annotated tag apache-carbondata-2.0.1-rc1 deleted (was 6178d36)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to annotated tag apache-carbondata-2.0.1-rc1
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


*** WARNING: tag apache-carbondata-2.0.1-rc1 was deleted! ***

   tag was  6178d36

This change permanently discards the following revisions:

 discard 13de1e8  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1



[carbondata] branch branch-2.0 updated: [maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
 new a4b62f2  [maven-release-plugin] prepare release 
apache-carbondata-2.0.1-rc1
a4b62f2 is described below

commit a4b62f28138e4353458b597d2ad075a73deb6b01
Author: kunal642 
AuthorDate: Mon Jun 1 14:58:05 2020 +0530

[maven-release-plugin] prepare release apache-carbondata-2.0.1-rc1
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index 25eac29..bae9c4c 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index c376e6e..e6f86c3 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index 4eed3da..eb237f9 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index bb1fe3a..cd7a3d6 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 8753d92..6bda3d6 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index a0ca71d..61cd65f 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index d184910..e1a0ff1 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index 0b7a57a..b178e7a 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 1948c47..28210d0 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index b6f338b..10978ac 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 2d14e7b..1aa242f 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index c05b334..a024262 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index 647a523..cf867ac 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1-SNAPSHOT
+2.0.1
 

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2295

2020-06-01 Thread Apache Jenkins Server
See 


Changes:


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[WARNING] The POM for 
org.apache.carbondata:carbondata-format:jar:2.1.0-SNAPSHOT is missing, no 
dependency information available
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar


Build failed in Jenkins: carbondata-master-spark-2.3 #2295

2020-06-01 Thread Apache Jenkins Server
See 


Changes:

[jacky.likun] [CARBONDATA-3840] Mark features as experimental


--
[...truncated 16.02 KB...]
[INFO] Deleting 

[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) 

[carbondata] branch branch-2.0 created (now 8947655)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at 8947655  [CARBONDATA-3840] Mark features as experimental

This branch includes the following new commits:

 new 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
 new 8947655  [CARBONDATA-3840] Mark features as experimental

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




[carbondata] branch branch-2.0 deleted (was deace64)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 was deace64  [CARBONDATA-3840] Mark features as experimental

This change permanently discards the following revisions:

 discard deace64  [CARBONDATA-3840] Mark features as experimental
 discard 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object



[carbondata] 01/02: [CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 316ea18a8e3825090f1e9083b67dee278bb9a98c
Author: akashrn5 
AuthorDate: Mon Jun 1 09:58:31 2020 +0530

[CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

Why is this PR needed?
Rename file fails in HDFS when the FS object is of FilterFileSystem,
(which basically can contain any filesystem to use as basic filesystem)

What changes were proposed in this PR?
While rename force, have a check for this FS object

This closes #3781
---
 .../apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java | 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
index ac508c3..7d36f0e 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
@@ -23,6 +23,7 @@ import org.apache.carbondata.common.logging.LogServiceFactory;
 
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FilterFileSystem;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hdfs.DistributedFileSystem;
 import org.apache.log4j.Logger;
@@ -77,6 +78,11 @@ public class HDFSCarbonFile extends AbstractDFSCarbonFile {
 ((DistributedFileSystem) fileSystem).rename(path, new 
Path(changetoName),
 org.apache.hadoop.fs.Options.Rename.OVERWRITE);
 return true;
+  } else if ((fileSystem instanceof FilterFileSystem) && 
(((FilterFileSystem) fileSystem)
+  .getRawFileSystem() instanceof DistributedFileSystem)) {
+((DistributedFileSystem) ((FilterFileSystem) 
fileSystem).getRawFileSystem())
+.rename(path, new Path(changetoName), 
org.apache.hadoop.fs.Options.Rename.OVERWRITE);
+return true;
   } else {
 return fileSystem.rename(path, new Path(changetoName));
   }



[carbondata] 02/02: [CARBONDATA-3840] Mark features as experimental

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit deace64bfc042db906246cbf01a4a5fb6a791f3d
Author: kunal642 
AuthorDate: Mon Jun 1 13:07:24 2020 +0530

[CARBONDATA-3840] Mark features as experimental

Why is this PR needed?
Mark features as experimental because they are subject to change in future.

What changes were proposed in this PR?
Mark features as experimental because they are subject to change in future.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3783
---
 README.md | 8 
 1 file changed, 8 insertions(+)

diff --git a/README.md b/README.md
index 2d9fcc7..0293c4c 100644
--- a/README.md
+++ b/README.md
@@ -69,6 +69,14 @@ CarbonData is built using Apache Maven, to [build 
CarbonData](https://github.com
 * [Carbon as Spark's 
Datasource](https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md)
 
 * [FAQs](https://github.com/apache/carbondata/blob/master/docs/faq.md) 
 
+## Experimental Features
+
+Some features are marked as experimental because the syntax/implementation 
might change in the future.
+1. Hybrid format table using Add Segment.
+2. Accelerating performance using MV on parquet/orc.
+3. Merge API for Spark DataFrame.
+4. Hive write for non-transactional table.
+
 ##  Integration
 * [Hive](https://github.com/apache/carbondata/blob/master/docs/hive-guide.md)
 * 
[Presto](https://github.com/apache/carbondata/blob/master/docs/prestodb-guide.md)



[carbondata] branch branch-2.0 created (now deace64)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at deace64  [CARBONDATA-3840] Mark features as experimental

This branch includes the following new commits:

 new 316ea18  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
 new deace64  [CARBONDATA-3840] Mark features as experimental

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




[carbondata] branch branch-2.0 deleted (was 5aeb768)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 was 5aeb768  [maven-release-plugin] prepare for next development iteration

This change permanently discards the following revisions:

 discard 5aeb768  [maven-release-plugin] prepare for next development iteration



[carbondata] 01/01: [maven-release-plugin] prepare for next development iteration

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git

commit 5aeb768319ec59098af68ada6801664ffe9ab894
Author: kunal642 
AuthorDate: Mon Jun 1 09:45:46 2020 +0530

[maven-release-plugin] prepare for next development iteration
---
 assembly/pom.xml| 2 +-
 common/pom.xml  | 2 +-
 core/pom.xml| 2 +-
 examples/flink/pom.xml  | 2 +-
 examples/spark/pom.xml  | 2 +-
 format/pom.xml  | 2 +-
 geo/pom.xml | 2 +-
 hadoop/pom.xml  | 2 +-
 index/bloom/pom.xml | 2 +-
 index/examples/pom.xml  | 2 +-
 index/lucene/pom.xml| 2 +-
 index/secondary-index/pom.xml   | 2 +-
 integration/flink-build/pom.xml | 2 +-
 integration/flink-proxy/pom.xml | 2 +-
 integration/flink/pom.xml   | 2 +-
 integration/hive/pom.xml| 2 +-
 integration/presto/pom.xml  | 2 +-
 integration/spark/pom.xml   | 2 +-
 mv/plan/pom.xml | 2 +-
 pom.xml | 4 ++--
 processing/pom.xml  | 2 +-
 sdk/sdk/pom.xml | 2 +-
 streaming/pom.xml   | 2 +-
 tools/cli/pom.xml   | 2 +-
 24 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/assembly/pom.xml b/assembly/pom.xml
index bae9c4c..97afb90 100644
--- a/assembly/pom.xml
+++ b/assembly/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/common/pom.xml b/common/pom.xml
index e6f86c3..c4e62f7 100644
--- a/common/pom.xml
+++ b/common/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/core/pom.xml b/core/pom.xml
index eb237f9..8367df8 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/examples/flink/pom.xml b/examples/flink/pom.xml
index cd7a3d6..23eb398 100644
--- a/examples/flink/pom.xml
+++ b/examples/flink/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/examples/spark/pom.xml b/examples/spark/pom.xml
index 6bda3d6..e60250f 100644
--- a/examples/spark/pom.xml
+++ b/examples/spark/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/format/pom.xml b/format/pom.xml
index 61cd65f..7f8599a 100644
--- a/format/pom.xml
+++ b/format/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/geo/pom.xml b/geo/pom.xml
index e1a0ff1..248936b 100644
--- a/geo/pom.xml
+++ b/geo/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/hadoop/pom.xml b/hadoop/pom.xml
index b178e7a..5dc0380 100644
--- a/hadoop/pom.xml
+++ b/hadoop/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../pom.xml
   
 
diff --git a/index/bloom/pom.xml b/index/bloom/pom.xml
index 28210d0..1114065 100644
--- a/index/bloom/pom.xml
+++ b/index/bloom/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/examples/pom.xml b/index/examples/pom.xml
index 10978ac..ab285ea 100644
--- a/index/examples/pom.xml
+++ b/index/examples/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/lucene/pom.xml b/index/lucene/pom.xml
index 1aa242f..a2ab3d6 100644
--- a/index/lucene/pom.xml
+++ b/index/lucene/pom.xml
@@ -4,7 +4,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/index/secondary-index/pom.xml b/index/secondary-index/pom.xml
index a024262..3cffbff 100644
--- a/index/secondary-index/pom.xml
+++ b/index/secondary-index/pom.xml
@@ -22,7 +22,7 @@
   
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
   
 
diff --git a/integration/flink-build/pom.xml b/integration/flink-build/pom.xml
index cf867ac..53f52a8 100644
--- a/integration/flink-build/pom.xml
+++ b/integration/flink-build/pom.xml
@@ -4,7 +4,7 @@
 
 org.apache.carbondata
 carbondata-parent
-2.0.1
+2.0.2-SNAPSHOT
 ../../pom.xml
 
 
diff --git a/integration/flink-proxy/pom.xml b/integration/flink-proxy/pom.xml
index 1a6e7fa..c000999 100644
--- a/integration/flink-proxy/pom.xml
+++ 

[carbondata] branch branch-2.0 created (now 5aeb768)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


  at 5aeb768  [maven-release-plugin] prepare for next development iteration

This branch includes the following new commits:

 new 5aeb768  [maven-release-plugin] prepare for next development iteration

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.




[carbondata] branch branch-2.0 deleted (was 5aeb768)

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a change to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/carbondata.git.


 was 5aeb768  [maven-release-plugin] prepare for next development iteration

This change permanently discards the following revisions:

 discard 5aeb768  [maven-release-plugin] prepare for next development iteration



[carbondata] branch master updated: [CARBONDATA-3840] Mark features as experimental

2020-06-01 Thread jackylk
This is an automated email from the ASF dual-hosted git repository.

jackylk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new 45b4781  [CARBONDATA-3840] Mark features as experimental
45b4781 is described below

commit 45b47814ff6eb838ddbe7025ed132cac4283d11b
Author: kunal642 
AuthorDate: Mon Jun 1 13:07:24 2020 +0530

[CARBONDATA-3840] Mark features as experimental

Why is this PR needed?
Mark features as experimental because they are subject to change in future.

What changes were proposed in this PR?
Mark features as experimental because they are subject to change in future.

Does this PR introduce any user interface change?
No

Is any new testcase added?
No

This closes #3783
---
 README.md | 8 
 1 file changed, 8 insertions(+)

diff --git a/README.md b/README.md
index 2d9fcc7..0293c4c 100644
--- a/README.md
+++ b/README.md
@@ -69,6 +69,14 @@ CarbonData is built using Apache Maven, to [build 
CarbonData](https://github.com
 * [Carbon as Spark's 
Datasource](https://github.com/apache/carbondata/blob/master/docs/carbon-as-spark-datasource-guide.md)
 
 * [FAQs](https://github.com/apache/carbondata/blob/master/docs/faq.md) 
 
+## Experimental Features
+
+Some features are marked as experimental because the syntax/implementation 
might change in the future.
+1. Hybrid format table using Add Segment.
+2. Accelerating performance using MV on parquet/orc.
+3. Merge API for Spark DataFrame.
+4. Hive write for non-transactional table.
+
 ##  Integration
 * [Hive](https://github.com/apache/carbondata/blob/master/docs/hive-guide.md)
 * 
[Presto](https://github.com/apache/carbondata/blob/master/docs/prestodb-guide.md)



Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2293

2020-06-01 Thread Apache Jenkins Server
See 


Changes:

[kunalkapoor642] [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[WARNING] The POM for 
org.apache.carbondata:carbondata-format:jar:2.1.0-SNAPSHOT is missing, no 
dependency information available
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar


Build failed in Jenkins: carbondata-master-spark-2.3 #2294

2020-06-01 Thread Apache Jenkins Server
See 


Changes:


--
[...truncated 16.31 KB...]
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/0/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ 
carbondata-common ---
[INFO] Installing 

 to 

Build failed in Jenkins: carbondata-master-spark-2.3 » Apache CarbonData :: Core #2294

2020-06-01 Thread Apache Jenkins Server
See 


Changes:


--
[INFO] 
[INFO] 
[INFO] Building Apache CarbonData :: Core 2.1.0-SNAPSHOT
[INFO] 
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/maven-metadata.xml
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.pom
[WARNING] The POM for 
org.apache.carbondata:carbondata-format:jar:2.1.0-SNAPSHOT is missing, no 
dependency information available
[INFO] Downloading: 
https://repo1.maven.org/maven2/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar
[INFO] Downloading: 
http://repository.apache.org/snapshots/org/apache/carbondata/carbondata-format/2.1.0-SNAPSHOT/carbondata-format-2.1.0-SNAPSHOT.jar


Build failed in Jenkins: carbondata-master-spark-2.3 #2293

2020-06-01 Thread Apache Jenkins Server
See 


Changes:

[kunalkapoor642] [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object


--
[...truncated 16.04 KB...]
[INFO] Deleting 

[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:clean (flatten.clean) @ carbondata-common 
---
[INFO] Deleting 

[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent (default-prepare-agent) @ 
carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
carbondata-common ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- flatten-maven-plugin:1.2.2:flatten (flatten) @ carbondata-common ---
[INFO] Generating flattened POM of project 
org.apache.carbondata:carbondata-common:jar:2.1.0-SNAPSHOT...
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 22 source files to 

[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.4:check (analyze-compile) > :findbugs @ 
carbondata-common >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:findbugs (findbugs) @ carbondata-common 
---
[INFO] Fork Value is true
[INFO] Done FindBugs Analysis
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.4:check (analyze-compile) < :findbugs @ 
carbondata-common <<<
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.4:check (analyze-compile) @ 
carbondata-common ---
[INFO] BugInstance size is 0
[INFO] Error size is 0
[INFO] No errors/warnings found
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
carbondata-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ 
carbondata-common ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ carbondata-common 
---
[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ carbondata-common ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
carbondata-common ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:prepare-agent-integration 
(default-prepare-agent-integration) @ carbondata-common ---
[INFO] argLine set to 
-javaagent:/home/jenkins/jenkins-slave/maven-repositories/1/org/jacoco/org.jacoco.agent/0.7.9/org.jacoco.agent-0.7.9-runtime.jar=destfile=
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ carbondata-common ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- scalastyle-maven-plugin:0.8.0:check (default) @ carbondata-common ---
[WARNING] sourceDirectory is not specified or does not exist 
value=
Saving to 
outputFile=
Processed 0 file(s)
Found 0 errors
Found 0 warnings
Found 0 infos
Finished in 1 ms
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report (default-report) @ 
carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:report-integration 
(default-report-integration) @ carbondata-common ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] 
[INFO] --- jacoco-maven-plugin:0.7.9:check (default-check) @ carbondata-common 
---
[INFO] Skipping JaCoCo execution due to missing execution data 

[INFO] 
[INFO] --- 

[carbondata] branch master updated: [CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

2020-06-01 Thread kunalkapoor
This is an automated email from the ASF dual-hosted git repository.

kunalkapoor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
 new c4f98c1  [CARBONDATA-3839]Fix rename file failed for FilterFileSystem 
DFS object
c4f98c1 is described below

commit c4f98c1e69491428092392f00d68afdd5014a1f5
Author: akashrn5 
AuthorDate: Mon Jun 1 09:58:31 2020 +0530

[CARBONDATA-3839]Fix rename file failed for FilterFileSystem DFS object

Why is this PR needed?
Rename file fails in HDFS when the FS object is of FilterFileSystem,
(which basically can contain any filesystem to use as basic filesystem)

What changes were proposed in this PR?
While rename force, have a check for this FS object

This closes #3781
---
 .../apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java | 6 ++
 1 file changed, 6 insertions(+)

diff --git 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
index ac508c3..7d36f0e 100644
--- 
a/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
+++ 
b/core/src/main/java/org/apache/carbondata/core/datastore/filesystem/HDFSCarbonFile.java
@@ -23,6 +23,7 @@ import org.apache.carbondata.common.logging.LogServiceFactory;
 
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FilterFileSystem;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.hdfs.DistributedFileSystem;
 import org.apache.log4j.Logger;
@@ -77,6 +78,11 @@ public class HDFSCarbonFile extends AbstractDFSCarbonFile {
 ((DistributedFileSystem) fileSystem).rename(path, new 
Path(changetoName),
 org.apache.hadoop.fs.Options.Rename.OVERWRITE);
 return true;
+  } else if ((fileSystem instanceof FilterFileSystem) && 
(((FilterFileSystem) fileSystem)
+  .getRawFileSystem() instanceof DistributedFileSystem)) {
+((DistributedFileSystem) ((FilterFileSystem) 
fileSystem).getRawFileSystem())
+.rename(path, new Path(changetoName), 
org.apache.hadoop.fs.Options.Rename.OVERWRITE);
+return true;
   } else {
 return fileSystem.rename(path, new Path(changetoName));
   }