[carbondata] branch master updated: [CARBONDATA-4094]: Fix fallback count(*) issue on partition table with index server

[indhumuthumurugesh]

This is an automated email from the ASF dual-hosted git repository.

indhumuthumurugesh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
     new 1dfcdec  [CARBONDATA-4094]: Fix fallback count(*) issue on partition 
table with index server
1dfcdec is described below

commit 1dfcdec6311e48d9a330c8a4aec8a5592106adba
Author: Vikram Ahuja <vikramahuja8...@gmail.com>
AuthorDate: Mon Dec 21 13:30:11 2020 +0530

    [CARBONDATA-4094]: Fix fallback count(*) issue on partition table with 
index server
    
    Why is this PR needed?
    The used asJava converts to java "in place", without copying the whole data 
to save
    time and memory and it just simply wraps the scala collection with a class 
that
    conforms to the java interface and thus java serializer is not able to 
serialize it.
    
    What changes were proposed in this PR?
    Converting it to list, so that it is able to serialize a list.
    
    Does this PR introduce any user interface change?
    No
    
    Is any new testcase added?
    No
    
    This closes #4061
---
 .../main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java | 4 ++++
 .../spark/src/main/scala/org/apache/spark/sql/CarbonCountStar.scala   | 2 +-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git 
a/hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java 
b/hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
index 9765913..4d917fb 100644
--- 
a/hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
+++ 
b/hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
@@ -404,6 +404,10 @@ public abstract class CarbonInputFormat<T> extends 
FileInputFormat<Void, T> {
       }
       return indexJob.executeCountJob(indexInputFormat, configuration);
     } catch (Exception e) {
+      if (CarbonProperties.getInstance().isFallBackDisabled()) {
+        LOG.error("Fallback is disabled");
+        throw e;
+      }
       LOG.error("Failed to get count from index server. Initializing 
fallback", e);
       IndexJob indexJob = IndexUtil.getEmbeddedJob();
       return indexJob.executeCountJob(indexInputFormat, configuration);
diff --git 
a/integration/spark/src/main/scala/org/apache/spark/sql/CarbonCountStar.scala 
b/integration/spark/src/main/scala/org/apache/spark/sql/CarbonCountStar.scala
index 6ca124f..678a0be 100644
--- 
a/integration/spark/src/main/scala/org/apache/spark/sql/CarbonCountStar.scala
+++ 
b/integration/spark/src/main/scala/org/apache/spark/sql/CarbonCountStar.scala
@@ -63,7 +63,7 @@ case class CarbonCountStar(
           sparkSession,
           TableIdentifier(
             carbonTable.getTableName,
-            Some(carbonTable.getDatabaseName))).map(_.asJava).orNull, false),
+            Some(carbonTable.getDatabaseName))).map(_.toList.asJava).orNull, 
false),
       carbonTable)
 
     if (CarbonProperties.isQueryStageInputEnabled) {