[GitHub] [carbondata] akashrn5 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


akashrn5 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r493205093



##
File path: 
integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
##
@@ -442,7 +442,11 @@ object AlterTableUtil {
 // update schema for long string columns
 updateSchemaForLongStringColumns(thriftTable, longStringColumns.get)
   } else if (propKeys.exists(_.equalsIgnoreCase("long_string_columns") && 
!set)) {
-updateSchemaForLongStringColumns(thriftTable, "")
+val varcharColumns = 
carbonTable.getTableInfo.getFactTable.getListOfColumns.asScala.toList

Review comment:
   here instead of looping all columns, just check whether the long string 
property present in `tblPropertiesMap`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


ajantha-bhat commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r493203786



##
File path: 
integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala
##
@@ -442,7 +442,11 @@ object AlterTableUtil {
 // update schema for long string columns
 updateSchemaForLongStringColumns(thriftTable, longStringColumns.get)
   } else if (propKeys.exists(_.equalsIgnoreCase("long_string_columns") && 
!set)) {
-updateSchemaForLongStringColumns(thriftTable, "")
+val varcharColumns = 
carbonTable.getTableInfo.getFactTable.getListOfColumns.asScala.toList

Review comment:
   why not directly check the table properties to see long_string property 
exist before or not ? If you have lot of columns, this will be slow compared to 
table property look up.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #3951: [CARBONDATA-4005] SI with cache level blocklet issue

2020-09-22 Thread GitBox


ShreelekhyaG commented on a change in pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#discussion_r493201285



##
File path: 
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -648,11 +648,20 @@ public static String getShortBlockId(String blockId) {
 .replace(DATA_PART_PREFIX, "").replace(CARBON_DATA_EXT, "");
 // to remove compressor name
 if (!blockId.equalsIgnoreCase(blockIdWithCompressorName)) {
-  int index = blockIdWithCompressorName.lastIndexOf(".");
+  int index = blockIdWithCompressorName.lastIndexOf(POINT);
+  int fileSeperatorIndex = 
blockIdWithCompressorName.lastIndexOf(File.separator);
   if (index != -1) {
-String replace =
-
blockIdWithCompressorName.replace(blockIdWithCompressorName.substring(index), 
"");
-return replace;
+String modifiedBlockId;

Review comment:
   We could use the same method for both partition and non-partition case. 
So removed `getShortBlockIdForPartitionTable `method.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (CARBONDATA-3996) Show table extended like command throws java.lang.ArrayIndexOutOfBoundsException

2020-09-22 Thread Akash R Nilugal (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akash R Nilugal resolved CARBONDATA-3996.
-
Resolution: Fixed

> Show table extended like command throws 
> java.lang.ArrayIndexOutOfBoundsException
> 
>
> Key: CARBONDATA-3996
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3996
> Project: CarbonData
>  Issue Type: Bug
>  Components: spark-integration
>Affects Versions: 2.0.0
>Reporter: Venugopal Reddy K
>Priority: Minor
> Fix For: 2.1.0
>
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> *Issue:*
> Show table extended like command throws 
> java.lang.ArrayIndexOutOfBoundsException
> *Steps to reproduce:*
> spark.sql("create table employee(id string, name string) stored as 
> carbondata")
> spark.sql("show table extended like 'emp*'").show(100, false)
> *Exception stack:*
>  
> {code:java}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 
> 3Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 3 at 
> org.apache.spark.sql.catalyst.expressions.GenericInternalRow.genericGet(rows.scala:201)
>  at 
> org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getAs(rows.scala:35)
>  at 
> org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getUTF8String(rows.scala:46)
>  at 
> org.apache.spark.sql.catalyst.expressions.GenericInternalRow.getUTF8String(rows.scala:195)
>  at 
> org.apache.spark.sql.catalyst.InternalRow$$anonfun$getAccessor$8.apply(InternalRow.scala:136)
>  at 
> org.apache.spark.sql.catalyst.InternalRow$$anonfun$getAccessor$8.apply(InternalRow.scala:136)
>  at 
> org.apache.spark.sql.catalyst.expressions.BoundReference.eval(BoundAttribute.scala:44)
>  at 
> org.apache.spark.sql.catalyst.expressions.UnaryExpression.eval(Expression.scala:389)
>  at 
> org.apache.spark.sql.catalyst.expressions.Alias.eval(namedExpressions.scala:152)
>  at 
> org.apache.spark.sql.catalyst.expressions.InterpretedMutableProjection.apply(Projection.scala:92)
>  at 
> org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$24$$anonfun$applyOrElse$23.apply(Optimizer.scala:1364)
>  at 
> org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$24$$anonfun$applyOrElse$23.apply(Optimizer.scala:1364)
>  at 
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
>  at 
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
>  at 
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>  at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35) at 
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at 
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at 
> org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$24.applyOrElse(Optimizer.scala:1364)
>  at 
> org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation$$anonfun$apply$24.applyOrElse(Optimizer.scala:1359)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$2.apply(TreeNode.scala:258)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$2.apply(TreeNode.scala:258)
>  at 
> org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:69)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:257)
>  at 
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDown(LogicalPlan.scala:29)
>  at 
> org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$class.transformDown(AnalysisHelper.scala:149)
>  at 
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
>  at 
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDown(LogicalPlan.scala:29)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformDown$1.apply(TreeNode.scala:263)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformDown$1.apply(TreeNode.scala:263)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:328)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode.mapProductIterator(TreeNode.scala:186)
>  at 
> org.apache.spark.sql.catalyst.trees.TreeNode.mapChildren(TreeNode.scala:326) 
> at 
> org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:263)
>  at 
> org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDown(LogicalPlan.scala:29)
>  at 
> org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$class.transformDown(AnalysisHelper.scala:149)
>  at 
> 

[GitHub] [carbondata] asfgit closed pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


asfgit closed pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (CARBONDATA-4000) Presto select query failure right after firing update query

2020-09-22 Thread Ajantha Bhat (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajantha Bhat resolved CARBONDATA-4000.
--
Resolution: Fixed

> Presto select query failure right after firing update query
> ---
>
> Key: CARBONDATA-4000
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4000
> Project: CarbonData
>  Issue Type: Bug
>  Components: core
>Reporter: Akshay
>Priority: Major
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> The putObject() method implementation was missing due to which select query 
> on *struct* complex data type after firing *update* queries were failing.
> Solution-
>  Added implementation in ColumnarVectorWrapperDirectWithDeleteDelta. Earlier 
> without this implementation control went to its parent class 
> AbstractCarbonColumnarVector,java which was unsupported.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (CARBONDATA-4000) Presto select query failure right after firing update query

2020-09-22 Thread Ajantha Bhat (Jira)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-4000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ajantha Bhat updated CARBONDATA-4000:
-
Fix Version/s: 2.1.0

> Presto select query failure right after firing update query
> ---
>
> Key: CARBONDATA-4000
> URL: https://issues.apache.org/jira/browse/CARBONDATA-4000
> Project: CarbonData
>  Issue Type: Bug
>  Components: core
>Reporter: Akshay
>Priority: Major
> Fix For: 2.1.0
>
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> The putObject() method implementation was missing due to which select query 
> on *struct* complex data type after firing *update* queries were failing.
> Solution-
>  Added implementation in ColumnarVectorWrapperDirectWithDeleteDelta. Earlier 
> without this implementation control went to its parent class 
> AbstractCarbonColumnarVector,java which was unsupported.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] asfgit closed pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


asfgit closed pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3948: [WIP] Analyze random 11 testcase failure in CI

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3948:
URL: https://github.com/apache/carbondata/pull/3948#issuecomment-697133208


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2448/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Zhangshunyu removed a comment on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


Zhangshunyu removed a comment on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697074445


   pls handle the filed testcases



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akashrn5 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


akashrn5 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-697132560


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Zhangshunyu commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


Zhangshunyu commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697074445







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3948: [WIP] Analyze random 11 testcase failure in CI

2020-09-22 Thread GitBox


ajantha-bhat commented on pull request #3948:
URL: https://github.com/apache/carbondata/pull/3948#issuecomment-697098260


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Klaus-xjp commented on pull request #3945: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


Klaus-xjp commented on pull request #3945:
URL: https://github.com/apache/carbondata/pull/3945#issuecomment-697071126


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kunal642 commented on pull request #3940: [CARBONDATA-3997] Fix decimal value reading issue for negative decimal numbers from presto

2020-09-22 Thread GitBox


kunal642 commented on pull request #3940:
URL: https://github.com/apache/carbondata/pull/3940#issuecomment-696644556


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


ShreelekhyaG commented on a change in pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#discussion_r492902403



##
File path: 
core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java
##
@@ -928,6 +928,11 @@ public void testTupeIDInUpdateScenarios() {
 Assert.assertEquals(CarbonTablePath.getShortBlockId(blockId), 
"0/0-0_0-0-0-1597409791503");
 blockId = 
"c3=aa/part-0-1001011_batchno0-0-0-1597411003332.snappy.carbondata";
 
Assert.assertEquals(CarbonTablePath.getShortBlockIdForPartitionTable(blockId), 
"c3=aa/0-1001011_0-0-0-1597411003332");
+// CACHE_LEVEL = BLOCKLET case

Review comment:
   Done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3948: [WIP] Analyze random 11 testcase failure in CI

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3948:
URL: https://github.com/apache/carbondata/pull/3948#issuecomment-696735655







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696747560







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3945: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3945:
URL: https://github.com/apache/carbondata/pull/3945#issuecomment-696554636







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] VenuReddy2103 commented on a change in pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


VenuReddy2103 commented on a change in pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#discussion_r492609629



##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/ShowTable/TestShowTable.scala
##
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.spark.testsuite.ShowTable
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import 
org.apache.carbondata.common.exceptions.sql.MalformedIndexCommandException
+
+/**
+  * Test class for show tables.
+  */
+class TestShowTable extends QueryTest with BeforeAndAfterAll with 
BeforeAndAfterEach {
+  val dbName = "testshowtable"
+  override def beforeAll: Unit = {
+sql(s"drop database if exists $dbName cascade")
+sql(s"create database $dbName")
+sql(s"use $dbName")
+  }
+
+  test("test show tables") {
+sql("create table employee(id string, name string) stored as carbondata")
+sql("create table employee_part(name string) partitioned by (grade int)")
+sql("create index employee_si on table employee(name) as 'carbondata'")
+sql("create materialized view employee_mv as select name from employee 
group by name")
+val df = sql("show tables").toDF
+val rows = df.collect()
+val schema = rows(0).schema
+assert(schema.length == 3)
+assert(schema(0).name.equals("database"))
+assert(schema(1).name.equals("tableName"))
+assert(schema(2).name.equals("isTemporary"))
+// show tables query can return views as well. Just validate if expected 
rows are present
+// and unexpected rows are not present.
+val expectedRows = Seq(Row(dbName, "employee", false),
+  Row(dbName, "employee_part", false),
+  Row(dbName, "employee_si", false))
+val expectedRowsDF = sqlContext.sparkSession
+  .createDataFrame(sqlContext.sparkContext.makeRDD[Row](expectedRows), 
schema)
+checkAnswer(df.intersect(expectedRowsDF), expectedRows)
+// check if mv table is not present
+val notExpectedRows = Seq(Row(dbName, "employee_mv", false))
+val notExpectedRowsDF = sqlContext.sparkSession
+  .createDataFrame(sqlContext.sparkContext.makeRDD[Row](notExpectedRows), 
schema)
+assert(df.intersect(notExpectedRowsDF).count() == 0)
+  }
+
+  test("test show table extended like") {
+sql("create table employee(id string, name string) stored as carbondata")

Review comment:
   employee_part table is non carbon table

##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/ShowTable/TestShowTable.scala
##
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.carbondata.spark.testsuite.ShowTable
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.test.util.QueryTest
+import org.scalatest.{BeforeAndAfterAll, BeforeAndAfterEach}
+
+import 
org.apache.carbondata.common.exceptions.sql.MalformedIndexCommandException
+
+/**
+  * Test class for show tables.
+  */
+class TestShowTable extends QueryTest with BeforeAndAfterAll with 
BeforeAndAfterEach {
+  val dbName = "testshowtable"
+  override def beforeAll: Unit = {
+sql(s"drop database if exists $dbName cascade")
+sql(s"create database $dbName")
+sql(s"use $dbName")
+  }
+
+  test("test show tables") {
+sql("create table employee(id string, 

[GitHub] [carbondata] asfgit closed pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

2020-09-22 Thread GitBox


asfgit closed pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] VenuReddy2103 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


VenuReddy2103 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696678341







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 commented on a change in pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 commented on a change in pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#discussion_r492730939



##
File path: 
core/src/main/java/org/apache/carbondata/core/scan/result/vector/impl/directread/ColumnarVectorWrapperDirectWithDeleteDeltaAndInvertedIndex.java
##
@@ -78,6 +80,17 @@ public void putNull(int rowId) {
 }
   }
 
+  @Override
+  public void putObject(int rowId, Object obj) {
+if (!deletedRows.get(rowId)) {
+  if (nullBits.get(rowId)) {
+columnVector.putNull(counter++);

Review comment:
   Have raised a separate jira to address this issue - CARBONDATA-4004





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-696669760







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] nihal0107 commented on pull request #3949: [TEMP] [WIP] Fix issue in concurrent SI global sort

2020-09-22 Thread GitBox


nihal0107 commented on pull request #3949:
URL: https://github.com/apache/carbondata/pull/3949#issuecomment-696758152


   retest this please.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for carbondata-spark test code

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3950:
URL: https://github.com/apache/carbondata/pull/3950#issuecomment-696761398







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akashrn5 commented on pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

2020-09-22 Thread GitBox


akashrn5 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696511140







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] asfgit closed pull request #3944: [CARBONDATA-4001] Fix SI global sort load on partition table

2020-09-22 Thread GitBox


asfgit closed pull request #3944:
URL: https://github.com/apache/carbondata/pull/3944


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 removed a comment on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 removed a comment on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696750855


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] asfgit closed pull request #3940: [CARBONDATA-3997] Fix decimal value reading issue for negative decimal numbers from presto

2020-09-22 Thread GitBox


asfgit closed pull request #3940:
URL: https://github.com/apache/carbondata/pull/3940


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Karan980 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


Karan980 commented on pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#issuecomment-696629075


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Karan980 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


Karan980 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r492735831



##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,24 @@ class VarcharDataTypesBasicTestCase extends QueryTest with 
BeforeAndAfterEach wi
 
 sql("DROP TABLE IF EXISTS varchar_complex_table")
   }
+
+  test("check new columns after modifying schema through alter table queries") 
{
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(

Review comment:
   Done

##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,24 @@ class VarcharDataTypesBasicTestCase extends QueryTest with 
BeforeAndAfterEach wi
 
 sql("DROP TABLE IF EXISTS varchar_complex_table")
   }
+
+  test("check new columns after modifying schema through alter table queries") 
{
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)

Review comment:
   Done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


Indhumathi27 commented on a change in pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#discussion_r492785607



##
File path: 
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -648,11 +648,20 @@ public static String getShortBlockId(String blockId) {
 .replace(DATA_PART_PREFIX, "").replace(CARBON_DATA_EXT, "");
 // to remove compressor name
 if (!blockId.equalsIgnoreCase(blockIdWithCompressorName)) {
-  int index = blockIdWithCompressorName.lastIndexOf(".");
+  int index = blockIdWithCompressorName.lastIndexOf(POINT);
+  int fileSeperatorIndex = 
blockIdWithCompressorName.lastIndexOf(File.separator);
   if (index != -1) {
-String replace =
-
blockIdWithCompressorName.replace(blockIdWithCompressorName.substring(index), 
"");
-return replace;
+String modifiedBlockId;

Review comment:
   Can extract common code from partition table and non-partition table to 
get block Id to new method

##
File path: 
core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java
##
@@ -928,6 +928,11 @@ public void testTupeIDInUpdateScenarios() {
 Assert.assertEquals(CarbonTablePath.getShortBlockId(blockId), 
"0/0-0_0-0-0-1597409791503");
 blockId = 
"c3=aa/part-0-1001011_batchno0-0-0-1597411003332.snappy.carbondata";
 
Assert.assertEquals(CarbonTablePath.getShortBlockIdForPartitionTable(blockId), 
"c3=aa/0-1001011_0-0-0-1597411003332");
+// CACHE_LEVEL = BLOCKLET case

Review comment:
   Please add a functional testcase for this scenario and check data after 
setting cache_level=blocklet





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696750855







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akashrn5 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


akashrn5 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r492651158



##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,24 @@ class VarcharDataTypesBasicTestCase extends QueryTest with 
BeforeAndAfterEach wi
 
 sql("DROP TABLE IF EXISTS varchar_complex_table")
   }
+
+  test("check new columns after modifying schema through alter table queries") 
{
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(

Review comment:
   as i can see, there are no long string columns, the table property 
`long_string_columns` doesnt have any value. SO how the unset long string 
affected. Basically we should avoid the flow going too far when there are no 
long string columns itself. can you check?

##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,24 @@ class VarcharDataTypesBasicTestCase extends QueryTest with 
BeforeAndAfterEach wi
 
 sql("DROP TABLE IF EXISTS varchar_complex_table")
   }
+
+  test("check new columns after modifying schema through alter table queries") 
{
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)

Review comment:
   also add a test case where long string column present in table property





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] marchpure commented on pull request #3939: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


marchpure commented on pull request #3939:
URL: https://github.com/apache/carbondata/pull/3939#issuecomment-696595812


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] jackylk commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


jackylk commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-696672614


   add to whitelist



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akashrn5 commented on pull request #3915: [CARBONDATA-3975]Fix wrong data from carbondata for binary column when read via hive

2020-09-22 Thread GitBox


akashrn5 commented on pull request #3915:
URL: https://github.com/apache/carbondata/pull/3915#issuecomment-696652560


   @kunal642 build passed, please review.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3949: [TEMP] [WIP] Fix issue in concurrent SI global sort

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3949:
URL: https://github.com/apache/carbondata/pull/3949#issuecomment-696740401







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kunal642 commented on a change in pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

2020-09-22 Thread GitBox


kunal642 commented on a change in pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#discussion_r492497322



##
File path: 
integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java
##
@@ -158,7 +158,9 @@ private static CarbonTable getCarbonTable(Configuration 
configuration, String pa
 try {
   splitList = carbonInputFormat.getSplits(jobContext);
 } catch (IOException ex) {
-  if (ex.getMessage().contains("No Index files are present in the table 
location :")) {
+  LOGGER.error("Unable to get splits: ", ex);
+  if (ex.getMessage().contains("No Index files are present in the table 
location :") ||

Review comment:
   This new added scenario is only failing in cluster





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#issuecomment-696554780







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kunal642 commented on pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

2020-09-22 Thread GitBox


kunal642 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696577595







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] jxxfxkp commented on pull request #3114: [CARBONDATA-3282] set hadoop conf to thread local for file factory usage in presto carbon

2020-09-22 Thread GitBox


jxxfxkp commented on pull request #3114:
URL: https://github.com/apache/carbondata/pull/3114#issuecomment-696586808


   There is no version of carbondata bloomfilter that can take effect on presto 
now.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Kejian-Li closed pull request #3889: Fix a spelling mistake

2020-09-22 Thread GitBox


Kejian-Li closed pull request #3889:
URL: https://github.com/apache/carbondata/pull/3889


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


ajantha-bhat commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696685344


   LGTM. 
   Will merge once build passes.
   
   Please create jira for inverted index issue and handle in separate PR



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3939: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3939:
URL: https://github.com/apache/carbondata/pull/3939#issuecomment-696490156







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kunal642 commented on pull request #3944: [CARBONDATA-4001] Fix SI global sort load on partition table

2020-09-22 Thread GitBox


kunal642 commented on pull request #3944:
URL: https://github.com/apache/carbondata/pull/3944#issuecomment-696639688


   LGTM



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akashrn5 commented on a change in pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

2020-09-22 Thread GitBox


akashrn5 commented on a change in pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#discussion_r492488092



##
File path: 
integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java
##
@@ -158,7 +158,9 @@ private static CarbonTable getCarbonTable(Configuration 
configuration, String pa
 try {
   splitList = carbonInputFormat.getSplits(jobContext);
 } catch (IOException ex) {
-  if (ex.getMessage().contains("No Index files are present in the table 
location :")) {
+  LOGGER.error("Unable to get splits: ", ex);
+  if (ex.getMessage().contains("No Index files are present in the table 
location :") ||

Review comment:
   can you add UT for these, so that later if anyone changes message, they 
should take care here.

##
File path: 
integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java
##
@@ -158,7 +158,9 @@ private static CarbonTable getCarbonTable(Configuration 
configuration, String pa
 try {
   splitList = carbonInputFormat.getSplits(jobContext);
 } catch (IOException ex) {
-  if (ex.getMessage().contains("No Index files are present in the table 
location :")) {
+  LOGGER.error("Unable to get splits: ", ex);
+  if (ex.getMessage().contains("No Index files are present in the table 
location :") ||
+  ex.getMessage().contains("CarbonData file is not present in the 
table location")) {

Review comment:
   As i can see from code, `CarbonData file is not present in the table 
location` is a FIleNotFoundException message, not IO, so this condition will be 
always false?
   





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#issuecomment-696812265







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3944: [CARBONDATA-4001] Fix SI global sort load on partition table

2020-09-22 Thread GitBox


ajantha-bhat commented on a change in pull request #3944:
URL: https://github.com/apache/carbondata/pull/3944#discussion_r492562276



##
File path: 
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
##
@@ -466,15 +466,15 @@ public IndexFilter getFilterPredicates(Configuration 
configuration) {
* get data blocks of given segment
*/
   protected List getDataBlocksOfSegment(JobContext job, 
CarbonTable carbonTable,
-  IndexFilter expression, List segmentIds,
+  IndexFilter expression, List validSegments,

Review comment:
   Actually when the main table was giving 0 rows. I was debugging this 
flow. And this naming is wrong. segmentID should be string, here it is not the 
segmentID but the complete Segment object. so renamed. Still you want to revert 
it ?

##
File path: 
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
##
@@ -466,15 +466,15 @@ public IndexFilter getFilterPredicates(Configuration 
configuration) {
* get data blocks of given segment
*/
   protected List getDataBlocksOfSegment(JobContext job, 
CarbonTable carbonTable,
-  IndexFilter expression, List segmentIds,
+  IndexFilter expression, List validSegments,

Review comment:
   Actually when the main table was giving 0 rows. I was debugging this 
flow. And this naming is wrong (confusing to debug). segmentID should be 
string, here it is not the segmentID but the complete Segment object. so 
renamed. Still you want to revert it ?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Klaus-xjp closed pull request #3930: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


Klaus-xjp closed pull request #3930:
URL: https://github.com/apache/carbondata/pull/3930


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696663738







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] kunal642 commented on a change in pull request #3944: [CARBONDATA-4001] Fix SI global sort load on partition table

2020-09-22 Thread GitBox


kunal642 commented on a change in pull request #3944:
URL: https://github.com/apache/carbondata/pull/3944#discussion_r492552371



##
File path: 
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
##
@@ -466,15 +466,15 @@ public IndexFilter getFilterPredicates(Configuration 
configuration) {
* get data blocks of given segment
*/
   protected List getDataBlocksOfSegment(JobContext job, 
CarbonTable carbonTable,
-  IndexFilter expression, List segmentIds,
+  IndexFilter expression, List validSegments,

Review comment:
   revert this change.. Not related to this PR

##
File path: 
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java
##
@@ -466,15 +466,15 @@ public IndexFilter getFilterPredicates(Configuration 
configuration) {
* get data blocks of given segment
*/
   protected List getDataBlocksOfSegment(JobContext job, 
CarbonTable carbonTable,
-  IndexFilter expression, List segmentIds,
+  IndexFilter expression, List validSegments,

Review comment:
   its fine.. ill merge
   





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3941: [CARBONDATA-3998] Fixed file not found exception for hive during insert

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3941:
URL: https://github.com/apache/carbondata/pull/3941#issuecomment-696565124







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3945: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3945:
URL: https://github.com/apache/carbondata/pull/3945#issuecomment-697115511







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3939: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3939:
URL: https://github.com/apache/carbondata/pull/3939#issuecomment-697114458


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4189/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3939: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3939:
URL: https://github.com/apache/carbondata/pull/3939#issuecomment-697113168


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2446/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ajantha-bhat commented on pull request #3948: [WIP] Analyze random 11 testcase failure in CI

2020-09-22 Thread GitBox


ajantha-bhat commented on pull request #3948:
URL: https://github.com/apache/carbondata/pull/3948#issuecomment-697098260


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Zhangshunyu commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


Zhangshunyu commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697078961


   Consider a scenario that user want to insert data and then update the table, 
the dml command is in time series which is sent 1 by 1, but the 2nd(update) cmd 
is sent by another driver and start to execute while the 1st(insert) is still 
in running, so it comes to the concurrent scenario described in this pr.
   If we allow update table who has segment with status 'INSERT_INPROGRESS', 
the 2nd cmd(update) will be executed successfully and the data inserted by the 
1st cmd would not updated by the 2nd cmd.
   Maybe in this scenario the result is not what the user expected.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Zhangshunyu removed a comment on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


Zhangshunyu removed a comment on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697074445


   pls handle the filed testcases



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Zhangshunyu commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


Zhangshunyu commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697074445


   pls handle the filed testcases



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Zhangshunyu commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


Zhangshunyu commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-697074579


   please handle the failed testcases



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Klaus-xjp commented on pull request #3945: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


Klaus-xjp commented on pull request #3945:
URL: https://github.com/apache/carbondata/pull/3945#issuecomment-697071126


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#issuecomment-696947234


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4188/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#issuecomment-696947129


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2445/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696917979


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4186/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696918364


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2443/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696884057







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3949: [TEMP] [WIP] Fix issue in concurrent SI global sort

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3949:
URL: https://github.com/apache/carbondata/pull/3949#issuecomment-696884059







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3945: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3945:
URL: https://github.com/apache/carbondata/pull/3945#issuecomment-696868894


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4182/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


ShreelekhyaG commented on a change in pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#discussion_r492902403



##
File path: 
core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java
##
@@ -928,6 +928,11 @@ public void testTupeIDInUpdateScenarios() {
 Assert.assertEquals(CarbonTablePath.getShortBlockId(blockId), 
"0/0-0_0-0-0-1597409791503");
 blockId = 
"c3=aa/part-0-1001011_batchno0-0-0-1597411003332.snappy.carbondata";
 
Assert.assertEquals(CarbonTablePath.getShortBlockIdForPartitionTable(blockId), 
"c3=aa/0-1001011_0-0-0-1597411003332");
+// CACHE_LEVEL = BLOCKLET case

Review comment:
   Done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] VenuReddy2103 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


VenuReddy2103 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696851559


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#issuecomment-696850231


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4181/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-696833256


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4180/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3945: [CARBONDATA-3991]Fix the set modified time function on S3 and Alluxio…

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3945:
URL: https://github.com/apache/carbondata/pull/3945#issuecomment-696824979


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2439/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#issuecomment-696820715


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4178/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-696814038


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2437/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#issuecomment-696812729


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2438/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#issuecomment-696812265


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2435/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696805184


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4177/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696802238


   > LGTM.
   > Will merge once build passes.
   > 
   > Please create jira for inverted index issue and handle in separate PR
   
   Done



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696800945


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4176/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for carbondata-spark test code

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3950:
URL: https://github.com/apache/carbondata/pull/3950#issuecomment-696777035


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4179/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696770718


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 removed a comment on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 removed a comment on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696750855


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


Indhumathi27 commented on a change in pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#discussion_r492786590



##
File path: 
core/src/test/java/org/apache/carbondata/core/util/CarbonUtilTest.java
##
@@ -928,6 +928,11 @@ public void testTupeIDInUpdateScenarios() {
 Assert.assertEquals(CarbonTablePath.getShortBlockId(blockId), 
"0/0-0_0-0-0-1597409791503");
 blockId = 
"c3=aa/part-0-1001011_batchno0-0-0-1597411003332.snappy.carbondata";
 
Assert.assertEquals(CarbonTablePath.getShortBlockIdForPartitionTable(blockId), 
"c3=aa/0-1001011_0-0-0-1597411003332");
+// CACHE_LEVEL = BLOCKLET case

Review comment:
   Please add a functional testcase for this scenario and check data after 
setting cache_level=blocklet





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3951: [WIP] SI with cache level blocklet issue

2020-09-22 Thread GitBox


Indhumathi27 commented on a change in pull request #3951:
URL: https://github.com/apache/carbondata/pull/3951#discussion_r492785607



##
File path: 
core/src/main/java/org/apache/carbondata/core/util/path/CarbonTablePath.java
##
@@ -648,11 +648,20 @@ public static String getShortBlockId(String blockId) {
 .replace(DATA_PART_PREFIX, "").replace(CARBON_DATA_EXT, "");
 // to remove compressor name
 if (!blockId.equalsIgnoreCase(blockIdWithCompressorName)) {
-  int index = blockIdWithCompressorName.lastIndexOf(".");
+  int index = blockIdWithCompressorName.lastIndexOf(POINT);
+  int fileSeperatorIndex = 
blockIdWithCompressorName.lastIndexOf(File.separator);
   if (index != -1) {
-String replace =
-
blockIdWithCompressorName.replace(blockIdWithCompressorName.substring(index), 
"");
-return replace;
+String modifiedBlockId;

Review comment:
   Can extract common code from partition table and non-partition table to 
get block Id to new method





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3950: [CARBONDATA-3889] Enable scalastyle check for carbondata-spark test code

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3950:
URL: https://github.com/apache/carbondata/pull/3950#issuecomment-696761398


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2436/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] nihal0107 commented on pull request #3949: [TEMP] [WIP] Fix issue in concurrent SI global sort

2020-09-22 Thread GitBox


nihal0107 commented on pull request #3949:
URL: https://github.com/apache/carbondata/pull/3949#issuecomment-696758152


   retest this please.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696754597


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2434/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] akkio-97 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


akkio-97 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696750855


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3943: [CARBONDATA-4000] Presto select query failure right after firing update query.

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3943:
URL: https://github.com/apache/carbondata/pull/3943#issuecomment-696747560


   Build Failed  with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2440/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3949: [TEMP] [WIP] Fix issue in concurrent SI global sort

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3949:
URL: https://github.com/apache/carbondata/pull/3949#issuecomment-696743786


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4175/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3949: [TEMP] [WIP] Fix issue in concurrent SI global sort

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3949:
URL: https://github.com/apache/carbondata/pull/3949#issuecomment-696740401


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2432/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3947: [CARBONDATA-4003] Improve IUD Concurrency

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3947:
URL: https://github.com/apache/carbondata/pull/3947#issuecomment-696739176







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3948: [WIP] Analyze random 11 testcase failure in CI

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3948:
URL: https://github.com/apache/carbondata/pull/3948#issuecomment-696738875


   Build Success with Spark 2.4.5, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2430/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3948: [WIP] Analyze random 11 testcase failure in CI

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3948:
URL: https://github.com/apache/carbondata/pull/3948#issuecomment-696735655


   Build Failed  with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4173/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (CARBONDATA-4005) SI with cache level blocklet issue

2020-09-22 Thread SHREELEKHYA GAMPA (Jira)
SHREELEKHYA GAMPA created CARBONDATA-4005:
-

 Summary: SI with cache level blocklet issue
 Key: CARBONDATA-4005
 URL: https://issues.apache.org/jira/browse/CARBONDATA-4005
 Project: CarbonData
  Issue Type: Bug
Reporter: SHREELEKHYA GAMPA


Select query on SI column returns blank resultset after changing the cache 
level to blocklet
PR: https://github.com/apache/carbondata/pull/3951



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3938: [CARBONDATA-3996]Fixed show table extended like command exception

2020-09-22 Thread GitBox


CarbonDataQA1 commented on pull request #3938:
URL: https://github.com/apache/carbondata/pull/3938#issuecomment-696728481


   Build Success with Spark 2.3.4, Please check CI 
http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4172/
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [carbondata] Karan980 commented on a change in pull request #3946: [CARBONDATA-4002] Fix removal of columns from table schema.

2020-09-22 Thread GitBox


Karan980 commented on a change in pull request #3946:
URL: https://github.com/apache/carbondata/pull/3946#discussion_r492735831



##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,24 @@ class VarcharDataTypesBasicTestCase extends QueryTest with 
BeforeAndAfterEach wi
 
 sql("DROP TABLE IF EXISTS varchar_complex_table")
   }
+
+  test("check new columns after modifying schema through alter table queries") 
{
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(

Review comment:
   Done

##
File path: 
integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/longstring/VarcharDataTypesBasicTestCase.scala
##
@@ -434,6 +437,24 @@ class VarcharDataTypesBasicTestCase extends QueryTest with 
BeforeAndAfterEach wi
 
 sql("DROP TABLE IF EXISTS varchar_complex_table")
   }
+
+  test("check new columns after modifying schema through alter table queries") 
{
+sql(
+  s"""
+ | CREATE TABLE if not exists $longStringTable(
+ | id INT, name STRING, description STRING, address STRING, note STRING
+ | ) STORED AS carbondata
+ | TBLPROPERTIES('sort_columns'='id,name')
+ |""".
+stripMargin)

Review comment:
   Done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




  1   2   >