[GitHub] [carbondata] maheshrajus commented on a change in pull request #3808: [CARBONDATA-3873] Secondary index compaction with maintable clean files causing exception

[GitBox]

maheshrajus commented on a change in pull request #3808:
URL: https://github.com/apache/carbondata/pull/3808#discussion_r450301066



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/load/Compactor.scala
##
@@ -121,10 +124,17 @@ object Compactor {
   segmentIdToLoadStartTimeMapping(validSegments.head),
   SegmentStatus.SUCCESS,
   carbonLoadModelForMergeDataFiles.getFactTimeStamp, 
rebuiltSegments.toList.asJava)
-
+siCompactionIndexList ::= indexCarbonTable
   } catch {
 case ex: Exception =>
   LOGGER.error(s"Compaction failed for SI table 
${secondaryIndex.indexName}", ex)
+  siCompactionIndexList.foreach { indexCarbonTable =>
+sparkSession.sql(
+  s"""ALTER TABLE ${carbonLoadModel.getDatabaseName}.${

Review comment:
   OK





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] maheshrajus commented on a change in pull request #3808: [CARBONDATA-3873] Secondary index compaction with maintable clean files causing exception

[GitBox]

maheshrajus commented on a change in pull request #3808:
URL: https://github.com/apache/carbondata/pull/3808#discussion_r450300989



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/load/Compactor.scala
##
@@ -121,10 +124,17 @@ object Compactor {
   segmentIdToLoadStartTimeMapping(validSegments.head),
   SegmentStatus.SUCCESS,
   carbonLoadModelForMergeDataFiles.getFactTimeStamp, 
rebuiltSegments.toList.asJava)
-
+siCompactionIndexList ::= indexCarbonTable
   } catch {
 case ex: Exception =>
   LOGGER.error(s"Compaction failed for SI table 
${secondaryIndex.indexName}", ex)
+  siCompactionIndexList.foreach { indexCarbonTable =>

Review comment:
   OK





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] maheshrajus commented on a change in pull request #3808: [CARBONDATA-3873] Secondary index compaction with maintable clean files causing exception

[GitBox]

maheshrajus commented on a change in pull request #3808:
URL: https://github.com/apache/carbondata/pull/3808#discussion_r450299932



##
File path: 
core/src/main/java/org/apache/carbondata/core/datastore/filesystem/AbstractDFSCarbonFile.java
##
@@ -455,7 +455,7 @@ public boolean createNewLockFile() throws IOException {
 try {
   listStatus = fileSystem.listStatus(path);
 } catch (IOException e) {
-  LOGGER.error("Exception occured: " + e.getMessage(), e);

Review comment:
   
   @akashrn5 @VenuReddy2103  i will add details in PR description





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org