[GitHub] [carbondata] akashrn5 commented on a change in pull request #3811: [CARBONDATA-3874] segment mismatch between maintable and SI table when load with concurrency

2020-07-06 Thread GitBox 


akashrn5 commented on a change in pull request #3811:
URL: https://github.com/apache/carbondata/pull/3811#discussion_r450365209



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/load/CarbonInternalLoaderUtil.java
##
@@ -304,14 +304,16 @@ public static boolean 
updateLoadMetadataWithMergeStatus(CarbonTable indexCarbonT
 
   /**
* Method to check if main table and SI have same number of valid segments 
or not
-   *
+   * @param mainTableLoadMetadataDetails

Review comment:
   no need of this, just method explanation is enough, as variable name 
suggest what info is it.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akashrn5 commented on a change in pull request #3811: [CARBONDATA-3874] segment mismatch between maintable and SI table when load with concurrency

2020-07-06 Thread GitBox 


akashrn5 commented on a change in pull request #3811:
URL: https://github.com/apache/carbondata/pull/3811#discussion_r450364155



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/events/SILoadEventListenerForFailedSegments.scala
##
@@ -78,20 +78,27 @@ class SILoadEventListenerForFailedSegments extends 
OperationEventListener with L
   .getTableMetadata(TableIdentifier(indexTableName,
 Some(carbonLoadModel.getDatabaseName))).storage.properties
   .getOrElse("isSITableEnabled", "true").toBoolean
+val indexTable = metaStore
+  .lookupRelation(Some(carbonLoadModel.getDatabaseName), 
indexTableName)(
+sparkSession)
+  .asInstanceOf[CarbonRelation]
+  .carbonTable
 
-if (!isLoadSIForFailedSegments) {
+val listOfLoadFolderDetailsArrayMainTbl: 
Array[LoadMetadataDetails] =

Review comment:
   `listOfLoadFolderDetailsArrayMainTbl`  same as above comment





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akashrn5 commented on a change in pull request #3811: [CARBONDATA-3874] segment mismatch between maintable and SI table when load with concurrency

2020-07-06 Thread GitBox 


akashrn5 commented on a change in pull request #3811:
URL: https://github.com/apache/carbondata/pull/3811#discussion_r450363325



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/command/SICreationCommand.scala
##
@@ -390,8 +391,13 @@ private[sql] case class CarbonCreateSecondaryIndexCommand(
   }
   val indexTablePath = CarbonTablePath
 
.getMetadataPath(tableInfo.getOrCreateAbsoluteTableIdentifier.getTablePath)
+  val listOfLoadFolderDetailsArrayMainTbl: Array[LoadMetadataDetails] =
+SegmentStatusManager.readLoadMetadata(carbonTable.getMetadataPath)
+  val listOfLoadFolderDetailsArraySI: Array[LoadMetadataDetails] =

Review comment:
   similar to above





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [carbondata] akashrn5 commented on a change in pull request #3811: [CARBONDATA-3874] segment mismatch between maintable and SI table when load with concurrency

2020-07-06 Thread GitBox 


akashrn5 commented on a change in pull request #3811:
URL: https://github.com/apache/carbondata/pull/3811#discussion_r450363174



##
File path: 
integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/command/SICreationCommand.scala
##
@@ -390,8 +391,13 @@ private[sql] case class CarbonCreateSecondaryIndexCommand(
   }
   val indexTablePath = CarbonTablePath
 
.getMetadataPath(tableInfo.getOrCreateAbsoluteTableIdentifier.getTablePath)
+  val listOfLoadFolderDetailsArrayMainTbl: Array[LoadMetadataDetails] =

Review comment:
   `listOfLoadFolderDetailsArrayMainTbl` variable name is big and not 
clear, can give a meaningful name like `mainTblLoadMetadataDetails`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org