marchpure commented on a change in pull request #3880:
URL: https://github.com/apache/carbondata/pull/3880#discussion_r466265360
##########
File path:
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonTableInputFormat.java
##########
@@ -389,14 +389,16 @@ protected FileSplit makeSplit(String segmentId, String
filePath, long start, lon
public void updateLoadMetaDataDetailsToSegments(List<Segment> validSegments,
List<org.apache.carbondata.hadoop.CarbonInputSplit> prunedSplits) {
+ Map<String, Segment> validSegmentsMap = validSegments.stream()
Review comment:
Appreciate for that good suggestion. But here we can only use Map
instead of Set.
Reason: The The pseudo-code for this function is shown as below.
// **0. segments.hashcode is segmentno, to when we compare 2
segments, only segmentno will be compared**
if (validSegments.contains(segmentInSplit)) {
**1. fetch the segment from validSegments**
Segment segmentInValidSegment <- fetch the segment from
validSegments
2.
segmentInSplit.setLoadMetadataDetails(segmentInValidSegment.getLoadMetadataDetails);
}
For SET. it's hard to fetch the segment from validSegments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
