abstractdog commented on a change in pull request #2771:
URL: https://github.com/apache/hive/pull/2771#discussion_r783012583
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java
##########
@@ -1230,12 +1232,16 @@ else if(statementId != parsedDelta.statementId) {
* that for any dir, either all files are acid or all are not.
*/
public static ParsedDelta parsedDelta(Path deltaDir, FileSystem fs) throws
IOException {
- return parsedDelta(deltaDir, fs, null);
+ return parsedDelta(deltaDir, fs, null, false, -1);
}
- private static ParsedDelta parsedDelta(Path deltaDir, FileSystem fs,
HdfsDirSnapshot dirSnapshot)
+ private static ParsedDelta parsedDelta(Path deltaDir, FileSystem fs,
HdfsDirSnapshot dirSnapshot,
+ boolean canTrim, long highWaterMark)
throws IOException {
ParsedDeltaLight deltaLight = ParsedDeltaLight.parse(deltaDir);
+ if(canTrim && !(deltaLight.minWriteId >= highWaterMark)){
Review comment:
please comment this part code, as this is the point of this patch as I
understood
so according to some circumstances (canTrim + watermark thing, about which I
have no idea :) ), getAcidState -> getChildState -> parsedDelta can
short-circuit, so I'm assuming this will prevent processing some deltas,
leading to performance improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
