Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22499 )
Change subject: IMPALA-13759: Fix Hive ACID INSERT OVERWRITE base detection ...................................................................... IMPALA-13759: Fix Hive ACID INSERT OVERWRITE base detection Base directory created by INSERT OVERWRITE / TRUNCATE should be treated differently than bases created by compaction because IOW/TRUNCATE bases must be accepted even if there is an earlier open writeId. This scenario can easily occur if there is a pending write to a single partition, as this doesn't block an IOW/TRUNCATE to another partition, while the global minOpenWrite affects whether the base is accepted. This change updates Impala logic to consider these bases valid similarly to Hive. Note that differentiating IOW/TRUNCATE from compaction is different than in Hive, as metadata files are not considered in Impala (IMPALA-13769). This can only cause problems when interacting with earlier Hive versions that did not use visibilityTxnId in the base path. I don't consider this to be a significant regression that should block the current critical fix. Testing: - added regression EE/FE tests Change-Id: I838eaf4f41bae148e558f64288a1370c0908efa4 Reviewed-on: http://gerrit.cloudera.org:8080/22499 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/util/AcidUtils.java M fe/src/test/java/org/apache/impala/util/AcidUtilsTest.java M tests/query_test/test_acid.py 3 files changed, 61 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/22499 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I838eaf4f41bae148e558f64288a1370c0908efa4 Gerrit-Change-Number: 22499 Gerrit-PatchSet: 4 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
