Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/10543 )
Change subject: IMPALA-6119: Fix issue with multiple partitions sharing same location ...................................................................... Patch Set 10: (1 comment) http://gerrit.cloudera.org:8080/#/c/10543/9/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/10543/9/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1440 PS9, Line 1440: dropPartitions(partitionsToRemove); This wouldn't work out unfortunately. When you modify the location of a partition than internally the modified partition is set dirty and then it is added to partitionsToRemove at L1439. Here we say that we remove all the other partitions on the same location, but as a result if we originally intended to set the location of a partition to an existing partition's location we end up dropping the other partition as well. Ouch :) In my opinion here we shouldn't call getPartitionsWithSameLocation. The reason we had to do this is that in L1399 Hive returns the available partitions incorrectly (In case we drop a partition with Hive the ones on the same location still appear as valid even though the directory beneath is dropped). So I'd go for not introducing a workaround for this in Impala, I rather opened a Jira for Hive to fix the root cause. Until it is taken care I'd disable the new test_drop_partition_from_hive test and enable it once Hive returns the correct list of partitions. What do you think? -- To view, visit http://gerrit.cloudera.org:8080/10543 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2a54bc8224bcefe65b83de2df58bb84629f2aa4a Gerrit-Change-Number: 10543 Gerrit-PatchSet: 10 Gerrit-Owner: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Fri, 08 Jun 2018 13:36:37 +0000 Gerrit-HasComments: Yes
