Hello Bharath Vissapragada, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5556 to look at the new patch set (#3). Change subject: IMPALA-3641: Fix catalogd RPC responses to DROP IF EXISTS. ...................................................................... IMPALA-3641: Fix catalogd RPC responses to DROP IF EXISTS. The main problem was that the catalogd's response to a some DROP IF EXISTS operations included a removed object that was applied to the requesting impalad's catalog cache. In particular, a DROP DATABASE IF EXISTS that did not actually drop anything in the catalogd still returned the object name in the RPC response as a removed object with the *current* catalog version (i.e., without incrementing the catalog version). The above behavior lead to a situation where a drop of a non-existent object overwrote a legitimate entry in an impalad's CatalogDeltaLog. Recall that the version of the dropped object was based on the current catalog version at some point in time, e.g., the same version of a legitimate entry in the CatalogDeltaLog. As a reminder, the CatalogDeltaLog protects deletions from being undone via updates from the statestore. So overwriting an object in the CatalogDeltaLog can lead to a dropped object appearing again with certain timing of a statestore update. Please see the JIRA for an analysis of logging output that shows the bug and its effect. The fix is simple: The RPC response of a DROP IF EXISTS should only contain a removed object if an object was actually removed from the catalogd. Testing: - Unfortunately, I have not been able to reproduce the issue locally despite vigorous attempts and despite knowing what the problem is. Our existing tests seem to reproduce the issue pretty reliably, so it's not clear whether a targeted test is feasible or needed. - An exhaustive test run passed. Change-Id: Icb1f31eb2ecf05b9b51ef4e12e6bb78f44d0cf84 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java 2 files changed, 77 insertions(+), 72 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/5556/3 -- To view, visit http://gerrit.cloudera.org:8080/5556 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Icb1f31eb2ecf05b9b51ef4e12e6bb78f44d0cf84 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com>