Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21087 )
Change subject: IMPALA-12851: Fix AllocWriteIdEvent process issue to add txnId-tableWriteIds mapping ...................................................................... IMPALA-12851: Fix AllocWriteIdEvent process issue to add txnId-tableWriteIds mapping During AllocWriteIdEvent process, txnId to tableWriteIds mapping is not added to catalog in the following cases: 1. When CREATE_TABLE event is followed by ALLOC_WRITE_ID_EVENT for the table in the same batch of MetastoreEventsProcessor.processEvents(), process AllocWriteIdEvent cannot find catalog table since CREATE_TABLE is not processed by the time of AllocWriteIdEvent object construction. 2. When catalog table is present. But it is not loaded. This patch fixes: 1. Removes the usage of get table from catalog in all the event constructors. Currently, AllocWriteIdEvent, ReloadEvent, CommitCompactionEvent get the catalog table in their constructors. 2. Adds the txnId to tableWriteIds mapping in catalog even when table is not loaded. And ensures the write ids are not added to table if it is a non-partitioned table. 3. Also fixed a bug in TableWriteId's hashCode() implementation that is breaking hashcode contract. Two same TableWriteId of different instances produce different hashcode though they are equal. 4. Fixed CatalogHmsSyncToLatestEventIdTest.cleanUp() issue. flagInvalidateCache and flagSyncToLatestEventId are incorrectly set in cleanUp. Testing: - Added tests in MetastoreEventsProcessorTest Change-Id: I8b1a918befd4ee694880fd4e3cc04cb55b64955f Reviewed-on: http://gerrit.cloudera.org:8080/21087 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/TableWriteId.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/catalog/metastore/CatalogHmsSyncToLatestEventIdTest.java 5 files changed, 198 insertions(+), 42 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21087 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8b1a918befd4ee694880fd4e3cc04cb55b64955f Gerrit-Change-Number: 21087 Gerrit-PatchSet: 11 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
