Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/22571 )
Change subject: IMPALA-13684: Improve waitForHmsEvent() to only wait for related events ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/22571/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/22571/3//COMMIT_MSG@14 PS3, Line 14: This patch improves this by only checking related events and wait until : the last event of them has been processed. > Is this patch proactively pull related events only? What happen with the ga This patch just pulls and checks events but won't cache or process them. It won't disrupt the job of EventProcessor. Just be more intelligent to check if anything needs to be waited. For unrelated events in between related events, currently we have to wait until EventProcessor process all of them. Once we have hierarchical metastore event processing (IMPALA-12709), we can improve this to only check queuing events for each tables. http://gerrit.cloudera.org:8080/#/c/22571/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/22571/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1675 PS3, Line 1675: // Step 4: Check table events : requiredEventId = getMinRequiredTableEventId(requiredEventId, db2Tables); > What happen if after draining events in Step 4, there happens to be new eve I think it's OK for a query to not see the data changes after it's submitted. This is the same behavior as using manual REFRESH before the query: Data changes after the REFRESH might not be reflected in the query result. The query could also fail if the table is just removed in Hive. -- To view, visit http://gerrit.cloudera.org:8080/22571 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic033b7e197cd19505653c3ff80c4857cc474bcfc Gerrit-Change-Number: 22571 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Anonymous Coward <k.venureddy2...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Comment-Date: Fri, 14 Mar 2025 12:03:58 +0000 Gerrit-HasComments: Yes
