Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23174 )
Change subject: IMPALA-14227: In HA failover, passive catalogd should apply pending HMS events before being active ...................................................................... Patch Set 7: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/23174/4//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/23174/4//COMMIT_MSG@11 PS4, Line 11: However, it could still use : a stale metadata cache when some pending HMS events generated by the : previous active catalogd are not applied yet. > Good point! It's impossible for the passive catalogd to keeps the metadata Out of scope for this patch, but I was thinking if it could be still possible to keep some prewarmed data even if there is no event processing or it has a hopelessly large lag. In this case all tables would need to be reloaded, but this could be still faster if we can reuse old file descriptors and reload all block locations. Implementing this may be a bit tricky - tables could be marked in catalog as "loaded but stale", and the reload could be initiated when some coordinator tries to load the table. This logic could be actually pretty efficient with event processor, as it could be checked whether there was any event on the given table, and avoid reloading it if the snapshot seems up to date. -- To view, visit http://gerrit.cloudera.org:8080/23174 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Icf4fcb0e27c14197f79625749949b47c033a5f31 Gerrit-Change-Number: 23174 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]> Gerrit-Comment-Date: Thu, 17 Jul 2025 09:15:07 +0000 Gerrit-HasComments: Yes
