Jason Fehr has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24275
Change subject: IMPALA-14949: Fix Catalogd Startup Deadlock ...................................................................... IMPALA-14949: Fix Catalogd Startup Deadlock Reset metadata handling code no longer yields up the write lock on versionLock_ in CatalogServiceCatalog during the Catalogd startup initial reset metadata. When Catalogd runs a global reset, the rebuildDbCache() function will pause and unlock versionLock_ at intervals defined by the --reset_metadata_lock_duration_ms startup flag. The purpose of this unlocking is to ensure a long running invalidate metadata does not block other metadata operations. During startup, Catalogd runs a global metadata reset. Other threads must wait until this initial reset finishes before they can process operations such as getPartialCatalogObject. If the initial reset takes longer than the duration in the --reset_metadata_lock_duration_ms startup flag, it will release its ownership of versionLock_ followed by immediately re-obtaining the write lock. In between, other threads take a read lock on versionLock_ but sleep until the initial reset completes. At that point, a deadlock situation occurs where the initial reset thread is waiting on a versionLock_ write lock while other threads that hold the versionLock_ read lock are waiting on the initial reset thread to complete. Manual testing replicated this issue and demonstrated the fix effectiveness. The replication required adding an arbitrary sleep in the CatalogServiceCatalog.rebuildDbCache function's while loop. Change-Id: Ibe71980d6fed5a95b00bc7b710c781f909545404 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/24275/1 -- To view, visit http://gerrit.cloudera.org:8080/24275 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ibe71980d6fed5a95b00bc7b710c781f909545404 Gerrit-Change-Number: 24275 Gerrit-PatchSet: 1 Gerrit-Owner: Jason Fehr <[email protected]>
