Hi all,

While running Hive 4.2.0 standalone metastore with CachedStore enabled
(~21k whitelisted tables), we found that a cold-started HMS accepts
connections but serves zero RPCs until prewarm completes (~10 minutes for
us).

Root cause: triggerPreWarm() and startCacheUpdateService() are both static
synchronized, so they share the CachedStore class monitor. The background
thread holds that monitor for the entire prewarm, while every new Thrift
worker thread calls startCacheUpdateService via setConf when constructing
its thread-local RawStore — so all workers block before reaching the
existing serve-from-rawstore fallback paths. Thread dumps show workers
BLOCKED on the class monitor at startCacheUpdateService. The code is
unchanged on master, so current releases are affected too.

Fix (verified in production): move
startCacheUpdateService/stopCacheUpdateService onto a dedicated lock object
and make cacheUpdateMaster volatile with a lock-free fast path. After the
fix, RPCs are served from the backing DB during prewarm (get_table ~30ms in
our tests) and switch to the cache when prewarm completes. We also have a
deterministic JUnit regression test.

Separately, we prototyped a config-gated multi-threaded prewarm (default 1
= current behavior, each worker with its own RawStore instance) Happy to
contribute that as a follow-up.

My ASF JIRA account request (username: neathypertxt) is pending review —
once it's created I'll file the issue and open a PR against master.

Thanks,
Vidit Gupta
Data Platform, Meesho

-- 
***
This communication is confidential, may be privileged, and is meant 
only for the intended recipient and purpose. No part of this email or any 
files transmitted with it can be shared, copied, forwarded, published 
online or offline, or used in any unauthorised manner. If you are not the 
intended recipient, please preserve the confidentiality of the contents, 
delete the e-mail and attachments (if any) from your system, and inform the 
sender immediately.
***

Reply via email to