Rushabh Shah created PHOENIX-7115: ------------------------------------- Summary: Create separate handler thread pool for invalidating server metadata cache Key: PHOENIX-7115 URL: https://issues.apache.org/jira/browse/PHOENIX-7115 Project: Phoenix Issue Type: Sub-task Reporter: Rushabh Shah Assignee: Rushabh Shah
MutableIndexFailureIT#testIndexWriteFailure is failing. See [this|https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1726/9/#showFailuresLink] build for reference. Currently all the invalidateServerMetadataCache invocations are handled by default RPC handler threads. We have 5 default handler threads configured in tests. This test makes sure that write to index table fails and since disableIndexOnWriteFailure is set to true, it will disable the index. We use around 10 different threads on the client side to write to base table and index table, so we use all the 5 handler threads to serve writes. Since writes to index fails, within the handler threads it will try to update the index state to DISABLE. On receiving the updateIndexState rpc, MetadataEndpointImpl will try to invalidate the server metadata cache on all the regionservers. The regionserver hosting index table doesn't have any available handler threads to serve invalidateServerMetadataCache requests. Hence the test fails. The root cause is all the read/write operations and invalidate server metadata cache operations share the same RPC handler pool. We need to have separate thread pool for invalidate server metadata cache operations -- This message was sent by Atlassian Jira (v8.20.10#820010)