[
https://issues.apache.org/jira/browse/PHOENIX-7776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18062716#comment-18062716
]
Richárd Antal commented on PHOENIX-7776:
----------------------------------------
I build HBase with HBASE-29843 and HBASE-29834 while testing.
I also had [some changes like
this|https://github.com/richardantal/phoenix-1/blob/dff56a6946585944d1e34dd28de1b135d60ff425/phoenix-core-client/src/main/java/org/apache/hadoop/hbase/ipc/controller/MetadataRpcController.java#L73-L77]
to make sure that we call the setPriority.
What I noticed is that in a few cases when accessing hbase:meta we use
MetadataRpcController instead of ServerRpcController
For example that I wrote in the description of the ticket :
{code:java}
2026-02-24T12:13:20,429 DEBUG
[RpcServer.Metadata.Fifo.handler=1,queue=0,port=58706]
client.ConnectionUtils(526): Start fetching meta region location from registry
2026-02-24T12:13:20,429 DEBUG
[MiniHBaseClusterRegionServer-EventLoopGroup-3-16] client.ConnectionUtils(534):
The fetched meta region location is [region=hbase:meta,,1.1588230740, {code}
the above is the same with hbase 2.6 and 3.0
but later
{code:java}
2026-02-24T12:13:20,440 DEBUG
[RpcServer.Metadata.Fifo.handler=0,queue=0,port=58706]
util.ReflectedFunctionCache(97): Populated cache for
org.apache.hadoop.hbase.filter.SingleColumnValueFilter in 0ms
2026-02-24T12:13:20,440 DEBUG
[RpcServer.Metadata.Fifo.handler=0,queue=0,port=58706]
util.ReflectedFunctionCache(97): Populated cache for
org.apache.hadoop.hbase.filter.BinaryComparator in 0ms{code}
Which used for HBase 2.6 looked like
{code:java}
2026-02-25T14:32:04,527 DEBUG [hconnection-0x5414fadd-shared-pool-0]
util.ReflectedFunctionCache(97): Populated cache for
org.apache.hadoop.hbase.filter.SingleColumnValueFilter in 6ms
2026-02-25T14:32:04,528 DEBUG [hconnection-0x5414fadd-shared-pool-0]
util.ReflectedFunctionCache(97): Populated cache for
org.apache.hadoop.hbase.filter.BinaryComparator in 0ms{code}
Also edited the stack traces for this in the ticket's description
> Fix hanging tests because of Metadata handlerss
> -----------------------------------------------
>
> Key: PHOENIX-7776
> URL: https://issues.apache.org/jira/browse/PHOENIX-7776
> Project: Phoenix
> Issue Type: Task
> Reporter: Richárd Antal
> Priority: Major
>
> With HBase 3 there are a lot of tests not finishing because we ran out of
> metadata handlers configure by METADATA_HANDLER_COUNT_ATTRIB.
> Few tests where it is easily visible are:
> ConcurrentGetTablesIT
> instead of 2 , usually passes with 20 failes with 18
> [https://github.com/richardantal/phoenix-1/blob/dff56a6946585944d1e34dd28de1b135d60ff425/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentGetTablesIT.java#L74]
> SingleMetadataHandlerIT
> With 1 metadata handler and 1 server side handler it hangs after
> {code:java}
> [RpcServer.Metadata.Fifo.handler=0,queue=0,port=58485]
> client.ConnectionUtils(526): Start fetching meta region location from
> registry{code}
> and there is no log line similar to
> {code:java}
> 2026-02-24T12:13:20,440 DEBUG
> [RpcServer.Metadata.Fifo.handler=0,queue=0,port=58706]
> util.ReflectedFunctionCache(97): Populated cache for
> org.apache.hadoop.hbase.filter.SingleColumnValueFilter in 0ms
> 2026-02-24T12:13:20,440 DEBUG
> [RpcServer.Metadata.Fifo.handler=0,queue=0,port=58706]
> util.ReflectedFunctionCache(97): Populated cache for
> org.apache.hadoop.hbase.filter.BinaryComparator in 0ms {code}
> With 2 metadata handlers and 0 server side handler it passes
> With this change it also passes:
> [https://github.com/richardantal/phoenix-1/blob/dff56a6946585944d1e34dd28de1b135d60ff425/phoenix-core/src/it/java/org/apache/phoenix/end2end/SingleMetadataHandlerIT.java#L57-L61]
> I've looked into it and there is a difference how we check the hbase:meta
> table
> For HBase 2.6
>
> {code:java}
> org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory.getController(ServerRpcControllerFactory.java:58)
> org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory.newController(ServerRpcControllerFactory.java:41)
> org.apache.hadoop.hbase.client.HTable.get(HTable.java:388)
> this.rpcControllerFactory.newController()
> if (rpcControllerFactory == null) {
> this.rpcControllerFactory =
> RpcControllerFactory.instantiate(configuration);
> } else {
> this.rpcControllerFactory = rpcControllerFactory;
> }
> {code}
>
> For HBase 3
> {code:java}
> org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.<init>(AsyncRpcRetryingCaller.java:91)
> this.controller = conn.rpcControllerFactory.newController();
> this.rpcControllerFactory =
> RpcControllerFactory.instantiate(conf);
> String rpcControllerFactoryClazz =
> configuration.get(CUSTOM_CONTROLLER_CONF_KEY,
> RpcControllerFactory.class.getName());
> set to ClientRpcControllerFactory.class.getName();{code}
>
> So for HBase 2.6 we use the previously set rpcControllerFactory, for HBase 3
> we use the CUSTOM_CONTROLLER_CONF_KEY which is ClientRpcControllerFactory and
> will create MetadataRpcController
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
