[
https://issues.apache.org/jira/browse/PHOENIX-7776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18062198#comment-18062198
]
Richárd Antal commented on PHOENIX-7776:
----------------------------------------
[~vjasani] Do you have an ida how we should handle this issue?
i've tried to increase the METADATA_HANDLER_COUNT_ATTRIB and when I run the
tests separately they are more likely to pass but when I run them together I
get out of memory errors and it just hangs..
You worked on PHOENIX-7370 so probably you have the best knowledge around this
area.
> Fix hanging tests because of Metadata handlerss
> -----------------------------------------------
>
> Key: PHOENIX-7776
> URL: https://issues.apache.org/jira/browse/PHOENIX-7776
> Project: Phoenix
> Issue Type: Task
> Reporter: Richárd Antal
> Priority: Major
>
> With HBase 3 there are a lot of tests not finishing because we ran out of
> metadata handlers configure by METADATA_HANDLER_COUNT_ATTRIB.
> Few tests where it is easily visible are:
> ConcurrentGetTablesIT
> instead of 2 , usually passes with 20 failes with 18
> [https://github.com/richardantal/phoenix-1/blob/dff56a6946585944d1e34dd28de1b135d60ff425/phoenix-core/src/it/java/org/apache/phoenix/end2end/ConcurrentGetTablesIT.java#L74]
> SingleMetadataHandlerIT
> With 1 metadata handler and 1 server side handler it hangs after
> {code:java}
> [RpcServer.Metadata.Fifo.handler=0,queue=0,port=58485]
> client.ConnectionUtils(526): Start fetching meta region location from
> registry{code}
> and there is no log line similar to
> {code:java}
> 2026-02-24T12:13:20,440 DEBUG
> [RpcServer.Metadata.Fifo.handler=0,queue=0,port=58706]
> util.ReflectedFunctionCache(97): Populated cache for
> org.apache.hadoop.hbase.filter.SingleColumnValueFilter in 0ms
> 2026-02-24T12:13:20,440 DEBUG
> [RpcServer.Metadata.Fifo.handler=0,queue=0,port=58706]
> util.ReflectedFunctionCache(97): Populated cache for
> org.apache.hadoop.hbase.filter.BinaryComparator in 0ms {code}
> With 2 metadata handlers and 0 server side handler it passes
> With this change it also passes:
> [https://github.com/richardantal/phoenix-1/blob/dff56a6946585944d1e34dd28de1b135d60ff425/phoenix-core/src/it/java/org/apache/phoenix/end2end/SingleMetadataHandlerIT.java#L57-L61]
> I've looked into it and there is a difference how we check the hbase:meta
> table
> For HBase 2.6
>
> {code:java}
> org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory.getController(ServerRpcControllerFactory.java:58)
> org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory.newController(ServerRpcControllerFactory.java:41)
> org.apache.hadoop.hbase.client.HTable.get(HTable.java:388)
> this.rpcControllerFactory.newController()
> if (rpcControllerFactory == null) {
> this.rpcControllerFactory =
> RpcControllerFactory.instantiate(configuration);
> } else {
> this.rpcControllerFactory = rpcControllerFactory;
> }
> {code}
>
> For HBase 3
> {code:java}
> org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.<init>(AsyncRpcRetryingCaller.java:91)
> this.controller = conn.rpcControllerFactory.newController();
> this.rpcControllerFactory =
> RpcControllerFactory.instantiate(conf);
> String rpcControllerFactoryClazz =
> configuration.get(CUSTOM_CONTROLLER_CONF_KEY,
> RpcControllerFactory.class.getName());
> set to ClientRpcControllerFactory.class.getName();{code}
>
> So for HBase 2.6 we use the previously set rpcControllerFactory, for HBase 3
> we use the CUSTOM_CONTROLLER_CONF_KEY which is ClientRpcControllerFactory and
> will create MetadataRpcController
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
