[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340752#comment-16340752
]
Pedro Boado commented on PHOENIX-4553:
--------------------------------------
Are you running a quickstart VM? That is basically the test that I ran
yesterday -logs are almost 100% the same-.
Only difference is network addressing - my VM uses a bridged network - .
But HMaster is definitely running and waiting for a RS to connect to it
{code}
2018-01-26 07:57:04,571 INFO org.apache.hadoop.hbase.master.ServerManager:
Waiting for region servers count to settle; currently checked in 0, slept for
324842 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
interval of 1500 ms.
{code}
And RS is definitely running and waiting to connect to HMaster
{code}
2018-01-26 07:56:34,777 INFO
SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
Authorization successful for hbase/quickstart.cloudera@CLOUDERA
(auth:KERBEROS) for protocol=interface
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingInterface
2018-01-26 07:56:35,616 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: error telling master we are
up com.google.protobuf.ServiceException: java.io.IOException: Call to
quickstart.cloudera/172.23.0.2:60000 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=22, waitTime=10001,
operationTimeout=10000 expired. at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:240)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at
org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2324)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:922)
at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Call
to quickstart.cloudera/172.23.0.2:60000 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=22, waitTime=10001,
operationTimeout=10000 expired. at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:292)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1273) at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
... 5 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call
id=22, waitTime=10001, operationTimeout=10000 expired. at
org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:73) at
org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1247) ... 6
more
{code}
I've run into similar issues before when running this VM with insufficient
resources - process delays were too high to keep a stable cluster running -
Have you noticed that the RS is starting way earlier than the master?
I don't think this is an issue - unless someone else reports it as well - .
I'll keep the ticket open for a while just in case.
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -----------------------------------------------------------------
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
> Reporter: Ihor Krysenko
> Priority: Minor
> Attachments: hbase-master.log, hbase-region.log, master-stderr.log,
> master-stdout.log, region-stderr.log, region-stdout.log
>
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=60000"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
> java.util.ServiceLoader.access$300(ServiceLoader.java:181) at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:369) at
> java.util.ServiceLoader$1.next(ServiceLoader.java:445) at
> org.apache.hadoop.security.SecurityUtil.getKerberosInfo(SecurityUtil.java:333)
> at
> org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:101)
> at org.apache.hadoop.hbase.ipc.RpcServer.authorize(RpcServer.java:2347) at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.authorizeConnection(RpcServer.java:1898)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.processOneRpc(RpcServer.java:1772)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1335)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1614)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1596)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:854)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:635)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:611)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
