[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394062#comment-16394062
]
Ihor Krysenko edited comment on PHOENIX-4553 at 3/10/18 6:25 AM:
-
[~pboado] I can check only in Monday, btw, I see, it will work, thanks.
was (Author: neospyk):
[~pboado] I can check only in Monday, btw, I see, it will work thanks.
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
>Reporter: Ihor Krysenko
>Assignee: Pedro Boado
>Priority: Minor
> Attachments: PHOENIX-4553-00.patch, hbase-master.log,
> hbase-region.log, master-stderr.log, master-stdout.log, region-stderr.log,
> region-stdout.log
>
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=6"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
> java.util.ServiceLoader.access$300(ServiceLoader.java:181) at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:369) at
> java.util.ServiceLoader$1.next(ServiceLoader.java:445) at
> org.apache.hadoop.security.SecurityUtil.getKerberosInfo(SecurityUtil.java:333)
> at
> org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:101)
> at org.apache.hadoop.hbase.ipc.RpcServer.authorize(RpcServer.java:2347) at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.authorizeConnection(RpcServer.java:1898)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.processOneRpc(RpcServer.java:1772)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1335)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1614)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1596)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:854)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:635)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:611)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393874#comment-16393874
]
Pedro Boado edited comment on PHOENIX-4553 at 3/10/18 12:43 AM:
[~neospyk] I've come across this issue again and noticed that the classpath
configured is actually wrong. Only
{code:java}
/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-server.jar
{code}
should be in HBase classpath. After amending it logs are clear again.
Can you please confirm that by amending file
/opt/cloudera/parcels/APACHE_PHOENIX/meta/phoenix_env.sh and replacing
{code}
APPENDSTRING=`echo ${MYLIBDIR}/*.jar | sed 's/ /:/g'
{code}
by
{code}
APPENDSTRING=`echo ${MYLIBDIR}/phoenix-*-server.jar | sed 's/ /:/g'
{code}
solves the issue with the warnings appearing during RS startup?
was (Author: pboado):
[~neospyk] I've come across this issue again and noticed that the classpath
configured is actually wrong. Only
{code:java}
/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-server.jar
{code}
should be in HBase classpath. After amending it logs are clear again.
Can you please confirm that by amending file
/opt/cloudera/parcels/APACHE_PHOENIX/meta/phoenix_env.sh and replacing
{code}
APPENDSTRING=`echo ${MYLIBDIR}/phoenix-*-server.jar | sed 's/ /:/g'
{code}
by
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
>Reporter: Ihor Krysenko
>Assignee: Pedro Boado
>Priority: Minor
> Attachments: hbase-master.log, hbase-region.log, master-stderr.log,
> master-stdout.log, region-stderr.log, region-stdout.log
>
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=6"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
> java.util.ServiceLoader.access$300(ServiceLoader.java:181) at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:369) at
> java.util.ServiceLoader$1.next(ServiceLoader.java:445) at
> org.apache.hadoop.security.SecurityUtil.getKerberosInfo(SecurityUtil.java:333)
> at
> org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:101)
> at org.apache.hadoop.hbase.ipc.RpcServer.authorize(RpcServer.java:2347) at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.authorizeConnection(RpcServer.java:1898)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.processOneRpc(RpcServer.java:1772)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1335)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1614)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1596)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:854)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:635)
> at
>
[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393874#comment-16393874
]
Pedro Boado edited comment on PHOENIX-4553 at 3/10/18 12:42 AM:
[~neospyk] I've come across this issue again and noticed that the classpath
configured is actually wrong. Only
{code:java}
/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-server.jar
{code}
should be in HBase classpath. After amending it logs are clear again.
Can you please confirm that by amending file
/opt/cloudera/parcels/APACHE_PHOENIX/meta/phoenix_env.sh and replacing
{code}
APPENDSTRING=`echo ${MYLIBDIR}/phoenix-*-server.jar | sed 's/ /:/g'
{code}
by
was (Author: pboado):
[~neospyk] I've come across this issue again and noticed that the classpath
configured is actually wrong. Only
{code:java}
/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-server.jar
{code}
should be in HBase classpath. After amending it logs are clear again.
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
>Reporter: Ihor Krysenko
>Assignee: Pedro Boado
>Priority: Minor
> Attachments: hbase-master.log, hbase-region.log, master-stderr.log,
> master-stdout.log, region-stderr.log, region-stdout.log
>
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=6"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
> java.util.ServiceLoader.access$300(ServiceLoader.java:181) at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:369) at
> java.util.ServiceLoader$1.next(ServiceLoader.java:445) at
> org.apache.hadoop.security.SecurityUtil.getKerberosInfo(SecurityUtil.java:333)
> at
> org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:101)
> at org.apache.hadoop.hbase.ipc.RpcServer.authorize(RpcServer.java:2347) at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.authorizeConnection(RpcServer.java:1898)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.processOneRpc(RpcServer.java:1772)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1335)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1614)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1596)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:854)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:635)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:611)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340752#comment-16340752
]
Pedro Boado edited comment on PHOENIX-4553 at 1/26/18 9:01 AM:
---
Are you running a quickstart VM? That is basically the test that I ran
yesterday - logs are almost 100% the same -.
Only difference is network addressing - my VM uses a bridged network - .
But HMaster is definitely running and waiting for a RS to connect to it
{code}
2018-01-26 07:57:04,571 INFO org.apache.hadoop.hbase.master.ServerManager:
Waiting for region servers count to settle; currently checked in 0, slept for
324842 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
interval of 1500 ms.
{code}
And RS is definitely running and waiting to connect to HMaster
{code}
2018-01-26 07:56:34,777 INFO
SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
Authorization successful for hbase/quickstart.cloudera@CLOUDERA
(auth:KERBEROS) for protocol=interface
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingInterface
2018-01-26 07:56:35,616 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: error telling master we are
up com.google.protobuf.ServiceException: java.io.IOException: Call to
quickstart.cloudera/172.23.0.2:6 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=22, waitTime=10001,
operationTimeout=1 expired. at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:240)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at
org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2324)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:922)
at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Call
to quickstart.cloudera/172.23.0.2:6 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=22, waitTime=10001,
operationTimeout=1 expired. at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:292)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1273) at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
... 5 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call
id=22, waitTime=10001, operationTimeout=1 expired. at
org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:73) at
org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1247) ... 6
more
{code}
I've run into similar issues before when running this VM with insufficient
resources - process delays were too high to keep a stable cluster running -
Have you noticed that the RS is starting way earlier than the master?
I don't think this is an issue - unless someone else reports it as well - .
I'll keep the ticket open for a while just in case.
was (Author: pboado):
Are you running a quickstart VM? That is basically the test that I ran
yesterday -logs are almost 100% the same-.
Only difference is network addressing - my VM uses a bridged network - .
But HMaster is definitely running and waiting for a RS to connect to it
{code}
2018-01-26 07:57:04,571 INFO org.apache.hadoop.hbase.master.ServerManager:
Waiting for region servers count to settle; currently checked in 0, slept for
324842 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms,
interval of 1500 ms.
{code}
And RS is definitely running and waiting to connect to HMaster
{code}
2018-01-26 07:56:34,777 INFO
SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
Authorization successful for hbase/quickstart.cloudera@CLOUDERA
(auth:KERBEROS) for protocol=interface
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingInterface
2018-01-26 07:56:35,616 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: error telling master we are
up com.google.protobuf.ServiceException: java.io.IOException: Call to
quickstart.cloudera/172.23.0.2:6 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=22, waitTime=10001,
operationTimeout=1 expired. at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:240)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at
[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340251#comment-16340251
]
Pedro Boado edited comment on PHOENIX-4553 at 1/25/18 11:36 PM:
- EDIT sorry, I misread the log -
[~jamestaylor] I rechecked (again) our parcels over a fresh CDH install and I
think they are fine. There must be some difference classpath wise with that
specific installation. I cannot reproduce the issue.
[~neospyk] can you please attach your full HBase Master classpath, please? It
should be displayed at the very beggining of the process startup . By the way I
don't see any warnings slf4j wise in my logs.
EDIT: I've just noticed you might be running a kerberised service. I've set up
kerberos in my testing environment and both HBase Master & RS are starting just
fine. Any additional feature to be enabled in CDH to consider?
This is my classpath
{code}
env:HBASE_CLASSPATH=/var/run/cloudera-scm-agent/process/147-hbase-MASTER:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/lib/hadoop/*:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/lib/hadoop/lib/*:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/bin/../lib/zookeeper/*:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/bin/../lib/zookeeper/lib/*:/usr/share/cmf/lib/plugins/tt-instrumentation-5.13.1.jar:/usr/share/cmf/lib/plugins/event-publish-5.13.1-shaded.jar:/usr/share/cmf/lib/plugins/navigator/cdh57/audit-plugin-cdh57-2.12.1-shaded.jar:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-queryserver.jar:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-server.jar:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar
{code}
was (Author: pboado):
- EDIT sorry, I misread the log -
[~jamestaylor] I rechecked (again) our parcels over a fresh CDH install and I
think they are fine. There must be some difference classpath wise with that
specific installation. I cannot reproduce the issue.
[~neospyk] can you please attach your full HBase Master classpath, please? It
should be displayed at the very beggining of the process startup . By the way I
don't see any warnings slf4j wise in my logs.
EDIT: I've just noticed you might be running a kerberised service. I've set up
kerberos in my testing environment and both HBase Master & RS are starting just
fine. Any additional feature to be enabled in CDH to consider?
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
>Reporter: Ihor Krysenko
>Priority: Minor
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=6"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
>
[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340251#comment-16340251
]
Pedro Boado edited comment on PHOENIX-4553 at 1/25/18 11:33 PM:
- EDIT sorry, I misread the log -
[~jamestaylor] I rechecked (again) our parcels over a fresh CDH install and I
think they are fine. There must be some difference classpath wise with that
specific installation. I cannot reproduce the issue.
[~neospyk] can you please attach your full HBase Master classpath, please? It
should be displayed at the very beggining of the process startup . By the way I
don't see any warnings slf4j wise in my logs.
EDIT: I've just noticed you might be running a kerberised service. I've set up
kerberos in my testing environment and both HBase Master & RS are starting just
fine. Any additional feature to be enabled in CDH to consider?
was (Author: pboado):
- EDIT sorry, I misread the log -
[~jamestaylor] I rechecked (again) our parcels over a fresh CDH install and I
think they are fine. There must be some difference classpath wise with that
specific installation. I cannot reproduce the issue.
[~neospyk] can you please attach your full HBase Master classpath, please? It
should be displayed at the very beggining of the process . By the way I don't
see any warnings slf4j wise.
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
>Reporter: Ihor Krysenko
>Priority: Minor
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=6"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
> java.util.ServiceLoader.access$300(ServiceLoader.java:181) at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:369) at
> java.util.ServiceLoader$1.next(ServiceLoader.java:445) at
> org.apache.hadoop.security.SecurityUtil.getKerberosInfo(SecurityUtil.java:333)
> at
> org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:101)
> at org.apache.hadoop.hbase.ipc.RpcServer.authorize(RpcServer.java:2347) at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.authorizeConnection(RpcServer.java:1898)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.processOneRpc(RpcServer.java:1772)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1335)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1614)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1596)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:854)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:635)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:611)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
[jira] [Comment Edited] (PHOENIX-4553) HBase Master could not start with activated APACHE_PHOENIX parcel
[
https://issues.apache.org/jira/browse/PHOENIX-4553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340251#comment-16340251
]
Pedro Boado edited comment on PHOENIX-4553 at 1/25/18 11:07 PM:
- EDIT sorry, I misread the log -
[~jamestaylor] I rechecked (again) our parcels over a fresh CDH install and I
think they are fine. There must be some difference classpath wise with that
specific installation. I cannot reproduce the issue.
[~neospyk] can you please attach your full HBase Master classpath, please? It
should be displayed at the very beggining of the process . By the way I don't
see any warnings slf4j wise.
was (Author: pboado):
[~neospyk] I just noticed the folder in your log
{{/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/ }} . That folder doesn't
match our parcel structure ( our parcel installs in
APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0 ) . Are you using our release parcels?
[~jamestaylor] I rechecked (again) our parcels over a fresh CDH install and I
think they are fine.
> HBase Master could not start with activated APACHE_PHOENIX parcel
> -
>
> Key: PHOENIX-4553
> URL: https://issues.apache.org/jira/browse/PHOENIX-4553
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
> Environment: CDH 5.11.2
> Apache phoenix 4.13.2-cdh5.11.2
>Reporter: Ihor Krysenko
>Priority: Minor
>
> After activation parcel HBase Master and Region could not start. Some
> problems with shaded thin-client, because if it remove from the parcel,
> everything work great.
> Please help.
> I think [GitHub
> commit|https://github.com/apache/phoenix/commit/e2c06b06fa1800b532e5d1ffa6f6ef8796cef213#diff-97e88e321a719f4389a2aa0e26fd0c8f]
> have influence on this bug.
> Below I put startup log for the HBaseMaster
> {code:java}
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/CDH-5.11.2-1.cdh5.11.2.p0.4/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-hive.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/cloudera/parcels/APACHE_PHOENIX-4.13.2-cdh5.11.2.p0.0/lib/phoenix/phoenix-4.13.2-cdh5.11.2-thin-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.slf4j.impl.Log4jLoggerFactory] Exception in thread
> "RpcServer.reader=1,bindAddress=0.0.0.0,port=6"
> java.util.ServiceConfigurationError: org.apache.hadoop.security.SecurityInfo:
> Provider
> org.apache.phoenix.shaded.org.apache.hadoop.security.AnnotatedSecurityInfo
> not a subtype at java.util.ServiceLoader.fail(ServiceLoader.java:231) at
> java.util.ServiceLoader.access$300(ServiceLoader.java:181) at
> java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:369) at
> java.util.ServiceLoader$1.next(ServiceLoader.java:445) at
> org.apache.hadoop.security.SecurityUtil.getKerberosInfo(SecurityUtil.java:333)
> at
> org.apache.hadoop.security.authorize.ServiceAuthorizationManager.authorize(ServiceAuthorizationManager.java:101)
> at org.apache.hadoop.hbase.ipc.RpcServer.authorize(RpcServer.java:2347) at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.authorizeConnection(RpcServer.java:1898)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.processOneRpc(RpcServer.java:1772)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.saslReadAndProcess(RpcServer.java:1335)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.process(RpcServer.java:1614)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1596)
> at org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:854)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:635)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:611)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
