[
https://issues.apache.org/jira/browse/HBASE-27057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540605#comment-17540605
]
George commented on HBASE-27057:
--------------------------------
2022-05-22 14:03:05.615Z [INFO ] [poolScheduler-3]
[c.b.s.s.c.h.GenerateRequestService] :--- Received scan jobs(1) of types
[ds_test_connection]
2022-05-22 14:03:05.616Z [INFO ] [Thread-425] [c.b.c.a.u.AESUtil] :---
MasterNode Unlimited cryptography enabled: AES: true , SHA: true , check hash:
4jCLdtQCrFRGsnHP+F491iow7W2PBBVzcvIGw0G3pBM=
2022-05-22 14:03:05.616Z [INFO ] [Thread-425]
[c.b.s.s.c.s.DataSourceConnectionService] :--- MasterNode connector type
hbase
2022-05-22 14:03:05.616Z [INFO ] [Thread-425]
[c.b.s.s.c.s.DataSourceConnectionService] :--- MasterNode source IS:
MasterNode
2022-05-22 14:03:05.616Z [INFO ] [Thread-425] [c.b.s.c.TaskExecution] :---
MasterNode Initializing task
2022-05-22 14:03:05.616Z [INFO ] [Thread-425] [c.b.s.c.TestConnectionTask] :---
MasterNode Initializing Test Connection Task
2022-05-22 14:03:05.617Z [INFO ] [Thread-425] [c.b.s.c.ScannerFactory] :---
MasterNode in ScannerFactory
2022-05-22 14:03:05.617Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode Certificate store management is configured for krb5ConfType file,
searching krb5ConfType file in certificate store
2022-05-22 14:03:05.618Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode Saving file [/etc/scanner/krb5.conf]
2022-05-22 14:03:05.618Z [INFO ] [Thread-425] [c.b.s.h.c.CommonUtils] :---
MasterNode Setting the java.security.krb5.conf system property to
'/etc/scanner/krb5.conf'
2022-05-22 14:03:05.647Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode Certificate store management is configured for keyTabType file,
searching keyTabType file in certificate store
2022-05-22 14:03:05.647Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode Saving file [/etc/scanner/big.keytab]
2022-05-22 14:03:05.647Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode Advanced configurations
2022-05-22 14:03:05.647Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode No advanced configuration
2022-05-22 14:03:05.650Z [INFO ] [Thread-425] [c.b.s.c.AESUtil] :---
MasterNode Unlimited cryptography enabled: AES: true , SHA: true , check hash:
4jCLdtQCrFRGsnHP+F491iow7W2PBBVzcvIGw0G3pBM=
2022-05-22 14:03:05.650Z [INFO ] [Thread-425]
[c.b.s.c.g.AbstractLocalDataSourceScanner] :--- MasterNode Sample mode
activated
2022-05-22 14:03:05.650Z [INFO ] [Thread-425]
[c.b.s.c.g.AbstractLocalDataSourceScanner] :--- MasterNode Max sample size:
100000
2022-05-22 14:03:05.650Z [INFO ] [Thread-425]
[c.b.s.h.c.HadoopUGIConnectorWrapper] :--- MasterNode Getting new User
Group Information...
2022-05-22 14:03:05.650Z [INFO ] [Thread-425] [c.b.s.h.c.CommonUtils] :---
MasterNode Kerberos authentication
2022-05-22 14:03:05.709Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode Advanced configurations
2022-05-22 14:03:05.709Z [INFO ] [Thread-425] [c.b.s.h.c.ConfigUtils] :---
MasterNode No advanced configuration
2022-05-22 14:03:05.709Z [WARN ] [Thread-425] [c.b.s.o.c.OrchestratorClient]
:--- MasterNode no classifier algorithm url found
2022-05-22 14:03:05.709Z [INFO ] [Thread-425]
[c.b.s.c.g.AbstractLocalDataSourceScanner] :--- MasterNode The enabled
classifier types are: [REGEX]
2022-05-22 14:03:05.709Z [WARN ] [Thread-425] [c.b.s.o.c.OrchestratorClient]
:--- MasterNode The dataSourceSupportedFeaturesUrl field was not supplied.
Will not update the hbase scanner type with [HasStructure, HasFieldAffinity,
HasDsar] supported features
2022-05-22 14:03:05.709Z [INFO ] [Thread-425] [c.b.s.c.TaskExecution] :---
MasterNode Task has been initialized successfully.
2022-05-22 14:03:05.709Z [INFO ] [Thread-425] [c.b.s.c.TaskExecution] :---
MasterNode Executing task
2022-05-22 14:03:05.709Z [INFO ] [Thread-425] [c.b.s.h.HBaseScanner] :---
MasterNode Getting HBase tables...
2022-05-22 14:03:05.710Z [INFO ] [Thread-425] [c.b.s.h.HBaseScanner] :---
MasterNode java11 support - Current java version is 11.0.14.1 change to
1.8.0_192
2022-05-22 14:03:05.711Z [INFO ] [Thread-425] [o.a.h.h.z.RecoverableZooKeeper]
:--- MasterNode Process identifier=hconnection-0x39acfe2b connecting to
ZooKeeper ensemble=10.2.0.244:21812022-05-22 14:03:05 INFO ZooKeeper:438 -
Initiating client connection, connectString=10.2.0.244:2181
sessionTimeout=90000 watcher=hconnection-0x39acfe2b0x0, quorum=10.2.0.244:2181,
baseZNode=/hbase
2022-05-22 14:03:05 INFO ClientCnxn:975 - Opening socket connection to server
ip-10-2-0-244.ec2.internal/10.2.0.244:2181. Will not attempt to authenticate
using SASL (unknown error)
2022-05-22 14:03:05 INFO ClientCnxn:852 - Socket connection established,
initiating session, client: /172.18.0.4:47928, server:
ip-10-2-0-244.ec2.internal/10.2.0.244:2181
2022-05-22 14:03:05 INFO ClientCnxn:1235 - Session establishment complete on
server ip-10-2-0-244.ec2.internal/10.2.0.244:2181, sessionid =
0xff80d872ec8b1bd0, negotiated timeout = 60000
2022-05-22 14:03:05.743Z [INFO ] [Thread-425]
[o.a.h.h.c.ConnectionManager$HConnectionImplementation] :--- MasterNode
Closing master protocol: MasterService
2022-05-22 14:03:05.743Z [INFO ] [Thread-425]
[o.a.h.h.c.ConnectionManager$HConnectionImplementation] :--- MasterNode
Closing zookeeper sessionid=0xff80d872ec8b1bd02022-05-22 14:03:05 INFO
ZooKeeper:684 - Session: 0xff80d872ec8b1bd0 closed
2022-05-22 14:03:05 INFO ClientCnxn:512 - EventThread shut down
2022-05-22 14:03:05.847Z [INFO ] [Thread-425]
[c.b.s.c.g.DataSourceConnectorAdapter] :--- MasterNode Objects available
for scanning (test connection): 5
2022-05-22 14:03:05.847Z [INFO ] [Thread-425] [c.b.s.c.TaskExecution] :---
MasterNode Closing task
2022-05-22 14:03:05.847Z [INFO ] [Thread-425]
[c.b.s.c.g.AbstractLocalDataSourceScanner] :--- MasterNode started
2022-05-22 14:03:05.848Z [INFO ] [Thread-425]
[c.b.s.c.g.AbstractLocalDataSourceScanner] :--- MasterNode closing the
orchestratorClient
2022-05-22 14:03:05.848Z [INFO ] [Thread-425] [c.b.s.o.c.OrchestratorClient]
:--- MasterNode started
2022-05-22 14:03:05.848Z [INFO ] [Thread-425] [c.b.s.o.c.OrchestratorClient]
:--- MasterNode closing the httpRestClient
2022-05-22 14:03:05.848Z [INFO ] [Thread-425] [c.b.s.c.HttpRESTClient] :---
MasterNode started
2022-05-22 14:03:05.848Z [INFO ] [Thread-425] [c.b.s.c.HttpRESTClient] :---
MasterNode closing the CloseableHttpClient
2022-05-22 14:03:05.848Z [INFO ] [Thread-425] [c.b.s.c.TaskExecution] :---
MasterNode Complete.
2022-05-22 14:03:38.217Z [INFO ] [poolScheduler-8]
[c.b.s.s.c.s.c.t.ThreadPoolUpdatesService] :--- Fetching thread pool
updates
> Issue with ZooKeeper configuration on CDP5
> ------------------------------------------
>
> Key: HBASE-27057
> URL: https://issues.apache.org/jira/browse/HBASE-27057
> Project: HBase
> Issue Type: Task
> Environment: Cloudera 5 with Kerberos / HBase
> Reporter: George
> Priority: Major
>
> Hello everyone, I am new here and in HBase as well, so sorry for my lack of
> knowledge, I tried to read but the documentation and the environment that I
> got as a test are confusing me. The explanation is here, I have got Cloudera
> 5 with Hbase with Kerberos, 2 master nodes, and 3 worker nodes on Ubuntu
> machines, we have to connect DataStore with special application with checks
> krb5.conf file and keytab after authentication goes and connect to DS and
> check the tables the metadata there. When I am trying to push the application
> to connect the master node zookeeper is failing, when I am trying to connect
> zookeper node from the same application I can scan and do what I need, but
> every time the connection goes to 2181 localhost, how to check where is the
> issue of configuration that masters done will give me access to the data
> stores?
>
> Here is log from master node:
> [2022-05-22 12:26:31.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> [2022-05-22 12:26:31.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.util.RetryCounter [MasterNode]- Sleeping 1000ms
> before retry #0...
> [2022-05-22 12:26:31.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> [2022-05-22 12:26:31.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.util.RetryCounter [MasterNode]- Sleeping 2000ms
> before retry #1...
> [2022-05-22 12:26:31.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> [2022-05-22 12:26:31.676Z] [ERROR] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> ZooKeeper exists failed after 2 attempts
> [2022-05-22 12:26:31.677Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.ZKUtil [MasterNode]-
> hconnection-0x46acd8780x0, quorum=10.2.0.30:2181, baseZNode=/hbase Unable to
> set watcher on znode (/hbase/hbaseid)
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> at
> com.bigid.scanner.hbase.HBaseScanner.getDataSourceObjects(HBaseScanner.java:60)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:108)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:105)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper.getDataSourceObjects(HadoopUGIConnectorWrapper.java:105)
> at
> com.bigid.scanner.connectors.generic.DataSourceConnectorAdapter.testConnection(DataSourceConnectorAdapter.java:289)
> at
> com.bigid.scanner.core.tasks.legacy.LegacyTestConnectionTask.execute(LegacyTestConnectionTask.java:43)
> at
> com.bigid.scanner.core.TestConnectionTask.execute(TestConnectionTask.java:36)
> at com.bigid.scanner.core.TaskExecution.execute(TaskExecution.java:36)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:57)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:20)
> at
> com.bigid.scanner.scannerboot.common.http.managers.TestConnectionSyncManager.lambda$runService$0(TestConnectionSyncManager.java:55)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> [2022-05-22 12:26:31.678Z] [ERROR] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher [MasterNode]-
> hconnection-0x46acd8780x0, quorum=10.2.0.30:2181, baseZNode=/hbase Received
> unexpected KeeperException, re-throwing exception
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> at
> com.bigid.scanner.hbase.HBaseScanner.getDataSourceObjects(HBaseScanner.java:60)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:108)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:105)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper.getDataSourceObjects(HadoopUGIConnectorWrapper.java:105)
> at
> com.bigid.scanner.connectors.generic.DataSourceConnectorAdapter.testConnection(DataSourceConnectorAdapter.java:289)
> at
> com.bigid.scanner.core.tasks.legacy.LegacyTestConnectionTask.execute(LegacyTestConnectionTask.java:43)
> at
> com.bigid.scanner.core.TestConnectionTask.execute(TestConnectionTask.java:36)
> at com.bigid.scanner.core.TaskExecution.execute(TaskExecution.java:36)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:57)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:20)
> at
> com.bigid.scanner.scannerboot.common.http.managers.TestConnectionSyncManager.lambda$runService$0(TestConnectionSyncManager.java:55)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> [2022-05-22 12:26:31.678Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.client.ZooKeeperRegistry [MasterNode]- Can't
> retrieve clusterId from Zookeeper
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source)
> at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> at
> com.bigid.scanner.hbase.HBaseScanner.getDataSourceObjects(HBaseScanner.java:60)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:108)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:105)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper.getDataSourceObjects(HadoopUGIConnectorWrapper.java:105)
> at
> com.bigid.scanner.connectors.generic.DataSourceConnectorAdapter.testConnection(DataSourceConnectorAdapter.java:289)
> at
> com.bigid.scanner.core.tasks.legacy.LegacyTestConnectionTask.execute(LegacyTestConnectionTask.java:43)
> at
> com.bigid.scanner.core.TestConnectionTask.execute(TestConnectionTask.java:36)
> at com.bigid.scanner.core.TaskExecution.execute(TaskExecution.java:36)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:57)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:20)
> at
> com.bigid.scanner.scannerboot.common.http.managers.TestConnectionSyncManager.lambda$runService$0(TestConnectionSyncManager.java:55)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> [2022-05-22 12:26:36.337Z] [INFO] scanner_file -
> 7a342cd4-cd4f-4a08-ab9f-8214b4f62f09 [poolScheduler-2]
> com.bigid.scanner.scannerboot.common.services.context.threads.ThreadPoolUpdatesService
> - Fetching thread pool updates
> [2022-05-22 12:26:41.675Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.util.RetryCounter [MasterNode]- Sleeping 1000ms
> before retry #0...
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.util.RetryCounter [MasterNode]- Sleeping 2000ms
> before retry #1...
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [ERROR] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> ZooKeeper exists failed after 2 attempts
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.ZKUtil [MasterNode]-
> hconnection-0x46acd8780x0, quorum=10.2.0.30:2181, baseZNode=/hbase Unable to
> set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:849)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:535)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1435)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1476)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1505)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1711)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3601)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:312)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:296)
> at
> com.bigid.scanner.hbase.HBaseScanner.getDataSourceObjects(HBaseScanner.java:62)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:108)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:105)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper.getDataSourceObjects(HadoopUGIConnectorWrapper.java:105)
> at
> com.bigid.scanner.connectors.generic.DataSourceConnectorAdapter.testConnection(DataSourceConnectorAdapter.java:289)
> at
> com.bigid.scanner.core.tasks.legacy.LegacyTestConnectionTask.execute(LegacyTestConnectionTask.java:43)
> at
> com.bigid.scanner.core.TestConnectionTask.execute(TestConnectionTask.java:36)
> at com.bigid.scanner.core.TaskExecution.execute(TaskExecution.java:36)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:57)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:20)
> at
> com.bigid.scanner.scannerboot.common.http.managers.TestConnectionSyncManager.lambda$runService$0(TestConnectionSyncManager.java:55)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> [2022-05-22 12:26:41.676Z] [ERROR] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher [MasterNode]-
> hconnection-0x46acd8780x0, quorum=10.2.0.30:2181, baseZNode=/hbase Received
> unexpected KeeperException, re-throwing exception
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:849)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:535)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1435)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1476)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1505)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1711)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3601)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:312)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:296)
> at
> com.bigid.scanner.hbase.HBaseScanner.getDataSourceObjects(HBaseScanner.java:62)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:108)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:105)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper.getDataSourceObjects(HadoopUGIConnectorWrapper.java:105)
> at
> com.bigid.scanner.connectors.generic.DataSourceConnectorAdapter.testConnection(DataSourceConnectorAdapter.java:289)
> at
> com.bigid.scanner.core.tasks.legacy.LegacyTestConnectionTask.execute(LegacyTestConnectionTask.java:43)
> at
> com.bigid.scanner.core.TestConnectionTask.execute(TestConnectionTask.java:36)
> at com.bigid.scanner.core.TaskExecution.execute(TaskExecution.java:36)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:57)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:20)
> at
> com.bigid.scanner.scannerboot.common.http.managers.TestConnectionSyncManager.lambda$runService$0(TestConnectionSyncManager.java:55)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
> [2022-05-22 12:26:41.676Z] [ERROR] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation
> [MasterNode]- Can't get connection to ZooKeeper: KeeperErrorCode =
> ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.util.RetryCounter [MasterNode]- Sleeping 1000ms
> before retry #0...
> [2022-05-22 12:26:41.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [poolScheduler-7]
> com.bigid.scanner.scannerboot.common.services.context.threads.ThreadPoolUpdatesService
> - Fetching thread pool updates
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [INFO] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.util.RetryCounter [MasterNode]- Sleeping 2000ms
> before retry #1...
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> Possibly transient ZooKeeper, quorum=10.2.0.30:2181,
> exception=org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> [2022-05-22 12:26:41.676Z] [ERROR] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper [MasterNode]-
> ZooKeeper exists failed after 2 attempts
> [2022-05-22 12:26:41.676Z] [WARN] scanner_file -
> 12750b5c-c00b-4eb9-8ec0-66a710ed822d [Thread-312]
> org.apache.hadoop.hbase.zookeeper.ZKUtil [MasterNode]-
> hconnection-0x46acd8780x0, quorum=10.2.0.30:2181, baseZNode=/hbase Unable to
> set watcher on znode (/hbase)
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(ConnectionManager.java:849)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.access$400(ConnectionManager.java:535)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1435)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1476)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1505)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1711)
> at
> org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3601)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:312)
> at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:296)
> at
> com.bigid.scanner.hbase.HBaseScanner.getDataSourceObjects(HBaseScanner.java:62)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:108)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper$4.run(HadoopUGIConnectorWrapper.java:105)
> at java.base/java.security.AccessController.doPrivileged(Native Method)
> at java.base/javax.security.auth.Subject.doAs(Unknown Source)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at
> com.bigid.scanner.hadoop.common.HadoopUGIConnectorWrapper.getDataSourceObjects(HadoopUGIConnectorWrapper.java:105)
> at
> com.bigid.scanner.connectors.generic.DataSourceConnectorAdapter.testConnection(DataSourceConnectorAdapter.java:289)
> at
> com.bigid.scanner.core.tasks.legacy.LegacyTestConnectionTask.execute(LegacyTestConnectionTask.java:43)
> at
> com.bigid.scanner.core.TestConnectionTask.execute(TestConnectionTask.java:36)
> at com.bigid.scanner.core.TaskExecution.execute(TaskExecution.java:36)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:57)
> at
> com.bigid.scanner.scannerboot.common.services.TestConnectionService.doService(TestConnectionService.java:20)
> at
> com.bigid.scanner.scannerboot.common.http.managers.TestConnectionSyncManager.lambda$runService$0(TestConnectionSyncManager.java:55)
> at
> java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown
> Source)
> at java.base/java.lang.Thread.run(Unknown Source)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
