ignore this question, problem solved. reason: hbase - nproc in
/etc/security/limits.d/hbase.conf is too small.
------------------ ???????? ------------------
??????: "????????????????-??????";<775620...@qq.com>;
????????: 2016??6??24??(??????) ????9:04
??????: "dev"<dev@kylin.apache.org>;
????: update hbase coprocessor
hi all: When I update hbase coprocessor with the CLI tool, it takes too
long to enable the hbase table and failed in the end. Is there any operation I
did wrong? The log is as fllows:
2016-06-23 16:43:08,326 INFO [main DeployCoprocessorCLI:112]: New coprocessor
jar:
hdfs://nameservice/kylin/kylin_metadata/coprocessor/kylin-coprocessor-1.5.2.1-1.jar
2016-06-23 16:43:08,326 INFO [main DeployCoprocessorCLI:193]: Disable
KYLIN_QOEGRNKI9S
2016-06-23 16:43:08,328 INFO [main HBaseAdmin:1007]: Started disable of
KYLIN_QOEGRNKI9S
2016-06-23 16:43:17,546 INFO [main HBaseAdmin:1062]: Disabled KYLIN_QOEGRNKI9S
2016-06-23 16:43:17,546 INFO [main DeployCoprocessorCLI:196]: Unset
coprocessor on KYLIN_QOEGRNKI9S
2016-06-23 16:43:17,548 INFO [main DeployCoprocessorCLI:186]: Add coprocessor
on KYLIN_QOEGRNKI9S
2016-06-23 16:43:21,989 INFO [main DeployCoprocessorCLI:224]: Enable
KYLIN_QOEGRNKI9S
2016-06-23 16:43:21,990 INFO [main HBaseAdmin:927]: Started enable of
KYLIN_QOEGRNKI9S
2016-06-23 23:37:25,879 ERROR [main DeployCoprocessorCLI:236]: Error processing
KYLIN_QOEGRNKI9S
java.io.IOException: Table 'KYLIN_QOEGRNKI9S' not yet enabled, after 24843849ms.
at
org.apache.hadoop.hbase.client.HBaseAdmin.waitUntilTableIsEnabled(HBaseAdmin.java:905)
at
org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:854)
at
org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:866)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:225)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessorOnHTables(DeployCoprocessorCLI.java:233)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:114)
2016-06-23 23:37:25,883 INFO [main DeployCoprocessorCLI:193]: Disable
KYLIN_5LT5IZPVZP
2016-06-23 23:37:25,886 INFO [main HBaseAdmin:1007]: Started disable of
KYLIN_5LT5IZPVZP
2016-06-23 23:37:26,031 ERROR [main DeployCoprocessorCLI:236]: Error processing
KYLIN_5LT5IZPVZP
org.apache.hadoop.hbase.TableNotEnabledException:
org.apache.hadoop.hbase.TableNotEnabledException: KYLIN_5LT5IZPVZP
at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.prepare(DisableTableHandler.java:99)
at
org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1656)
at
org.apache.hadoop.hbase.master.MasterRpcServices.disableTable(MasterRpcServices.java:485)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:44235)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:744)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:226)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:240)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:140)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3678)
at
org.apache.hadoop.hbase.client.HBaseAdmin.disableTableAsync(HBaseAdmin.java:1004)
at
org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:1037)
at
org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:1072)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:194)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessorOnHTables(DeployCoprocessorCLI.java:233)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:114)
Caused by:
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.TableNotEnabledException):
org.apache.hadoop.hbase.TableNotEnabledException: KYLIN_5LT5IZPVZP
at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.prepare(DisableTableHandler.java:99)
at
org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1656)
at
org.apache.hadoop.hbase.master.MasterRpcServices.disableTable(MasterRpcServices.java:485)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:44235)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:744)
at
org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1200)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.disableTable(MasterProtos.java:46614)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.disableTable(ConnectionManager.java:1887)
at
org.apache.hadoop.hbase.client.HBaseAdmin$9.call(HBaseAdmin.java:1009)
at
org.apache.hadoop.hbase.client.HBaseAdmin$9.call(HBaseAdmin.java:1004)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
... 7 more
2016-06-23 23:37:26,032 INFO [main DeployCoprocessorCLI:193]: Disable
KYLIN_NO8517FW2C
2016-06-23 23:37:26,033 INFO [main HBaseAdmin:1007]: Started disable of
KYLIN_NO8517FW2C
2016-06-23 23:37:29,108 INFO [main HBaseAdmin:1062]: Disabled KYLIN_NO8517FW2C
2016-06-23 23:37:29,108 INFO [main DeployCoprocessorCLI:196]: Unset
coprocessor on KYLIN_NO8517FW2C
2016-06-23 23:37:29,111 INFO [main DeployCoprocessorCLI:186]: Add coprocessor
on KYLIN_NO8517FW2C
2016-06-23 23:37:29,566 INFO [main DeployCoprocessorCLI:224]: Enable
KYLIN_NO8517FW2C
2016-06-23 23:37:29,567 INFO [main HBaseAdmin:927]: Started enable of
KYLIN_NO8517FW2C
2016-06-24 06:31:33,265 ERROR [main DeployCoprocessorCLI:236]: Error processing
KYLIN_NO8517FW2C
java.io.IOException: Table 'KYLIN_NO8517FW2C' not yet enabled, after 24843626ms.
at
org.apache.hadoop.hbase.client.HBaseAdmin.waitUntilTableIsEnabled(HBaseAdmin.java:905)
at
org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:854)
at
org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:866)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:225)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessorOnHTables(DeployCoprocessorCLI.java:233)
at
org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:114)
2016-06-24 06:31:33,265 INFO [main DeployCoprocessorCLI:193]: Disable
KYLIN_R6T2LNGTAQ
2016-06-24 06:31:33,267 INFO [main HBaseAdmin:1007]: Started disable of
KYLIN_R6T2LNGTAQ
2016-06-24 06:31:36,405 INFO [main HBaseAdmin:1062]: Disabled KYLIN_R6T2LNGTAQ
2016-06-24 06:31:36,405 INFO [main DeployCoprocessorCLI:196]: Unset
coprocessor on KYLIN_R6T2LNGTAQ
2016-06-24 06:31:36,407 INFO [main DeployCoprocessorCLI:186]: Add coprocessor
on KYLIN_R6T2LNGTAQ
2016-06-24 06:31:36,812 INFO [main DeployCoprocessorCLI:224]: Enable
KYLIN_R6T2LNGTAQ
2016-06-24 06:31:36,813 INFO [main HBaseAdmin:927]: Started enable of
KYLIN_R6T2LNGTAQ
2016-06-24 06:31:39,949 INFO [main HBaseAdmin:856]: Enabled table
KYLIN_R6T2LNGTAQ
2016-06-24 06:31:39,949 INFO [main DeployCoprocessorCLI:193]: Disable
KYLIN_C200Z0U1H8
2016-06-24 06:31:39,950 INFO [main HBaseAdmin:1007]: Started disable of
KYLIN_C200Z0U1H8
2016-06-24 06:31:43,056 INFO [main HBaseAdmin:1062]: Disabled KYLIN_C200Z0U1H8
2016-06-24 06:31:43,056 INFO [main DeployCoprocessorCLI:196]: Unset
coprocessor on KYLIN_C200Z0U1H8
2016-06-24 06:31:43,058 INFO [main DeployCoprocessorCLI:186]: Add coprocessor
on KYLIN_C200Z0U1H8
2016-06-24 06:31:43,432 INFO [main DeployCoprocessorCLI:224]: Enable
KYLIN_C200Z0U1H8
2016-06-24 06:31:43,433 INFO [main HBaseAdmin:927]: Started enable of
KYLIN_C200Z0U1H8
