Thanks for sharing the root cause. :-) On Fri, Jun 24, 2016 at 11:35 AM, 移动苏州研发中心-陈雷雷 <775620...@qq.com> wrote:
> ignore this question, problem solved. reason: hbase - nproc in > /etc/security/limits.d/hbase.conf is too small. > > > > > ------------------ 原始邮件 ------------------ > 发件人: "移动苏州研发中心-陈雷雷";<775620...@qq.com>; > 发送时间: 2016年6月24日(星期五) 上午9:04 > 收件人: "dev"<dev@kylin.apache.org>; > > 主题: update hbase coprocessor > > > > hi all: When I update hbase coprocessor with the CLI tool, it takes > too long to enable the hbase table and failed in the end. Is there any > operation I did wrong? The log is as fllows: > > > 2016-06-23 16:43:08,326 INFO [main DeployCoprocessorCLI:112]: New > coprocessor jar: > hdfs://nameservice/kylin/kylin_metadata/coprocessor/kylin-coprocessor-1.5.2.1-1.jar > 2016-06-23 16:43:08,326 INFO [main DeployCoprocessorCLI:193]: Disable > KYLIN_QOEGRNKI9S > 2016-06-23 16:43:08,328 INFO [main HBaseAdmin:1007]: Started disable of > KYLIN_QOEGRNKI9S > 2016-06-23 16:43:17,546 INFO [main HBaseAdmin:1062]: Disabled > KYLIN_QOEGRNKI9S > 2016-06-23 16:43:17,546 INFO [main DeployCoprocessorCLI:196]: Unset > coprocessor on KYLIN_QOEGRNKI9S > 2016-06-23 16:43:17,548 INFO [main DeployCoprocessorCLI:186]: Add > coprocessor on KYLIN_QOEGRNKI9S > 2016-06-23 16:43:21,989 INFO [main DeployCoprocessorCLI:224]: Enable > KYLIN_QOEGRNKI9S > 2016-06-23 16:43:21,990 INFO [main HBaseAdmin:927]: Started enable of > KYLIN_QOEGRNKI9S > 2016-06-23 23:37:25,879 ERROR [main DeployCoprocessorCLI:236]: Error > processing KYLIN_QOEGRNKI9S > java.io.IOException: Table 'KYLIN_QOEGRNKI9S' not yet enabled, after > 24843849ms. > at > org.apache.hadoop.hbase.client.HBaseAdmin.waitUntilTableIsEnabled(HBaseAdmin.java:905) > at > org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:854) > at > org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:866) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:225) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessorOnHTables(DeployCoprocessorCLI.java:233) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:114) > 2016-06-23 23:37:25,883 INFO [main DeployCoprocessorCLI:193]: Disable > KYLIN_5LT5IZPVZP > 2016-06-23 23:37:25,886 INFO [main HBaseAdmin:1007]: Started disable of > KYLIN_5LT5IZPVZP > 2016-06-23 23:37:26,031 ERROR [main DeployCoprocessorCLI:236]: Error > processing KYLIN_5LT5IZPVZP > org.apache.hadoop.hbase.TableNotEnabledException: > org.apache.hadoop.hbase.TableNotEnabledException: KYLIN_5LT5IZPVZP > at > org.apache.hadoop.hbase.master.handler.DisableTableHandler.prepare(DisableTableHandler.java:99) > at > org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1656) > at > org.apache.hadoop.hbase.master.MasterRpcServices.disableTable(MasterRpcServices.java:485) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:44235) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:744) > > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:226) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:240) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:140) > at > org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3678) > at > org.apache.hadoop.hbase.client.HBaseAdmin.disableTableAsync(HBaseAdmin.java:1004) > at > org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:1037) > at > org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:1072) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:194) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessorOnHTables(DeployCoprocessorCLI.java:233) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:114) > Caused by: > org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.TableNotEnabledException): > org.apache.hadoop.hbase.TableNotEnabledException: KYLIN_5LT5IZPVZP > at > org.apache.hadoop.hbase.master.handler.DisableTableHandler.prepare(DisableTableHandler.java:99) > at > org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1656) > at > org.apache.hadoop.hbase.master.MasterRpcServices.disableTable(MasterRpcServices.java:485) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:44235) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) > at java.lang.Thread.run(Thread.java:744) > > > at > org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1200) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:216) > at > org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:300) > at > org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.disableTable(MasterProtos.java:46614) > at > org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$5.disableTable(ConnectionManager.java:1887) > at > org.apache.hadoop.hbase.client.HBaseAdmin$9.call(HBaseAdmin.java:1009) > at > org.apache.hadoop.hbase.client.HBaseAdmin$9.call(HBaseAdmin.java:1004) > at > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126) > ... 7 more > 2016-06-23 23:37:26,032 INFO [main DeployCoprocessorCLI:193]: Disable > KYLIN_NO8517FW2C > 2016-06-23 23:37:26,033 INFO [main HBaseAdmin:1007]: Started disable of > KYLIN_NO8517FW2C > 2016-06-23 23:37:29,108 INFO [main HBaseAdmin:1062]: Disabled > KYLIN_NO8517FW2C > 2016-06-23 23:37:29,108 INFO [main DeployCoprocessorCLI:196]: Unset > coprocessor on KYLIN_NO8517FW2C > 2016-06-23 23:37:29,111 INFO [main DeployCoprocessorCLI:186]: Add > coprocessor on KYLIN_NO8517FW2C > 2016-06-23 23:37:29,566 INFO [main DeployCoprocessorCLI:224]: Enable > KYLIN_NO8517FW2C > 2016-06-23 23:37:29,567 INFO [main HBaseAdmin:927]: Started enable of > KYLIN_NO8517FW2C > 2016-06-24 06:31:33,265 ERROR [main DeployCoprocessorCLI:236]: Error > processing KYLIN_NO8517FW2C > java.io.IOException: Table 'KYLIN_NO8517FW2C' not yet enabled, after > 24843626ms. > at > org.apache.hadoop.hbase.client.HBaseAdmin.waitUntilTableIsEnabled(HBaseAdmin.java:905) > at > org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:854) > at > org.apache.hadoop.hbase.client.HBaseAdmin.enableTable(HBaseAdmin.java:866) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessor(DeployCoprocessorCLI.java:225) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.resetCoprocessorOnHTables(DeployCoprocessorCLI.java:233) > at > org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI.main(DeployCoprocessorCLI.java:114) > 2016-06-24 06:31:33,265 INFO [main DeployCoprocessorCLI:193]: Disable > KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:33,267 INFO [main HBaseAdmin:1007]: Started disable of > KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:36,405 INFO [main HBaseAdmin:1062]: Disabled > KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:36,405 INFO [main DeployCoprocessorCLI:196]: Unset > coprocessor on KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:36,407 INFO [main DeployCoprocessorCLI:186]: Add > coprocessor on KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:36,812 INFO [main DeployCoprocessorCLI:224]: Enable > KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:36,813 INFO [main HBaseAdmin:927]: Started enable of > KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:39,949 INFO [main HBaseAdmin:856]: Enabled table > KYLIN_R6T2LNGTAQ > 2016-06-24 06:31:39,949 INFO [main DeployCoprocessorCLI:193]: Disable > KYLIN_C200Z0U1H8 > 2016-06-24 06:31:39,950 INFO [main HBaseAdmin:1007]: Started disable of > KYLIN_C200Z0U1H8 > 2016-06-24 06:31:43,056 INFO [main HBaseAdmin:1062]: Disabled > KYLIN_C200Z0U1H8 > 2016-06-24 06:31:43,056 INFO [main DeployCoprocessorCLI:196]: Unset > coprocessor on KYLIN_C200Z0U1H8 > 2016-06-24 06:31:43,058 INFO [main DeployCoprocessorCLI:186]: Add > coprocessor on KYLIN_C200Z0U1H8 > 2016-06-24 06:31:43,432 INFO [main DeployCoprocessorCLI:224]: Enable > KYLIN_C200Z0U1H8 > 2016-06-24 06:31:43,433 INFO [main HBaseAdmin:927]: Started enable of > KYLIN_C200Z0U1H8 >
