hi all! the error log just tell you that your hadoop jobhistory service is not turn on, nothing else!
I think you should follow luke suggestion! best regards! wangxianbin1...@gmail.com 发件人： 胡志华(万里通科技及数据中心商务智能团队数据分析组) 发送时间： 2015-12-30 09:23 收件人： 'dev@kylin.apache.org' 主题： 答复: long time create HTable without error Hi Luke, Thanks for your suggestion, but sandbox is suitable for personal experiment. We would like to use kylin on our own clusters(hadoop/hbase/hive), that's our actual usage scenario. I have refered to kylin.apache.org from beginning to the end, I can't find more suggest from the installation guide. My hbase cluster can create table, import data. But the kylin.log showed something. Sample cube building stopped at step 13 Create HTable , I will pick up some error log from kylin.log, [pool-7-thread-2]:[2015-12-29 21:08:47,804][ERROR][org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:100)] - java.net.ConnectException: Call From dw-server20-30.xs.paic.com.cn/172.16.20.30 to bihbasemaster:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused java.io.IOException: java.net.ConnectException: to bihbasemaster:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:343) at org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:380) at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:557) at org.apache.hadoop.mapreduce.Job$7.run(Job.java:761) at org.apache.hadoop.mapreduce.Job$7.run(Job.java:758) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:758) at org.apache.kylin.job.common.HadoopCmdOutput.updateJobCounter(HadoopCmdOutput.java:86) at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:156) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.net.ConnectException: Call From dw-server20-30.xs.paic.com.cn/172.16.20.30 to bihbasemaster:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732) at org.apache.hadoop.ipc.Client.call(Client.java:1480) at org.apache.hadoop.ipc.Client.call(Client.java:1407) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy52.getCounters(Unknown Source) at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getCounters(MRClientProtocolPBClientImpl.java:166) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) -----邮件原件----- 发件人: Luke Han [mailto:luke...@gmail.com] 发送时间: 2015年12月30日 8:30 收件人: dev@kylin.apache.org 主题: Re: long time create HTable without error Hi Zhihua, For more detail, please check installation guide first: http://kylin.apache.org/docs/install/index.html <http://kylin.apache.org/docs/install/index.html> No matter which distribution you pickup, but please just try to play with suggested one for quick start and then try to add your dataset. Thanks. Best Regards! --------------------- Luke Han On Tue, Dec 29, 2015 at 11:12 PM, Luke Han <luke...@gmail.com> wrote: > I would like to suggest start experimentation with Hortonworks sandbox > (HDP) and Kylin binary package first, when you have more insight about > the entire process and how Kylin interactive with Hadoop/Hive/HBase, > then move to your own cluster with apache only distribution. > > And please double check in your kylin.log file, there's should have > more detail idea about what's wrong. > > Thanks. > > > Best Regards! > --------------------- > > Luke Han > > On Tue, Dec 29, 2015 at 10:08 PM, 胡志华(万里通科技及数据中心商务智能团队数据分析组) < > huzhihua...@pingan.com.cn> wrote: > >> Hi all, >> >> Hadoop2.7.1 >> Hbase0.98.16 >> Hive1.2.1 >> >> My sample cube Is running with error and error, this time >> stoped at step 13 “Create HTable” for a long time without any error >> notification >> >> >> I found the log of hamster, as below >> >> >> 2015-12-29 21:18:02,345 INFO [FifoRpcScheduler.handler1-thread-18] >> master.HMaster: Client=root/null create 'KYLIN_VSPE13YZKF', >> {TABLE_ATTRIBUTES => {coprocessor$1 => >> 'hdfs://bihbasemaster/kylin/kylin_metadata/coprocessor/kylin-coproces >> sor-1.2-1.jar|org.apache.kylin.storage.hbase.coprocessor.endpoint.IIE >> ndpoint|1000|', >> coprocessor$2 => >> 'hdfs://bihbasemaster/kylin/kylin_metadata/coprocessor/kylin-coproces >> sor-1.2-1.jar|org.apache.kylin.storage.hbase.coprocessor.observer.Agg >> regateRegionObserver|1001|', METADATA => {'KYLIN_HOST' => >> 'kylin_metadata', 'SPLIT_POLICY' => >> 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'} >> }, {NAME => 'F1', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => >> 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => >> 'LZO', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => >> 'FALSE', BLOCKSIZE => '4194304', IN_MEMORY => 'false', BLOCKCACHE => >> 'true'}, {NAME => 'F2', DATA_BLOCK_ENCODING => 'FAST_DIFF', >> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', >> COMPRESSION => 'LZO', MIN_VERSIONS => '0', TTL => 'FOREVER', >> KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '4194304', IN_MEMORY => >> 'false', BLOCKCACHE => 'true'} >> 2015-12-29 21:18:02,368 INFO >> [MASTER_TABLE_OPERATIONS-dw-server20-30:60000-0] >> handler.CreateTableHandler: Create table KYLIN_VSPE13YZKF >> 2015-12-29 21:18:02,393 INFO >> [RegionOpenAndInitThread-KYLIN_VSPE13YZKF-1] regionserver.HRegion: >> creating HRegion KYLIN_VSPE13YZKF HTD == 'KYLIN_VSPE13YZKF', >> {TABLE_ATTRIBUTES => >> {coprocessor$1 => >> 'hdfs://bihbasemaster/kylin/kylin_metadata/coprocessor/kylin-coproces >> sor-1.2-1.jar|org.apache.kylin.storage.hbase.coprocessor.endpoint.IIE >> ndpoint|1000|', >> coprocessor$2 => >> 'hdfs://bihbasemaster/kylin/kylin_metadata/coprocessor/kylin-coproces >> sor-1.2-1.jar|org.apache.kylin.storage.hbase.coprocessor.observer.Agg >> regateRegionObserver|1001|', METADATA => {'KYLIN_HOST' => >> 'kylin_metadata', 'SPLIT_POLICY' => >> 'org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy'} >> }, {NAME => 'F1', DATA_BLOCK_ENCODING => 'FAST_DIFF', BLOOMFILTER => >> 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => >> 'LZO', MIN_VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => >> 'FALSE', BLOCKSIZE => '4194304', IN_MEMORY => 'false', BLOCKCACHE => >> 'true'}, {NAME => 'F2', DATA_BLOCK_ENCODING => 'FAST_DIFF', >> BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', >> COMPRESSION => 'LZO', MIN_VERSIONS => '0', TTL => 'FOREVER', >> KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '4194304', IN_MEMORY => >> 'false', BLOCKCACHE => 'true'} RootDir = >> hdfs://bihbasemaster/hbase/.tmp Table name == KYLIN_VSPE13YZKF >> 2015-12-29 21:18:02,410 INFO >> [RegionOpenAndInitThread-KYLIN_VSPE13YZKF-1] regionserver.HRegion: >> Closed KYLIN_VSPE13YZKF,,1451395082310.ce534b19cceff6d3c8f5866fbbf9fd47. >> 2015-12-29 21:18:02,417 INFO >> [MASTER_TABLE_OPERATIONS-dw-server20-30:60000-0] catalog.MetaEditor: >> Added 1 >> 2015-12-29 21:18:02,418 INFO >> [MASTER_TABLE_OPERATIONS-dw-server20-30:60000-0] master.AssignmentManager: >> Bulk assigning 1 region(s) across 4 server(s), round-robin=true >> 2015-12-29 21:18:02,424 INFO >> [dw-server20-30.wlt,60000,1451383344930-GeneralBulkAssigner-3] >> master.AssignmentManager: dw-server20-34.wlt,60020,1451383345797 >> unassigned >> znodes=1 of total=1 >> 2015-12-29 21:18:02,424 INFO >> [dw-server20-30.wlt,60000,1451383344930-GeneralBulkAssigner-3] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=OFFLINE, ts=1451395082418, server=null} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=PENDING_OPEN, >> ts=1451395082424, server=dw-server20-34.wlt,60020,1451383345797} >> 2015-12-29 21:18:02,435 INFO >> [MASTER_TABLE_OPERATIONS-dw-server20-30:60000-0] master.AssignmentManager: >> Bulk assigning done >> 2015-12-29 21:18:02,438 INFO [AM.ZK.Worker-pool2-t127] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=PENDING_OPEN, ts=1451395082424, >> server=dw-server20-34.wlt,60020,1451383345797} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=OPENING, ts=1451395082438, >> server=dw-server20-34.wlt,60020,1451383345797} >> 2015-12-29 21:18:02,444 INFO >> [MASTER_TABLE_OPERATIONS-dw-server20-30:60000-0] >> handler.CreateTableHandler: failed. Null >> 2015-12-29 21:18:02,481 INFO [AM.ZK.Worker-pool2-t128] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=OPENING, ts=1451395082438, >> server=dw-server20-34.wlt,60020,1451383345797} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=CLOSED, ts=1451395082481, >> server=dw-server20-34.wlt,60020,1451383345797} >> 2015-12-29 21:18:02,481 INFO [AM.ZK.Worker-pool2-t128] >> master.AssignmentManager: Setting node as OFFLINED in ZooKeeper for >> region {ENCODED => ce534b19cceff6d3c8f5866fbbf9fd47, NAME => >> 'KYLIN_VSPE13YZKF,,1451395082310.ce534b19cceff6d3c8f5866fbbf9fd47.', >> STARTKEY => '', ENDKEY => ''} >> 2015-12-29 21:18:02,481 INFO [AM.ZK.Worker-pool2-t128] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=CLOSED, ts=1451395082481, >> server=dw-server20-34.wlt,60020,1451383345797} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=OFFLINE, ts=1451395082481, >> server=dw-server20-34.wlt,60020,1451383345797} >> 2015-12-29 21:18:02,484 INFO [AM.ZK.Worker-pool2-t128] >> master.AssignmentManager: Assigning >> KYLIN_VSPE13YZKF,,1451395082310.ce534b19cceff6d3c8f5866fbbf9fd47. to >> dw-server20-31.wlt,60020,1451383345912 >> 2015-12-29 21:18:02,484 INFO [AM.ZK.Worker-pool2-t128] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=OFFLINE, ts=1451395082481, >> server=dw-server20-34.wlt,60020,1451383345797} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=PENDING_OPEN, >> ts=1451395082484, server=dw-server20-31.wlt,60020,1451383345912} >> 2015-12-29 21:18:02,500 INFO [AM.ZK.Worker-pool2-t129] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=PENDING_OPEN, ts=1451395082484, >> server=dw-server20-31.wlt,60020,1451383345912} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=OPENING, ts=1451395082500, >> server=dw-server20-31.wlt,60020,1451383345912} >> 2015-12-29 21:18:02,546 INFO [AM.ZK.Worker-pool2-t131] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=OPENING, ts=1451395082501, >> server=dw-server20-31.wlt,60020,1451383345912} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=CLOSED, ts=1451395082546, >> server=dw-server20-31.wlt,60020,1451383345912} >> 2015-12-29 21:18:02,546 INFO [AM.ZK.Worker-pool2-t131] >> master.AssignmentManager: Setting node as OFFLINED in ZooKeeper for >> region {ENCODED => ce534b19cceff6d3c8f5866fbbf9fd47, NAME => >> 'KYLIN_VSPE13YZKF,,1451395082310.ce534b19cceff6d3c8f5866fbbf9fd47.', >> STARTKEY => '', ENDKEY => ''} >> 2015-12-29 21:18:02,546 INFO [AM.ZK.Worker-pool2-t131] >> master.RegionStates: Transition {ce534b19cceff6d3c8f5866fbbf9fd47 >> state=CLOSED, ts=1451395082546, >> server=dw-server20-31.wlt,60020,1451383345912} to >> {ce534b19cceff6d3c8f5866fbbf9fd47 state=OFFLINE, ts=1451395082546, >> server=dw-server20-31.wlt,60020,1451383345912} >> >> >> >> >> >> ********************************************************************* >> *********************************************************** >> The information in this email is confidential and may be legally >> privileged. If you have received this email in error or are not the >> intended recipient, please immediately notify the sender and delete >> this message from your computer. Any use, distribution, or copying of >> this email other than by the intended recipient is strictly >> prohibited. All messages sent to and from us may be monitored to >> ensure compliance with internal policies and to protect our business. >> Emails are not secure and cannot be guaranteed to be error free as >> they can be intercepted, amended, lost or destroyed, or contain >> viruses. Anyone who communicates with us by email is taken to accept these >> risks. >> >> 收发邮件者请注意： >> 本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。 >> 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 >> >> ********************************************************************* >> *********************************************************** >> > > ******************************************************************************************************************************** The information in this email is confidential and may be legally privileged. If you have received this email in error or are not the intended recipient, please immediately notify the sender and delete this message from your computer. Any use, distribution, or copying of this email other than by the intended recipient is strictly prohibited. All messages sent to and from us may be monitored to ensure compliance with internal policies and to protect our business. Emails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, lost or destroyed, or contain viruses. Anyone who communicates with us by email is taken to accept these risks. 收发邮件者请注意： 本邮件含保密信息，若误收本邮件，请务必通知发送人并直接删去，不得使用、传播或复制本邮件。 进出邮件均受到本公司合规监控。邮件可能发生被截留、被修改、丢失、被破坏或包含计算机病毒等不安全情况。 ********************************************************************************************************************************
