Hi, I'm not quite sure about your settings because we may have messed up the terms. How many machines do you have in your settings? Correct me if I'm wrong: Machine 1: a sandbox which works acts as the "hadoop cluster" Machine 2: a hadoop client machine which installed the client libraries and is running Kylin.sh Machine 3: you working laptop/PC ?
The config 'kylin.job.run.as.remote.cmd" might be confusing, it should not be set to "true" unless you're NOT running Kylin.sh on a hadoop client machine (Thus kylin instance has to ssh to another real hadoop client machine to execute hbase,hive,hadoop commands). So normally, if you're running Kylin.sh on "Machine 2", you should leave 'kylin.job.run.as.remote.cmd" to false On Wed, Aug 26, 2015 at 10:46 AM, Diego Pinheiro <[email protected]> wrote: > Hi Bin Mahone, > > sorry for the late reply. Thank you for your support. I didn't know > about Kylin instances. It is really interesting. > > However, let me ask you, I was setting up my hadoop client machine > with Kylin to communicate to my sandbox. But things are not working > well. > > I have installed hadoop 2.4.0, hbase 0.98.0 and hive 0.13.1. All them > are working and I can access my "remote server" from my client machine > (actually, I set kylin as sandbox since all my hadoop cli is pointing > to my sandbox). Then, Kylin was built and everything was ok until I > tried to build the cube. > > I got the following errors always in the second step of cube build: > > [pool-5-thread-2]:[2015-08-25 > > 19:16:03,679][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)] > - error check status > java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) > at > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:198) > at > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at java.net.Socket.connect(Socket.java:528) > at java.net.Socket.<init>(Socket.java:425) > at java.net.Socket.<init>(Socket.java:280) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) > at > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) > at > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at > org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:78) > at > org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:55) > at > org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56) > at > org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:136) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > org.apache.kylin.job.exception.ExecuteException: > java.lang.NullPointerException > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:73) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:105) > ... 6 more > > Do you have any thoughts about these errors? (detailed log is attached) > > > On Fri, Aug 21, 2015 at 3:21 AM, hongbin ma <[email protected]> wrote: > > the document is summarized at > > http://kylin.incubator.apache.org/docs/install/kylin_cluster.html > > > > On Fri, Aug 21, 2015 at 1:51 PM, hongbin ma <[email protected]> > wrote: > > > >> hi Diego > >> > >> the config "kylin.job.run.as.remote.cmd" is somehow ambiguous , it is > >> enabled when you cannot run Kylin server on the same machine as your > hadoop > >> CLI, for example, if you're starting Kylin from you local IDE, and you > >> hadoop CLI is a sandbox in another machine, this is the "remote" case. > >> > >> In most of the production deployments we suggest using '"non-remote" > mode, > >> that is, kylin instance is started on the hadoop CLI. The picture > depicts > >> the scenario: > >> > https://github.com/apache/incubator-kylin/blob/0.7/website/images/install/on_cli_install_scene.png > >> > >> Kylin instances are stateless, the runtime state is saved in its > >> "Metadata Store" in hbase (kylin.metadata.url config in > >> conf/kylin.properties). For load balance considerations it is possible > to > >> start multiple Kylin instances sharing the same metadata store (thus > >> sharing the same state on table schemas, job status, cube status, etc.) > >> > >> Each of the kylin instances has a kylin.server.mode entry in > >> conf/kylin.properties specifying the runtime mode, it has three > options: 1. > >> "job" for running job engine only 2. "query" for running query engine > only > >> and 3 "all" for running both. Notice that only one server can run the > job > >> engine("all" mode or "job" mode), the others must all be "query" mode. > >> > >> A typical scenario is depicted in the attachment chart. > >> > >> > >> > >> > >> > >> -- > >> Regards, > >> > >> *Bin Mahone | 马洪宾* > >> Apache Kylin: http://kylin.io > >> Github: https://github.com/binmahone > >> > > > > > > > > -- > > Regards, > > > > *Bin Mahone | 马洪宾* > > Apache Kylin: http://kylin.io > > Github: https://github.com/binmahone > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
