I see...my settings are the two first machines you pointed out: Machine 1: a sandbox which works acts as the "hadoop cluster" Machine 2: a hadoop client machine which installed the client libraries and is running Kylin.sh
I will take a look at my kylin.properties to check 'kylin.job.run.as.remote.cmd". Unfortunately, I can't do it right now, but, as soon as I checked it. I will let you know. On Tue, Aug 25, 2015 at 11:11 PM, hongbin ma <[email protected]> wrote: > Hi, I'm not quite sure about your settings because we may have messed up > the terms. > How many machines do you have in your settings? Correct me if I'm wrong: > Machine 1: a sandbox which works acts as the "hadoop cluster" > Machine 2: a hadoop client machine which installed the client libraries > and is running Kylin.sh > Machine 3: you working laptop/PC ? > > The config 'kylin.job.run.as.remote.cmd" might be confusing, it should not > be set to "true" unless you're NOT running Kylin.sh on a hadoop client > machine (Thus kylin instance has to ssh to another real hadoop client > machine to execute hbase,hive,hadoop commands). So normally, if you're > running Kylin.sh on "Machine 2", you should leave > 'kylin.job.run.as.remote.cmd" to false > > > > On Wed, Aug 26, 2015 at 10:46 AM, Diego Pinheiro <[email protected]> > wrote: > >> Hi Bin Mahone, >> >> sorry for the late reply. Thank you for your support. I didn't know >> about Kylin instances. It is really interesting. >> >> However, let me ask you, I was setting up my hadoop client machine >> with Kylin to communicate to my sandbox. But things are not working >> well. >> >> I have installed hadoop 2.4.0, hbase 0.98.0 and hive 0.13.1. All them >> are working and I can access my "remote server" from my client machine >> (actually, I set kylin as sandbox since all my hadoop cli is pointing >> to my sandbox). Then, Kylin was built and everything was ok until I >> tried to build the cube. >> >> I got the following errors always in the second step of cube build: >> >> [pool-5-thread-2]:[2015-08-25 >> >> 19:16:03,679][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)] >> - error check status >> java.net.ConnectException: Connection refused >> at java.net.PlainSocketImpl.socketConnect(Native Method) >> at >> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) >> at >> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:198) >> at >> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) >> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >> at java.net.Socket.connect(Socket.java:579) >> at java.net.Socket.connect(Socket.java:528) >> at java.net.Socket.<init>(Socket.java:425) >> at java.net.Socket.<init>(Socket.java:280) >> at >> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) >> at >> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) >> at >> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) >> at >> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) >> at >> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) >> at >> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) >> at >> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) >> at >> org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:78) >> at >> org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:55) >> at >> org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56) >> at >> org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:136) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) >> at >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) >> at >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> >> org.apache.kylin.job.exception.ExecuteException: >> java.lang.NullPointerException >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110) >> at >> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) >> at >> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.lang.NullPointerException >> at >> org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:73) >> at >> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:105) >> ... 6 more >> >> Do you have any thoughts about these errors? (detailed log is attached) >> >> >> On Fri, Aug 21, 2015 at 3:21 AM, hongbin ma <[email protected]> wrote: >> > the document is summarized at >> > http://kylin.incubator.apache.org/docs/install/kylin_cluster.html >> > >> > On Fri, Aug 21, 2015 at 1:51 PM, hongbin ma <[email protected]> >> wrote: >> > >> >> hi Diego >> >> >> >> the config "kylin.job.run.as.remote.cmd" is somehow ambiguous , it is >> >> enabled when you cannot run Kylin server on the same machine as your >> hadoop >> >> CLI, for example, if you're starting Kylin from you local IDE, and you >> >> hadoop CLI is a sandbox in another machine, this is the "remote" case. >> >> >> >> In most of the production deployments we suggest using '"non-remote" >> mode, >> >> that is, kylin instance is started on the hadoop CLI. The picture >> depicts >> >> the scenario: >> >> >> https://github.com/apache/incubator-kylin/blob/0.7/website/images/install/on_cli_install_scene.png >> >> >> >> Kylin instances are stateless, the runtime state is saved in its >> >> "Metadata Store" in hbase (kylin.metadata.url config in >> >> conf/kylin.properties). For load balance considerations it is possible >> to >> >> start multiple Kylin instances sharing the same metadata store (thus >> >> sharing the same state on table schemas, job status, cube status, etc.) >> >> >> >> Each of the kylin instances has a kylin.server.mode entry in >> >> conf/kylin.properties specifying the runtime mode, it has three >> options: 1. >> >> "job" for running job engine only 2. "query" for running query engine >> only >> >> and 3 "all" for running both. Notice that only one server can run the >> job >> >> engine("all" mode or "job" mode), the others must all be "query" mode. >> >> >> >> A typical scenario is depicted in the attachment chart. >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> Regards, >> >> >> >> *Bin Mahone | 马洪宾* >> >> Apache Kylin: http://kylin.io >> >> Github: https://github.com/binmahone >> >> >> > >> > >> > >> > -- >> > Regards, >> > >> > *Bin Mahone | 马洪宾* >> > Apache Kylin: http://kylin.io >> > Github: https://github.com/binmahone >> > > > > -- > Regards, > > *Bin Mahone | 马洪宾* > Apache Kylin: http://kylin.io > Github: https://github.com/binmahone
