in this case, you should leave kylin.job.run.as.remote.cmd alone. On Wed, Aug 26, 2015 at 11:52 AM, Diego Pinheiro <[email protected]> wrote:
> I see...my settings are the two first machines you pointed out: > > Machine 1: a sandbox which works acts as the "hadoop cluster" > Machine 2: a hadoop client machine which installed the client libraries > and is running Kylin.sh > > I will take a look at my kylin.properties to check > 'kylin.job.run.as.remote.cmd". Unfortunately, I can't do it right now, > but, as soon as I checked it. I will let you know. > > > On Tue, Aug 25, 2015 at 11:11 PM, hongbin ma <[email protected]> wrote: > > Hi, I'm not quite sure about your settings because we may have messed up > > the terms. > > How many machines do you have in your settings? Correct me if I'm wrong: > > Machine 1: a sandbox which works acts as the "hadoop cluster" > > Machine 2: a hadoop client machine which installed the client libraries > > and is running Kylin.sh > > Machine 3: you working laptop/PC ? > > > > The config 'kylin.job.run.as.remote.cmd" might be confusing, it should > not > > be set to "true" unless you're NOT running Kylin.sh on a hadoop client > > machine (Thus kylin instance has to ssh to another real hadoop client > > machine to execute hbase,hive,hadoop commands). So normally, if you're > > running Kylin.sh on "Machine 2", you should leave > > 'kylin.job.run.as.remote.cmd" to false > > > > > > > > On Wed, Aug 26, 2015 at 10:46 AM, Diego Pinheiro < > [email protected]> > > wrote: > > > >> Hi Bin Mahone, > >> > >> sorry for the late reply. Thank you for your support. I didn't know > >> about Kylin instances. It is really interesting. > >> > >> However, let me ask you, I was setting up my hadoop client machine > >> with Kylin to communicate to my sandbox. But things are not working > >> well. > >> > >> I have installed hadoop 2.4.0, hbase 0.98.0 and hive 0.13.1. All them > >> are working and I can access my "remote server" from my client machine > >> (actually, I set kylin as sandbox since all my hadoop cli is pointing > >> to my sandbox). Then, Kylin was built and everything was ok until I > >> tried to build the cube. > >> > >> I got the following errors always in the second step of cube build: > >> > >> [pool-5-thread-2]:[2015-08-25 > >> > >> > 19:16:03,679][ERROR][org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)] > >> - error check status > >> java.net.ConnectException: Connection refused > >> at java.net.PlainSocketImpl.socketConnect(Native Method) > >> at > >> > java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > >> at > >> > java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:198) > >> at > >> > java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > >> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > >> at java.net.Socket.connect(Socket.java:579) > >> at java.net.Socket.connect(Socket.java:528) > >> at java.net.Socket.<init>(Socket.java:425) > >> at java.net.Socket.<init>(Socket.java:280) > >> at > >> > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) > >> at > >> > org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) > >> at > >> > org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > >> at > >> > org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > >> at > >> > org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > >> at > >> > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > >> at > >> > org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > >> at > >> > org.apache.kylin.job.tools.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:78) > >> at > >> > org.apache.kylin.job.tools.HadoopStatusGetter.get(HadoopStatusGetter.java:55) > >> at > >> > org.apache.kylin.job.tools.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56) > >> at > >> > org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:136) > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > >> at > >> > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > >> at > >> > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) > >> at > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> at > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> at java.lang.Thread.run(Thread.java:745) > >> > >> org.apache.kylin.job.exception.ExecuteException: > >> java.lang.NullPointerException > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:110) > >> at > >> > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:106) > >> at > >> > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:133) > >> at > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> at > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: java.lang.NullPointerException > >> at > >> > org.apache.kylin.job.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:73) > >> at > >> > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:105) > >> ... 6 more > >> > >> Do you have any thoughts about these errors? (detailed log is attached) > >> > >> > >> On Fri, Aug 21, 2015 at 3:21 AM, hongbin ma <[email protected]> > wrote: > >> > the document is summarized at > >> > http://kylin.incubator.apache.org/docs/install/kylin_cluster.html > >> > > >> > On Fri, Aug 21, 2015 at 1:51 PM, hongbin ma <[email protected]> > >> wrote: > >> > > >> >> hi Diego > >> >> > >> >> the config "kylin.job.run.as.remote.cmd" is somehow ambiguous , it is > >> >> enabled when you cannot run Kylin server on the same machine as your > >> hadoop > >> >> CLI, for example, if you're starting Kylin from you local IDE, and > you > >> >> hadoop CLI is a sandbox in another machine, this is the "remote" > case. > >> >> > >> >> In most of the production deployments we suggest using '"non-remote" > >> mode, > >> >> that is, kylin instance is started on the hadoop CLI. The picture > >> depicts > >> >> the scenario: > >> >> > >> > https://github.com/apache/incubator-kylin/blob/0.7/website/images/install/on_cli_install_scene.png > >> >> > >> >> Kylin instances are stateless, the runtime state is saved in its > >> >> "Metadata Store" in hbase (kylin.metadata.url config in > >> >> conf/kylin.properties). For load balance considerations it is > possible > >> to > >> >> start multiple Kylin instances sharing the same metadata store (thus > >> >> sharing the same state on table schemas, job status, cube status, > etc.) > >> >> > >> >> Each of the kylin instances has a kylin.server.mode entry in > >> >> conf/kylin.properties specifying the runtime mode, it has three > >> options: 1. > >> >> "job" for running job engine only 2. "query" for running query engine > >> only > >> >> and 3 "all" for running both. Notice that only one server can run the > >> job > >> >> engine("all" mode or "job" mode), the others must all be "query" > mode. > >> >> > >> >> A typical scenario is depicted in the attachment chart. > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> Regards, > >> >> > >> >> *Bin Mahone | 马洪宾* > >> >> Apache Kylin: http://kylin.io > >> >> Github: https://github.com/binmahone > >> >> > >> > > >> > > >> > > >> > -- > >> > Regards, > >> > > >> > *Bin Mahone | 马洪宾* > >> > Apache Kylin: http://kylin.io > >> > Github: https://github.com/binmahone > >> > > > > > > > > -- > > Regards, > > > > *Bin Mahone | 马洪宾* > > Apache Kylin: http://kylin.io > > Github: https://github.com/binmahone > -- Regards, *Bin Mahone | 马洪宾* Apache Kylin: http://kylin.io Github: https://github.com/binmahone
