Do you see any other log under logs/ directory?
This also can happen when JAVA_HOME is not set to JDK7.
Thanks,
moon
On Tue, Nov 10, 2015 at 3:33 AM Bill Sparks <jspa...@cray.com> wrote:
> Sorry I missed your original response.
>
> I export the SPARK_HOME to the install location and set the MASTER
> environment variable to the mesos cluster. Yes we are using mesos as the
> scheduler.
>
> E.g.
>
> export SPARK_HOME=/etc/spark/spark-1.5.1
> export
> MASTER="mesos://zk://maryann2:2181,maryann3:2181,maryann4:2181/mesos"
>
> grep master /etc/spark/spark-1.5.1/conf/spark-defaults.conf
> spark.master
> mesos://zk://maryann2:2181,maryann3:2181,maryann4:2181/mesos
>
>
> Not sure about the following which is (I think) the true error. I don't
> get any interpreter logs, so I'm assuming it's not even getting started.
>
>
> INFO [2015-11-09 12:31:48,272] ({qtp1241236050-99}
> NotebookServer.java[onMessage]:101) - RECEIVE << PING
> ERROR [2015-11-09 12:31:48,648] ({Thread-14}
> RemoteInterpreterEventPoller.java[run]:63) - Can't get
> RemoteInterpreterEvent
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:61)
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> ... 7 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
> ... 8 more
>
>
> --
> Jonathan (Bill) Sparks
> Software Architecture
> Cray Inc.
>
> From: Pablo Torre <pablotorr...@gmail.com>
> Reply-To: "users@zeppelin.incubator.apache.org" <
> users@zeppelin.incubator.apache.org>
> Date: Monday, November 9, 2015 8:35 AM
> To: "users@zeppelin.incubator.apache.org" <
> users@zeppelin.incubator.apache.org>
> Subject: Re: zeppelin startup warnings
>
> Hi Moon,
>
> I have the same problem. I am doing this in a script:
>
> SPARK_PROFILE=1.4
> SPARK_VERSION=1.4.1
> HADOOP_PROFILE=2.4
> HADOOP_VERSION=2.4.0
>
>
>
> curl -sL --retry 3 \
> "
> http://mirrors.ibiblio.org/apache/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE.tgz";
> \
> | gunzip \
> | tar x -C /usr/ \
> && ln -s /usr/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE /usr/spark \
> && rm -rf /usr/spark/examples \
> && rm /usr/spark/lib/spark-examples*.jar
>
>
>
> and then in my zeppelin configuration file I have this:
>
> export SPARK_HOME="/usr/spark"
>
> 2015-11-08 7:09 GMT+01:00 moon soo Lee <m...@apache.org>:
>
>> This error may able to occur when SparkContext got some error to connect
>> to Spark cluster.
>>
>> Could you share how you configured Zeppelin?
>> Have you export SPARK_HOME and set 'master' property in 'Interpreter'
>> page?
>>
>> Thanks,
>> moon
>>
>>
>> On Fri, Nov 6, 2015 at 1:03 AM Bill Sparks <jspa...@cray.com> wrote:
>>
>>> Hello.
>>>
>>> I'm trying to run the master branch on CentOS6.5.
>>>
>>> My compile completes and the server start successfully, in the log file
>>> I see the following warnings. Question, are these expected? Also I'm uses
>>> spark with Mesos, I've followed the wiki instruction s but get the
>>> following stack trace on a simple inspection of the spark contect variable
>>> sc. Note the spark-shell executes successfully. I'm thinking it has
>>> something to do with the thrift server not running or … not even getting to
>>> executing the spark job.
>>>
>>> Regards
>>> -Bill
>>>
>>> Startup Warnings (running as root):
>>>
>>> [root@maryann4 incubator-zeppelin]# ./bin/zeppelin.sh
>>> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1024m;
>>> support was removed in 8.0
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in
>>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.PackagesResourceConfig
>>> init
>>> INFO: Scanning for root resource and provider classes in the packages:
>>> org.apache.zeppelin.rest
>>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
>>> logClasses
>>> INFO: Root resource classes found:
>>> class org.apache.zeppelin.rest.NotebookRestApi
>>> class org.apache.zeppelin.rest.ZeppelinRestApi
>>> class org.apache.zeppelin.rest.InterpreterRestApi
>>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
>>> init
>>> INFO: No provider classes found.
>>> Nov 05, 2015 9:59:13 AM
>>> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
>>> INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012
>>> 05:14 PM'
>>> Nov 05, 2015 9:59:14 AM com.sun.jersey.spi.inject.Errors
>>> processErrorMessages
>>> WARNING: The following warnings have been detected with resource and/or
>>> provider classes:
>>> WARNING: A sub-resource method, public javax.ws.rs.core.Response
>>> org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String)
>>> throws java.io.IOException, with URI template, "/", is treated as a
>>> resource method
>>> WARNING: A sub-resource method, public javax.ws.rs.core.Response
>>> org.apache.zeppelin.rest.NotebookRestApi.getNotebookList() throws
>>> java.io.IOException, with URI template, "/", is treated as a resource method
>>> WARNING: A HTTP GET method, public javax.ws.rs.core.Response
>>> org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String),
>>> should not consume any entity.
>>>
>>> Error via spark interpreter …
>>>
>>> java.net.ConnectException: Connection refused at
>>> java.net.PlainSocketImpl.socketConnect(Native Method) at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
>>> at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>>> at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
>>> java.net.Socket.connect(Socket.java:589) at
>>> org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>>> at
>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>>> at
>>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>>> at
>>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>>> at
>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>>> at
>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257)
>>> at
>>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at
>>> org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>> --
>>> Jonathan (Bill) Sparks
>>> Software Architecture
>>>
>>>
>
>
> --
> Pablo Torre.
> Freelance software engineer and Ruby on Rails developer.
> Oleiros (Coruña)
> *Personal site <http://www.pablotorrerodriguez.com>*
> My blog <http://www.aboutechnologies.com>
>
