I am not an expert in Java, but could there be an issue using the markdown
and the hive interpreters together because they share a variable name (md =
markdown object in %markdown and md = metatdata in %hive)
markdown:
public void open() { md = new Markdown4jProcessor(); }
hive:
try { ResultSetMetaData md = res.getMetaData(); for (int i = 1; i <
md.getColumnCount()
+ 1; i++) { if (i == 1) { msg.append(md.getColumnName(i)); } else { msg.
append("\t" + md.getColumnName(i)); } }
On Fri, Jun 19, 2015 at 6:56 AM, John Omernik <j...@omernik.com> wrote:
> Hey all,
>
> I am working with three primary interpreters, %md, %pyspark, and %hive.
> What I am noticing is with my current config, sometimes an interpreter will
> start other times, I'll get an errors below. I wish I could say what the
> rhyme or reason was.
>
> If I get the errors, then I have to restart Zeppelin before it will work
> (or even attempt to work). I've tried clicking "restart interpreter" in the
> interpreters tab, it seems to work, but when I go back to a notebook I get
> "Scheduler
> already terminated"
>
> What's interesting here, is other than a restart, I can run the cells (I
> have three one for each interpreter) in different orders and get different
> results, sometimes if I run %hive first, it works, then %pyspark, that will
> work too then %md will fail. (Note these are the SAME commands, on the same
> server, same config etc).
>
> Other times, I can get them to run no matter what, it's very inconsistent,
> and combined with the fact that once an interpreter fails, there is no
> getting it back until the whole server is restarted.
>
> Also of note here: I am running a recently compiled version of this (I
> downloaded this on Wed) using git clone)
>
> Any help would be appreciated in determining how to troubleshoot this!
>
> John
>
>
>
>
>
>
> Example from %md
>
> *In Notebook error*
>
>
>
> %md
> #For the Love of Jeezy Pete
>
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:135)
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:249)
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:202)
> org.apache.zeppelin.scheduler.Job.run(Job.java:170)
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> java.util.concurrent.FutureTask.run(FutureTask.java:262)
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:745)
>
> *In Running Shell Window (where I ran bin/zeppelin.sh)*
>
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:135)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:249)
>
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
>
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:202)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:170)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:138)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:133)
>
> ... 12 more
>
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:185)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>
> ... 19 more
>
> Caused by: java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:579)
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:180)
>
> ... 20 more
>
> *from interpreter log file:*
>
> INFO [2015-06-19 06:44:29,134] ({Thread-0}
> RemoteInterpreterServer.java[run]:95) - Starting remote interpreter server
> on port 54930
>
>
> *From Zeppelin Log file:*
>
> INFO [2015-06-19 06:44:19,329] ({pool-1-thread-2}
> SchedulerFactory.java[jobStarted]:132) - Job
> paragraph_1434713440246_1991176208 started by scheduler
> remoteinterpreter_328619575
>
> INFO [2015-06-19 06:44:19,331] ({pool-1-thread-2}
> Paragraph.java[jobRun]:194) - run paragraph 20150619-063040_649381067 using
> md org.apache.zeppelin.interpreter.LazyOpenInterpreter@38946f29
>
> INFO [2015-06-19 06:44:19,341] ({pool-1-thread-2}
> RemoteInterpreterProcess.java[reference]:107) - Run interpreter process
> /mapr/brewpot/mesos/zeppelin/0.5.0-incubating-SNAPSHOT/bin/interpreter.sh
> -d /mapr/brewpot/mesos/zeppelin/0.5.0-incubating-SNAPSHOT/interpreter/md -p
> 54930
>
> ERROR [2015-06-19 06:44:24,399] ({Thread-35}
> RemoteScheduler.java[getStatus]:226) - Can't get status information
>
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:138)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:224)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:183)
>
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:185)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>
> ... 8 more
>
> Caused by: java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:579)
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:180)
>
> ... 9 more
>
> ERROR [2015-06-19 06:44:24,399] ({pool-1-thread-2} Job.java[run]:183) -
> Job failed
>
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:135)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:249)
>
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
>
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:202)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:170)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
>
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:138)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:133)
>
> ... 12 more
>
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:185)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>
> ... 19 more
>
> Caused by: java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:579)
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:180)
>
> ... 20 more
>
>
>
>
>
