Could you check the interpreter log ? The issue of not be able to
connecting to thrift port could happen due to network issue, like firewall
or you use VPN, etc.
小野圭二 <onoke...@gmail.com>于2018年3月5日周一 下午12:32写道：
> Hi,
>
> I show the all log on this error.There are 2 type error. The 1 is at the
> first run. The 2 is at the second run. Seems like could not established to
> Thrift callback port 51348 that was set at the first run.
> First of all, does it need to establish the callback port of Thrift
> server?
> The notebook script was just
> %pyspark
> print 'aaa'
> Any advice appreciate.
>
> 1.
> INFO [2018-03-05 13:11:13,914] ({qtp1651855867-13}
> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U INFO [2018-03-05
> 13:11:13,934] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:109) -
> Job 20180302-135047_99102973 started by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session
> INFO [2018-03-05 13:11:13,936] ({pool-2-thread-2}
> Paragraph.java[jobRun]:397) - run paragraph 20180302-135047_99102973 using
> pyspark
> RemoteInterpreter_org.apache.zeppelin.spark.PySparkInterpreter_shared_session
> INFO [2018-03-05 13:11:13,936] ({pool-2-thread-2}
> ManagedInterpreterGroup.java[getOrCreateInterpreterProcess]:57) - Create
> InterperterProcess for InterpreterGroup: 2CV1983WX:shared_process WARN
> [2018-03-05 13:11:14,010] ({pool-2-thread-2}
> InterpreterSetting.java[setupPropertiesForSparkR]:796) - sparkr.zip is not
> found, SparkR may not work. INFO [2018-03-05 13:11:14,017]
> ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:123) -
> Choose port 51347 for RemoteInterpreterProcess INFO [2018-03-05
> 13:11:14,023] ({pool-2-thread-2}
> RemoteInterpreterManagedProcess.java[start]:130) - Thrift server for
> callback will start. Port: 51348 INFO [2018-03-05 13:11:14,049]
> ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:205) - Run
> interpreter process [/Users/onokeiji/zeppelin/zeppelin/bin/interpreter.sh,
> -d, /Users/onokeiji/zeppelin/zeppelin/interpreter/spark, -c, 172.16.20.10,
> -p, 51348, -l, /Users/onokeiji/zeppelin/zeppelin/local-repo/2CV1983WX, -g,
> spark] INFO [2018-03-05 13:11:14,238] ({Exec Default Executor}
> RemoteInterpreterManagedProcess.java[onProcessComplete]:258) - Interpreter
> process exited 0 ERROR [2018-03-05 13:12:14,058] ({pool-2-thread-2}
> Job.java[run]:184) - Job failed
> org.apache.zeppelin.interpreter.InterpreterException: Cannot run
> interpreter at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:220)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:101)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:153)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:182) at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) ERROR [2018-03-05 13:12:14,075]
> ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2219) - Error
> org.apache.zeppelin.interpreter.InterpreterException: Cannot run
> interpreter at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:220)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:101)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:153)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:182) at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748) WARN [2018-03-05 13:12:14,075]
> ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2227) - Job
> 20180302-135047_99102973 is finished, status: ERROR, exception:
> org.apache.zeppelin.interpreter.InterpreterException: Cannot run
> interpreter, result: Cannot run interpreter INFO [2018-03-05 13:12:14,103]
> ({pool-2-thread-2} VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U
> INFO [2018-03-05 13:12:14,109] ({pool-2-thread-2}
> SchedulerFactory.java[jobFinished]:115) - Job 20180302-135047_99102973
> finished by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session
>
> 2.
> INFO [2018-03-05 13:15:32,736] ({qtp1651855867-13}
> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U
> INFO [2018-03-05 13:15:32,751] ({pool-2-thread-2}
> SchedulerFactory.java[jobStarted]:109) - Job 20180302-135047_99102973
> started by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session
> INFO [2018-03-05 13:15:32,753] ({pool-2-thread-2}
> Paragraph.java[jobRun]:397) - run paragraph 20180302-135047_99102973 using
> pyspark
> RemoteInterpreter_org.apache.zeppelin.spark.PySparkInterpreter_shared_session
> ERROR [2018-03-05 13:15:32,882] ({pool-2-thread-2} Job.java[run]:184) -
> Job failed
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:152)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:182)
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:73)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:144)
> ... 13 more
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused (Connection refused)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> ... 20 more
> Caused by: java.net.ConnectException: Connection refused (Connection
> refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
> ... 21 more
> ERROR [2018-03-05 13:15:32,884] ({pool-2-thread-2}
> NotebookServer.java[afterStatusChange]:2219) - Error
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:152)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:182)
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:73)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:144)
> ... 13 more
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused (Connection refused)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> ... 20 more
> Caused by: java.net.ConnectException: Connection refused (Connection
> refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
> ... 21 more
> WARN [2018-03-05 13:15:32,885] ({pool-2-thread-2}
> NotebookServer.java[afterStatusChange]:2227) - Job 20180302-135047_99102973
> is finished, status: ERROR, exception:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused), result:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> INFO [2018-03-05 13:15:32,910] ({pool-2-thread-2}
> VFSNotebookRepo.java[save]:221) - Saving note:2D6S6TB3U
> INFO [2018-03-05 13:15:32,914] ({pool-2-thread-2}
> SchedulerFactory.java[jobFinished]:115) - Job 20180302-135047_99102973
> finished by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter-shared_session
>
> 2018-03-02 15:07 GMT+09:00 小野圭二 <onoke...@gmail.com>:
>
>> Hi Jeff,
>>
>> No one graved 63039 so far on Mac.
>> Yes, this issue has been reproduced as changing the port NO randomly.
>>
>> Ono
>>
>>
>> 2018-03-02 14:49 GMT+09:00 Jeff Zhang <zjf...@gmail.com>:
>>
>>>
>>> It is possible that 63039 is used by other processes, as Mac can not
>>> detect whether this port is used or not from my experience.
>>>
>>> And can you reproduce this issue consistently ? Because each time
>>> zeppelin will chose a random port as long as you don't specify the
>>> portRange, so if you reproduce it consistently, it might due to other
>>> issues.
>>>
>>>
>>>
>>>
>>> 小野圭二 <onoke...@gmail.com>于2018年3月2日周五 下午1:32写道：
>>>
>>>> Hi there,
>>>>
>>>> I am so confusing now, let me know why if You know this, please.
>>>>
>>>> The env:
>>>> Mac: High Sierra ver.10.13.3
>>>> Zep: 0.7.3
>>>> Spark: 2.2.0
>>>> Java: 1.8
>>>> notebook script: %pyspark print 'aaa' (just it for
>>>> test)
>>>>
>>>> In this env, i got the error in log as
>>>>
>>>> ERROR ({pool-2-thread-2} Job.java[run]:184) - Job failed
>>>>
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException:
>>>> java.net.ConnectException: Connection refused (Connection refused)
>>>>
>>>>
>>>> Indeed, before running notebook, RemoteInterpreterManagedProcess.java
>>>> prepares a Port for callback to Thrift.
>>>>
>>>>
>>>> INFO ({pool-2-thread-1}
>>>> RemoteInterpreterManagedProcess.java[start]:130) - Thrift server for
>>>> callback will start. Port: 63039
>>>>
>>>>
>>>> I guess, the reason why the error was Zeppelin could not attache to the
>>>> Thrift PORT.
>>>>
>>>>
>>>> And my confusion is, when i run the same module on CentOS, it worked
>>>> fine.
>>>> CentOS: 7
>>>> Zep: same
>>>> Spark: same
>>>> Java: same
>>>> notebook script: same
>>>>
>>>> And there was no the Thift port description in the log file. From this
>>>> consequence, the fine working was cause of did not use Thrift port on
>>>> CentOS.
>>>>
>>>>
>>>> Do You know actually there is different behavior between Mac and
>>>> CentOS? And i am so appreciated if You know the workaround on Mac.
>>>>
>>>>
>>>> Best,
>>>> Ono
>>>>
>>>
>>
>
