option to zeppelin notebook run if one of the paragraph fails
Let's say I have multiple paragraphs in a note, it seems that those paragraphs run sequentially one after another, and the subsequent paragraph continues to run if any previous one fails, is there any way to configure the notebook to stop further runs if any paragraph fails? Thanks,Richard
Re: RE: Zeppelin 0.7.2 integration with Presto 0.184
the issue I am seeing is related to this: [ZEPPELIN-2891] Impossible to use
jdbc interface with presto-jdbc >= 0.180 - ASF JIRA
|
|
| |
[ZEPPELIN-2891] Impossible to use jdbc interface with presto-jdbc >= 0.1...
|
|
|
and I have also filed an issue against presto, but people believes that should
be fixed on Zeppelin side.
Is strict config checking really needed? · Issue #9254 · prestodb/presto
|
|
|
| | |
|
|
|
| |
Is strict config checking really needed? · Issue #9254 · prestodb/presto
A strict config check was added in this commit
defabcb#diff-fc7374097aa800bd529f476b35716019 Any unrecognized co...
|
|
|
On Wednesday, November 1, 2017, 9:15:20 AM PDT, Richard Xin
<richardxin...@yahoo.com> wrote:
the problem looks like the strict check introduced a few month ago by presto
throws exception when unrecognized properties are added, and default.url is
unrecognized by presto
prestodb/presto
|
|
|
| | |
|
|
|
| |
prestodb/presto
presto - Distributed SQL query engine for big data
|
|
|
On Wednesday, November 1, 2017, 2:31:04 AM PDT, David Howell
<david.how...@zipmoney.com.au> wrote:
#yiv3292286389 -- filtered {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2
4;}#yiv3292286389 p.yiv3292286389MsoNormal, #yiv3292286389
li.yiv3292286389MsoNormal, #yiv3292286389 div.yiv3292286389MsoNormal
{margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;}#yiv3292286389 h2
{margin-right:0cm;margin-left:0cm;font-size:18.0pt;}#yiv3292286389 a:link,
#yiv3292286389 span.yiv3292286389MsoHyperlink
{color:blue;text-decoration:underline;}#yiv3292286389 a:visited, #yiv3292286389
span.yiv3292286389MsoHyperlinkFollowed
{color:#954F72;text-decoration:underline;}#yiv3292286389 pre
{margin:0cm;margin-bottom:.0001pt;font-size:10.0pt;}#yiv3292286389
span.yiv3292286389Heading2Char {font-weight:bold;}#yiv3292286389
span.yiv3292286389HTMLPreformattedChar {}#yiv3292286389
p.yiv3292286389ydpa2ef8223card-description, #yiv3292286389
li.yiv3292286389ydpa2ef8223card-description, #yiv3292286389
div.yiv3292286389ydpa2ef8223card-description
{margin-right:0cm;margin-left:0cm;font-size:11.0pt;}#yiv3292286389
.yiv3292286389MsoChpDefault {}#yiv3292286389 filtered {margin:72.0pt 72.0pt
72.0pt 72.0pt;}#yiv3292286389 div.yiv3292286389WordSection1 {}#yiv3292286389
I’ve done this previously, my answer
herehttps://stackoverflow.com/a/44238195/1335793
I was using an older version though.
Did you try appending catalog, or a catalog & schema to the url?
Error message suggests problem is with the “default.url” in the interpreter
settings.
On another note, AWS has released Athena (managed version of presto) so when
it's available in my region I will look at using that.
Dave
From: Richard Xin
Sent: Wednesday, 1 November 2017 9:35 AM
To: Users
Subject: Zeppelin 0.7.2 integration with Presto 0.184
I am not sure Zeppelin 0.7.2 is compatible with Presto 0.184here are the steps:
1. create new presto interpreter
2. %presto
show tables
java.sql.SQLException: Unrecognized connection property 'url'at
com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:292)at
com.facebook.presto.jdbc.PrestoDriverUri.(PrestoDriverUri.java:89)at
com.facebook.presto.jdbc.PrestoDriverUri.(PrestoDriverUri.java:79)at
com.facebook.presto.jdbc.PrestoDriver.connect(PrestoDriver.java:86)at
java.sql.DriverManager.getConnection(DriverManager.java:664)at
java.sql.DriverManager.getConnection(DriverManager.java:208)at
org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:836)at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:434)at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361)at
org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
...
it seems that presto has some strict property name checking, I believe it's not
uncommon that people want to use presto in Zeppelin, is there any work-around?
I am on AWS EMR
related presto code:prestodb/presto
|
|
|
| | |
|
|
|
| |
prestodb/presto
presto - Distributed SQL query engine for big data
|
|
|
Re: RE: Zeppelin 0.7.2 integration with Presto 0.184
the problem looks like the strict check introduced a few month ago by presto
throws exception when unrecognized properties are added, and default.url is
unrecognized by presto
prestodb/presto
|
|
|
| | |
|
|
|
| |
prestodb/presto
presto - Distributed SQL query engine for big data
|
|
|
On Wednesday, November 1, 2017, 2:31:04 AM PDT, David Howell
<david.how...@zipmoney.com.au> wrote:
#yiv9822418077 #yiv9822418077 -- _filtered #yiv9822418077
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv9822418077
#yiv9822418077 p.yiv9822418077MsoNormal, #yiv9822418077
li.yiv9822418077MsoNormal, #yiv9822418077 div.yiv9822418077MsoNormal
{margin:0cm;margin-bottom:.0001pt;font-size:11.0pt;}#yiv9822418077 h2
{margin-right:0cm;margin-left:0cm;font-size:18.0pt;}#yiv9822418077 a:link,
#yiv9822418077 span.yiv9822418077MsoHyperlink
{color:blue;text-decoration:underline;}#yiv9822418077 a:visited, #yiv9822418077
span.yiv9822418077MsoHyperlinkFollowed
{color:#954F72;text-decoration:underline;}#yiv9822418077 pre
{margin:0cm;margin-bottom:.0001pt;font-size:10.0pt;}#yiv9822418077
span.yiv9822418077Heading2Char {font-weight:bold;}#yiv9822418077
span.yiv9822418077HTMLPreformattedChar {}#yiv9822418077
p.yiv9822418077ydpa2ef8223card-description, #yiv9822418077
li.yiv9822418077ydpa2ef8223card-description, #yiv9822418077
div.yiv9822418077ydpa2ef8223card-description
{margin-right:0cm;margin-left:0cm;font-size:11.0pt;}#yiv9822418077
.yiv9822418077MsoChpDefault {} _filtered #yiv9822418077 {margin:72.0pt 72.0pt
72.0pt 72.0pt;}#yiv9822418077 div.yiv9822418077WordSection1 {}#yiv9822418077
I’ve done this previously, my answer
herehttps://stackoverflow.com/a/44238195/1335793
I was using an older version though.
Did you try appending catalog, or a catalog & schema to the url?
Error message suggests problem is with the “default.url” in the interpreter
settings.
On another note, AWS has released Athena (managed version of presto) so when
it's available in my region I will look at using that.
Dave
From: Richard Xin
Sent: Wednesday, 1 November 2017 9:35 AM
To: Users
Subject: Zeppelin 0.7.2 integration with Presto 0.184
I am not sure Zeppelin 0.7.2 is compatible with Presto 0.184here are the steps:
1. create new presto interpreter
2. %presto
show tables
java.sql.SQLException: Unrecognized connection property 'url'at
com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:292)at
com.facebook.presto.jdbc.PrestoDriverUri.(PrestoDriverUri.java:89)at
com.facebook.presto.jdbc.PrestoDriverUri.(PrestoDriverUri.java:79)at
com.facebook.presto.jdbc.PrestoDriver.connect(PrestoDriver.java:86)at
java.sql.DriverManager.getConnection(DriverManager.java:664)at
java.sql.DriverManager.getConnection(DriverManager.java:208)at
org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)at
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:836)at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:434)at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361)at
org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
...
it seems that presto has some strict property name checking, I believe it's not
uncommon that people want to use presto in Zeppelin, is there any work-around?
I am on AWS EMR
related presto code:prestodb/presto
|
|
|
| | |
|
|
|
| |
prestodb/presto
presto - Distributed SQL query engine for big data
|
|
|
Zeppelin 0.7.2 integration with Presto 0.184
I am not sure Zeppelin 0.7.2 is compatible with Presto 0.184here are the steps: 1. create new presto interpreter 2. %presto show tables java.sql.SQLException: Unrecognized connection property 'url' at com.facebook.presto.jdbc.PrestoDriverUri.validateConnectionProperties(PrestoDriverUri.java:292) at com.facebook.presto.jdbc.PrestoDriverUri.(PrestoDriverUri.java:89) at com.facebook.presto.jdbc.PrestoDriverUri.(PrestoDriverUri.java:79) at com.facebook.presto.jdbc.PrestoDriver.connect(PrestoDriver.java:86) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:208) at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79) at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:836) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:434) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361) at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129) ... it seems that presto has some strict property name checking, I believe it's not uncommon that people want to use presto in Zeppelin, is there any work-around? I am on AWS EMR related presto code:prestodb/presto | | | | | | | | | | | prestodb/presto presto - Distributed SQL query engine for big data | | |
run zeppelin notebook from outside Zeppelin UI or programmatically
Hi, is there a recommended way to run zeppelin notebook from outside Zeppelin UI or command-line or programmatically? I found one post, but looks pretty hacky to me. How to run zeppelin notebook from command line (automatically) | | | | | | | | | | | How to run zeppelin notebook from command line (automatically) How do we run the notebook from command line? Further to 1, how would I pass command line arguments into the not... | | | Thanks,
Re: Zeppelin postgres interpreter throws NPE
[ZEPPELIN-2828] unhandled NullPointerException in PostgreSqlInterpreter when
unable to connect - ASF JIRA
|
|
| |
[ZEPPELIN-2828] unhandled NullPointerException in PostgreSqlInterpreter ...
|
|
|
On Thursday, August 3, 2017, 4:53:05 PM PDT, Jianfeng (Jeff) Zhang
<jzh...@hortonworks.com> wrote:
Agree, you can create a ticket for better exception message. And the error is
better to be displayed in frontend
Best Regard,Jeff Zhang
From: Richard Xin <richardxin...@yahoo.com>
Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Date: Friday, August 4, 2017 at 4:40 AM
To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Subject: Re: Zeppelin postgres interpreter throws NPE
oh, just figured it out myselfit seems that the property I aded in interpreter
config postgres.database is not recognized by zeppelin, I solved it by adding
dbname to url.
anyway, the exception is better to be handled and emit more friendly error
message.
On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin
<richardxin...@yahoo.com> wrote:
on AWS EMR1. Tested remote postgres connection using psql, works OK2. added
postges interpreter via Interpreter UI3. create new note:%postgresql (or
%psql)
select * from test.batch_report;
got NPE
4. here are what's in the log:
WARN [2017-08-03 20:22:46,859] ({qtp459296537-18}
LoginRestApi.java[postLogin]:119) -
{"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21}
NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 :
richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter
instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22}
InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter
instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2}
SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734
started by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) -
run paragraph 20170803-180535_552293631 using psql
org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2}
RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process
[/usr/lib/zeppelin/bin/interpreter.sh, -d,
/usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l,
/usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2}
RemoteInterpreter.java[init]:221) - Create remote interpreter
org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2}
RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local
angular object registry from ZeppelinServer to remote interpreter group
2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2}
NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is
finished, status: ERROR, exception: null, result: %text
java.lang.NullPointerException
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPo
Re: Zeppelin postgres interpreter throws NPE
oh, just figured it out myselfit seems that the property I aded in interpreter
config postgres.database is not recognized by zeppelin, I solved it by adding
dbname to url.
anyway, the exception is better to be handled and emit more friendly error
message.
On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin
<richardxin...@yahoo.com> wrote:
on AWS EMR1. Tested remote postgres connection using psql, works OK2. added
postges interpreter via Interpreter UI3. create new note:%postgresql (or
%psql)
select * from test.batch_report;
got NPE
4. here are what's in the log:
WARN [2017-08-03 20:22:46,859] ({qtp459296537-18}
LoginRestApi.java[postLogin]:119) -
{"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21}
NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 :
richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter
instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22}
InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter
instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2}
SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734
started by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362)
- run paragraph 20170803-180535_552293631 using psql
org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2}
RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process
[/usr/lib/zeppelin/bin/interpreter.sh, -d,
/usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l,
/usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2}
RemoteInterpreter.java[init]:221) - Create remote interpreter
org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2}
RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local
angular object registry from ZeppelinServer to remote interpreter group
2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2}
NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is
finished, status: ERROR, exception: null, result: %text
java.lang.NullPointerException
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2}
SchedulerFactory.java[jobFinished]:137) - Job
paragraph_1501783535283_1713771734 finished by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
Zeppelin postgres interpreter throws NPE
on AWS EMR1. Tested remote postgres connection using psql, works OK2. added
postges interpreter via Interpreter UI3. create new note:%postgresql (or
%psql)
select * from test.batch_report;
got NPE
4. here are what's in the log:
WARN [2017-08-03 20:22:46,859] ({qtp459296537-18}
LoginRestApi.java[postLogin]:119) -
{"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21}
NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 :
richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter
instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22}
InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter
instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22}
InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter
org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2}
SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734
started by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362)
- run paragraph 20170803-180535_552293631 using psql
org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2}
RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process
[/usr/lib/zeppelin/bin/interpreter.sh, -d,
/usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l,
/usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2}
RemoteInterpreter.java[init]:221) - Create remote interpreter
org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2}
RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local
angular object registry from ZeppelinServer to remote interpreter group
2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2}
NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is
finished, status: ERROR, exception: null, result: %text
java.lang.NullPointerException
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
at
org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2}
SchedulerFactory.java[jobFinished]:137) - Job
paragraph_1501783535283_1713771734 finished by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
Re: configure Zeppelin to use existing external hive metastore
It works, Thanks Jeff!
On Wednesday, August 2, 2017, 6:24:34 PM PDT, Jeff Zhang <zjf...@gmail.com>
wrote:
I suppose the %sql means %spark.sql, in that case you need to modify the
hive-site.xml under SPARK_CONF_DIR
Richard Xin <richardxin...@yahoo.com>于2017年8月3日周四 上午9:21写道：
on AWS EMRI am trying to bind zeppelin to an existing hive metastore, I
modified hive-site.xml under /etc/hive/conf to pointing to existing metastore,
and I tested with commandline hive, it works as expected.but under zeppelin
%sql is still connected to previous hive metastore, I added line export
HIVE_CONF_DIR=/etc/hive/conf
inside zeppelin-env.sh
And /etc/spark/conf's spark-env.sh has following line by default:export
HIVE_CONF_DIR=${HIVE_CONF_DIR:-/etc/hive/conf}
What did I miss?
Thanks
configure Zeppelin to use existing external hive metastore
on AWS EMRI am trying to bind zeppelin to an existing hive metastore, I
modified hive-site.xml under /etc/hive/conf to pointing to existing metastore,
and I tested with commandline hive, it works as expected.but under zeppelin
%sql is still connected to previous hive metastore, I added line export
HIVE_CONF_DIR=/etc/hive/conf
inside zeppelin-env.sh
And /etc/spark/conf's spark-env.sh has following line by default:export
HIVE_CONF_DIR=${HIVE_CONF_DIR:-/etc/hive/conf}
What did I miss?
Thanks
activeDirectoryRealm.groupRolesMap
I am facing some hurdle with activeDirectoryRealm.groupRolesMapthe following is
the content of my shiro.ini...activeDirectoryRealm.groupRolesMap =
"CN=Zeppelin-Admin,OU=Zeppelin,OU=Applications,OU=Groups,DC=directory,DC=[domain_here],DC=com":"admin","CN=ZeppelinZepZeppelinpelin-Devs,OU=Zepplin,OU=Applications,OU=Groups,DC=directory,DC=[domain_here],DC=com":"developer","CN=Zeppelin-Analyst,OU=Zeppelin,OU=Applications,OU=Groups,DC=directory,DC=DC=[domain_here],DC=com":"datascientist"
activeDirectoryRealm.authorizationCachingEnabled =
falseactiveDirectoryRealm.principalSuffix = @directory.mydomain.com...
[roles]
admin = *
datascientist = *developer = *
[urls]uncomment the below urls that you want to hide./api/version = anon
/api/interpreter/** = authc, roles[admin]/** = authc
My AD account is member of
"CN=Zeppelin-Admin,OU=Zeppelin,OU=Applications,OU=Groups,DC=directory,DC=[domain_here],DC=com",
but when I login, I saw followings in the log:
WARN [2017-07-26 00:14:10,981] (
{qtp1287712235-15} LoginRestApi.java[postLogin]:119) -
{"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"b681cbbb-8a10-40c8-9ba8-c46ee59efd42","roles":"[]"}}
please note roles node is empty, I was expecting "admin" in the role list, does
anyone have similar issue? is my config activeDirectoryRealm.groupRolesMap
correct?
Thanks,Richard Xin
Interpreter menu stop working when Active Directory integration enabled
It looks like a bug to me. [ZEPPELIN-2810] Interpreter menu stop working when Active Directory integration enabled - ASF JIRA | | | | [ZEPPELIN-2810] Interpreter menu stop working when Active Directory inte... | | |
Zeppelin's integration with Active Directory integration
I just configured Zeppelin for AD integration, I am facing an issue that it only support full DN as user namefor example:It works when login as "CN=John Doe,OU=Employees,DC=directory,DC=test,DC=com", but login failed when using friendly AD user name such as "john.doe", what's the likely root cause for that? My Zeppelin (0.7.2) is hosted on AWS' EMR. Thanks,Richard Xin
Re: Zeppelin directory structure for S3 notebook storage
correction:I tried both "user" and "username",
and the S3 directory structure is actually like this:
{bucket}/username/notebook/{notebook-id}/note.json instead of advertised
"{bucket}/{username}/{notebook-id}/note.json"
Please advise.Thanks
On Friday, July 21, 2017, 3:02:15 PM PDT, Richard Xin <richardxin...@yahoo.com>
wrote:
oops, accidentally clicked "send" ...
1. enabled default security by renaming shiro.ini (default user admin, user1,2
...)2. enabled S3 storage, and my config in zeppelin-site.xml
zeppelin.notebook.s3.user username
user name for s3 folder structure
I noticed that my s3 has following directory
structure:{bucket}/username/{notebook-id}/ (where username didn't change for
different users)
I was expecting {bucket}/{username}/{notebook-id}/, such
as{bucket}/admin/{notebook-id}/ and {bucket}/user1/{notebook-id}/
Did I miss anything?Thanks
On Friday, July 21, 2017, 2:58:09 PM PDT, Richard Xin <richardxin...@yahoo.com>
wrote:
1. enabled default security by renaming shiro.ini (default user admin, user1,2
...)2. enabled S3 storage, and my config in
zeppelin.notebook.s3.user username
user name for s3 folder structure
Re: Zeppelin directory structure for S3 notebook storage
oops, accidentally clicked "send" ...
1. enabled default security by renaming shiro.ini (default user admin, user1,2
...)2. enabled S3 storage, and my config in zeppelin-site.xml
zeppelin.notebook.s3.user username
user name for s3 folder structure
I noticed that my s3 has following directory
structure:{bucket}/username/{notebook-id}/ (where username didn't change for
different users)
I was expecting {bucket}/{username}/{notebook-id}/, such
as{bucket}/admin/{notebook-id}/ and {bucket}/user1/{notebook-id}/
Did I miss anything?Thanks
On Friday, July 21, 2017, 2:58:09 PM PDT, Richard Xin <richardxin...@yahoo.com>
wrote:
1. enabled default security by renaming shiro.ini (default user admin, user1,2
...)2. enabled S3 storage, and my config in
zeppelin.notebook.s3.user username
user name for s3 folder structure
Zeppelin directory structure for S3 notebook storage
1. enabled default security by renaming shiro.ini (default user admin, user1,2 ...)2. enabled S3 storage, and my config in zeppelin.notebook.s3.user username user name for s3 folder structure
Re: Zeppelin notebook.repo.GitNotebookRepo
Khalid, thanks for the reply.the problem I am facing is we could use either local git or S3 without git support. Any idea how I could get version control on S3? if I change storage to S3, I can't use version control. -Richard On Thursday, July 20, 2017, 12:39:05 AM PDT, Khalid Huseynov <kha...@apache.org> wrote: Hi Richard, Actually that setting is only for local storage that allows git/revision related actions (e.g. commit, change revision). There's a separate setting for S3 though, you can find details in here. On Thu, Jul 20, 2017 at 9:47 AM, Richard Xin <richardxin...@yahoo.com> wrote: for newly installed Zeppelin (0.7.2), I renamed zeppelin-site.xml.template to zeppelin-site.xml, and restart zeppelin, zeppelin-site.xml already have the following enabled: zeppelin.notebook. storage org.apache.zeppelin. notebook.repo.GitNotebookRepo< /value> versioned notebook persistence layer implementation looks like local git repo is enabled, can this be point to remote git repo? such as github.com or even S3 will be great.Thanks,Richard
Why anonymous login disabled for zeppelin (EMR)
I started AWS EMR instance, initially anonymous login was enabled, i renamed zeppelin-site.xml.template to zeppelin-site.xml and restarted zeppelin, I noticed that anonymous login was disabled. shouldn't config in zeppelin-site.xml.template be all default values? and I checked the file, it does have this: zeppelin.anonymous.allowed true Anonymous user allowed by default Thanks in advance for the help.
Re: restarting zeppelin on EMR causing exceptions
thanks, Jonathan, the Zeppelin's native restart command (sudo
zeppelin-daemon.sh restart) doesn't work for EMR, I saw error "channel 7: open
failed: connect failed: Connection refused" using native restart command, it
seems that it doesn't stop all dependency process cleanly.
after running native "sudo zeppelin-daemon.sh restart" on EMR, the cluster is
not easy to recover, I tried to kill -9 the zeppelin process and restart using
sudo start on EMR, but still not workingsudo stop/start zeppelin does not have
issues, works as expected.
-Richard
On Wednesday, July 19, 2017, 5:35:57 PM PDT, Jonathan Kelly
<jonathaka...@gmail.com> wrote:
EMR uses upstart to manage all daemons/services running on the cluster, so the
best way to restart Zeppelin on EMR is to use "sudo stop zeppelin; sudo start
zeppelin". (Note: normally you'd use "sudo restart", but this doesn't actually
work due to the way that upstart is configured for all of these processes on
EMR.)
~ Jonathan
On Wed, Jul 19, 2017 at 4:11 PM Steven Kirtzic
<steven.kirtzic.f...@statefarm.com> wrote:
Hey Moon,
Can you remove me from this mailing list? Thanks,
-Steven
From: moon soo Lee <m...@apache.org>
Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Date: Wednesday, July 19, 2017 at 5:39 PM
To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Subject: Re: restarting zeppelin on EMR causing exceptions
Hi,
Can you try restart Zeppelin with 'zeppelin' user permission?
sudo -u zeppelin /usr/lib/zeppelin/bin/zeppelin-daemon.sh restart
Thanks,
moon
On Wed, Jul 19, 2017 at 3:21 PM Richard Xin <richardxin...@yahoo.com> wrote:
I was doing config changes on EMR (EMR Release label:emr-5.7.0) Zeppelin
(Zeppelin 0.7.2) and struggled with Zeppelin server restart, finally I was able
to narrow down to the steps to reproduce below (it's reproducible for any newly
created EMR if following the steps below):
1) provision a EMR cluster (1 master node + 2 core nodes) with Spark, Hadoop,
hive and Zeppelin
2) ssh to the master node
3) curl http://ec2-34-209-164-132.us-west-2.compute.amazonaws.com:8890/#/
4) sudo /usr/lib/zeppelin/bin/zeppelin-daemon.sh restart
5) do #3 again
you will see errors:
HTTP ERROR: 503
Problem accessing /. Reason:
Service Unavailable
6) tail Zeppelin log under /var/log/zeppelin/
you will see following errors:
WARN [2017-07-19 21:34:57,425] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED org.eclipse.jetty.server.Server@4ed38226: java.net.BindException:
Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:189)
Zeppelin notebook.repo.GitNotebookRepo
for newly installed Zeppelin (0.7.2), I renamed zeppelin-site.xml.template to zeppelin-site.xml, and restart zeppelin, zeppelin-site.xml already have the following enabled: zeppelin.notebook.storage org.apache.zeppelin.notebook.repo.GitNotebookRepo versioned notebook persistence layer implementation looks like local git repo is enabled, can this be point to remote git repo? such as github.com or even S3 will be great.Thanks,Richard
Re: restarting zeppelin on EMR causing exceptions
nope, not working
[ec2-user@ip-xxx-xxx-xx-xxx ~]$ sudo -u zeppelin
/usr/lib/zeppelin/bin/zeppelin-daemon.sh restart
find: failed to restore initial working directory: Permission denied
Zeppelin is not running
Zeppelin start [ OK ]
Zeppelin process died [FAILED]
On Wednesday, July 19, 2017, 3:39:37 PM PDT, moon soo Lee <m...@apache.org>
wrote:
Hi,
Can you try restart Zeppelin with 'zeppelin' user permission?
sudo -u zeppelin /usr/lib/zeppelin/bin/zeppelin-daemon.sh restart
Thanks,
moon
On Wed, Jul 19, 2017 at 3:21 PM Richard Xin <richardxin...@yahoo.com> wrote:
I was doing config changes on EMR (EMR Release label:emr-5.7.0) Zeppelin
(Zeppelin 0.7.2) and struggled with Zeppelin server restart, finally I was able
to narrow down to the steps to reproduce below (it's reproducible for any newly
created EMR if following the steps below):
1) provision a EMR cluster (1 master node + 2 core nodes) with Spark, Hadoop,
hive and Zeppelin
2) ssh to the master node
3) curl http://ec2-34-209-164-132.us-west-2.compute.amazonaws.com:8890/#/
4) sudo /usr/lib/zeppelin/bin/zeppelin-daemon.sh restart
5) do #3 again
you will see errors:
HTTP ERROR: 503
Problem accessing /. Reason:
Service Unavailable
6) tail Zeppelin log under /var/log/zeppelin/
you will see following errors:
WARN [2017-07-19 21:34:57,425] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED org.eclipse.jetty.server.Server@4ed38226: java.net.BindException:
Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:189)
restarting zeppelin on EMR causing exceptions
I was doing config changes on EMR (EMR Release label:emr-5.7.0) Zeppelin
(Zeppelin 0.7.2) and struggled with Zeppelin server restart, finally I was able
to narrow down to the steps to reproduce below (it's reproducible for any newly
created EMR if following the steps below):
1) provision a EMR cluster (1 master node + 2 core nodes) with Spark, Hadoop,
hive and Zeppelin
2) ssh to the master node
3) curl http://ec2-34-209-164-132.us-west-2.compute.amazonaws.com:8890/#/
4) sudo /usr/lib/zeppelin/bin/zeppelin-daemon.sh restart
5) do #3 again
you will see errors:
HTTP ERROR: 503
Problem accessing /. Reason:
Service Unavailable
6) tail Zeppelin log under /var/log/zeppelin/
you will see following errors:
WARN [2017-07-19 21:34:57,425] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED org.eclipse.jetty.server.Server@4ed38226: java.net.BindException:
Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:189)
Re: can Zeppelin runParagraph from different notebooks?
Thanks Jeff,It worked in some cases, However it seems it has following
issues:example:in Note A I have following paragraph:val message = "Hello
Zeppelin"
println("Note 1 = " + message)
Test Case 1===
in Note B: Paragraph 1:%spark
z.run("2CN3UDXMZ", "20170612-231131_191205958")
// works but it does not print out message "Note 1 = " ...
Paragraph 2:
println("P2: message == " + message)
// Works OK
Test Case 2===
in Note B:
Paragraph 1:%spark
z.run("2CN3UDXMZ", "20170612-231131_191205958")
println("Note2: " + message)
//:316: error: not found: value message// newly defined variables can
not be used in the same paragraph of z.run?
My question is, is the behavior I described expected? Or did I missing
something obvious?
Thanks,RichardX
On Monday, June 26, 2017, 3:23:52 PM PDT, Jianfeng (Jeff) Zhang
<jzh...@hortonworks.com> wrote:
z.run
Best Regard,Jeff Zhang
From: Richard Xin <richardxin...@yahoo.com>
Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Date: Tuesday, June 27, 2017 at 6:04 AM
To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>, "m...@apache.org"
<m...@apache.org>
Subject: Re: can Zeppelin runParagraph from different notebooks?
%spark
run("2CN3UDXMZ", "20170609-233658_1498522009"); (which is spark script written
in scala)
:288: error: not found: value runrun("2CN3UDXMZ",
"20170609-233658_1498522009");^
Zeppelin Version 0.7.1
Did I miss anything?Thanks,
On Tuesday, June 13, 2017, 8:48:54 PM PDT, Jianfeng (Jeff) Zhang
<jzh...@hortonworks.com> wrote:
Please use 0.7.x
Best Regard,Jeff Zhang
From: Richard Xin <richardxin...@yahoo.com>
Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Date: Wednesday, June 14, 2017 at 8:48 AM
To: "m...@apache.org" <m...@apache.org>, "users@zeppelin.apache.org"
<users@zeppelin.apache.org>
Subject: Re: can Zeppelin runParagraph from different notebooks?
it doesn't work, which version you are using?
On Tuesday, June 13, 2017, 5:41:06 PM PDT, moon soo Lee <m...@apache.org> wrote:
In spark interpreter, you can try
%sparkrun(NOTE_ID, PARAGRAPH_ID)
Hope this helps.
Thanks,moon
On Mon, Jun 12, 2017 at 9:52 AM Richard Xin <richardxin...@yahoo.com> wrote:
Angular (frontend API)
|
|
|
| | |
|
|
|
| |
Angular (frontend API)
The Apache Software Foundation
Angular (frontend API)
|
|
|
This method doesn't seem to support running paragraphs from different
notebooks, did I miss anything?
Re: can Zeppelin runParagraph from different notebooks?
%spark
run("2CN3UDXMZ", "20170609-233658_1498522009"); (which is spark script written
in scala)
:288: error: not found: value run run("2CN3UDXMZ",
"20170609-233658_1498522009"); ^
Zeppelin Version 0.7.1
Did I miss anything?Thanks,
On Tuesday, June 13, 2017, 8:48:54 PM PDT, Jianfeng (Jeff) Zhang
<jzh...@hortonworks.com> wrote:
Please use 0.7.x
Best Regard,Jeff Zhang
From: Richard Xin <richardxin...@yahoo.com>
Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org>
Date: Wednesday, June 14, 2017 at 8:48 AM
To: "m...@apache.org" <m...@apache.org>, "users@zeppelin.apache.org"
<users@zeppelin.apache.org>
Subject: Re: can Zeppelin runParagraph from different notebooks?
it doesn't work, which version you are using?
On Tuesday, June 13, 2017, 5:41:06 PM PDT, moon soo Lee <m...@apache.org> wrote:
In spark interpreter, you can try
%sparkrun(NOTE_ID, PARAGRAPH_ID)
Hope this helps.
Thanks,moon
On Mon, Jun 12, 2017 at 9:52 AM Richard Xin <richardxin...@yahoo.com> wrote:
Angular (frontend API)
|
|
|
| | |
|
|
|
| |
Angular (frontend API)
The Apache Software Foundation
Angular (frontend API)
|
|
|
This method doesn't seem to support running paragraphs from different
notebooks, did I miss anything?
Re: NPE in SparkInterpreter.java
just try to understand the comment in the code:// Some case,
scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws an NPE
what are the "Some case", it sounds like Zeppelin developers are not sure how
it happened and how to prevent it?
I am trying to make a strong case for my company to switch from other notebook
application to Zeppelin, Zeppelin looks good and only this issue concerns me.
I'm looking forward for any insights, thanks.On Monday, June 26, 2017, 11:56:45
AM PDT, Serega Sheypak <serega.shey...@gmail.com> wrote:
Hi, I have more or less the same symptom
if (Utils.isScala2_10()) {
binder = (Map<String, Object>) getValue("_binder");
} else {
binder = (Map<String, Object>) getLastObject();
}
binder.put("sc", sc); // EXCEPTION HERE
java.lang.NullPointerException at
org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:887) at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:489)
at org.apache.zeppelin.scheduler.Job.run(Job.java:181) at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at
java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I'm going to check this one
stuff:https://stackoverflow.com/questions/43289067/getting-nullpointerexception-when-running-spark-code-in-zeppelin-0-7-1
2017-06-26 20:41 GMT+02:00 Richard Xin <richardxin...@yahoo.com>:
Anyone?
On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin
<richardxin...@yahoo.com> wrote:
apache/zeppelin
|
|
|
| | |
|
|
|
| |
apache/zeppelin
zeppelin - Mirror of Apache Zeppelin
|
|
|
in the code I see // Some case, scala.tools.nsc.interpreter.
IMain$ReadEvalPrint.call throws an NPE
We had issues that Zeppelin note errored out immediately without any hint
messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin
0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for this
error and how to prevent it from happening again?
Thanks,
Re: NPE in SparkInterpreter.java
Anyone? On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin <richardxin...@yahoo.com> wrote: apache/zeppelin | | | | | | | | | | | apache/zeppelin zeppelin - Mirror of Apache Zeppelin | | | in the code I see // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws an NPE We had issues that Zeppelin note errored out immediately without any hint messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for this error and how to prevent it from happening again? Thanks,
NPE in SparkInterpreter.java
apache/zeppelin | | | | | | | | | | | apache/zeppelin zeppelin - Mirror of Apache Zeppelin | | | in the code I see // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws an NPE We had issues that Zeppelin note errored out immediately without any hint messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for this error and how to prevent it from happening again? Thanks,
Zeppelin 0.6.0 (on EMR) stops responding after several runs (NPE error), and back online after restart zeppelin
it happened several times already, worked again after restart Zeppelin
I see consistently similar error when died
ERROR [2017-06-14 17:59:59,705] ({pool-2-thread-2}
SparkInterpreter.java[putLatestVarInResourcePool]:1253) -
java.lang.NullPointerException
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
at
org.apache.zeppelin.spark.SparkInterpreter.getLastObject(SparkInterpreter.java:1114)
at
org.apache.zeppelin.spark.SparkInterpreter.putLatestVarInResourcePool(SparkInterpreter.java:1249)
at
org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:1232)
at
org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:1144)
at
org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:1137)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:95)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:490)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
Re: can Zeppelin runParagraph from different notebooks?
it doesn't work, which version you are using? On Tuesday, June 13, 2017, 5:41:06 PM PDT, moon soo Lee <m...@apache.org> wrote: In spark interpreter, you can try %sparkrun(NOTE_ID, PARAGRAPH_ID) Hope this helps. Thanks,moon On Mon, Jun 12, 2017 at 9:52 AM Richard Xin <richardxin...@yahoo.com> wrote: Angular (frontend API) | | | | | | | | | | | Angular (frontend API) The Apache Software Foundation Angular (frontend API) | | | This method doesn't seem to support running paragraphs from different notebooks, did I miss anything?
