date:20160728

Re: Drill JDBC Interpreter: Connection Refused

2016-07-28 Thread Alexander Bezzubov

Hi Kevin,

this is very strange indeed.

Could you please try to delete ./logs/* restart Zeppelin and try again, and
then post 2 full log files, of both zeppelin server and drill interpreter
I.e on gist or pastebin and share a link here?

This should help to debug the situation.

--
Alex


On Fri, Jul 29, 2016, 01:17 Kevin Verhoeven 
wrote:

> Thanks Jongyoul,
>
>
>
> I tried your suggestion and used %drill with:
>
>
>
> drill.driver
>
> drill.url
>
> drill.user
>
> drill.password
>
>
>
> However, the query still failed with a “Connection refused” error.
>
>
>
> My drill.url is: jdbc:drill:zk=localhost:2181/drill/drillbits1
>
>
>
> ZooKeeper and Drill both work, I’ve run many queries through Drill’s UI.
> Only Zeppelin throws an error.
>
>
>
> Do you have any other suggestions?
>
>
>
> Thanks,
>
>
>
> Kevin
>
>
>
> The error I found in the log:
>
>
>
> ERROR [2016-07-28 09:13:37,639] ({Thread-97}
> RemoteScheduler.java[getStatus]:256) - Can't get status information
>
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:96)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:254)
>
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:212)
>
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>
> ... 8 more
>
> Caused by: java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:589)
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
>
> ... 9 more
>
>
>
> *From:* Jongyoul Lee [mailto:jongy...@gmail.com]
> *Sent:* Thursday, July 28, 2016 3:02 AM
>
>
> *To:* users@zeppelin.apache.org
> *Subject:* Re: Drill JDBC Interpreter: Connection Refused
>
>
>
> Hello Kevin,
>
>
>
> I've got to know what the problem is.
>
>
>
> If you are running query with "%drill ...", you can set
>
>
>
> drill.url
>
> drill.user
>
> drill.password
>
>
>
> If you set default.*, you should use "%jdbc ..."
>
>
>
> It's a little bit confused, and it will be improved by 0.7.0.
>
>
>
> Hope this help,
>
> Jongyoul
>
>
>
> On Thu, Jul 28, 2016 at 7:56 AM, Kevin Verhoeven <
> kevin.verhoe...@ds-iq.com> wrote:
>
> I have the same problem. I am running Zeppelin version 0.6.0 and Drill
> 1.5. When I attempt a Drill query from Zeppelin I receive a "Connection
> refused" error. The query works successfully in Drill. My Interpreter
> configurations:
>
>
>
> default.url: jdbc:drill:zk=my*ServerName*:8121/drill/drillbits1
>
> default.user:
>
> default.password:
>
>
>
> Dependency: org.apache.drill.exec:drill-jdbc:1.5.0
>
>
>
> The query I run is very simple, but the Zeppelin paragraph runs for a few
> seconds and then returns the connection refused error. I pasted the full
> error below.
>
>
>
> Any recommendations?
>
>
>
> Thanks!
>
>
>
> Kevin
>
>
>
> Zeppelin Paragraph:
>
>
>
> %drill
>
> SELECT * FROM asdf LIMIT 100
>
>
>
> Zeppelin Error:
>
>
>
> java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
>
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>
> at
> java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
>

Re: classnotfoundexception using zeppelin with spark authentication

2016-07-28 Thread Jonathan Esterhazy

yes, this PR fixes my classnotfoundexceptions. awesome!

any idea when it might make it into a release?

On Wed, Jul 27, 2016 at 11:25 PM, Jeff Zhang  wrote:

> I think it is fixed in https://github.com/apache/zeppelin/pull/1237,
> please try that.
>
> On Thu, Jul 28, 2016 at 1:58 AM, Jonathan Esterhazy <
> jonathan.esterh...@gmail.com> wrote:
>
>> I just tried this with using zeppelin master, and the classnotfound error
>> is still there.
>>
>>
>> On Thu, Jul 7, 2016 at 7:07 AM, Hyung Sung Shim 
>> wrote:
>>
>>> Hi. Jonathan.
>>> If it is possible, could you use the zeppelin master on the github?
>>> Zeppelin is working well with spark authentication when i tested.
>>>
>>>
>>>
>>> 2016-07-02 3:05 GMT+09:00 Jonathan Esterhazy <
>>> jonathan.esterh...@gmail.com>:
>>>
 Hyung, thx for your help. I opened these:

 https://issues.apache.org/jira/browse/ZEPPELIN-1096 (this scala
 problem)
 https://issues.apache.org/jira/browse/ZEPPELIN-1097 (similar looking
 python problem)

 LMK if I can provide more info or help in some way.

 On Fri, Jul 1, 2016 at 5:08 AM, Hyung Sung Shim 
 wrote:

> Hi Jonathan.
> Unfortunately I got same error in my test bed.
> Do you mind create an jira issue for this?
>
> 2016-07-01 3:19 GMT+09:00 Jonathan Esterhazy <
> jonathan.esterh...@gmail.com>:
>
>> I added this and still get the same exception. The same property is
>> also set in spark-defaults.conf.
>>
>> After that didn't work, I also tried adding --conf
>> spark.authenticate=true and --conf
>> spark.authenticate.enableSaslEncryption=true, to match the other related
>> settings in spark-defaults.conf. Still get the same 
>> classnotfoundexception.
>>
>>
>> On Thu, Jun 30, 2016 at 10:45 AM, Hyung Sung Shim 
>> wrote:
>>
>>> Please add the *export SPARK_SUBMIT_OPTIONS="--conf
>>> spark.authenticate.secret=secret"* in conf/zeppelin-env.sh and
>>> restart zeppelin and retry your code.
>>>
>>>
>>> 2016-06-30 23:34 GMT+09:00 Jonathan Esterhazy <
>>> jonathan.esterh...@gmail.com>:
>>>
 yes it does. i only see this problem in zeppelin.

 On Thu, Jun 30, 2016 at 7:05 AM, Hyung Sung Shim  wrote:

> Hi Jonathan.
> It's not easy to build the test environments but I am working on
> this.
> I have question for you.
> Does your code working well on spark-shell in the
> spark.authenticate mode?
>
> 2016-06-30 22:47 GMT+09:00 Jonathan Esterhazy <
> jonathan.esterh...@gmail.com>:
>
>> Hyung, did you have any luck w/ zeppelin + spark authentication?
>> I'm quite stumped.
>>
>> thx.
>>
>> On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim <
>> hss...@nflabs.com> wrote:
>>
>>> Thank you.
>>> Let me try.
>>>
>>> 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy <
>>> jonathan.esterh...@gmail.com>:
>>>
 Hyung,

 Yes, here they are.

 zeppelin-env.sh:

 export ZEPPELIN_PORT=8890
 export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf
 export ZEPPELIN_LOG_DIR=/var/log/zeppelin
 export ZEPPELIN_PID_DIR=/var/run/zeppelin
 export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid
 export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook
 export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps
 export MASTER=yarn-client
 export SPARK_HOME=/usr/lib/spark
 export HADOOP_CONF_DIR=/etc/hadoop/conf
 export
 CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/*
 :/usr/share/aws/emr/emrfs/auxlib/*"
 export JAVA_HOME=/usr/lib/jvm/java-1.8.0
 export ZEPPELIN_NOTEBOOK_S3_BUCKET=mybucket
 export ZEPPELIN_NOTEBOOK_S3_USER=zeppelin
 export
 ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo

 spark-defaults.conf:

 spark.master yarn
 spark.driver.extraClassPath
  
 /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf

 :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/*
 spark.driver.extraLibraryPath
  /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native
 spark.executor.extraClassPath

Re: Pass Credentials through JDBC

2016-07-28 Thread Benjamin Kim

Thank you. I’ll take a look.


> On Jul 28, 2016, at 8:16 AM, Jongyoul Lee  wrote:
> 
> You can find more information on 
> https://issues.apache.org/jira/browse/ZEPPELIN-1146 
> 
> 
> Hope this help,
> Jongyoul
> 
> On Fri, Jul 29, 2016 at 12:08 AM, Benjamin Kim  > wrote:
> Hi Jonyoul,
> 
> How would I enter credentials with the current version of Zeppelin? Do you 
> know of a way to make it work now?
> 
> Thanks,
> Ben
> 
>> On Jul 28, 2016, at 8:06 AM, Jongyoul Lee > > wrote:
>> 
>> Hi,
>> 
>> In my plan, this is a next step after 
>> https://issues.apache.org/jira/browse/ZEPPELIN-1210 
>> . But for now, there's 
>> no way to pass your credentials with hiding them. I hope that would be 
>> included in 0.7.0.
>> 
>> Regards,
>> Jongyoul
>> 
>> On Thu, Jul 28, 2016 at 11:22 PM, Benjamin Kim > > wrote:
>> How do I pass username and password to JDBC connections such as Phoenix and 
>> Hive that are my own? Can my credentials be passed from Shiro after logging 
>> in? Or do I have to set them at the Interpreter level without sharing them? 
>> I wish there was more information on this.
>> 
>> Thanks,
>> Ben
>> 
>> 
>> 
>> -- 
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net 
> 
> 
> 
> 
> -- 
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net

RE: Drill JDBC Interpreter: Connection Refused

2016-07-28 Thread Kevin Verhoeven

Thanks Jongyoul,

I tried your suggestion and used %drill with:

drill.driver
drill.url
drill.user
drill.password

However, the query still failed with a “Connection refused” error.

My drill.url is: jdbc:drill:zk=localhost:2181/drill/drillbits1

ZooKeeper and Drill both work, I’ve run many queries through Drill’s UI. Only 
Zeppelin throws an error.

Do you have any other suggestions?

Thanks,

Kevin

The error I found in the log:

ERROR [2016-07-28 09:13:37,639] ({Thread-97} 
RemoteScheduler.java[getStatus]:256) - Can't get status information
org.apache.zeppelin.interpreter.InterpreterException: 
org.apache.thrift.transport.TTransportException: java.net.ConnectException: 
Connection refused
at 
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at 
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at 
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at 
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at 
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at 
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at 
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:96)
at 
org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:254)
at 
org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:212)
Caused by: org.apache.thrift.transport.TTransportException: 
java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at 
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 8 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 9 more

From: Jongyoul Lee [mailto:jongy...@gmail.com]
Sent: Thursday, July 28, 2016 3:02 AM
To: users@zeppelin.apache.org
Subject: Re: Drill JDBC Interpreter: Connection Refused

Hello Kevin,

I've got to know what the problem is.

If you are running query with "%drill ...", you can set

drill.url
drill.user
drill.password

If you set default.*, you should use "%jdbc ..."

It's a little bit confused, and it will be improved by 0.7.0.

Hope this help,
Jongyoul

On Thu, Jul 28, 2016 at 7:56 AM, Kevin Verhoeven 
> wrote:
I have the same problem. I am running Zeppelin version 0.6.0 and Drill 1.5. 
When I attempt a Drill query from Zeppelin I receive a "Connection refused" 
error. The query works successfully in Drill. My Interpreter configurations:

default.url: jdbc:drill:zk=myServerName:8121/drill/drillbits1
default.user:
default.password:

Dependency: org.apache.drill.exec:drill-jdbc:1.5.0

The query I run is very simple, but the Zeppelin paragraph runs for a few 
seconds and then returns the connection refused error. I pasted the full error 
below.

Any recommendations?

Thanks!

Kevin

Zeppelin Paragraph:

%drill
SELECT * FROM asdf LIMIT 100

Zeppelin Error:

java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at 
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at 
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
at 
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
at 
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at 
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at 
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at 
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at

Re: Pass Credentials through JDBC

2016-07-28 Thread Jongyoul Lee

You can find more information on
https://issues.apache.org/jira/browse/ZEPPELIN-1146

Hope this help,
Jongyoul

On Fri, Jul 29, 2016 at 12:08 AM, Benjamin Kim  wrote:

> Hi Jonyoul,
>
> How would I enter credentials with the current version of Zeppelin? Do you
> know of a way to make it work now?
>
> Thanks,
> Ben
>
> On Jul 28, 2016, at 8:06 AM, Jongyoul Lee  wrote:
>
> Hi,
>
> In my plan, this is a next step after
> https://issues.apache.org/jira/browse/ZEPPELIN-1210. But for now, there's
> no way to pass your credentials with hiding them. I hope that would be
> included in 0.7.0.
>
> Regards,
> Jongyoul
>
> On Thu, Jul 28, 2016 at 11:22 PM, Benjamin Kim  wrote:
>
>> How do I pass username and password to JDBC connections such as Phoenix
>> and Hive that are my own? Can my credentials be passed from Shiro after
>> logging in? Or do I have to set them at the Interpreter level without
>> sharing them? I wish there was more information on this.
>>
>> Thanks,
>> Ben
>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Setting default interpreter at notebook level

2016-07-28 Thread Jongyoul Lee

Hi Abul,

Concerning "defaultInterpreter", it is a feature of current master and
doesn't work at 0.6.0. Sorry for wrong information. And for now, we don't
have any specific plan for supporting different default interpreter with
same interpreter setting. Thus, in your case, %r tags is a proper way for
now. I also don't think it's the best. I hope Zeppelin support this
feature, too.

Regards,
Jongyoul

On Thu, Jul 28, 2016 at 10:54 PM, Abul Basar  wrote:

> Hello Jongyoul,
>
> I could not find the file interpreter-setting.json, but I found a file
> conf/interpreter.json. I added a property "default":"true"
> for interpreter "org.apache.zeppelin.spark.SparkRInterpreter". I restarted
> zeppelin demon service. But, R did not work as default interpreter.
>
> Then I changed conf/zeppelin-site.xml to alter the sequence of
> the interpreter, putting org.apache.zeppelin.spark.SparkRInterpreter as
> first interpreter. This worked. But this solution is not practical. If want
> to work on a R notebook and a scala notebook in parallel, this mechanism
> require me to touch conf/zeppelin-site.xml file or keep on using %r tags
> on each cell.
>
> Thanks!
>
> -AB
>
> On Mon, Jul 25, 2016 at 3:30 PM, Jongyoul Lee  wrote:
>
>> Hello Abul,
>>
>> Changing orders within a group dynamically is not supported yet. You can
>> change it by making interpreter-setting.json in a resources directory In a
>> interpreter-setting.json, you can find a property named `default`. If it's
>> true, That will be a default interpreter in a group. If you don't want to
>> compile Zeppelin again, copy interpreter-setting.json into
>> interpreter/spark/ and open it and change it. It will also have a same
>> effect.
>>
>> Hope this help,
>> Jongyoul
>>
>> On Mon, Jul 25, 2016 at 4:39 PM, Abul Basar  wrote:
>>
>>> Hi Krishnaprasad,
>>>
>>> Yes, I have played around with that feature. What I found is "spark,
>>> pyspark, r, sql" are grouped together. I use Zeppelin for Spark projects.
>>> So I need to set one of these sub-categories as default. Most often I use
>>> scala for Spark. But I should be able to create a notebook using r (which
>>> essentially is SparkR) as a default. Please let me know if I am missing
>>> something.
>>>
>>> Thanks!
>>> - AB
>>>
>>> On Mon, Jul 25, 2016 at 12:45 PM, Krishnaprasad A S <
>>> krishna.pra...@flytxt.com> wrote:
>>>
 Hi Abul,
  You can change the default interpreter for each notebook through
 zeppelin web UI.
 Go to the notebook and then settings(up right corner), there you can
 find Interpreter binding option. You can reorder the interpreters by drag
 and drop. The first one will be default.

 Hope this helps.

 Regards,
 Krishnaprasad

 On Mon, Jul 25, 2016 at 12:01 PM, Abul Basar  wrote:

> I know there is a way to set up a default interpreter at Zepplin using 
> zeppelin.interpreters
> property in conf/zeppelin-site.xml. The setting is global is nature.
>
> But, is it possible to create a notebook level setting for
> interpreter? For example, in a notebook I want to set the default
> interpreter at R so that for every code block i do not have to start with
> "%spark.r", while on another notebook, I want to set the default
> interpreter as Scala.
>
> I am using v0.6
>
> AB
>



 --
 Krishnaprasad A S
 Lead Engineer
 Flytxt
 Skype: krishnaprasadas
 M: +91 8907209454 | O: +91 471.3082753 | F: +91 471.2700202
 www.flytxt.com | Visit our blog  | Follow us
  | Connect on LinkedIn
 

>>>
>>>
>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>
>


-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Pass Credentials through JDBC

2016-07-28 Thread Jongyoul Lee

Hi,

In my plan, this is a next step after
https://issues.apache.org/jira/browse/ZEPPELIN-1210. But for now, there's
no way to pass your credentials with hiding them. I hope that would be
included in 0.7.0.

Regards,
Jongyoul

On Thu, Jul 28, 2016 at 11:22 PM, Benjamin Kim  wrote:

> How do I pass username and password to JDBC connections such as Phoenix
> and Hive that are my own? Can my credentials be passed from Shiro after
> logging in? Or do I have to set them at the Interpreter level without
> sharing them? I wish there was more information on this.
>
> Thanks,
> Ben

-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Issue with Spark + Zeppelin on Mesos - Failed to create local dir

2016-07-28 Thread Jongyoul Lee

It looks like error of Spark's side. Does it works normally by spark shell?

On Thu, Jul 28, 2016 at 7:23 AM, Michael Sells  wrote:

> Trying to get Zeppelin running on Mesos and I'm consistently hitting the
> following error when I try to create a dataframe/rdd from a file.
>
> java.io.IOException: Failed to create local dir in
> /tmp/blockmgr-82f31798-dd17-4907-a039-d1c90bf12a80/0e.
> at
> org.apache.spark.storage.DiskBlockManager.getFile(DiskBlockManager.scala:73)
> at org.apache.spark.storage.DiskStore.contains(DiskStore.scala:161)
> at org.apache.spark.storage.BlockManager.org
> $apache$spark$storage$BlockManager$$getCurrentBlockStatus(BlockManager.scala:391)
> at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:817)
> at
> org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:645)
> at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1003)
>
> Running Mesos 0.28, Zeppelin 0.6.0, Spark 1.6.1. This seems to happen
> whenever I try to read data from any source. Errors out just trying to
> create a dataframe or rdd like:
>
> sc.textFile("s3://filepath")
>
> Any pointers on what might be off here? I've tried changing the temp dir
> around and opening permissions. Everything I see indicates it should be
> able to write there. Any help would be appreciated.
>
> Thanks,
> Mike
>



-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Drill JDBC Interpreter: Connection Refused

Re: classnotfoundexception using zeppelin with spark authentication

Re: Pass Credentials through JDBC

RE: Drill JDBC Interpreter: Connection Refused

Re: Pass Credentials through JDBC

Re: Setting default interpreter at notebook level

Re: Pass Credentials through JDBC

Re: Issue with Spark + Zeppelin on Mesos - Failed to create local dir

8 matches

Site Navigation

Mail list logo

Footer information