date:20160912

[jira] [Created] (ZEPPELIN-1432) Support cancellation of paragraph execution

2016-09-12 Thread Jeff Zhang (JIRA)

Jeff Zhang created ZEPPELIN-1432:


 Summary: Support cancellation of paragraph execution
 Key: ZEPPELIN-1432
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1432
 Project: Zeppelin
  Issue Type: Sub-task
Reporter: Jeff Zhang
Assignee: Jeff Zhang


Although the cancel api of LivyInterpreter is implemented, but it just cancel 
to poll the result, didn't cancel the real execution in livy side. So it would 
better to cancel the execution on livy side as well. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ZEPPELIN-1431) Improvments for LivyInterpreter

2016-09-12 Thread Jeff Zhang (JIRA)

Jeff Zhang created ZEPPELIN-1431:


 Summary: Improvments for LivyInterpreter
 Key: ZEPPELIN-1431
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1431
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Jeff Zhang


Create this umbrella ticket for LivyInterpreter improvements. \cc 
[~prabhjyotsi...@apache.com]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ZEPPELIN-1430) Display appId and webui link in LivyInterpreter's output

2016-09-12 Thread Jeff Zhang (JIRA)

Jeff Zhang created ZEPPELIN-1430:


 Summary: Display appId and webui link in LivyInterpreter's output
 Key: ZEPPELIN-1430
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1430
 Project: Zeppelin
  Issue Type: Improvement
  Components: Interpreters
Affects Versions: 0.6.1
Reporter: Jeff Zhang
Assignee: Jeff Zhang


For now, it is hard to figure out what the yarn application of the livy session 
represent, it would be better to display the appId and webui link in the output 
of  LivyInterpreter for diagnosing purpose. It can also be applied to the 
native SparkInterpreter, but could be done in another ticket. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1420: [ZEPPELIN-1421] Fix dead link in docs/README.md

2016-09-12 Thread zjffdu

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1420
  
Thanks @AhyoungRyu 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1396: [ZEPPELIN-1405] ConnectionPool for JDBCInterpre...

2016-09-12 Thread astroshim

Github user astroshim closed the pull request at:

https://github.com/apache/zeppelin/pull/1396


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1396: [ZEPPELIN-1405] ConnectionPool for JDBCInterpreter.

2016-09-12 Thread astroshim

Github user astroshim commented on the issue:

https://github.com/apache/zeppelin/pull/1396
  
re-trigger CI


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1404: ZEPPELIN-1411. UDF with pyspark not working - object h...

2016-09-12 Thread Leemoonsoo

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1404
  
@zjffdu Right, it looks like AbstractTestRestApi need to be improved when 
CI is not defined.
So far, i think you can try download and run spark standalone cluster in 
this way

```
./testing/downloadSpark.sh 1.6.2 2.6
./testing/startSparkCluster.sh 1.6.2 2.6
```

And then try run the test cases, so `getSparkHome()` can find sparkHome.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1425: Improving Matplotlib Integration with Zeppelin

2016-09-12 Thread Leemoonsoo

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1425
  
Realized that there's a very similar contribution #1422 created few days 
before.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1425: Improving Matplotlib Integration with Zeppelin

2016-09-12 Thread Leemoonsoo

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1425
  
Tried and working well.
Could you try trigger CI again and see if it goes to green?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1413: [MINOR] Format Fix for InterpreterResultTest.java

2016-09-12 Thread Leemoonsoo

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1413
  
Thanks @zjffdu.
LGTM and merge if there're no more dicsussions


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1363: [ZEPPELIN-1371]add text/numeric conversion support to ...

2016-09-12 Thread Peilin-Yang

Github user Peilin-Yang commented on the issue:

https://github.com/apache/zeppelin/pull/1363
  
I made several changes for this commit:

1. put the style from notebook.css to paragraph.css
2. do not make any assumption/auto-detection of the original type of the 
data. Basically I drop the changes made in #1371 
3.  now the data type is column based. I added the validators for numbers 
and dates. Cells that fail to be converted to the required type will be shown 
with red background.

 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1429) Auto completion issue with Spark 2.0

2016-09-12 Thread Cyril Scetbon (JIRA)

Cyril Scetbon created ZEPPELIN-1429:
---

 Summary: Auto completion issue with Spark 2.0
 Key: ZEPPELIN-1429
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1429
 Project: Zeppelin
  Issue Type: Bug
  Components: GUI
Affects Versions: 0.6.1
 Environment: - OsX 10.11.6
Reporter: Cyril Scetbon


Completion does not work in Zeppelin 0.6.1 (at least). [~astroshim] thinks it's 
an issue with spark-2.0 supporting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1400: [ZEPPELIN-1407] Fix Scala 2.11 build

2016-09-12 Thread minahlee

Github user minahlee commented on the issue:

https://github.com/apache/zeppelin/pull/1400
  
@lresende thank you for verification. I am reverting this PR then.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]

2016-09-12 Thread echarles

Github user echarles commented on the issue:

https://github.com/apache/zeppelin/pull/932
  
@bzz pushed my local merge. Anything else that should be done before you 
merge?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1408: [ZEPPELIN-808] Changed Permissions on Notebooks do not...

2016-09-12 Thread kavinkumarks

Github user kavinkumarks commented on the issue:

https://github.com/apache/zeppelin/pull/1408
  
@khalidhuseynov I have made the changes and committed them.I have handled 
the exception in runParagraph() instead of throwing custom exception from 
VFSNotebookRepo class since the exception need to be handled i.e 
FileSystemException (subclass of IOException) is already handled in different 
ways for other scenarios and in different implementations of NotebookRepo.

Thanks,
Kavin
MailTo: kavin.ku...@imaginea.com


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1404: ZEPPELIN-1411. UDF with pyspark not working - o...

2016-09-12 Thread zjffdu

GitHub user zjffdu reopened a pull request:

https://github.com/apache/zeppelin/pull/1404

ZEPPELIN-1411. UDF with pyspark not working - object has no attribute 
'parseDataType'

### What is this PR for?
The root cause is that SQLContext's signature changes in spark 2.0. 
Spark 1.6
```
def __init__(self, sparkContext, sqlContext=None):
```
Spark 2.0
```
def __init__(self, sparkContext, sparkSession=None, jsqlContext=None):
```
So we need to create SQLContext using named parameters, otherwise it would 
take intp.getSQLContext() as sparkSession which cause the issue.

### What type of PR is it?
[Bug Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1411

### How should this be tested?
Tested using the example code in ZEPPELIN-1411. 

### Screenshots (if appropriate)

![image](https://cloud.githubusercontent.com/assets/164491/18260139/9bd702c0-741d-11e6-8b23-946c38a794c3.png)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1411

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1404.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1404


commit ae12ea03737d943ee20ac6398346852046c260d4
Author: Jeff Zhang 
Date:   2016-09-06T02:26:05Z

ZEPPELIN-1411. UDF with pyspark not working - object has no attribute 
'parseDataType'

commit 632f1481c8ec6c9bc818e77f2663f0980be4446b
Author: Jeff Zhang 
Date:   2016-09-09T06:21:37Z

add unit test




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1404: ZEPPELIN-1411. UDF with pyspark not working - o...

2016-09-12 Thread zjffdu

Github user zjffdu closed the pull request at:

https://github.com/apache/zeppelin/pull/1404


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1400: [ZEPPELIN-1407] Fix Scala 2.11 build

2016-09-12 Thread lresende

Github user lresende commented on the issue:

https://github.com/apache/zeppelin/pull/1400
  
I have verified that with that by reverting my changes 
-PScala2.10/Scala2.11 have the proper behavior and my original issue was that 
indeed my build alias had -D for the profile activation. Sorry for the 
confusion, please go ahead and revert this change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1428) Improving Matplotlib integration with zeppelin Pyspark

2016-09-12 Thread Ishmeet Kaur (JIRA)

Ishmeet Kaur created ZEPPELIN-1428:
--

 Summary: Improving Matplotlib integration with zeppelin Pyspark
 Key: ZEPPELIN-1428
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1428
 Project: Zeppelin
  Issue Type: Improvement
Reporter: Ishmeet Kaur


Currently plotting via matplotlib in the zeppelin notebook with the pyspark 
interpreter can be done by 

{code}
%pyspark
import matplotlib
matplotlib.use('Agg') 
import matplotlib.pyplot as plt; plt.rcdefaults()
import numpy as np
import matplotlib.pyplot as plt
import StringIO

def show(p):
img = StringIO.StringIO()
p.savefig(img, format='svg')
img.seek(0)
print "%html " + img.buf + ""

# Example data
people = ('Tom', 'Dick', 'Harry', 'Slim', 'Jim')
y_pos = np.arange(len(people))
performance = 3 + 10 * np.random.rand(len(people))
error = np.random.rand(len(people))

plt.barh(y_pos, performance, xerr=error, align='center', alpha=0.4)
plt.yticks(y_pos, people)
plt.xlabel('Performance')
plt.title('How fast do you want to go today?')

show(plt)
{code}

The show() functionality for plotting thin the PyZeppelinContext class is not 
present in pyspark.
Instead of defining the show() function explicitly,we need to add the 
functionality of show_matplotlib() as in bootstrap.py to the z.show() function.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Issues in using Remote Interpreter

2016-09-12 Thread Sachin Janani

Hi Eric,Sourav,
I tried with latest HEAD and it seems to be broken but it works properly
with zeppelin 0.6.1.

On Mon, Sep 12, 2016 at 6:53 PM, Eric Charles  wrote:

> Another more classical way to achieve this is to start the remote
> interpreter via the shell command (example `./bin/interpreter.sh -p 3034 -d
> markdown`).
>
> When I configure the markdown interpreter to connect to localhost:3034, I
> receive an thrift exception (*). My thrift server and client are normally
> compatible.
> Anyone running remote interperter with latest HEAD?
>
>
>  INFO [2016-09-12 15:22:38,530] ({pool-2-thread-2}
> SchedulerFactory.java[jobFinished]:137) - Job
> paragraph_1423836981412_-1007008116 finished by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter2A94M5J1Z1515
> 930344
> ERROR [2016-09-12 15:22:38,714] ({qtp2076287037-51}
> ResourcePoolUtils.java[getAllResourcesExcept]:64) - Negative length:
> -2147418111
> org.apache.thrift.TApplicationException: Negative length: -2147418111
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:
> 111)
> at org.apache.thrift.TServiceClient.receiveBase(
> TServiceClient.java:71)
> at
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
> Client.recv_resourcePoolGetAll(RemoteInterpreterService.java:486)
> at
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
> Client.resourcePoolGetAll(RemoteInterpreterService.java:474)
> at
> org.apache.zeppelin.resource.ResourcePoolUtils.getAllResourcesExcept(
> ResourcePoolUtils.java:58)
> at
> org.apache.zeppelin.resource.ResourcePoolUtils.getAllResources(
> ResourcePoolUtils.java:36)
> at org.apache.zeppelin.helium.Helium.suggestApp(Helium.java:153)
> at org.apache.zeppelin.rest.HeliumRestApi.suggest(
> HeliumRestApi.java:74)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at
> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(
> AbstractInvoker.java:180)
> at
> org.apache.cxf.service.invoker.AbstractInvoker.
> invoke(AbstractInvoker.java:96)
>
>
>
> On Fri, Aug 19, 2016 at 6:47 PM, Sourav Mazumder <
> sourav.mazumde...@gmail.com> wrote:
>
> > Hi,
> >
> > From the documentation it is not very clear how exactly the remote
> > interpreter works.
> >
> > Here is what I tried. But it didn't work. I am using zeppelin 0.6.1
> binary
> > and spark 2.0.
> >
> > 1. In a remote spark cluster
> > a)  I created a spark interpreter (copying from the default spark
> > interpreter) with master as Yarn-Client
> > b). I started the same in a notebook (ensuring that this interpreter is
> > enabled) using following code -
> > import org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
> >
> > val interpreter = new RemoteInterpreterServer(51000)
> > interpreter.start()
> > c). When I go to the yarn application manager I can see this new
> > interpreter is running. Also the log file shows that the remote
> interpreter
> > has started in 51000 port
> >
> > 2. Next in a client machine (a mac laptop) I started Zeppelin.
> > a) I created a new Spark interpreter copying from existing one. There I
> > enabled the checkbox - Connect to Existing Process and provided host ip
> and
> > port.
> > b) I have kept the master as local
> > c) Now I selected this interpreter for Zeppelin Tutorial notebook and ran
> > the same.
> > d) The first paragraph with spark interpreter (Load Data in Table) worked
> > fine. Also I saw the log in the server shows that it is successfully
> > parsing the bank table.
> > e) But the next paragraph (%sql select age, count(1) value from bank
> where
> > age < 30 group by age order by age) gives error as follwos -
> >
> > Caused by: java.lang.ClassCastException: cannot assign instance of
> > scala.collection.immutable.List$SerializationProxy to field
> > org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$dependencies_ of type
> > scala.collection.Seq in instance of org.apache.spark.rdd.
> MapPartitionsRDD
> >
> > at
> > java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(
> > ObjectStreamClass.java:2089)
> >
> > at
> > java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
> >
> > at
> > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1999)
> >
> > at
> > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
> >
> > at
> > java.io.ObjectInputStream.readOrdinaryObject(
> ObjectInputStream.java:1801)
> >
> > at
> > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> >
> > at
> > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
> >
> > at
> > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
> >
> > at
> >

Re: Issues in using Remote Interpreter

2016-09-12 Thread Hyung Sung Shim

Hello.
ZeppelinServer launch the InterpreterServer just before paragraph job is
started in the code automatically so you don't need to start
InterpreterServer on the paragraph.

I got same error when I was same situation with Eric Charles.
It's a bug i think so could you file the jira issue for this?



2016-09-12 22:23 GMT+09:00 Eric Charles :

> Another more classical way to achieve this is to start the remote
> interpreter via the shell command (example `./bin/interpreter.sh -p 3034 -d
> markdown`).
>
> When I configure the markdown interpreter to connect to localhost:3034, I
> receive an thrift exception (*). My thrift server and client are normally
> compatible.
> Anyone running remote interperter with latest HEAD?
>
>
>  INFO [2016-09-12 15:22:38,530] ({pool-2-thread-2}
> SchedulerFactory.java[jobFinished]:137) - Job
> paragraph_1423836981412_-1007008116 finished by scheduler
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter2A94M5J1Z1515
> 930344
> ERROR [2016-09-12 15:22:38,714] ({qtp2076287037-51}
> ResourcePoolUtils.java[getAllResourcesExcept]:64) - Negative length:
> -2147418111
> org.apache.thrift.TApplicationException: Negative length: -2147418111
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:
> 111)
> at org.apache.thrift.TServiceClient.receiveBase(
> TServiceClient.java:71)
> at
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
> Client.recv_resourcePoolGetAll(RemoteInterpreterService.java:486)
> at
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
> Client.resourcePoolGetAll(RemoteInterpreterService.java:474)
> at
> org.apache.zeppelin.resource.ResourcePoolUtils.getAllResourcesExcept(
> ResourcePoolUtils.java:58)
> at
> org.apache.zeppelin.resource.ResourcePoolUtils.getAllResources(
> ResourcePoolUtils.java:36)
> at org.apache.zeppelin.helium.Helium.suggestApp(Helium.java:153)
> at org.apache.zeppelin.rest.HeliumRestApi.suggest(
> HeliumRestApi.java:74)
> at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at
> org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(
> AbstractInvoker.java:180)
> at
> org.apache.cxf.service.invoker.AbstractInvoker.
> invoke(AbstractInvoker.java:96)
>
>
>
> On Fri, Aug 19, 2016 at 6:47 PM, Sourav Mazumder <
> sourav.mazumde...@gmail.com> wrote:
>
> > Hi,
> >
> > From the documentation it is not very clear how exactly the remote
> > interpreter works.
> >
> > Here is what I tried. But it didn't work. I am using zeppelin 0.6.1
> binary
> > and spark 2.0.
> >
> > 1. In a remote spark cluster
> > a)  I created a spark interpreter (copying from the default spark
> > interpreter) with master as Yarn-Client
> > b). I started the same in a notebook (ensuring that this interpreter is
> > enabled) using following code -
> > import org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
> >
> > val interpreter = new RemoteInterpreterServer(51000)
> > interpreter.start()
> > c). When I go to the yarn application manager I can see this new
> > interpreter is running. Also the log file shows that the remote
> interpreter
> > has started in 51000 port
> >
> > 2. Next in a client machine (a mac laptop) I started Zeppelin.
> > a) I created a new Spark interpreter copying from existing one. There I
> > enabled the checkbox - Connect to Existing Process and provided host ip
> and
> > port.
> > b) I have kept the master as local
> > c) Now I selected this interpreter for Zeppelin Tutorial notebook and ran
> > the same.
> > d) The first paragraph with spark interpreter (Load Data in Table) worked
> > fine. Also I saw the log in the server shows that it is successfully
> > parsing the bank table.
> > e) But the next paragraph (%sql select age, count(1) value from bank
> where
> > age < 30 group by age order by age) gives error as follwos -
> >
> > Caused by: java.lang.ClassCastException: cannot assign instance of
> > scala.collection.immutable.List$SerializationProxy to field
> > org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$dependencies_ of type
> > scala.collection.Seq in instance of org.apache.spark.rdd.
> MapPartitionsRDD
> >
> > at
> > java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(
> > ObjectStreamClass.java:2089)
> >
> > at
> > java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
> >
> > at
> > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1999)
> >
> > at
> > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
> >
> > at
> > java.io.ObjectInputStream.readOrdinaryObject(
> ObjectInputStream.java:1801)
> >
> > at
> > java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> >
> > at
> >

[GitHub] zeppelin issue #930: Make path for .spark-dist folder always accessible [ZEP...

2016-09-12 Thread echarles

Github user echarles commented on the issue:

https://github.com/apache/zeppelin/pull/930
  
Closing this PR, I can not reproduce the issue anymore.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #930: Make path for .spark-dist folder always accessib...

2016-09-12 Thread echarles

Github user echarles closed the pull request at:

https://github.com/apache/zeppelin/pull/930


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Issues in using Remote Interpreter

2016-09-12 Thread Eric Charles

Another more classical way to achieve this is to start the remote
interpreter via the shell command (example `./bin/interpreter.sh -p 3034 -d
markdown`).

When I configure the markdown interpreter to connect to localhost:3034, I
receive an thrift exception (*). My thrift server and client are normally
compatible.
Anyone running remote interperter with latest HEAD?


 INFO [2016-09-12 15:22:38,530] ({pool-2-thread-2}
SchedulerFactory.java[jobFinished]:137) - Job
paragraph_1423836981412_-1007008116 finished by scheduler
org.apache.zeppelin.interpreter.remote.RemoteInterpreter2A94M5J1Z1515930344
ERROR [2016-09-12 15:22:38,714] ({qtp2076287037-51}
ResourcePoolUtils.java[getAllResourcesExcept]:64) - Negative length:
-2147418111
org.apache.thrift.TApplicationException: Negative length: -2147418111
at
org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_resourcePoolGetAll(RemoteInterpreterService.java:486)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.resourcePoolGetAll(RemoteInterpreterService.java:474)
at
org.apache.zeppelin.resource.ResourcePoolUtils.getAllResourcesExcept(ResourcePoolUtils.java:58)
at
org.apache.zeppelin.resource.ResourcePoolUtils.getAllResources(ResourcePoolUtils.java:36)
at org.apache.zeppelin.helium.Helium.suggestApp(Helium.java:153)
at org.apache.zeppelin.rest.HeliumRestApi.suggest(HeliumRestApi.java:74)
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at
org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)



On Fri, Aug 19, 2016 at 6:47 PM, Sourav Mazumder <
sourav.mazumde...@gmail.com> wrote:

> Hi,
>
> From the documentation it is not very clear how exactly the remote
> interpreter works.
>
> Here is what I tried. But it didn't work. I am using zeppelin 0.6.1 binary
> and spark 2.0.
>
> 1. In a remote spark cluster
> a)  I created a spark interpreter (copying from the default spark
> interpreter) with master as Yarn-Client
> b). I started the same in a notebook (ensuring that this interpreter is
> enabled) using following code -
> import org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
>
> val interpreter = new RemoteInterpreterServer(51000)
> interpreter.start()
> c). When I go to the yarn application manager I can see this new
> interpreter is running. Also the log file shows that the remote interpreter
> has started in 51000 port
>
> 2. Next in a client machine (a mac laptop) I started Zeppelin.
> a) I created a new Spark interpreter copying from existing one. There I
> enabled the checkbox - Connect to Existing Process and provided host ip and
> port.
> b) I have kept the master as local
> c) Now I selected this interpreter for Zeppelin Tutorial notebook and ran
> the same.
> d) The first paragraph with spark interpreter (Load Data in Table) worked
> fine. Also I saw the log in the server shows that it is successfully
> parsing the bank table.
> e) But the next paragraph (%sql select age, count(1) value from bank where
> age < 30 group by age order by age) gives error as follwos -
>
> Caused by: java.lang.ClassCastException: cannot assign instance of
> scala.collection.immutable.List$SerializationProxy to field
> org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$dependencies_ of type
> scala.collection.Seq in instance of org.apache.spark.rdd.MapPartitionsRDD
>
> at
> java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(
> ObjectStreamClass.java:2089)
>
> at
> java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1999)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at
> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
>
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
>
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
>
> at
> java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
>
> at java.io.ObjectInputStream.readObject(ObjectInputStream.
> java:371)
>
> at
> scala.collection.immutable.List$SerializationProxy.
> readObject(List.scala:479)
>
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>
> at
>

[GitHub] zeppelin issue #1400: [ZEPPELIN-1407] Fix Scala 2.11 build

2016-09-12 Thread minahlee

Github user minahlee commented on the issue:

https://github.com/apache/zeppelin/pull/1400
  
As moon mentioned, if you use build instruction in 
https://github.com/apache/zeppelin#example it doesn't bring any issue.
One of the reason I changed activation rule in #1251 was that using -D 
won't be scalable for scala-2.12 support in the future.
@lresende if you are not strongly against to build with -P and if you can 
confirm that there is no issue to build using `-Pscala-2.11` not `-Dscala-2.11` 
in c88010fd0a744d026b75d02bc62bc6dc690f0f9a, I would like to revert this change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1427) Scala z.show() doesn't work on v.0.6.1 ?

2016-09-12 Thread Jonathan Type (JIRA)

Jonathan Type created ZEPPELIN-1427:
---

 Summary: Scala z.show() doesn't work on v.0.6.1 ?
 Key: ZEPPELIN-1427
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1427
 Project: Zeppelin
  Issue Type: Bug
  Components: GUI
Affects Versions: 0.6.1
 Environment: zeppelin
Reporter: Jonathan Type
 Fix For: 0.6.1


The Scala Interpreter doesn't allow to show part of tables with command 
z.show(x,5) on the version 0.6.1. 
It was working well with previous version of zeppelin.
Now, it only shows list of variables and type of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1404: ZEPPELIN-1411. UDF with pyspark not working - object h...

2016-09-12 Thread zjffdu

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1404
  
@Leemoonsoo , I follow the above command, but seems it doesn't work. I 
check `AbstractTestRestApi`, It seems pyspark related job would only run either 
in travis CI or set in spark standalone with SPARK_HOME is setup (`pyspark` 
needs to be set as true).  Do I understand correctly ?
```
 // ci environment runs spark cluster for testing
  // so configure zeppelin use spark cluster
  if ("true".equals(System.getenv("CI"))) {
// assume first one is spark
InterpreterSetting sparkIntpSetting = null;
for(InterpreterSetting intpSetting : 
ZeppelinServer.notebook.getInterpreterFactory().get()) {
  if (intpSetting.getName().equals("spark")) {
sparkIntpSetting = intpSetting;
  }
}

// set spark master and other properties
sparkIntpSetting.getProperties().setProperty("master", "spark://" + 
getHostname() + ":7071");
sparkIntpSetting.getProperties().setProperty("spark.cores.max", 
"2");

// set spark home for pyspark
sparkIntpSetting.getProperties().setProperty("spark.home", 
getSparkHome());
pySpark = true;
sparkR = true;

ZeppelinServer.notebook.getInterpreterFactory().restart(sparkIntpSetting.getId());
  } else {
// assume first one is spark
InterpreterSetting sparkIntpSetting = null;
for(InterpreterSetting intpSetting : 
ZeppelinServer.notebook.getInterpreterFactory().get()) {
  if (intpSetting.getName().equals("spark")) {
sparkIntpSetting = intpSetting;
  }
}

String sparkHome = getSparkHome();
if (sparkHome != null) {
  sparkIntpSetting.getProperties().setProperty("master", "spark://" 
+ getHostname() + ":7071");
  sparkIntpSetting.getProperties().setProperty("spark.cores.max", 
"2");
  // set spark home for pyspark
  sparkIntpSetting.getProperties().setProperty("spark.home", 
sparkHome);
  pySpark = true;
  sparkR = true;
}
```
 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1420: [ZEPPELIN-1421] Fix dead link in docs/README.md

2016-09-12 Thread kavinkumarks

Github user kavinkumarks commented on the issue:

https://github.com/apache/zeppelin/pull/1420
  
@AhyoungRyu the changes look good!

Thanks,
Kavin
MailTo: kavin.ku...@imaginea.com


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1432) Support cancellation of paragraph execution

[jira] [Created] (ZEPPELIN-1431) Improvments for LivyInterpreter

[jira] [Created] (ZEPPELIN-1430) Display appId and webui link in LivyInterpreter's output

[GitHub] zeppelin issue #1420: [ZEPPELIN-1421] Fix dead link in docs/README.md

[GitHub] zeppelin pull request #1396: [ZEPPELIN-1405] ConnectionPool for JDBCInterpre...

[GitHub] zeppelin issue #1396: [ZEPPELIN-1405] ConnectionPool for JDBCInterpreter.

[GitHub] zeppelin issue #1404: ZEPPELIN-1411. UDF with pyspark not working - object h...

[GitHub] zeppelin issue #1425: Improving Matplotlib Integration with Zeppelin

[GitHub] zeppelin issue #1425: Improving Matplotlib Integration with Zeppelin

[GitHub] zeppelin issue #1413: [MINOR] Format Fix for InterpreterResultTest.java

[GitHub] zeppelin issue #1363: [ZEPPELIN-1371]add text/numeric conversion support to ...

[jira] [Created] (ZEPPELIN-1429) Auto completion issue with Spark 2.0

[GitHub] zeppelin issue #1400: [ZEPPELIN-1407] Fix Scala 2.11 build

[GitHub] zeppelin issue #932: Replace CXF with Jersey2 [ZEPPELIN-903]

[GitHub] zeppelin issue #1408: [ZEPPELIN-808] Changed Permissions on Notebooks do not...

[GitHub] zeppelin pull request #1404: ZEPPELIN-1411. UDF with pyspark not working - o...

[GitHub] zeppelin pull request #1404: ZEPPELIN-1411. UDF with pyspark not working - o...

[GitHub] zeppelin issue #1400: [ZEPPELIN-1407] Fix Scala 2.11 build

[jira] [Created] (ZEPPELIN-1428) Improving Matplotlib integration with zeppelin Pyspark

Re: Issues in using Remote Interpreter

Re: Issues in using Remote Interpreter

[GitHub] zeppelin issue #930: Make path for .spark-dist folder always accessible [ZEP...

[GitHub] zeppelin pull request #930: Make path for .spark-dist folder always accessib...

Re: Issues in using Remote Interpreter

[GitHub] zeppelin issue #1400: [ZEPPELIN-1407] Fix Scala 2.11 build

[jira] [Created] (ZEPPELIN-1427) Scala z.show() doesn't work on v.0.6.1 ?

[GitHub] zeppelin issue #1404: ZEPPELIN-1411. UDF with pyspark not working - object h...

[GitHub] zeppelin issue #1420: [ZEPPELIN-1421] Fix dead link in docs/README.md

28 matches

Site Navigation

Mail list logo

Footer information