[jira] [Created] (ZEPPELIN-1797) CI build fails on DependencyResolverTest.testLoad \w spark-csv
Alexander Bezzubov created ZEPPELIN-1797: Summary: CI build fails on DependencyResolverTest.testLoad \w spark-csv Key: ZEPPELIN-1797 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1797 Project: Zeppelin Issue Type: Bug Affects Versions: 0.7.0 Reporter: Alexander Bezzubov Few PRs failed on Spark 1.x profiles \w Scala 2.10 on {{DependencyResolverTest.testLoad}} loading spark-csv_2.10 - PR [https://github.com/apache/zeppelin/pull/1689|#1689], [https://api.travis-ci.org/jobs/183505569/log.txt?deansi=true|logs] - PR [|#], [|logs] It might be actually an expected behavior as profiles \w Scala 2.11 do fail tests \w Scala 2.10 artefacts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1793) Import\export between Zeppelin and Jupyter notebook formats
Alexander Bezzubov created ZEPPELIN-1793: Summary: Import\export between Zeppelin and Jupyter notebook formats Key: ZEPPELIN-1793 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1793 Project: Zeppelin Issue Type: Improvement Reporter: Alexander Bezzubov Fix For: 0.7.0 As a user doing data analytics in Notebook environment, I would like not to be locked up with a single notebook implementation. Right now, Apache Zeppelin has it's own notebook format {{note.json}}, and it is different from a Jupyther one, so user is locked up. In order to avoid it, Zeppelin user should be able to: - import new Jupyter\IPython {.ipnb} notebook to Zeppelin, so it gets converted to a {note.json} and saved under {{./notebooks/}} - export existing Zeppelin notebook as Jupyter format, so it gets converted and downloaded as Jupyter {{.ipnb}} file This can be further broken down to sub-tasks (in order of priority): - backend: implement an {{.ipnb <-> note.json}} convertor, \w UnitTests - backend: implement REST API for import\export notebooks in Zeppelin - frontend: add GUI for Import/Export to Zeppelin webapp As soon as backend is ready, it will be easy for find people working on frontend, willing to contribute Import/Export GUI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1761) CI: pySparkDepLoaderTest fails the build
Alexander Bezzubov created ZEPPELIN-1761: Summary: CI: pySparkDepLoaderTest fails the build Key: ZEPPELIN-1761 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1761 Project: Zeppelin Issue Type: Bug Reporter: Alexander Bezzubov Test in {{zeppelin-server}} fails {code} Results : Failed tests: ZeppelinSparkClusterTest.pySparkDepLoaderTest:430 expected: but was: Tests run: 82, Failures: 1, Errors: 0, Skipped: 1 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project zeppelin-server: There are test failures. [ERROR] [ERROR] Please refer to /home/travis/build/apache/zeppelin/zeppelin-server/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] {code} Failures: - PR [#1731|https://github.com/apache/zeppelin/pull/1731], [logs|https://api.travis-ci.org/jobs/181725932/log.txt?deansi=true] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1738) IgniteSqlInterpreterTest fails the build \w Scala 2.11
Alexander Bezzubov created ZEPPELIN-1738: Summary: IgniteSqlInterpreterTest fails the build \w Scala 2.11 Key: ZEPPELIN-1738 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1738 Project: Zeppelin Issue Type: Bug Reporter: Alexander Bezzubov Build \w Scala 2.11 fails {code} ./dev/change_scala_version.sh 2.11 mvn clean package -Pspark-2.0 -Phadoop-2.3 -Ppyspark -Psparkr -Pscalding -Pexamples -Pscala-2.11 -pl \!zeppelin-web,\!zeppelin-distribution [15:25:59] Ignite node stopped OK [name=test, uptime=00:00:00:106] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.75 sec <<< FAILURE! - in org.apache.zeppelin.ignite.IgniteSqlInterpreterTest testSql(org.apache.zeppelin.ignite.IgniteSqlInterpreterTest) Time elapsed: 1.51 sec <<< FAILURE! java.lang.AssertionError: expected: but was: at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at org.apache.zeppelin.ignite.IgniteSqlInterpreterTest.testSql(IgniteSqlInterpreterTest.java:95) Results : Failed tests: IgniteSqlInterpreterTest.testSql:95 expected: but was: Tests run: 4, Failures: 1, Errors: 0, Skipped: 0 {code} Reproduced locally and on [CI|] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1703) More OpenPower support: skip PhantomJS installation if tests are skipped
Alexander Bezzubov created ZEPPELIN-1703: Summary: More OpenPower support: skip PhantomJS installation if tests are skipped Key: ZEPPELIN-1703 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1703 Project: Zeppelin Issue Type: Improvement Components: front-end Affects Versions: 0.6.2 Reporter: Alexander Bezzubov Assignee: Alexander Bezzubov Fix For: 0.7.0 Right now zeppelin-web {{mvn package}} runs {{npm install}} and alwasy downloads\installs PhantomJS with lots of native dependencies, which do not play nice with non-intel architectures. As discussed with [~asanjar] - it would be nice to let users build Zeppelin frontend and skip this headache, if the tests are skipped though {{mvn package -DskipTests}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1656) z.show in Python interpreter does not work
Alexander Bezzubov created ZEPPELIN-1656: Summary: z.show in Python interpreter does not work Key: ZEPPELIN-1656 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1656 Project: Zeppelin Issue Type: Bug Components: python-interpreter Affects Versions: 0.7.0 Reporter: Alexander Bezzubov Fix For: 0.7.0 After ZEPPELIN-1345, even without {{py4j}} installed, simple paragraphs that use `z.show()` can not be run {{code}} import pandas as pd df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None) df.columns=['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth', 'Name'] z.show(df) {{code}} results in {{code}} raceback (most recent call last): File "", line 1, in AttributeError: 'list' object has no attribute 'show' Traceback (most recent call last): File "", line 1, in AttributeError: 'list' object has no attribute '_displayhook' {{code}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1655) Dynamic forms in Python interpreter do not work
Alexander Bezzubov created ZEPPELIN-1655: Summary: Dynamic forms in Python interpreter do not work Key: ZEPPELIN-1655 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1655 Project: Zeppelin Issue Type: Bug Components: python-interpreter Affects Versions: 0.7.0 Reporter: Alexander Bezzubov Fix For: 0.7.0 ZEPPELIN-1345 updates [Python `PyZeppelinContext` | https://github.com/apache/zeppelin/blob/master/python/src/main/resources/bootstrap.py#L121] and [Pyspark `PyZeppelinContext ` |https://github.com/apache/zeppelin/blob/master/spark/src/main/resources/python/zeppelin_pyspark.py#L53] but does not touch [Python `Py4jZeppelinContext`|https://github.com/apache/zeppelin/blob/master/python/src/main/resources/bootstrap_input.py#L28] - wich is used for dynamic forms, in case `py4j` is installed on the system for things like ``` z.select("Choose a letter", ([a,"a"], [b,"b"], [c,"c"] ``` In latter case Python interpreter becomes not useable - any line results in ``` x = 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'Py4jZeppelinContext' object has no attribute '_displayhook' ``` -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1646) Machine Learning workflow in Apache Zeppelin
Alexander Bezzubov created ZEPPELIN-1646: Summary: Machine Learning workflow in Apache Zeppelin Key: ZEPPELIN-1646 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1646 Project: Zeppelin Issue Type: Bug Reporter: Alexander Bezzubov This is umbrella for all automation\features we would like to have to accomodate machine learning workflow in Zeppelin notebook-style environment. Here is a rough ideas list: - training progress indicator in GUI - model visualizations - model deployment automation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1361) Python interpreter - add support for AngularDisplay system
Alexander Bezzubov created ZEPPELIN-1361: Summary: Python interpreter - add support for AngularDisplay system Key: ZEPPELIN-1361 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1361 Project: Zeppelin Issue Type: Improvement Components: python-interpreter Reporter: Alexander Bezzubov Fix For: 0.7.0 As discussed on mailing list dev@z.a.o subj "improving matplotlib integration in zeppelin" - in order to complete ZEPPELIN-1344 and give ability for one Python paragraph to update content in another one - we can use AngularDisplay system, same as in Apache Spark interpreter. {code} So without rerun another paragraph, it's possible to update result of a paragraph from another. Any interpreter can get AngularObjectRegistry[3] from InterpreterContext [4], and AngularObjectRegistry allows create object / add event hook to communicate with front-end. [3] https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/display/AngularObjectRegistry.java [4] https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterContext.java#L115 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1347) Release 0.6.2
Alexander Bezzubov created ZEPPELIN-1347: Summary: Release 0.6.2 Key: ZEPPELIN-1347 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1347 Project: Zeppelin Issue Type: Wish Reporter: Alexander Bezzubov Fix For: 0.6.2 There are few quite important bugs found in 0.6.1 (linked to this issue) it would be great to have them resolved ASAP and release 0.6.2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1325) Better Python interpreter impl
Alexander Bezzubov created ZEPPELIN-1325: Summary: Better Python interpreter impl Key: ZEPPELIN-1325 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1325 Project: Zeppelin Issue Type: Improvement Components: python-interpreter Affects Versions: 0.7.0 Reporter: Alexander Bezzubov We have basic Python interpreter implementation, but there are number issues that are very hard to address keeping it's existing architecture. Without changing user-facing APIs (like `z.show`, `%python.sql` etc) the goal here is to have a more stable and generic basic Python interpreter implementation, that - accept all valid python syntax ZEPPELIN-1133 - have accurate runtime error detection ZEPPELIN-1114 - support windows platform ZEPPELIN-893 - provides ResourcePool support (`z.get` & `z.set`) The approach I'm exploring is: to use custom python script that {code}eval(paragrpah){code} in the Python process and communicates to Interpreter process, instead of just reading\writing to stdin\stout of the default repl. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1286) CI for Spark 1.6 fails on master & all PRs
Alexander Bezzubov created ZEPPELIN-1286: Summary: CI for Spark 1.6 fails on master & all PRs Key: ZEPPELIN-1286 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1286 Project: Zeppelin Issue Type: Bug Reporter: Alexander Bezzubov 2nd CI profile, responsible for Spark 1.6 fails on all PRs {code} Running org.apache.zeppelin.spark.SparkInterpreterTest SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/travis/.m2/repository/org/slf4j/slf4j-log4j12/1.7.10/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/travis/build/apache/zeppelin/zeppelin-interpreter/target/zeppelin-interpreter-0.7.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/travis/build/apache/zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] log4j:WARN No appenders could be found for logger (org.apache.zeppelin.interpreter.Interpreter). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Tests run: 14, Failures: 0, Errors: 14, Skipped: 0, Time elapsed: 10.953 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkInterpreterTest emptyConfigurationVariablesOnlyForNonSparkProperties(org.apache.zeppelin.spark.SparkInterpreterTest) Time elapsed: 7.078 sec <<< ERROR! java.lang.NullPointerException: null at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:308) at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:220) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:731) at org.apache.zeppelin.spark.SparkInterpreterTest.setUp(SparkInterpreterTest.java:93) shareSingleSparkContext(org.apache.zeppelin.spark.SparkInterpreterTest) Time elapsed: 2.309 sec <<< ERROR! java.lang.NullPointerException: null at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:308) at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:220) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:731) at org.apache.zeppelin.spark.SparkInterpreterTest.shareSingleSparkContext(SparkInterpreterTest.java:245) testBasicIntp(org.apache.zeppelin.spark.SparkInterpreterTest) Time elapsed: 0.002 sec <<< ERROR! Tests in error: SparkInterpreterTest.setUp:93 » NullPointer SparkInterpreterTest.shareSingleSparkContext:245 » NullPointer SparkInterpreterTest.testBasicIntp:136 » NullPointer SparkInterpreterTest.testCreateDataFrame:182 » NullPointer SparkInterpreterTest.testDisableImplicitImport:280 » NullPointer SparkInterpreterTest.testEnableImplicitImport:264 » NullPointer SparkInterpreterTest.testEndWithComment:171 » NullPointer SparkInterpreterTest.testListener:176 » NullPointer SparkInterpreterTest.testNextLineComments:160 » NullPointer SparkInterpreterTest.testNextLineCompanionObject:166 » NullPointer SparkInterpreterTest.testNextLineInvocation:155 » NullPointer SparkInterpreterTest.testReferencingUndefinedVal:220 » NullPointer SparkInterpreterTest.testSparkSql:200 » NullPointer SparkInterpreterTest.testZShow:193 » NullPointer SparkSqlInterpreterTest.setUp:61 » NullPointer SparkSqlInterpreterTest.setUp:61 » NullPointer SparkSqlInterpreterTest.setUp:61 » NullPointer Tests run: 22, Failures: 0, Errors: 17, Skipped: 0 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1115) Python: add SQL for DataFrame support with Table Display system
Alexander Bezzubov created ZEPPELIN-1115: Summary: Python: add SQL for DataFrame support with Table Display system Key: ZEPPELIN-1115 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1115 Project: Zeppelin Issue Type: Improvement Components: python-interpreter Reporter: Alexander Bezzubov Assignee: Alexander Bezzubov Fix For: 0.7.0 In spark interpreter group we have {{%sql}} interpreter which supports Table Display system, we use it in the tutorial notebook and it's very convenient for data explorations. The idea is to have the same kind of support for Python interpreter group i.e `%python.sql` but only for Pandas [DataFrame](https://github.com/yhat/pandasql). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1073) Falky Test - ParagraphActionsIT.testClearOutputButton
Alexander Bezzubov created ZEPPELIN-1073: Summary: Falky Test - ParagraphActionsIT.testClearOutputButton Key: ZEPPELIN-1073 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1073 Project: Zeppelin Issue Type: Bug Reporter: Alexander Bezzubov CI fails i.e on - [#1094|https://github.com/apache/zeppelin/pull/1094], [logs|https://s3.amazonaws.com/archive.travis-ci.org/jobs/140563773/log.txt] {code} Driver info: driver.version: unknown at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206) at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158) at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:647) at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:353) at org.openqa.selenium.remote.RemoteWebDriver.findElementByXPath(RemoteWebDriver.java:490) at org.openqa.selenium.By$ByXPath.findElement(By.java:361) at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:345) at org.apache.zeppelin.AbstractZeppelinIT$1.apply(AbstractZeppelinIT.java:98) at org.apache.zeppelin.AbstractZeppelinIT$1.apply(AbstractZeppelinIT.java:96) at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:238) at org.apache.zeppelin.AbstractZeppelinIT.pollingWait(AbstractZeppelinIT.java:96) at org.apache.zeppelin.AbstractZeppelinIT.waitForParagraph(AbstractZeppelinIT.java:70) at org.apache.zeppelin.integration.ParagraphActionsIT.testClearOutputButton(ParagraphActionsIT.java:285) Caused by: org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"xpath","selector":"(//div[@ng-controller=\"ParagraphCtrl\"])[1]//div[contains(@class, 'control')]//span[1][contains(.,'FINISHED')]"} Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 143.455 sec - in org.apache.zeppelin.integration.ZeppelinIT Results : Tests in error: ParagraphActionsIT.testClearOutputButton:285->AbstractZeppelinIT.waitForParagraph:70->AbstractZeppelinIT.pollingWait:96 » Timeout Tests run: 15, Failures: 0, Errors: 1, Skipped: 0 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1048) Python: pandas support using Table display system
Alexander Bezzubov created ZEPPELIN-1048: Summary: Python: pandas support using Table display system Key: ZEPPELIN-1048 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1048 Project: Zeppelin Issue Type: Improvement Reporter: Alexander Bezzubov Assignee: Alexander Bezzubov Zeppelin has [table display system](http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/displaysystem/basicdisplaysystem.html#table) for some interpreters i.e Spark and JDBC, etc. It would be nice to have same level of support for python interpreter using Pandas -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1027) add python tutorial notebook
Alexander Bezzubov created ZEPPELIN-1027: Summary: add python tutorial notebook Key: ZEPPELIN-1027 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1027 Project: Zeppelin Issue Type: Improvement Reporter: Alexander Bezzubov Assignee: Alexander Bezzubov We have Apache Spark and R tutorials, this task is about building one for Python users -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-1026) Pick editor's syntax highlight based on default binded interpreter
Alexander Bezzubov created ZEPPELIN-1026: Summary: Pick editor's syntax highlight based on default binded interpreter Key: ZEPPELIN-1026 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1026 Project: Zeppelin Issue Type: Improvement Reporter: Alexander Bezzubov Depending on what interpreter is binded for the notebook first, syntax of the paragraph editors should be highlighted differently. I.e one might choose %python to be default so we should highlight it as python. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-929) CASE II "Notebook->Create New Note" is not visible after click on Notebooks in header
Alexander Bezzubov created ZEPPELIN-929: --- Summary: CASE II "Notebook->Create New Note" is not visible after click on Notebooks in header Key: ZEPPELIN-929 URL: https://issues.apache.org/jira/browse/ZEPPELIN-929 Project: Zeppelin Issue Type: Sub-task Reporter: Alexander Bezzubov -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-928) CASE I: `ZeppelinIT.testSparkInterpreterDependencyLoading` Save button not found after adding dependence to Spark interpreter
Alexander Bezzubov created ZEPPELIN-928: --- Summary: CASE I: `ZeppelinIT.testSparkInterpreterDependencyLoading` Save button not found after adding dependence to Spark interpreter Key: ZEPPELIN-928 URL: https://issues.apache.org/jira/browse/ZEPPELIN-928 Project: Zeppelin Issue Type: Sub-task Reporter: Alexander Bezzubov -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ZEPPELIN-927) Update Twitter project branding, removing "incubating"
Alexander Bezzubov created ZEPPELIN-927: --- Summary: Update Twitter project branding, removing "incubating" Key: ZEPPELIN-927 URL: https://issues.apache.org/jira/browse/ZEPPELIN-927 Project: Zeppelin Issue Type: Improvement Reporter: Alexander Bezzubov Fix For: 0.6.0 Right now twitter background artwork does include project name an "incubating" label. As soon as project has graduated to TLP, we need to have same artwork but without an "incubating" label. -- This message was sent by Atlassian JIRA (v6.3.4#6332)