[GitHub] zeppelin pull request #3060: [ZEPPELIN-3600] Add REST API to change logger l...
Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/3060 ---
[jira] [Created] (ZEPPELIN-3617) Allow to specify saving resourceId as paragraph property
Jeff Zhang created ZEPPELIN-3617: Summary: Allow to specify saving resourceId as paragraph property Key: ZEPPELIN-3617 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3617 Project: Zeppelin Issue Type: Improvement Reporter: Jeff Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin pull request #3044: ZEPPELIN-3563. Add pool to paragraph property t...
Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/3044 ---
[GitHub] zeppelin issue #3064: [ZEPPELIN-3596] Saving resources from pool to SQL
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3064 Thanks @oxygen311 for the contribution, but I think it is already done in ZEPPELIN-3377 via ZeppelinContext variable interpolation ---
Re: [DISCUSS] Add Language Server Protocol to Apache Zeppelin
HI Maxim, Thanks for the info. I take a look at the LSP site, but didn't find the implementation for SQL, I guess you are working on sql implementation. Here's my several concerns. 1. How much effort to integrate it into zeppelin ? My current thinking is that we could make AutoComplete as plugin that user can choose whatever implementing, such as the zeppelin interpreter built-in or the LSP. 2. How much better is LSP compared to the current auto completion feature. e.g. Auto completion of scala is based on scala shell, and this works well. 3. What is the licence of LSP ? 4. What is the binary size of LSP if we include it into binary distribution of zeppelin ? Belousov Maksim Eduardovich 于2018年7月11日周三 下午7:35写道： > Hello, dev! > > > Our team would like to add LSP [1] to Zeppelin to get more perfect > autocomplete and other functionality. > We are working on SQL language server and want to implement it in JDBC > interpreter. > > Let see python interpreter. > Now ipython(ipyspark) give good autocomplete out of box. > There are two ways when LSP in Zeppelin will available: > 1. user or operations team will add and configure desired LSP server > 2. Zeppelin will include one of the python language server, for example [2] > > > What way is more appropriate? > Is LSP will be excessive in Zeppelin? > > > > > 1. https://langserver.org > 2. https://github.com/palantir/python-language-server? > > > > Regards, > > Maxim >
[GitHub] zeppelin issue #3060: [ZEPPELIN-3600] Add REST API to change logger level dy...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/3060 Will merge it ---
[jira] [Created] (ZEPPELIN-3616) Paragraph editor sections auto-collapse
Paul Brenner created ZEPPELIN-3616: -- Summary: Paragraph editor sections auto-collapse Key: ZEPPELIN-3616 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3616 Project: Zeppelin Issue Type: Bug Affects Versions: 0.8.0 Reporter: Paul Brenner We are using 0.8 release and noticed that the editor section of paragraphs will randomly collapse when you leave a notebook open for a while. Clicking "hide editor" followed by "show editor" will bring them back but it is quite annoying and wasn't there in the pre-release version we were using. Currently we have not figured out a specific process to reproduce this. Will update this ticket if we can figure out how to reliably reproduce. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin issue #3055: ZEPPELIN-3587. Interpret paragarph text as whole code ...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3055 ping @felixcheung ---
[jira] [Created] (ZEPPELIN-3615) Add data types to z.show(df)
Alexey Zabelkin created ZEPPELIN-3615: - Summary: Add data types to z.show(df) Key: ZEPPELIN-3615 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3615 Project: Zeppelin Issue Type: Improvement Affects Versions: 0.8.0 Reporter: Alexey Zabelkin In pyton/ipython/pyspark/ipyspark there is a z.show(df) function that prints out the Pandas DataFrame results. It's good to extend this function to know data types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3614) Helium visualizations for offline (disconnected from the internet) environment
Rani Y. created ZEPPELIN-3614: - Summary: Helium visualizations for offline (disconnected from the internet) environment Key: ZEPPELIN-3614 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3614 Project: Zeppelin Issue Type: Bug Components: helium Affects Versions: 0.9.0 Reporter: Rani Y. I've downloaded and compiled 0.9. I cannot find the helium directory, or how to use visualization offline in those settings. Is there any short explanation on how to download, and add to local registry visualizations ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin pull request #3064: [ZEPPELIN-3596] Saving resources from pool to S...
GitHub user oxygen311 opened a pull request:
https://github.com/apache/zeppelin/pull/3064
[ZEPPELIN-3596] Saving resources from pool to SQL
### What is this PR for?
> It's information in resourcePool, but it is only available for viewing.
It would be nice if we can save this data to SQL via JDBC Driver.
Now you can access to `ResourcePool` data from SQL query. For example, if
you write `SELECT * FROM
{ResourcePool.note_id=SOME_NOTE_ID.paragraph_id=SOME_PARAGRAPH_ID};` and it
will be correct SQL query.
How it works:
1. Creates and fills a table named like a `PARAGRAPH_ID`;
2. Expression `{*}` replaces with actual table name;
3. Updated SQL query is running.
Also, you can not specify the note_id, then the note_id of the current
notebook will be used. For example, `SELECT * FROM
{ResourcePool.paragraph_id=SOME_PARAGRAPH_ID};`.
### What type of PR is it?
Improvement
### What is the Jira issue?
[Zeppelin
3596](https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-3596?filter=allopenissues)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/TinkoffCreditSystems/zeppelin ZEPPELIN-3596
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/3064.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3064
commit 6033d8fddb4b7f21f268a79143baa5903bf6c759
Author: oxygen311
Date: 2018-07-10T10:18:37Z
Add `preparePoolData` function
commit 882a4ee9aa128d2d06d85841a4ea50e4d01f4aaa
Author: oxygen311
Date: 2018-07-10T16:06:45Z
Code refactoring
commit af8747b6c19e7ab94e23352ab8162d72c9a19fe0
Author: oxygen311
Date: 2018-07-11T09:41:28Z
Refactoring
commit 1a32689b101d5974f5e31d1d883befe9d0142eec
Author: oxygen311
Date: 2018-07-11T11:41:03Z
Add tests
commit d5e57ad5cc5e19b77aded00efcc2a567283f1b1b
Author: oxygen311
Date: 2018-07-11T11:55:47Z
Remove assignment operation from cycle
---
[DISCUSS] Add Language Server Protocol to Apache Zeppelin
Hello, dev! Our team would like to add LSP [1] to Zeppelin to get more perfect autocomplete and other functionality. We are working on SQL language server and want to implement it in JDBC interpreter. Let see python interpreter. Now ipython(ipyspark) give good autocomplete out of box. There are two ways when LSP in Zeppelin will available: 1. user or operations team will add and configure desired LSP server 2. Zeppelin will include one of the python language server, for example [2] What way is more appropriate? Is LSP will be excessive in Zeppelin? 1. https://langserver.org 2. https://github.com/palantir/python-language-server? Regards, Maxim
[jira] [Created] (ZEPPELIN-3613) Cluster Session module design
liuxun created ZEPPELIN-3613: Summary: Cluster Session module design Key: ZEPPELIN-3613 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3613 Project: Zeppelin Issue Type: Sub-task Components: zeppelin-server Affects Versions: 0.9.0 Reporter: liuxun Fix For: 0.9.0 h3. Since the Interpreter process in Zeppelin is a resident process, the Zeppelin-Server service natively saves the runtime environment of the user, Note, and the Thrift remote connection of the corresponding interpreter process through the Session, but in a distributed environment, you It is not possible to determine which Zeppelin-Server service the user will log in to through the domain name, so the Zeppelin-Server service must be able to support users who are still on the A server to continue to use it in the B server. For this reason, we have two implementations: # Cluster unified cache Session The Session created in each Zeppelin-Server service is stored in a unified manner through Cluster MetaData, and the session information is obtained from Cluster MetaData regardless of which Zeppelin-Server service the user logs in. Advantages and Disadvantages: The code size is large, and it is complicated to synchronize the session state logic in each server. The metadata stored in Cluster MetaData is also more; # Rebuild Session The user who originally uses the A server will have the user's session. When the user logs in to the B server, the Zeppelin-Server service of the B server re-establishes the user in the B server according to the user and the Note information combined with the Interpreter metadata in the Cluster MetaData. Session Advantages and Disadvantages: The code implementation is relatively simple, and there is no need to synchronize the session state in each server. The process of rebuilding the Session is very fast and simple; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3612) Cluster High availability module design
liuxun created ZEPPELIN-3612: Summary: Cluster High availability module design Key: ZEPPELIN-3612 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3612 Project: Zeppelin Issue Type: Sub-task Components: zeppelin-server Affects Versions: 0.9.0 Reporter: liuxun Fix For: 0.9.0 h3. In the case of a partial Zeppelin-Server service process or server exception, the service can continue to be served; the Zeppelin-Server service can sense the availability of all Interpreter processes in the server cluster; # *Raft protocol* The Raft protocol ensures that only N/2+1 servers in the cluster need to be in a normal state without affecting the service. # *Interpreter process monitoring* The Interpreter process creates a process heartbeat thread through the ClusterMonitor class, and periodically sends the Interpreter process heartbeat information and the IP and port information of the Thrift interface to the cluster. When the Interpreter process is closed, the process of deleting the process metadata is sent to the cluster. The process health check thread is created in the Zeppelin-Server through the ClusterMonitor class. The heartbeat of all Interpreter processes in the Cluster MetaData is periodically checked. If the timeout expires, the process metadata is deleted to prevent the Interpreter process from being abnormal. # *Interpreter process rebuild* When the interpreter process is created, the Zeppelin-Server detects the session information of the Interpreter process. First, it checks whether the process is valid. If it is not available, the corresponding session is cleared and the Interpreter process is re-created. Preventing an Interpreter process from being abnormal or a server exception on the process causes Interpreter to be unavailable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3611) High utilization module design
liuxun created ZEPPELIN-3611: Summary: High utilization module design Key: ZEPPELIN-3611 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3611 Project: Zeppelin Issue Type: Sub-task Components: zeppelin-server Affects Versions: 0.9.0 Reporter: liuxun Fix For: 0.9.0 h3. When you create an Interpreter process in Zeppelin-Server, you can create an Interpreter process in another Zeppelin-Server from a remote server with cluster resources, and establish a session. # *Server resource reporting* Zeppelin-Server collects server hardware CPU and memory usage, periodically reports to the cluster, and stores metadata in Cluster MetaData. # *Create an interpreter process remotely* When Zeppelin-Server creates an Interpreter process, first find out the server with idle resources from Cluster MetaData, and create an Interpreter process remotely through Thrift. # *Reuse the Interpreter process* When Zeppelin-Server creates an Interpreter process, it first looks up the Cluster MetaData to see if the required Interpreter process already exists. If it already exists, it obtains the IP and port of the Interpreter process's Thrift through the Cluster MetaData to create the user's note and Interpreter. Session association; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3610) Cluster Raft module design
liuxun created ZEPPELIN-3610: Summary: Cluster Raft module design Key: ZEPPELIN-3610 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3610 Project: Zeppelin Issue Type: Sub-task Components: zeppelin-server Affects Versions: 0.9.0 Reporter: liuxun Fix For: 0.9.0 By using the Raft protocol, multiple Zeppelin-Server groups are built into a Zeppelin cluster, the cluster State Machine is maintained through the Raft protocol, and the services in the cluster are agreed upon. The Zeppelin-Server and Zeppelin-Interperter services and processes are stored in the Cluster MetaData. Metadata information; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-3609) Refactor `ZeppelinServer` to use `jersey-hk2`
Jongyoul Lee created ZEPPELIN-3609: -- Summary: Refactor `ZeppelinServer` to use `jersey-hk2` Key: ZEPPELIN-3609 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3609 Project: Zeppelin Issue Type: Improvement Reporter: Jongyoul Lee `hk2` is already included in `zeppelin-server` module. we can simplify initialization code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin issue #3057: [ZEPPELIN 3582] Add type data to result of query from ...
Github user oxygen311 commented on the issue: https://github.com/apache/zeppelin/pull/3057 @zjffdu we plan to do it in next pool requests. ---
[GitHub] zeppelin pull request #3063: ZEPPELIN-3606. Reduce verbose InterpreterNotFou...
Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/3063 ---
[jira] [Created] (ZEPPELIN-3608) Call getEditorSetting only when space is typed
Jeff Zhang created ZEPPELIN-3608: Summary: Call getEditorSetting only when space is typed Key: ZEPPELIN-3608 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3608 Project: Zeppelin Issue Type: Improvement Reporter: Jeff Zhang Currently, when I type "%spark", zeppelin would try to get editor setting for `s`, `sp`, `spa`, `spar`, `spark`. Which could cause many exception in zeppelin server side (ZEPPELIN-3606). Actually it is not necessary to call getEditorSetting so many times, we should call it after user type space, that means we should only call it after `%spark ` -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zeppelin issue #3063: ZEPPELIN-3606. Reduce verbose InterpreterNotFoundExcep...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/3063 Will merge it soon as it is a very easy fix. ---
[GitHub] zeppelin pull request #3063: ZEPPELIN-3606. Reduce verbose InterpreterNotFou...
GitHub user zjffdu opened a pull request: https://github.com/apache/zeppelin/pull/3063 ZEPPELIN-3606. Reduce verbose InterpreterNotFoundException for getEditorSetting ### What is this PR for? Simple PR to reduce the verbose exception for getEditorSetting. ### What type of PR is it? [Bug Fix | Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3606 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-3606 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/3063.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3063 commit cdea27da6cff7e1b8159d01cc8f1010551493ff5 Author: Jeff Zhang Date: 2018-07-11T05:29:12Z ZEPPELIN-3606. Reduce verbose InterpreterNotFoundException for getEdtiorSetting ---
[GitHub] zeppelin issue #3019: [ZEPPELIN-3540] Show Spark inline UI in Non-Secure mod...
Github user shahsank3t commented on the issue: https://github.com/apache/zeppelin/pull/3019 @jongyoul @zjffdu @prabhjyotsingh Updated the code by reusing modal and other available methods. Please review it. ---
[jira] [Created] (ZEPPELIN-3607) Tab completion doesn't work when no interpreter is specified
Jeff Zhang created ZEPPELIN-3607:
Summary: Tab completion doesn't work when no interpreter is
specified
Key: ZEPPELIN-3607
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3607
Project: Zeppelin
Issue Type: Bug
Reporter: Jeff Zhang
How to reproduce.
1. create note using spark as the default interpreter.
2. run the following code without specifying interpreter (that means it would
use spark interpreter)
{code}
sc.version
{code}
3. Then delete `version`, and type tab. No completion returned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] zeppelin issue #3060: [ZEPPELIN-3600] Change logger level dynamically
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/3060 maybe it's better to clarify that we are adding the Admin REST API in this change and also allow logging lever to be changed? ---
