[GitHub] zeppelin issue #497: ZEPPELIN-143: Git as a versioned notebook storage
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/497 Haven't had the time to look into that further. TBH, I don't think storing notebooks as text files is a good strategy - not in the environment we use Zeppelin, anyway. We're likely moving towards some sort of centralized storage (maybe a KV storage), in order to be able to scale horizontally ---
[GitHub] zeppelin issue #2599: [ZEPPELIN-2952] encrypt credentials.json with AES
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2599 done :) ---
[GitHub] zeppelin pull request #2599: [ZEPPELIN-2952] encrypt credentials.json with A...
GitHub user herval opened a pull request: https://github.com/apache/zeppelin/pull/2599 [ZEPPELIN-2952] encrypt credentials.json with AES ### What is this PR for? Support encrypting passwords using a private key ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2952 ### How should this be tested? - Set the env variable `ZEPPELIN_CREDENTIALS_ENCRYPT_KEY=something` - Save a few credentials - Check that the `credentials.json` file is storing encrypted passwords - Restart server using the same env variable for `ZEPPELIN_CREDENTIALS_ENCRYPT_KEY` - The credentials should still be decryptable ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes You can merge this pull request into a Git repository by running: $ git pull https://github.com/herval/zeppelin encrypt-credentials Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2599.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2599 commit c3e0ead0a4ba18d1131ffa008858593030b4776a Author: Herval Freire <hfre...@twitter.com> Date: 2017-08-04T17:51:37Z encrypt credentials.json with AES ---
[jira] [Created] (ZEPPELIN-2952) Support encrypting passwords on credentials.json
Herval Freire created ZEPPELIN-2952: --- Summary: Support encrypting passwords on credentials.json Key: ZEPPELIN-2952 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2952 Project: Zeppelin Issue Type: Improvement Reporter: Herval Freire -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zeppelin issue #2440: [ZEPPELIN-2587] allow logging in if you're anonymous
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2440 Hi folks, Please let me know if I should close this. Tks! ---
[GitHub] zeppelin issue #2498: [ZEPPELIN-2796] Fetching LDAP roles
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2498 Hi folks, Please let me know if I should close this. Tks! ---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 Hi folks, Please let me know if I should close this. Tks! ---
[GitHub] zeppelin pull request #2480: [ZEPPELIN-1249] build all submodules w/ the sam...
Github user herval closed the pull request at: https://github.com/apache/zeppelin/pull/2480 ---
[GitHub] zeppelin issue #2480: [ZEPPELIN-1249] build all submodules w/ the same scala...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2480 @zjffdu what's the benefit of *not* doing it, given it's a simple change? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2503: [ZEPPELIN-2808] remember me support
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2503 any arguments against following the current way most sites do it (which is having "remember me" on by default, instead of a checkbox option)? Showing/hiding the checkbox depending on a config will introduce quite a lot of config code, so unless there's a reasoning not to always support remembering credentials, I'd say it's a more sensible way to go --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2145: [ZEPPELIN-2271] encoding password of credentials
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2145 https://github.com/herval/zeppelin/tree/encrypt-credentials --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2503: [ZEPPELIN-2808] remember me support
Github user herval commented on the issue:
https://github.com/apache/zeppelin/pull/2503
Hmm It shouldnât work without the Cookie manager. The checkbox will still
show, though, which begs the question if it should be a default? (Otherwise
weâll have to hide the checkbox based on some shiro config)
From: Kun <notificati...@github.com>
Sent: Friday, August 4, 2017 7:24:19 PM
To: apache/zeppelin
Cc: Herval Freire; Author
Subject: Re: [apache/zeppelin] [ZEPPELIN-2808] remember me support (#2503)
@1ambda commented on this pull request.
In
conf/shiro.ini.template<https://github.com/apache/zeppelin/pull/2503#discussion_r131512560>:
> @@ -46,6 +46,12 @@ user3 = password4, role2
#ldapRealm.userDnTemplate = uid={0},ou=Users,dc=COMPANY,dc=COM
#ldapRealm.contextFactory.authenticationMechanism = simple
+### If you want "remember me" to keep used logged in between sessions
Hi, without enalbing these lines, remember me feature works. Is it intended?
â
You are receiving this because you authored the thread.
Reply to this email directly, view it on
GitHub<https://github.com/apache/zeppelin/pull/2503#pullrequestreview-54506236>,
or mute the
thread<https://github.com/notifications/unsubscribe-auth/AAAV6lPF0-F6K1GJDFMHV_1RElwPb64Yks5sU9JTgaJpZM4OjEhR>.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2145: [ZEPPELIN-2271] encoding password of credentials
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2145 I've implemented something slightly similar here, except I'm actually encrypting the entire file using AES, using a passphrase you provide (which can live in memory) it's not ideal, but as others stated, it's better than no protection at all. thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2484: [ZEPPELIN-2711] basic metrics for paragraphs & noteboo...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2484 @jongyoul good q. I tried another stats libraries I've seen used around (airlift), but had a hard time reconciling dependencies, so went with the smallest footprint one + implementing the servlet myself. It looks like metrics-core is [pretty self contained too](https://mvnrepository.com/artifact/com.codahale.metrics/metrics-core/3.0.2) - I'd be fine giving it a try, if folks prefer it for some reason (I'm honestly not super up-to-speed on what's popular these days, so I'm glad to defer this decision :-)) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #497: ZEPPELIN-143: Git as a versioned notebook storage
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/497 @p0wl I started remote git support here: https://github.com/herval/zeppelin/tree/remote-git-support - it's a WIP, if someone wants to take over (it would work w/ github or any other git repo) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2503: [ZEPPELIN-2808] remember me support
Github user herval commented on the issue:
https://github.com/apache/zeppelin/pull/2503
That popup is not the siteâs âremember meâ - github actually keeps
you logged in by default (close the browser and reopen, and youâre still
logged, no need to re-authenticate)
You can see some examples of the âremember meâ/âkeep logged inâ
Iâm talking about on this ticket here:
http://www.uxforthemasses.com/login-page/
From: Kun <notificati...@github.com>
Sent: Tuesday, July 25, 2017 8:25:01 PM
To: apache/zeppelin
Cc: Herval Freire; Author
Subject: Re: [apache/zeppelin] [ZEPPELIN-2808] remember me support (#2503)
I don't have a preference, but in my experience, many web apps rely on the
browser provided remember-me feature. Here is an example of Github sign-in page
* https://github.com/login
Chrome
[image]<https://user-images.githubusercontent.com/4968473/28603134-460fcad2-71fd-11e7-8008-09e0e02e08e0.png>
Safari
[image]<https://user-images.githubusercontent.com/4968473/28603151-6036d1a8-71fd-11e7-9dc2-7fed43dff894.png>
â
You are receiving this because you authored the thread.
Reply to this email directly, view it on
GitHub<https://github.com/apache/zeppelin/pull/2503#issuecomment-317936689>, or
mute the
thread<https://github.com/notifications/unsubscribe-auth/AAAV6kptiwD_SODuxu5bCFknSdCau5Y0ks5sRrGNgaJpZM4OjEhR>.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2503: [ZEPPELIN-2808] remember me support
GitHub user herval opened a pull request: https://github.com/apache/zeppelin/pull/2503 [ZEPPELIN-2808] remember me support ### What is this PR for? Add support for "remember me" cookies, so we can activate LDAP without forcing users to re-login every time ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2808 ### How should this be tested? - Activate "rememberMe" by following the steps on the `shiro.ini.template` - Log in with the "remember me" option checked - Close the browser - Reopen the browser - You should still be logged in ### Questions: * While this is standard behavior on many web-apps, some folks (as exemplified on the Shiro documentation) find it "not secure". I'd like to know Zeppelin folks' opinion on this. * The "remember me" checkbox will show, regardless of whether you configure the rememberMeManager or not. Should this be a default? Should we hide it from the UI based on some config? You can merge this pull request into a Git repository by running: $ git pull https://github.com/herval/zeppelin remember-me Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2503.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2503 commit a0a5437d75282b60686f915436cb96e4cb661a8d Author: Herval Freire <hfre...@twitter.com> Date: 2017-07-25T20:07:25Z [ZEPPELIN-2808] remember me support --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (ZEPPELIN-2808) "Remember Me" for Login
Herval Freire created ZEPPELIN-2808: --- Summary: "Remember Me" for Login Key: ZEPPELIN-2808 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2808 Project: Zeppelin Issue Type: Improvement Reporter: Herval Freire Long-lived cookies (so user stays logged in if they close/reopen the browser) are common across webapps, and natively supported by Shiro. Adding support for those on Zeppelin should be trivial, and would prevent users from having to re-login every time they access their notebooks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 @soralee @khalidhuseynov would you or anyone else like to give this a new try? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2484: [ZEPPELIN-2711] basic metrics for paragraphs & noteboo...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2484 Hi folks - any thoughts on this approach? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2498: [ZEPPELIN-2796] Fetching LDAP roles
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2498 Hi folks, Any thoughts on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2498: [ZEPPELIN-2796] Fetching LDAP roles
GitHub user herval opened a pull request:
https://github.com/apache/zeppelin/pull/2498
[ZEPPELIN-2796] Fetching LDAP roles
### What is this PR for?
This actually fetches the user's LDAP roles, when using the LdapRealm, so
they're accessible on Zeppelin (via the `SecurityUtils`).
As I debugged through it, I found some weird replacement code, so I wrote
some tests around the `LdapRealm` and fixed what I believe was wrong there.
Eg.: `expandTemplate` would expand, for instance, `uid={0}` into
`uid={0}yourusername`.
In order to make the actual LDAP calls for roles, I exposed the
`queryForAuthorizationInfo` method and used it on `SecurityUtils`.
### What type of PR is it?
Bug Fix / Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2796
### How should this be tested?
I tested this in our LDAP environment. My shiro.ini looked more or less
like this:
```
[main]
ldapRealm = org.apache.zeppelin.realm.LdapRealm
ldapRealm.contextFactory.url = ldap://ldap.foo
ldapRealm.contextFactory.authenticationMechanism = simple
ldapRealm.userDnTemplate = uid={0},cn=users,dc=ods,dc=foo
ldapRealm.searchBase = dc=ods,dc=foo,dc=corp
ldapRealm.userSearchBase = cn=users,dc=ods,dc=foo
ldapRealm.groupSearchBase = cn=groups,dc=ods,dc=foo
ldapRealm.authorizationEnabled = true
ldapRealm.groupObjectClass = posixGroup
ldapRealm.memberAttribute = memberUid
ldapRealm.userObjectClass = posixAccount
ldapRealm.memberOfAttribute = memberOf
ldapReal.groupSearchFilter = '(memberUid={0})'
securityManager.realms = $ldapRealm
sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
securityManager.sessionManager = $sessionManager
securityManager.sessionManager.globalSessionTimeout = 8640
shiro.loginUrl = /api/login
[urls]
/api/version = anon
/api/metrics = anon
/api/interpreter/repository/** = authc
/api/configurations/** = authc
/api/credential/** = authc
/api/notebook-repositories = authc
# /** = anon
/** = authc
```
you can tell the role fetching worked inspecting the web call to `/ticker`,
which will now return the full roles list
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/herval/zeppelin ldap-roles
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2498.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2498
commit ed1b0f850236de9fedc34fbeacc074212d6e6840
Author: Herval Freire <hfre...@twitter.com>
Date: 2017-07-19T18:23:43Z
expose roles from ldap + fix substitution code
commit 4d3ed58fb32cf66d984767e65ac71b69471e1920
Author: Herval Freire <hfre...@twitter.com>
Date: 2017-07-20T17:05:38Z
expose roles from ldap + fix substitution code
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Created] (ZEPPELIN-2796) LDAP fetching of roles not working?
Herval Freire created ZEPPELIN-2796:
---
Summary: LDAP fetching of roles not working?
Key: ZEPPELIN-2796
URL: https://issues.apache.org/jira/browse/ZEPPELIN-2796
Project: Zeppelin
Issue Type: Bug
Reporter: Herval Freire
I tried setting up LDAP support. While authentication works, I couldn't get any
roles lists back. This led me down SHIRO-492 and the code on `SecurityUtils`,
which uses a `getListRoles()` function that simply returns a list you can
configure on shiro.ini ("rolesByGroup") - so no LDAP call is actually made.
I could get it to work by using the (currently protected)
`queryForAuthorizationInfo`, so my question here is if there's any historical
reason not to use that to fetch user's roles?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (ZEPPELIN-2792) Hide UI elements the user doesn't have credentials for
Herval Freire created ZEPPELIN-2792: --- Summary: Hide UI elements the user doesn't have credentials for Key: ZEPPELIN-2792 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2792 Project: Zeppelin Issue Type: New Feature Reporter: Herval Freire The user shouldn't be able to see certain tabs (eg interpreters or job list) if they don't have credentials support to those. That leads to a lot of confusion and could likely lead to a more manageable UI -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ZEPPELIN-2791) Support "super users"
Herval Freire created ZEPPELIN-2791: --- Summary: Support "super users" Key: ZEPPELIN-2791 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2791 Project: Zeppelin Issue Type: New Feature Reporter: Herval Freire In some cases, we want to be able to have unrestricted access to certain roles. Eg.: allow a user/role to be a "member" or "owner" of all notebooks. Any thoughts? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ZEPPELIN-2790) Storing Notebooks and Configs on database
Herval Freire created ZEPPELIN-2790: --- Summary: Storing Notebooks and Configs on database Key: ZEPPELIN-2790 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2790 Project: Zeppelin Issue Type: New Feature Reporter: Herval Freire At Twitter, we run mostly all of our services following the 12-factor approach (https://12factor.net/). Zeppelin is one of the few exceptions we can't really containerize, since: - it stores notebook data on local disk - it stores configuration on disk One option we considered is migrating the whole thing to git (with remote pushes). The issue, then, becomes concurrency. So the saner option we're left with is using a database backend to store that information. In the interest of time to build / easier compatibility with the JSON mode, what we're considering is basically storing some metadata + a JSON "blob" on your database of choice (configurable w/ jdbc driver). We didn't start commiting resources to building this yet - I'd like to probe the opensource community's interest on this, as well as any remarks, suggestions or no-gos you'd have in mind. Please advise! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zeppelin issue #2440: [ZEPPELIN-2587] allow logging in if you're anonymous
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2440 hmm I'm finding it difficult to get all the integration tests to pass. Inclined to test only for the presence of the button. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 ping - please advise --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 hmm I'll see if I can somehow split "report mode" and "view only mode". I didnt experience the Helium issue, personally --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2480: [ZEPPELIN-1249] build all submodules w/ the same scala...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2480 sounds good, Iâll do that then _ From: Felix Cheung <notificati...@github.com<mailto:notificati...@github.com>> Sent: Wednesday, July 12, 2017 9:34 AM Subject: Re: [apache/zeppelin] [ZEPPELIN-1249] build all submodules w/ the same scala version (#2480) To: apache/zeppelin <zeppe...@noreply.github.com<mailto:zeppe...@noreply.github.com>> Cc: Herval Freire <hervalfre...@gmail.com<mailto:hervalfre...@gmail.com>>, Author <aut...@noreply.github.com<mailto:aut...@noreply.github.com>> I realize 2.10 is EOL but as mentioned, from a while back many platforms are still running only on 2.10. Even Spark still supports 2.10 as of the latest release 2.2.0 from yesterday. The reason is many enterprises have applications that cannot be easily migrated to 2.11. It is possible that some users are only using a small number of interpreters (as you do) and they need a particular scala version. So instead of forcing all submodule to always use the same scala version, how about set a global scala version as the default but each submodule still has its own spark.scala.version that can be set in the maven command line when building? â You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://github.com/apache/zeppelin/pull/2480#issuecomment-314824983>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAAV6pwoowBIvtcK7sFeNYkcva3qjV-Iks5sNPWAgaJpZM4OTkY9>. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 all tests passing now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2484: [ZEPPELIN-2711] basic metrics for paragraphs & noteboo...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2484 This is the basic set of functionality needed to start monitoring some common metrics - Comments & observations welcome! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2480: [ZEPPELIN-1249] build all submodules w/ the same scala...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2480 Regarding verification, how do you suggest I go about that? We've been using this setup internally for some time, but we only use a few of the interpreters (spark, scalding, jdbc, md) - not sure if that could lead to any sort of runtime issue on other interpreters, but I find it highly unlikely.. Can you clarify on the flexibility bit? Not sure I understand what's the benefit of using different versions at the same time (not to mention Scala 2.10 is EOL since 2016). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2484: [ZEPPELIN-2711] [WIP] basic metrics for paragra...
GitHub user herval opened a pull request: https://github.com/apache/zeppelin/pull/2484 [ZEPPELIN-2711] [WIP] basic metrics for paragraphs & notebook view/create/run ### What is this PR for? This exposes JMX metrics for a few operations on Zeppelin. The first step here is to expose them via JMX, then later introduce a JSON endpoint (which would make it easier to integrate these metrics on common monitoring systems such as Prometheus, Graphite, Influx, etc I'm putting this out as WIP before finishing the tests & JSON bits in order to kick-off any design discussion and address comments sooner than later ### What type of PR is it? Feature ### Todos * [ ] - Tests * [ ] - JSON endpoint ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2711 ### How should this be tested? TODO ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? Yes * Is there breaking changes for older versions? No * Does this needs documentation? Yes You can merge this pull request into a Git repository by running: $ git pull https://github.com/herval/zeppelin metrics Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2484.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2484 commit 41bfbe3adf3b6a99c8b8794ae225ec68e98b5848 Author: Herval Freire <hfre...@twitter.com> Date: 2017-07-11T00:21:50Z basic metrics for paragraphs & notebook view/create/run --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2481: [HOTFIX][ZEPPELIN-2760] fix JDBC regression caused aft...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2481 oh, Indeed. There's some inheritance there ð¶ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2481: [HOTFIX][ZEPPELIN-2760] fix JDBC regression caused aft...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2481 I actually don't see any test on `JdbcInterpreterTest` covering that code path at all - I think the test suite is passing simply because that bit is not tested, and feel it should be --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2481: [HOTFIX][ZEPPELIN-2760] fix JDBC regression caused aft...
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2481 would it be worth adding a test around this? I imagine the tests would fail with a "class not found", so it would be easy to add one that proves that you don't get the error if auth.type is empty --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2480: [ZEPPELIN-1249] build all submodules w/ the sam...
GitHub user herval opened a pull request: https://github.com/apache/zeppelin/pull/2480 [ZEPPELIN-1249] build all submodules w/ the same scala version ### What is this PR for? This uses the same Scala version throughout modules, allowing us to use only Scala 2.11. ### What type of PR is it? Improvement ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1249 https://issues.apache.org/jira/browse/ZEPPELIN-2681 ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/herval/zeppelin fix-scala11-support Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2480.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2480 commit 5dfceb69544ac9dc7beb7d93a2d7bcd208e29f7f Author: Herval Freire <hfre...@twitter.com> Date: 2017-06-30T01:15:02Z build all submodules w/ the same scala version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (ZEPPELIN-2760) Can't run JDBC interpreters due to class not found
Herval Freire created ZEPPELIN-2760:
---
Summary: Can't run JDBC interpreters due to class not found
Key: ZEPPELIN-2760
URL: https://issues.apache.org/jira/browse/ZEPPELIN-2760
Project: Zeppelin
Issue Type: Bug
Reporter: Herval Freire
Priority: Blocker
This is a rehash of https://issues.apache.org/jira/browse/ZEPPELIN-1326 -
appears to have been caused by this commit:
https://github.com/apache/zeppelin/commit/e1f0a3205eb5aede0b2d80c9d3de59b3f47b699c
{code}
java.lang.NoClassDefFoundError:
org/apache/hadoop/security/UserGroupInformation$AuthenticationMethod
at
org.apache.zeppelin.jdbc.security.JDBCSecurityImpl.getAuthtype(JDBCSecurityImpl.java:64)
at
org.apache.zeppelin.jdbc.JDBCInterpreter.isKerboseEnabled(JDBCInterpreter.java:213)
at
org.apache.zeppelin.interpreter.KerberosInterpreter.open(KerberosInterpreter.java:64)
at
org.apache.zeppelin.jdbc.JDBCInterpreter.open(JDBCInterpreter.java:173)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:499)
at org.apache.zeppelin.scheduler.Job.run(Job.java:181)
at
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zeppelin pull request #:
Github user herval commented on the pull request: https://github.com/apache/zeppelin/commit/e1f0a3205eb5aede0b2d80c9d3de59b3f47b699c#commitcomment-23016085 Related ticket: https://issues.apache.org/jira/browse/ZEPPELIN-1326 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 Rebased to latest master - please advise --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue:
https://github.com/apache/zeppelin/pull/2439
Hmm Iâll try again.
Btw, did you `mvn clean` after switching branches? I think changes on the
js/html files donât get picked up unless you do that...
From: Sora Lee <notificati...@github.com>
Sent: Tuesday, July 4, 2017 1:23:03 AM
To: apache/zeppelin
Cc: Herval Freire; Mention
Subject: Re: [apache/zeppelin] [ZEPPELIN-2680] allow opening notebook as a
reader (#2439)
I could reproduce this problem on master branch but, I could not see to
resolve on your branch.
It has same error.
Could you test it out on your branch?
â
You are receiving this because you were mentioned.
Reply to this email directly, view it on
GitHub<https://github.com/apache/zeppelin/pull/2439#issuecomment-312813608>, or
mute the
thread<https://github.com/notifications/unsubscribe-auth/AAAV6sqTBAbMYY9AyGP9UiaoO7sogLEuks5sKfZngaJpZM4OFljo>.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 @soralee thanks for taking the time :-) there's one more step to your list: 1. login as user1 2. create a note 3. add a paragraph **4. change the visualization to a chart (instead of table)** 5. set owners and writers field set user1 6. set readers field sets user2 7. logout user1 and login user2. 8. back to note and check whether popup is appeared or not. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 any thoughts on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 any additional thoughts on this? Is it mergeable? @khalidhuseynov @tinkoff-dwh @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue:
https://github.com/apache/zeppelin/pull/2428
I am using the original setting (getMaxResults). I had to set it in two
calls because setFetchSize will determine that there is a ânextâ page, but
if I only do âsetMaxResultsâ, it will not display the truncation warning
(as it translates to a âlimitâ)
From: Felix Cheung <notificati...@github.com>
Sent: Thursday, June 29, 2017 9:30:44 PM
To: apache/zeppelin
Cc: Herval Freire; Mention
Subject: Re: [apache/zeppelin] [ZEPPELIN-1470] limiting results from jdbc
(#2428)
got it. I agree it's generally better to restrict the job at the source
perhaps you can use the existing settings but apply it at the JDBC instead?
it would be easier to switch existing user over the new behavior this way -
actually, let's see what other thinks about this approach.
â
You are receiving this because you were mentioned.
Reply to this email directly, view it on
GitHub<https://github.com/apache/zeppelin/pull/2428#issuecomment-312172202>, or
mute the
thread<https://github.com/notifications/unsubscribe-auth/AAAV6lwqItW73HbpQPIjLnKd6M4nnvryks5sJHn0gaJpZM4OBqFf>.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Created] (ZEPPELIN-2711) Metrics for service monitoring
Herval Freire created ZEPPELIN-2711: --- Summary: Metrics for service monitoring Key: ZEPPELIN-2711 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2711 Project: Zeppelin Issue Type: New Feature Reporter: Herval Freire I'm putting this out mainly to start a discussion on what approach we want to take here. For starter, we have two options in terms of distribution: - support JMX - support some sort of custom metrics mechanism Following the lines of projects such as Presto, we can expose these as JSON for quick consumption The second point is how we want to store those metrics: - Do we want metrics per execution? - Do we want quantiles/aggregated stats? I'm not sure what's the usual volume of data on a Zeppelin notebook - for our internal purposes, individual stats may be enough. Finally, there's the question as to which stats libraries to use. I'm currently between airlift (in case we go for aggregated + expose as json). Thoughts? Is this something the project is accepting contributions on (or has a roadmap for)? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 @tinkoff-dwh it's not duplicate. The bug here is that if you don't do `statement.setMaxRows`, some drivers (eg postgres) will try to load everything when you issue a `select *`. Using `setFetchSize` won't affect that (I tested it). So the solution I found is to set the limit to (fetch size + 1). There's a function that will prune the results *after they're back from JDBC* (`getResults`) - I'm guessing we could remove that, if all it did was checking that limit, but don't think it's worth pursuing at this time. TLDR - the current limit check is only applied _after_ you get results. If you query a big enough table, you'll never get the results (and will kill Zeppelin in the process) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428  --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 hmm.. It actually seems that setFetchSize does not limit the query execution (so it takes forever & hangs Zeppelin) Figuring out if there's a way to show results are truncated + use setMaxRows --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2439 Build is green: https://travis-ci.org/herval/zeppelin/builds/247345573 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2440: [ZEPPELIN-2587] allow logging in if you're anonymous
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2440 @zjffdu added one, let me know if that covers it --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2440: [ZEPPELIN-2587] allow logging in if youre anony...
GitHub user herval opened a pull request: https://github.com/apache/zeppelin/pull/2440 [ZEPPELIN-2587] allow logging in if youre anonymous ### What is this PR for? The login button is not showing (since 0.7.1) if you allow anonymous access. This shows it again. ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN/ZEPPELIN-2587 ### How should this be tested? - Start Zeppelin with anonymous support - The login button should now show on the bar ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/herval/zeppelin allow-login Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2440.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2440 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2439: [ZEPPELIN-2680] allow opening notebook as a rea...
GitHub user herval opened a pull request:
https://github.com/apache/zeppelin/pull/2439
[ZEPPELIN-2680] allow opening notebook as a reader
### What is this PR for?
We have a few notebooks that have a list of writers & owners, but no
restricted readers list.
When you try to open these notebooks as a reader AND they have a chart the
app sends a COMMIT_PARAGRAPH for each, which causes a write permission popup to
show ("Insufficient privileges to write note").
I tracked down the issue to the result.controller.js
(commitParagraphResult) - propagating the "viewOnly" state fixes that behavior.
### What type of PR is it?
Bug Fix
### How should this be tested?
- Create a notebook. Add a paragraph with a graph. Set the "owner" of the
notebook to something other than yourself. You won't be able to load the
notebook without the security popup anymore (unless logging in as the owner)
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/herval/zeppelin dont-write-as-a-reader
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2439.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2439
commit 3b369fce0d0703fed61a2b7babe0ca3b85ffbc08
Author: Herval Freire <hfre...@twitter.com>
Date: 2017-06-26T17:00:21Z
[ZEPPELIN-2680] allow opening notebook as a reader
### What is this PR for?
We have a few notebooks that have a list of writers & owners, but no
restricted readers list.
When you try to open these notebooks as a reader AND they have a chart the
app sends a COMMIT_PARAGRAPH for each, which causes a write permission popup to
show ("Insufficient privileges to write note").
I tracked down the issue to the result.controller.js
(commitParagraphResult) - propagating the "viewOnly" state fixes that behavior.
### What type of PR is it?
Bug Fix
### How should this be tested?
- Create a notebook. Add a paragraph with a graph. Set the "owner" of the
notebook to something other than yourself. You won't be able to load the
notebook without the security popup anymore (unless logging in as the owner)
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
No
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 @tinkoff-dwh build #3 is failing in both my master & zeppelin master: https://travis-ci.org/apache/zeppelin - restarting didn't make it pass: https://travis-ci.org/apache/zeppelin/jobs/246113686 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (ZEPPELIN-2681) Fix Scala 2.11 compatibility
Herval Freire created ZEPPELIN-2681: --- Summary: Fix Scala 2.11 compatibility Key: ZEPPELIN-2681 URL: https://issues.apache.org/jira/browse/ZEPPELIN-2681 Project: Zeppelin Issue Type: Improvement Reporter: Herval Freire Priority: Minor In the current code, -Pscala-2.11 will only affect a few submodules (eg Spark). Most things are hardcoded to Scala 2.10. Is there any reason to keep it like this now? I'm thinking about modifying the poms to use the same Scala version on all goals - saw a brief discussion here https://issues.apache.org/jira/browse/ZEPPELIN-605 and was wondering if that'd be ok, before I start digging into it -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ZEPPELIN-2680) Can't open notebook as a reader
Herval Freire created ZEPPELIN-2680:
---
Summary: Can't open notebook as a reader
Key: ZEPPELIN-2680
URL: https://issues.apache.org/jira/browse/ZEPPELIN-2680
Project: Zeppelin
Issue Type: Bug
Reporter: Herval Freire
We have a few notebooks that have a list of writers & owners, but no restricted
readers list.
When you try to open these notebooks _as a reader AND they have a chart_ the
app sends a COMMIT_PARAGRAPH for each, which causes a write permission popup to
show ("Insufficient privileges to write note").
I tracked down the issue to the result.controller.js (commitParagraphResult)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 It seems apache/zeppelin master (which is what this branch is based on) is failing the build - should I rebase it to 0.7 instead, or wait for master to be fixed? Please let me know what should I do next here (this is my first contribution to the project, pardon the ignorance) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 Why +1? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 I'm also not exactly sure how to write a test for this, as the JDBCInterpreterTest doesn't use anything in the line of mocks (and the outside behavior - limiting the number of returned rows - is already guaranteed by some filtering code. Any ideas @tinkoff-dwh ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #2428: [ZEPPELIN-1470] limiting results from jdbc
Github user herval commented on the issue: https://github.com/apache/zeppelin/pull/2428 @tinkoff-dwh you're right, picked the wrong method - using setFetchSize now --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin pull request #2428: [ZEPPELIN-1470] limiting results from jdbc
GitHub user herval opened a pull request: https://github.com/apache/zeppelin/pull/2428 [ZEPPELIN-1470] limiting results from jdbc ### What is this PR for? One thing we tracked down is that if you issue a large query on a very large table, it will simply try to load all results (and then cap them on Zeppelin's side), which seems suboptimal (and will freeze the server). Setting this on the JDBC level seems to solve the problem. ### What type of PR is it? Bug Fix ### Todos * [ ] - Task ### How should this be tested? - Create or use a table with a very large number of rows. In our tests, I simply created a: ``` createdb zeppelin_test psql zeppelin_test create table too_many_rows(n int) ``` And added 5m rows to it. Making a paragraph like this will hang without this: ``` %zeppelin_test select * from too_many_rows ``` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/herval/zeppelin hfreire/limit-row-count Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2428.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2428 commit f574f84706e4b345b62bebd456964928c4b787e8 Author: Herval Freire <hfre...@twitter.com> Date: 2017-06-21T23:54:29Z limiting results from jdbc --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
