[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1322 Then I think we should either revert this PR or fix it for spark interpreter as well. Because spark interpreter is the most important interpreter of zeppelin IMO. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 @zjffdu Yes, you are right, with SPARK_HOME/SPARK_SUBMIT it doesn't work. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1322 Sorry for late comment. I was in vacation in the last 2 weeks. I found this didn't work for spark interpreter. @prabhjyotsingh Did you try it for spark interpreter and other interpreters ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 Closing this, will open a new one with merge of https://github.com/apache/zeppelin/pull/1265. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user astroshim commented on the issue: https://github.com/apache/zeppelin/pull/1322 I got following checkstyle error while building source. ``` [INFO] There are 1 checkstyle errors. [ERROR] NotebookServer.java[1381] (sizes) LineLength: Line is longer than 100 characters (found 102). ``` @prabhjyotsingh Could you fix this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 @Leemoonsoo yes thats a good suggestion. Let me try and do it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1322
Instead of `USE_SSH_IMPERSONATION`, how about let user customize
impersonation method?
For example,
```
ZEPPELIN_INTERPRETER_IMPERSONATION_CMD="su - ${ZEPPELIN_USER_NAME}"
```
by default. but user can override this env variable, like
```
ZEPPELIN_INTERPRETER_IMPERSONATION_CMD="ssh -p12345
${ZEPPELIN_USER_NAME}@localhost"
```
It gives more flexibility i think. (e.g. give additional options like -p.
use different command to impersonate)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 Agreed @echarles, the `doAs` part will be a problem, until [ZEPPELIN-1340](https://issues.apache.org/jira/browse/ZEPPELIN-1340) is resolved. Until then for security we may have to run half interpreter with "User Impersonate" enable from UI (for example shell, python interpreter), and for others use the standard `doAs` already implemented (like livy, spark, jdbc) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 Btw, for the hadoop case (or spark on yarn case), this PR may give an issue for `doAs`. Typically, you configure `hadoop.proxyuser.foo.hosts` and `hadoop.proxyuser.foo.group`, `foo` being the os/kerberos user under which you run your java code that calls doAs. If we run ssh/su as the front-end user, we will not fullfill what the hadoop/yarn cluster is expecting. We thus should have two checkboxes: - One for the OS/kerberos impersonation (this PR only adresses OS). - The other for Hadoop impersonation. If you select one, I would expect the other one to be disabled. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 Sure, In this PR I was only thinking about the otherwise case i.e. in the environment where hadoop dependencies where not present, and hence start interpreter as end-web-user. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1322 @prabhjyotsingh I agree @echarles's idea. Interpreter tries to find hadoop dependencies first and if it passes, it uses `doAs`. Otherwise, let's talk about how to do it. How do you think of it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 Yes, you are right, let me do it right away. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 ... and make ZEPPELIN-1320 a subtask of ZEPPELIN-1337 ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 To make ZEPPELIN-1337 Umbrella for multiple user support for zeppelin more readable, should we rename the following: - ZEPPELIN-1340: "Run Hadoop-based interpreter process on Kerberos as web front end user" - ZEPPELIN-1320: "Run zeppelin interpreter process as web front end user" --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 @echarles , Yes agreed, will need to update in doc, and a extra toolbar near the check box where user can enable User Impersonate. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user echarles commented on the issue: https://github.com/apache/zeppelin/pull/1322 Whatever `su` or `ssh` is used, I feel the main trick is the user provisioning on the host running the interpreter. Until now, the shiro authentication system had no impact on the user provisioning. This PR changes this. I guess we all agree and are aware that adding user `foo` to shiro.ini, and enabling impersonation, will require to run `adduser foo` manually. We should make this clear in the doc but also stress it in the UI (with a hover, or a clear text/link near the User Impersonate. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 > It's better to use RUNAS ~ and su - ~ @jongyoul How about I make `use RUNAS ~ and su - ~` by default, but if in `zeppelin-env.sh` a property say `USE_SSH_IMPERSONATION` is set to true, then it will use `ssh web-user@localhost` in this way user gets to decide, what is best suited for their user case. >Could you check this PR support `scoped` as well which uses multiple threads in one process? Yes I've checked this with Shell and Python interpreter it was working as expected. @Leemoonsoo, yes agreed, I too think this options should be there, and have implemented it as well. If you take a look at GIF attached in this PR description, it's doing that you are asking for :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/1322 If i add one more, What do you guys think about adding an option `Impersonate` in the interpreter setting on GUI? That'll give user flexibility of selecting current behavior (without impersonation) and new behavior. Otherwise, this PR will make incompatible user behavior change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1322 @prabhjyotsingh Without issues above, Could you check this PR support `scoped` as well which uses multiple threads in one process? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1322 @prabhjyotsingh I don't know how to support different users' environments fully, actually. But I think it's better to use `RUNAS ~` and `su - ~` and using `ssh` without password make some security issues. In case Mesos, it uses that way to support restrict resources. But I never see using `ssh` without password. How do you think of it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1322
Yes, I thought about the usage in mac and windows, and initially started of
with using `RUNAS ${userName}` for windows and `su - ${userName}` for *nix
systems, but then it requires zeppelin server to run as root. Hence,
implemented with `ssh ${userName}@localhost`.
Have not thought about the cases in which system admin uses virtual users.
Now since with this, we are able to propagate end web user to
RemoteInterpreterManagedProcess.start, we can choose to use some other
mechanism in `interpreter.sh`/`interpreter.cmd` instead of "ssh", or may be
make it configurable using some extra config in "zeppelin-env.sh"
What do you recommend, that would be a secure and all full proof mechanism
by which we can run interpreter as different user ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1322 I agree that it's simple way to use ssh to support impersonation. but I'm worried about it. First, we should consider not to use ssh server in a local machine. It's disabled on Mac by default and in case of Windows users, they might not have any ssh server. Second, even if all of users can use connect their machine via ssh, all of users' name should be the same as system users. AFAIK, Some Zeppelin use cases, the system admin uses virtual users as well. Do you think of it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1322: [ZEPPELIN-1320] Run zeppelin interpreter process as we...
Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1322 CI green! Ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
