Repository: zeppelin Updated Branches: refs/heads/master 21a084b17 -> b86148c61
[ZEPPELIN-1149] %sh interpreter kerberos support ### What is this PR for? Zeppelin Shell interpreter should work in a Kerberos environment ### What type of PR is it? [Feature] ### What is the Jira issue? * [ZEPPELIN-1149](https://issues.apache.org/jira/browse/ZEPPELIN-1149) ### How should this be tested? In JDBC interpreter setting add following properties - shell.auth.type = KERBEROS - shell.principal = principal value - shell.keytab.location = keytab location Now try and run any shell command (example: hdfs dfs -ls) it should return with valid results. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Rohit Choudhary <[email protected]> Closes #1163 from rconline/ZEPPELIN-1149 and squashes the following commits: f6af0f6 [Rohit Choudhary] typo fix 3648a1c [Rohit Choudhary] fix CI, resolve merge conflict 1924b99 [Rohit Choudhary] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1149 9143b47 [Rohit Choudhary] Removed Hadoop common dependency and changed switch to String based Kerberos/Simple. 60e6d21 [Rohit Choudhary] Addressing feedback from felixcheung b0300e8 [Rohit Choudhary] changing property name to "zeppelin.*" to make it match with rest of the project. 0a77e80 [Rohit Choudhary] shell interpreter doc 5190791 [Rohit Choudhary] [ZEPPELIN-1149] %sh interpreter kerberos support Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/b86148c6 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/b86148c6 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/b86148c6 Branch: refs/heads/master Commit: b86148c614fe55daa7b775e7800f28ad5efaac7e Parents: 21a084b Author: Rohit Choudhary <[email protected]> Authored: Thu Jul 14 11:49:43 2016 +0530 Committer: Prabhjyot Singh <[email protected]> Committed: Mon Jul 25 10:39:56 2016 +0530 ---------------------------------------------------------------------- docs/interpreter/shell.md | 93 ++++++++++++-------- .../apache/zeppelin/shell/ShellInterpreter.java | 6 +- .../shell/security/ShellSecurityImpl.java | 59 +++++++++++++ .../src/main/resources/interpreter-setting.json | 18 ++++ 4 files changed, 136 insertions(+), 40 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b86148c6/docs/interpreter/shell.md ---------------------------------------------------------------------- diff --git a/docs/interpreter/shell.md b/docs/interpreter/shell.md index f86e830..abcaf1c 100644 --- a/docs/interpreter/shell.md +++ b/docs/interpreter/shell.md @@ -1,40 +1,55 @@ ---- -layout: page -title: "Shell Interpreter" -description: "Shell Interpreter" -group: interpreter ---- -{% include JB/setup %} - -# Shell interpreter for Apache Zeppelin - -<div id="toc"></div> - -## Overview -Shell interpreter uses [Apache Commons Exec](https://commons.apache.org/proper/commons-exec) to execute external processes. -In Zeppelin notebook, you can use ` %sh ` in the beginning of a paragraph to invoke system shell and run commands. - -> **Note :** Currently each command runs as the user Zeppelin server is running as. - -## Configuration -At the "Interpreters" menu in Zeppelin dropdown menu, you can set the property value for Shell interpreter. - -<table class="table-configuration"> - <tr> - <th>Name</th> - <th>Value</th> - <th>Description</th> - </tr> - <tr> - <td>shell.command.timeout.millisecs</td> - <td>60000</td> - <td>Shell command time out in millisecs</td> - </tr> -</table> - -## Example -The following example demonstrates the basic usage of Shell in a Zeppelin notebook. - -<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/shell-example.png" /> - +--- +layout: page +title: "Shell Interpreter" +description: "Shell Interpreter" +group: interpreter +--- +{% include JB/setup %} + +# Shell interpreter for Apache Zeppelin + +<div id="toc"></div> + +## Overview +Shell interpreter uses [Apache Commons Exec](https://commons.apache.org/proper/commons-exec) to execute external processes. +In Zeppelin notebook, you can use ` %sh ` in the beginning of a paragraph to invoke system shell and run commands. + +> **Note :** Currently each command runs as the user Zeppelin server is running as. + +## Configuration +At the "Interpreters" menu in Zeppelin dropdown menu, you can set the property value for Shell interpreter. + +<table class="table-configuration"> + <tr> + <th>Name</th> + <th>Value</th> + <th>Description</th> + </tr> + <tr> + <td>shell.command.timeout.millisecs</td> + <td>60000</td> + <td>Shell command time out in millisecs</td> + </tr> + <tr> + <td>zeppelin.shell.auth.type</td> + <td></td> + <td>Types of authentications' methods supported are SIMPLE, and KERBEROS</td> + </tr> + <tr> + <td>zeppelin.shell.principal</td> + <td></td> + <td>The principal name to load from the keytab</td> + </tr> + <tr> + <td>zeppelin.shell.keytab.location</td> + <td></td> + <td>The path to the keytab file</td> + </tr> +</table> + +## Example +The following example demonstrates the basic usage of Shell in a Zeppelin notebook. + +<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/shell-example.png" /> + If you need further information about **Zeppelin Interpreter Setting** for using Shell interpreter, please read [What is interpreter setting?](../manual/interpreters.html#what-is-interpreter-setting) section first. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b86148c6/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java ---------------------------------------------------------------------- diff --git a/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java b/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java index a4cf550..9619de5 100644 --- a/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java +++ b/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java @@ -38,6 +38,7 @@ import org.apache.zeppelin.interpreter.InterpreterResult.Code; import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion; import org.apache.zeppelin.scheduler.Scheduler; import org.apache.zeppelin.scheduler.SchedulerFactory; +import org.apache.zeppelin.shell.security.ShellSecurityImpl; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -57,8 +58,11 @@ public class ShellInterpreter extends Interpreter { @Override public void open() { - LOGGER.info("Command timeout property: {}", TIMEOUT_PROPERTY); + LOGGER.info("Command timeout property: {}", getProperty(TIMEOUT_PROPERTY)); executors = new HashMap<String, DefaultExecutor>(); + if (!StringUtils.isAnyEmpty(getProperty("zeppelin.shell.auth.type"))) { + ShellSecurityImpl.createSecureConfiguration(getProperty(), shell); + } } @Override http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b86148c6/shell/src/main/java/org/apache/zeppelin/shell/security/ShellSecurityImpl.java ---------------------------------------------------------------------- diff --git a/shell/src/main/java/org/apache/zeppelin/shell/security/ShellSecurityImpl.java b/shell/src/main/java/org/apache/zeppelin/shell/security/ShellSecurityImpl.java new file mode 100644 index 0000000..bb3ce86 --- /dev/null +++ b/shell/src/main/java/org/apache/zeppelin/shell/security/ShellSecurityImpl.java @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.shell.security; + +import org.apache.commons.exec.CommandLine; +import org.apache.commons.exec.DefaultExecutor; +import org.apache.zeppelin.interpreter.InterpreterException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Properties; + + +/*** + * Shell security helper + */ +public class ShellSecurityImpl { + + private static Logger LOGGER = LoggerFactory.getLogger(ShellSecurityImpl.class); + + public static void createSecureConfiguration(Properties properties, String shell) { + + String authType = properties.getProperty("zeppelin.shell.auth.type") + .trim().toUpperCase(); + + switch (authType) { + case "KERBEROS": + CommandLine cmdLine = CommandLine.parse(shell); + cmdLine.addArgument("-c", false); + String kinitCommand = String.format("kinit -k -t %s %s", + properties.getProperty("zeppelin.shell.keytab.location"), + properties.getProperty("zeppelin.shell.principal")); + cmdLine.addArgument(kinitCommand, false); + DefaultExecutor executor = new DefaultExecutor(); + + try { + int exitVal = executor.execute(cmdLine); + } catch (Exception e) { + LOGGER.error("Unable to run kinit for zeppelin user " + kinitCommand, e); + throw new InterpreterException(e); + } + } + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/b86148c6/shell/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/shell/src/main/resources/interpreter-setting.json b/shell/src/main/resources/interpreter-setting.json index 5e9a051..78621df 100644 --- a/shell/src/main/resources/interpreter-setting.json +++ b/shell/src/main/resources/interpreter-setting.json @@ -9,6 +9,24 @@ "propertyName": "shell.command.timeout.millisecs", "defaultValue": "60000", "description": "Shell command time out in millisecs. Default = 60000" + }, + "zeppelin.shell.auth.type": { + "envName": null, + "propertyName": "zeppelin.shell.auth.type", + "defaultValue": "", + "description": "If auth type is needed, Example: KERBEROS" + }, + "zeppelin.shell.keytab.location": { + "envName": null, + "propertyName": "zeppelin.shell.keytab.location", + "defaultValue": "", + "description": "Kerberos keytab location" + }, + "zeppelin.shell.principal": { + "envName": null, + "propertyName": "zeppelin.shell.principal", + "defaultValue": "", + "description": "Kerberos principal" } } }
