Repository: zeppelin Updated Branches: refs/heads/master 6f9012b40 -> a3ca80031
[ZEPPELIN-1026] set syntax highlight based on default bound interpreter ### What is this PR for? This is complete work of #1148. Comments and tasks on #1148 has been handled in this PR. - Add syntax language information in `interpreter-setting.json` - When user type `%replName` in paragraph, back-end check if the interpreter name with `replName` exists, and return language information to front-end if it does - If user doesn't specify `%replName`, default interpreter's language will be used - Using alias name for paragraph syntax highlight ### What type of PR is it? [Bug Fix | Improvement] ### What is the Jira issue? [ZEPPELIN-1026](https://issues.apache.org/jira/browse/ZEPPELIN-1026) ### How should this be tested? 1. Create new note and make markdown interpreter to be default. 2. See if markdown syntax is applied. ### Screenshots (if appropriate) #### Case 1. When the default interpreter set to python interpreter. **Before** Has `scala` as syntax highlight language when %python is not set. <img width="665" alt="screen shot 2016-07-07 at 10 46 20 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png";> **After** Has `python` as syntax highlight language even when %python is not set. <img width="666" alt="screen shot 2016-07-07 at 10 44 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png";> #### Case 2. When use alias name as repl name. **Before** <img width="742" alt="screen shot 2016-09-08 at 4 22 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png";> **After** <img width="741" alt="screen shot 2016-09-08 at 4 34 57 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png";> ### Further possible improvements There are still several cases that Zeppelin doesn't handle syntax highlight well. These can be handled with another jira ticket/PR. 1. When default bound interpreter changes, syntax highlight is not changed accordingly 2. When copy/paste code, syntax highlight won't be applied properly since Zeppelin only checks changes when cursor is in first line. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes(for creating new interpreter) Author: Mina Lee <[email protected]> Closes #1415 from minahlee/ZEPPELIN-1026 and squashes the following commits: c66fb0e [Mina Lee] Move getEditorSetting to InterpreterFactory class 2d56222 [Mina Lee] Add description about default syntax highlight in doc 08ccad9 [Mina Lee] Fix test 0874522 [Mina Lee] Change condition for triggering 'getAndSetEditorSetting' to reduce front-end <-> back-end communication 9e4f2e9 [Mina Lee] Change the way to read interpreter language from interpreter-setting.json after #1145 75543b3 [Mina Lee] Add test 565d9d0 [Mina Lee] [DOC] Setting syntax highlight when writing new interpreter 20132ca [Mina Lee] Get paragraph editor mode from backend 52f4207 [Mina Lee] Align comments for readability 26cbbb8 [Mina Lee] Add editor field Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/a3ca8003 Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/a3ca8003 Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/a3ca8003 Branch: refs/heads/master Commit: a3ca80031121a495f2946bd6209710dc7b6e330d Parents: 6f9012b Author: Mina Lee <[email protected]> Authored: Mon Sep 19 19:05:39 2016 +0200 Committer: Mina Lee <[email protected]> Committed: Fri Sep 23 16:47:26 2016 +0900 ---------------------------------------------------------------------- .../src/main/resources/interpreter-setting.json | 3 + docs/development/writingzeppelininterpreter.md | 20 ++- .../src/main/resources/interpreter-setting.json | 3 + .../src/main/resources/interpreter-setting.json | 3 + .../src/main/resources/interpreter-setting.json | 12 ++ .../src/main/resources/interpreter-setting.json | 3 + .../src/main/resources/interpreter-setting.json | 3 + .../src/main/resources/interpreter-setting.json | 5 +- .../src/main/resources/interpreter-setting.json | 12 ++ .../sparkr-resources/interpreter-setting.json | 15 ++ .../zeppelin/interpreter/Interpreter.java | 6 + .../apache/zeppelin/socket/NotebookServer.java | 21 ++- zeppelin-web/bower.json | 4 +- .../notebook/paragraph/paragraph.controller.js | 96 +++++++------ .../websocketEvents/websocketEvents.factory.js | 6 +- .../websocketEvents/websocketMsg.service.js | 9 ++ .../interpreter/InterpreterFactory.java | 68 +++++++-- .../zeppelin/interpreter/InterpreterInfo.java | 15 +- .../zeppelin/notebook/socket/Message.java | 143 ++++++++++--------- .../interpreter/InterpreterFactoryTest.java | 60 +++++++- .../org/apache/zeppelin/notebook/NoteTest.java | 2 - .../apache/zeppelin/notebook/NotebookTest.java | 34 ++--- .../interpreter/mock/interpreter-setting.json | 12 ++ 23 files changed, 398 insertions(+), 157 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/bigquery/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/bigquery/src/main/resources/interpreter-setting.json b/bigquery/src/main/resources/interpreter-setting.json index 3e524ed..fb44063 100644 --- a/bigquery/src/main/resources/interpreter-setting.json +++ b/bigquery/src/main/resources/interpreter-setting.json @@ -22,6 +22,9 @@ "defaultValue": "100000", "description": "Maximum number of rows to fetch from BigQuery" } + }, + "editor": { + "language": "sql" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/docs/development/writingzeppelininterpreter.md ---------------------------------------------------------------------- diff --git a/docs/development/writingzeppelininterpreter.md b/docs/development/writingzeppelininterpreter.md index 4ae1cbc..acf72e8 100644 --- a/docs/development/writingzeppelininterpreter.md +++ b/docs/development/writingzeppelininterpreter.md @@ -48,7 +48,7 @@ There are three locations where you can store your interpreter group, name and o {ZEPPELIN_INTERPRETER_DIR}/{YOUR_OWN_INTERPRETER_DIR}/interpreter-setting.json ``` -Here is an example of `interpreter-setting.json` on your own interpreter. +Here is an example of `interpreter-setting.json` on your own interpreter. Note that if you don't specify editor object, your interpreter will use plain text mode for syntax highlighting. ```json [ @@ -69,6 +69,9 @@ Here is an example of `interpreter-setting.json` on your own interpreter. "defaultValue": "property2DefaultValue", "description": "Property 2 description" }, ... + }, + "editor": { + "language": "your-syntax-highlight-language" } }, { @@ -81,8 +84,8 @@ Finally, Zeppelin uses static initialization with the following: ``` static { - Interpreter.register("MyInterpreterName", MyClassName.class.getName()); - } + Interpreter.register("MyInterpreterName", MyClassName.class.getName()); +} ``` **Static initialization is deprecated and will be supported until 0.6.0.** @@ -96,15 +99,20 @@ some interpreter specific code... ``` ## Programming Languages for Interpreter -If the interpreter uses a specific programming language ( like Scala, Python, SQL ), it is generally recommended to add a syntax highlighting supported for that to the notebook paragraph editor. +If the interpreter uses a specific programming language (like Scala, Python, SQL), it is generally recommended to add a syntax highlighting supported for that to the notebook paragraph editor. To check out the list of languages supported, see the `mode-*.js` files under `zeppelin-web/bower_components/ace-builds/src-noconflict` or from [github.com/ajaxorg/ace-builds](https://github.com/ajaxorg/ace-builds/tree/master/src-noconflict). If you want to add a new set of syntax highlighting, 1. Add the `mode-*.js` file to <code>[zeppelin-web/bower.json](https://github.com/apache/zeppelin/blob/master/zeppelin-web/bower.json)</code> ( when built, <code>[zeppelin-web/src/index.html](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/index.html)</code> will be changed automatically. ). -2. Add to the list of `editorMode` in <code>[zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js)</code> - it follows the pattern 'ace/mode/x' where x is the name. -3. Add to the code that checks for `%` prefix and calls `session.setMode(editorMode.x)` in `setParagraphMode` located in <code>[zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js](https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js)</code>. +2. Add `editor` object to `interpreter-setting.json` file. If you want to set your language to `java` for example, add: + + ``` + "editor": { + "language": "java" + } + ``` ## Install your interpreter binary http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/flink/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/flink/src/main/resources/interpreter-setting.json b/flink/src/main/resources/interpreter-setting.json index 067e7b8..e9bd126 100644 --- a/flink/src/main/resources/interpreter-setting.json +++ b/flink/src/main/resources/interpreter-setting.json @@ -16,6 +16,9 @@ "defaultValue": "6123", "description": "port of running JobManager." } + }, + "editor": { + "language": "scala" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/jdbc/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/jdbc/src/main/resources/interpreter-setting.json b/jdbc/src/main/resources/interpreter-setting.json index 289d4f8..abdb719 100644 --- a/jdbc/src/main/resources/interpreter-setting.json +++ b/jdbc/src/main/resources/interpreter-setting.json @@ -154,6 +154,9 @@ "defaultValue": "org.postgresql.Driver", "description": "" } + }, + "editor": { + "language": "sql" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/livy/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/livy/src/main/resources/interpreter-setting.json b/livy/src/main/resources/interpreter-setting.json index bc5a275..bad6830 100644 --- a/livy/src/main/resources/interpreter-setting.json +++ b/livy/src/main/resources/interpreter-setting.json @@ -87,6 +87,9 @@ "defaultValue": "", "description": "Adding extra libraries to livy interpreter" } + }, + "editor": { + "language": "scala" } }, { @@ -105,6 +108,9 @@ "defaultValue": "false", "description": "Execute multiple SQL concurrently if set true." } + }, + "editor": { + "language": "sql" } }, { @@ -112,6 +118,9 @@ "name": "pyspark", "className": "org.apache.zeppelin.livy.LivyPySparkInterpreter", "properties": { + }, + "editor": { + "language": "python" } }, { @@ -119,6 +128,9 @@ "name": "sparkr", "className": "org.apache.zeppelin.livy.LivySparkRInterpreter", "properties": { + }, + "editor": { + "language": "r" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/markdown/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/markdown/src/main/resources/interpreter-setting.json b/markdown/src/main/resources/interpreter-setting.json index 78ad735..38a6a76 100644 --- a/markdown/src/main/resources/interpreter-setting.json +++ b/markdown/src/main/resources/interpreter-setting.json @@ -10,6 +10,9 @@ "defaultValue": "markdown4j", "description": "Markdown Parser Type. Available values: markdown4j, pegdown. Default = markdown4j" } + }, + "editor": { + "language": "markdown" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/python/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/python/src/main/resources/interpreter-setting.json b/python/src/main/resources/interpreter-setting.json index 4b12cad..d3df586 100644 --- a/python/src/main/resources/interpreter-setting.json +++ b/python/src/main/resources/interpreter-setting.json @@ -16,6 +16,9 @@ "defaultValue": "1000", "description": "Max number of dataframe rows to display." } + }, + "editor": { + "language": "python" } }, { http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/shell/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/shell/src/main/resources/interpreter-setting.json b/shell/src/main/resources/interpreter-setting.json index 78621df..d1996fa 100644 --- a/shell/src/main/resources/interpreter-setting.json +++ b/shell/src/main/resources/interpreter-setting.json @@ -28,6 +28,9 @@ "defaultValue": "", "description": "Kerberos principal" } + }, + "editor": { + "language": "sh" } } -] \ No newline at end of file +] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/spark/src/main/resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/spark/src/main/resources/interpreter-setting.json b/spark/src/main/resources/interpreter-setting.json index d87a6c7..55d25bd 100644 --- a/spark/src/main/resources/interpreter-setting.json +++ b/spark/src/main/resources/interpreter-setting.json @@ -54,6 +54,9 @@ "defaultValue": "local[*]", "description": "Spark master uri. ex) spark://masterhost:7077" } + }, + "editor": { + "language": "scala" } }, { @@ -85,6 +88,9 @@ "defaultValue": "true", "description": "Import implicits, UDF collection, and sql if set true. true by default." } + }, + "editor": { + "language": "sql" } }, { @@ -104,6 +110,9 @@ "defaultValue": "spark-packages,http://dl.bintray.com/spark-packages/maven,false;";, "description": "A list of 'id,remote-repository-URL,is-snapshot;' for each remote repository." } + }, + "editor": { + "language": "scala" } }, { @@ -117,6 +126,9 @@ "defaultValue": "python", "description": "Python command to run pyspark with" } + }, + "editor": { + "language": "python" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/spark/src/main/sparkr-resources/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/spark/src/main/sparkr-resources/interpreter-setting.json b/spark/src/main/sparkr-resources/interpreter-setting.json index f884fe4..338c861 100644 --- a/spark/src/main/sparkr-resources/interpreter-setting.json +++ b/spark/src/main/sparkr-resources/interpreter-setting.json @@ -54,6 +54,9 @@ "defaultValue": "local[*]", "description": "Spark master uri. ex) spark://masterhost:7077" } + }, + "editor": { + "language": "scala" } }, { @@ -85,6 +88,9 @@ "defaultValue": "true", "description": "Import implicits, UDF collection, and sql if set true. true by default." } + }, + "editor": { + "language": "sql" } }, { @@ -104,6 +110,9 @@ "defaultValue": "spark-packages,http://dl.bintray.com/spark-packages/maven,false;";, "description": "A list of 'id,remote-repository-URL,is-snapshot;' for each remote repository." } + }, + "editor": { + "language": "scala" } }, { @@ -117,6 +126,9 @@ "defaultValue": "python", "description": "Python command to run pyspark with" } + }, + "editor": { + "language": "python" } }, { @@ -148,6 +160,9 @@ "defaultValue": "out.format = 'html', comment = NA, echo = FALSE, results = 'asis', message = F, warning = F", "description": "" } + }, + "editor": { + "language": "r" } } ] http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java index 6d7d660..9678b46 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java @@ -251,6 +251,7 @@ public abstract class Interpreter { private String className; private boolean defaultInterpreter; private Map<String, InterpreterProperty> properties; + private Map<String, Object> editor; private String path; public RegisteredInterpreter(String name, String group, String className, @@ -266,6 +267,7 @@ public abstract class Interpreter { this.className = className; this.defaultInterpreter = defaultInterpreter; this.properties = properties; + this.editor = new HashMap<>(); } public String getName() { @@ -292,6 +294,10 @@ public abstract class Interpreter { return properties; } + public Map<String, Object> getEditor() { + return editor; + } + public void setPath(String path) { this.path = path; } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java index 8b6329a..3e54e05 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java @@ -250,6 +250,9 @@ public class NotebookServer extends WebSocketServlet implements case SAVE_INTERPRETER_BINDINGS: saveInterpreterBindings(conn, messagereceived); break; + case EDITOR_SETTING: + getEditorSetting(conn, messagereceived); + break; default: break; } @@ -1457,7 +1460,7 @@ public class NotebookServer extends WebSocketServlet implements } /** - * This callback is for praragraph that runs on RemoteInterpreterProcess + * This callback is for paragraph that runs on RemoteInterpreterProcess * @param paragraph * @param out * @param output @@ -1569,11 +1572,23 @@ public class NotebookServer extends WebSocketServlet implements if (id.equals(interpreterGroupId)) { broadcast( note.getId(), - new Message(OP.ANGULAR_OBJECT_REMOVE).put("name", name).put( - "noteId", noteId).put("paragraphId", paragraphId)); + new Message(OP.ANGULAR_OBJECT_REMOVE) + .put("name", name) + .put("noteId", noteId) + .put("paragraphId", paragraphId)); } } } } + + private void getEditorSetting(NotebookSocket conn, Message fromMessage) + throws IOException { + String replName = (String) fromMessage.get("magic"); + String noteId = getOpenNoteId(conn); + Message resp = new Message(OP.EDITOR_SETTING); + resp.put("editor", notebook().getInterpreterFactory().getEditorSetting(noteId, replName)); + conn.send(serializeMessage(resp)); + return; + } } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/bower.json ---------------------------------------------------------------------- diff --git a/zeppelin-web/bower.json b/zeppelin-web/bower.json index 5ec18a7..aeabd79 100644 --- a/zeppelin-web/bower.json +++ b/zeppelin-web/bower.json @@ -24,7 +24,7 @@ "angular-elastic": "~2.4.2", "angular-elastic-input": "~2.2.0", "angular-xeditable": "0.1.12", - "highlightjs": "^9.2.0", + "highlightjs": "^9.4.0", "lodash": "~3.9.3", "angular-filter": "~0.5.4", "ngtoast": "~2.0.0", @@ -62,7 +62,7 @@ "highlight.pack.js", "styles/github.css" ], - "version": "8.4.0", + "version": "9.4.0", "name": "highlightjs" } } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js index bd3b6b3..8c46810 100644 --- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js +++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js @@ -15,13 +15,14 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $rootScope, $route, $window, $routeParams, $location, $timeout, $compile, - $http, websocketMsgSrv, baseUrlSrv, ngToast, + $http, $q, websocketMsgSrv, baseUrlSrv, ngToast, saveAsService, esriLoader) { var ANGULAR_FUNCTION_OBJECT_NAME_PREFIX = '_Z_ANGULAR_FUNC_'; $scope.parentNote = null; $scope.paragraph = null; $scope.originalText = ''; $scope.editor = null; + $scope.magic = null; var paragraphScope = $rootScope.$new(true, $rootScope); @@ -78,15 +79,6 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r var angularObjectRegistry = {}; - var editorModes = { - 'ace/mode/python': /^%(\w*\.)?(pyspark|python)\s*$/, - 'ace/mode/scala': /^%(\w*\.)?spark\s*$/, - 'ace/mode/r': /^%(\w*\.)?(r|sparkr|knitr)\s*$/, - 'ace/mode/sql': /^%(\w*\.)?\wql/, - 'ace/mode/markdown': /^%md/, - 'ace/mode/sh': /^%sh/ - }; - // Controller init $scope.init = function(newParagraph, note) { $scope.paragraph = newParagraph; @@ -538,7 +530,7 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r $scope.aceChanged = function() { $scope.dirtyText = $scope.editor.getSession().getValue(); $scope.startSaveTimer(); - $scope.setParagraphMode($scope.editor.getSession(), $scope.dirtyText, $scope.editor.getCursorPosition()); + setParagraphMode($scope.editor.getSession(), $scope.dirtyText, $scope.editor.getCursorPosition()); }; $scope.aceLoaded = function(_editor) { @@ -576,37 +568,11 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r // not applying emacs key binding while the binding override Ctrl-v. default behavior of paste text on windows. } - $scope.setParagraphMode = function(session, paragraphText, pos) { - // Evaluate the mode only if the first 30 characters of the paragraph have been modified or the the position is undefined. - if ((typeof pos === 'undefined') || (pos.row === 0 && pos.column < 30)) { - // If paragraph loading, use config value if exists - if ((typeof pos === 'undefined') && $scope.paragraph.config.editorMode) { - session.setMode($scope.paragraph.config.editorMode); - } else { - // Defaults to spark mode - var newMode = 'ace/mode/scala'; - // Test first against current mode - var oldMode = session.getMode().$id; - if (!editorModes[oldMode] || !editorModes[oldMode].test(paragraphText)) { - for (var key in editorModes) { - if (key !== oldMode) { - if (editorModes[key].test(paragraphText)) { - $scope.paragraph.config.editorMode = key; - session.setMode(key); - return true; - } - } - } - $scope.paragraph.config.editorMode = newMode; - session.setMode(newMode); - } - } - } - }; - var remoteCompleter = { getCompletions: function(editor, session, pos, prefix, callback) { - if (!$scope.editor.isFocused()) { return;} + if (!$scope.editor.isFocused()) { + return; + } pos = session.getTextRange(new Range(0, 0, pos.row, pos.column)).length; var buf = session.getValue(); @@ -662,7 +628,7 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r autoAdjustEditorHeight(_editor.container.id); }); - $scope.setParagraphMode($scope.editor.getSession(), $scope.editor.getSession().getValue()); + setParagraphMode($scope.editor.getSession(), $scope.editor.getSession().getValue()); // autocomplete on '.' /* @@ -737,6 +703,53 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r } }; + var getAndSetEditorSetting = function(session, interpreterName) { + var deferred = $q.defer(); + websocketMsgSrv.getEditorSetting(interpreterName); + $timeout( + $scope.$on('editorSetting', function(event, data) { + deferred.resolve(data); + } + ), 1000); + + deferred.promise.then(function(editorSetting) { + if (!_.isEmpty(editorSetting.editor)) { + var mode = 'ace/mode/' + editorSetting.editor.language; + $scope.paragraph.config.editorMode = mode; + session.setMode(mode); + } + }); + }; + + var setParagraphMode = function(session, paragraphText, pos) { + // Evaluate the mode only if the the position is undefined + // or the first 30 characters of the paragraph have been modified + // or cursor position is at beginning of second line.(in case user hit enter after typing %magic) + if ((typeof pos === 'undefined') || (pos.row === 0 && pos.column < 30) || (pos.row === 1 && pos.column === 0)) { + // If paragraph loading, use config value if exists + if ((typeof pos === 'undefined') && $scope.paragraph.config.editorMode) { + session.setMode($scope.paragraph.config.editorMode); + } else { + var magic; + // set editor mode to default interpreter syntax if paragraph text doesn't start with '%' + // TODO(mina): dig into the cause what makes interpreterBindings has no element + if (!paragraphText.startsWith('%') && ((typeof pos !== 'undefined') && pos.row === 0 && pos.column === 1) || + (typeof pos === 'undefined') && $scope.$parent.interpreterBindings.length !== 0) { + magic = $scope.$parent.interpreterBindings[0].name; + getAndSetEditorSetting(session, magic); + } else { + var replNameRegexp = /%(.+?)\s/g; + var match = replNameRegexp.exec(paragraphText); + if (match && $scope.magic !== match[1]) { + magic = match[1].trim(); + $scope.magic = magic; + getAndSetEditorSetting(session, magic); + } + } + } + } + }; + var autoAdjustEditorHeight = function(id) { var editor = $scope.editor; var height = editor.getSession().getScreenLength() * editor.renderer.lineHeight + @@ -2259,7 +2272,6 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r var noteId = $route.current.pathParams.noteId; $http.get(baseUrlSrv.getRestApiBase() + '/helium/suggest/' + noteId + '/' + $scope.paragraph.id) .success(function(data, status, headers, config) { - console.log('Suggested apps %o', data); $scope.suggestion = data.body; }) .error(function(err, status, headers, config) { http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js b/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js index e4f45db..36e90b4 100644 --- a/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js +++ b/zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js @@ -76,8 +76,8 @@ angular.module('zeppelinWebApp').factory('websocketEvents', action: function(dialog) { dialog.close(); angular.element('#loginModal').modal({ - show: 'true' - }); + show: 'true' + }); } }, { label: 'Cancel', @@ -97,6 +97,8 @@ angular.module('zeppelinWebApp').factory('websocketEvents', $rootScope.$broadcast('updateProgress', data); } else if (op === 'COMPLETION_LIST') { $rootScope.$broadcast('completionList', data); + } else if (op === 'EDITOR_SETTING') { + $rootScope.$broadcast('editorSetting', data); } else if (op === 'ANGULAR_OBJECT_UPDATE') { $rootScope.$broadcast('angularObjectUpdate', data); } else if (op === 'ANGULAR_OBJECT_REMOVE') { http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js ---------------------------------------------------------------------- diff --git a/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js b/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js index 3dcdc3e..3b27bce 100644 --- a/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js +++ b/zeppelin-web/src/components/websocketEvents/websocketMsg.service.js @@ -180,6 +180,15 @@ angular.module('zeppelinWebApp').service('websocketMsgSrv', function($rootScope, }); }, + getEditorSetting: function(replName) { + websocketEvents.sendNewEvent({ + op: 'EDITOR_SETTING', + data: { + magic: replName + } + }); + }, + isConnected: function() { return websocketEvents.isConnected(); }, http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java index 47ff946..40dcc30 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java @@ -49,6 +49,8 @@ import java.util.Properties; import java.util.Set; import com.google.common.base.Preconditions; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Maps; import com.google.gson.Gson; import com.google.gson.GsonBuilder; import com.google.gson.reflect.TypeToken; @@ -223,7 +225,8 @@ public class InterpreterFactory implements InterpreterGroupFactory { InterpreterInfo interpreterInfo; for (RegisteredInterpreter r : Interpreter.registeredInterpreters.values()) { interpreterInfo = - new InterpreterInfo(r.getClassName(), r.getName(), r.isDefaultInterpreter()); + new InterpreterInfo(r.getClassName(), r.getName(), r.isDefaultInterpreter(), + r.getEditor()); add(r.getGroup(), interpreterInfo, convertInterpreterProperties(r.getProperties()), r.getPath()); } @@ -335,7 +338,7 @@ public class InterpreterFactory implements InterpreterGroupFactory { for (RegisteredInterpreter registeredInterpreter : registeredInterpreters) { InterpreterInfo interpreterInfo = new InterpreterInfo(registeredInterpreter.getClassName(), registeredInterpreter.getName(), - registeredInterpreter.isDefaultInterpreter()); + registeredInterpreter.isDefaultInterpreter(), registeredInterpreter.getEditor()); Properties properties = new Properties(); Map<String, InterpreterProperty> p = registeredInterpreter.getProperties(); @@ -370,10 +373,11 @@ public class InterpreterFactory implements InterpreterGroupFactory { fis.close(); String json = sb.toString(); - InterpreterInfoSaving info = gson.fromJson(json, InterpreterInfoSaving.class); + InterpreterInfoSaving infoSaving = gson.fromJson(json, InterpreterInfoSaving.class); - for (String k : info.interpreterSettings.keySet()) { - InterpreterSetting setting = info.interpreterSettings.get(k); + for (String k : infoSaving.interpreterSettings.keySet()) { + InterpreterSetting setting = infoSaving.interpreterSettings.get(k); + List<InterpreterInfo> infos = setting.getInterpreterInfos(); // Always use separate interpreter process // While we decided to turn this feature on always (without providing @@ -391,15 +395,23 @@ public class InterpreterFactory implements InterpreterGroupFactory { depClassPath = interpreterSettingObject.getPath(); setting.setPath(depClassPath); + for (InterpreterInfo info : infos) { + if (info.getEditor() == null) { + Map<String, Object> editor = getEditorFromSettingByClassName(interpreterSettingObject, + info.getClassName()); + info.setEditor(editor); + } + } + setting.setInterpreterGroupFactory(this); loadInterpreterDependencies(setting); interpreterSettings.put(k, setting); } - this.interpreterBindings = info.interpreterBindings; + this.interpreterBindings = infoSaving.interpreterBindings; - if (info.interpreterRepositories != null) { - for (RemoteRepository repo : info.interpreterRepositories) { + if (infoSaving.interpreterRepositories != null) { + for (RemoteRepository repo : infoSaving.interpreterRepositories) { if (!depResolver.getRepos().contains(repo)) { this.interpreterRepositories.add(repo); } @@ -407,6 +419,17 @@ public class InterpreterFactory implements InterpreterGroupFactory { } } + public Map<String, Object> getEditorFromSettingByClassName(InterpreterSetting intpSetting, + String className) { + List<InterpreterInfo> intpInfos = intpSetting.getInterpreterInfos(); + for (InterpreterInfo intpInfo : intpInfos) { + if (className.equals(intpInfo.getClassName())) { + return intpInfo.getEditor(); + } + } + return ImmutableMap.of("language", (Object) "text"); + } + private void loadInterpreterDependencies(final InterpreterSetting setting) { setting.setStatus(InterpreterSetting.Status.DOWNLOADING_DEPENDENCIES); @@ -1261,6 +1284,35 @@ public class InterpreterFactory implements InterpreterGroupFactory { this.env = env; } + public Map<String, Object> getEditorSetting(String noteId, String replName) { + Interpreter intp = getInterpreter(noteId, replName); + Map<String, Object> editor = Maps.newHashMap( + ImmutableMap.<String, Object>builder() + .put("language", "text").build()); + String defaultSettingName = getDefaultInterpreterSetting(noteId).getName(); + String group = StringUtils.EMPTY; + try { + List<InterpreterSetting> intpSettings = getInterpreterSettings(noteId); + for (InterpreterSetting intpSetting : intpSettings) { + String[] replNameSplit = replName.split("\\."); + if (replNameSplit.length == 2) { + group = replNameSplit[0]; + } + // when replName is 'name' of interpreter + if (defaultSettingName.equals(intpSetting.getName())) { + editor = getEditorFromSettingByClassName(intpSetting, intp.getClassName()); + } + // when replName is 'alias name' of interpreter or 'group' of interpreter + if (replName.equals(intpSetting.getName()) || group.equals(intpSetting.getName())) { + editor = getEditorFromSettingByClassName(intpSetting, intp.getClassName()); + break; + } + } + } catch (NullPointerException e) { + logger.warn("Couldn't get interpreter editor language"); + } + return editor; + } private Interpreter getDevInterpreter() { if (devInterpreter == null) { http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java index c104b9d..e9088e9 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterInfo.java @@ -19,6 +19,8 @@ package org.apache.zeppelin.interpreter; import com.google.gson.annotations.SerializedName; +import java.util.Map; + /** * Information of interpreters in this interpreter setting. * this will be serialized for conf/interpreter.json and REST api response. @@ -27,11 +29,14 @@ public class InterpreterInfo { private String name; @SerializedName("class") private String className; private boolean defaultInterpreter = false; + private Map<String, Object> editor; - InterpreterInfo(String className, String name, boolean defaultInterpreter) { + InterpreterInfo(String className, String name, boolean defaultInterpreter, + Map<String, Object> editor) { this.className = className; this.name = name; this.defaultInterpreter = defaultInterpreter; + this.editor = editor; } public String getName() { @@ -50,6 +55,14 @@ public class InterpreterInfo { return defaultInterpreter; } + public Map<String, Object> getEditor() { + return editor; + } + + public void setEditor(Map<String, Object> editor) { + this.editor = editor; + } + @Override public boolean equals(Object obj) { if (!(obj instanceof InterpreterInfo)) { http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java index 9175e30..b314e62 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java @@ -21,43 +21,43 @@ import java.util.HashMap; import java.util.Map; /** - * Zeppelin websocker massage template class. + * Zeppelin websocket massage template class. */ public class Message { /** * Representation of event type. */ public static enum OP { - GET_HOME_NOTE, // [c-s] load note for home screen + GET_HOME_NOTE, // [c-s] load note for home screen - GET_NOTE, // [c-s] client load note - // @param id note id + GET_NOTE, // [c-s] client load note + // @param id note id - NOTE, // [s-c] note info - // @param note serlialized Note object + NOTE, // [s-c] note info + // @param note serlialized Note object - PARAGRAPH, // [s-c] paragraph info - // @param paragraph serialized paragraph object + PARAGRAPH, // [s-c] paragraph info + // @param paragraph serialized paragraph object - PROGRESS, // [s-c] progress update - // @param id paragraph id - // @param progress percentage progress - - NEW_NOTE, // [c-s] create new notebook - DEL_NOTE, // [c-s] delete notebook - // @param id note id - CLONE_NOTE, // [c-s] clone new notebook - // @param id id of note to clone - // @param name name fpor the cloned note - IMPORT_NOTE, // [c-s] import notebook - // @param object notebook + PROGRESS, // [s-c] progress update + // @param id paragraph id + // @param progress percentage progress + + NEW_NOTE, // [c-s] create new notebook + DEL_NOTE, // [c-s] delete notebook + // @param id note id + CLONE_NOTE, // [c-s] clone new notebook + // @param id id of note to clone + // @param name name fpor the cloned note + IMPORT_NOTE, // [c-s] import notebook + // @param object notebook NOTE_UPDATE, - RUN_PARAGRAPH, // [c-s] run paragraph - // @param id paragraph id - // @param paragraph paragraph content.ie. script - // @param config paragraph config - // @param params paragraph params + RUN_PARAGRAPH, // [c-s] run paragraph + // @param id paragraph id + // @param paragraph paragraph content.ie. script + // @param config paragraph config + // @param params paragraph params COMMIT_PARAGRAPH, // [c-s] commit paragraph // @param id paragraph id @@ -69,72 +69,77 @@ public class Message { CANCEL_PARAGRAPH, // [c-s] cancel paragraph run // @param id paragraph id - MOVE_PARAGRAPH, // [c-s] move paragraph order - // @param id paragraph id - // @param index index the paragraph want to go + MOVE_PARAGRAPH, // [c-s] move paragraph order + // @param id paragraph id + // @param index index the paragraph want to go INSERT_PARAGRAPH, // [c-s] create new paragraph below current paragraph // @param target index - COMPLETION, // [c-s] ask completion candidates - // @param id - // @param buf current code - // @param cursor cursor position in code + EDITOR_SETTING, // [c-s] ask paragraph editor setting + // @param magic magic keyword written in paragraph + // ex) spark.spark or spark + + COMPLETION, // [c-s] ask completion candidates + // @param id + // @param buf current code + // @param cursor cursor position in code - COMPLETION_LIST, // [s-c] send back completion candidates list - // @param id - // @param completions list of string + COMPLETION_LIST, // [s-c] send back completion candidates list + // @param id + // @param completions list of string - LIST_NOTES, // [c-s] ask list of note - RELOAD_NOTES_FROM_REPO, // [c-s] reload notes from repo + LIST_NOTES, // [c-s] ask list of note + RELOAD_NOTES_FROM_REPO, // [c-s] reload notes from repo - NOTES_INFO, // [s-c] list of note infos - // @param notes serialized List<NoteInfo> object + NOTES_INFO, // [s-c] list of note infos + // @param notes serialized List<NoteInfo> object PARAGRAPH_REMOVE, PARAGRAPH_CLEAR_OUTPUT, - PARAGRAPH_APPEND_OUTPUT, // [s-c] append output - PARAGRAPH_UPDATE_OUTPUT, // [s-c] update (replace) output + PARAGRAPH_APPEND_OUTPUT, // [s-c] append output + PARAGRAPH_UPDATE_OUTPUT, // [s-c] update (replace) output PING, AUTH_INFO, - ANGULAR_OBJECT_UPDATE, // [s-c] add/update angular object - ANGULAR_OBJECT_REMOVE, // [s-c] add angular object del + ANGULAR_OBJECT_UPDATE, // [s-c] add/update angular object + ANGULAR_OBJECT_REMOVE, // [s-c] add angular object del - ANGULAR_OBJECT_UPDATED, // [c-s] angular object value updated, + ANGULAR_OBJECT_UPDATED, // [c-s] angular object value updated, - ANGULAR_OBJECT_CLIENT_BIND, // [c-s] angular object updated from AngularJS z object + ANGULAR_OBJECT_CLIENT_BIND, // [c-s] angular object updated from AngularJS z object - ANGULAR_OBJECT_CLIENT_UNBIND, // [c-s] angular object unbind from AngularJS z object + ANGULAR_OBJECT_CLIENT_UNBIND, // [c-s] angular object unbind from AngularJS z object - LIST_CONFIGURATIONS, // [c-s] ask all key/value pairs of configurations - CONFIGURATIONS_INFO, // [s-c] all key/value pairs of configurations - // @param settings serialized Map<String, String> object + LIST_CONFIGURATIONS, // [c-s] ask all key/value pairs of configurations + CONFIGURATIONS_INFO, // [s-c] all key/value pairs of configurations + // @param settings serialized Map<String, String> object - CHECKPOINT_NOTEBOOK, // [c-s] checkpoint notebook to storage repository - // @param noteId - // @param checkpointName + CHECKPOINT_NOTEBOOK, // [c-s] checkpoint notebook to storage repository + // @param noteId + // @param checkpointName - LIST_REVISION_HISTORY, // [c-s] list revision history of the notebook - // @param noteId - NOTE_REVISION, // [c-s] get certain revision of note - // @param noteId - // @param revisionId + LIST_REVISION_HISTORY, // [c-s] list revision history of the notebook + // @param noteId + NOTE_REVISION, // [c-s] get certain revision of note + // @param noteId + // @param revisionId - APP_APPEND_OUTPUT, // [s-c] append output - APP_UPDATE_OUTPUT, // [s-c] update (replace) output - APP_LOAD, // [s-c] on app load - APP_STATUS_CHANGE, // [s-c] on app status change + APP_APPEND_OUTPUT, // [s-c] append output + APP_UPDATE_OUTPUT, // [s-c] update (replace) output + APP_LOAD, // [s-c] on app load + APP_STATUS_CHANGE, // [s-c] on app status change - LIST_NOTEBOOK_JOBS, // [c-s] get notebook job management infomations - LIST_UPDATE_NOTEBOOK_JOBS, // [s-c] get job management informations + LIST_NOTEBOOK_JOBS, // [c-s] get notebook job management infomations + LIST_UPDATE_NOTEBOOK_JOBS, // [c-s] get job management informations for until unixtime UNSUBSCRIBE_UPDATE_NOTEBOOK_JOBS, // [c-s] unsubscribe job information for job management - GET_INTERPRETER_BINDINGS, // [c-s] get interpreter bindings - // @param noteID - SAVE_INTERPRETER_BINDINGS, // [c-s] save interpreter bindings - // @param noteID - // @param selectedSettingIds - INTERPRETER_BINDINGS // [s-c] interpreter bindings + // @param unixTime + GET_INTERPRETER_BINDINGS, // [c-s] get interpreter bindings + // @param noteID + SAVE_INTERPRETER_BINDINGS, // [c-s] save interpreter bindings + // @param noteID + // @param selectedSettingIds + INTERPRETER_BINDINGS // [s-c] interpreter bindings } public OP op; http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java index 5cdda05..09031a5 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/InterpreterFactoryTest.java @@ -34,12 +34,22 @@ import org.apache.zeppelin.dep.DependencyResolver; import org.apache.zeppelin.interpreter.mock.MockInterpreter1; import org.apache.zeppelin.interpreter.mock.MockInterpreter2; import org.apache.zeppelin.interpreter.remote.RemoteInterpreter; +import org.apache.zeppelin.notebook.JobListenerFactory; +import org.apache.zeppelin.notebook.Note; +import org.apache.zeppelin.notebook.Notebook; +import org.apache.zeppelin.notebook.repo.NotebookRepo; +import org.apache.zeppelin.notebook.repo.VFSNotebookRepo; +import org.apache.zeppelin.scheduler.SchedulerFactory; +import org.apache.zeppelin.search.SearchService; import org.junit.After; import org.junit.Before; import org.junit.Test; +import org.quartz.SchedulerException; import org.sonatype.aether.RepositoryException; import static org.junit.Assert.*; +import static org.mockito.Mockito.mock; +import org.mockito.Mock; public class InterpreterFactoryTest { @@ -47,13 +57,19 @@ public class InterpreterFactoryTest { private File tmpDir; private ZeppelinConfiguration conf; private InterpreterContext context; + private Notebook notebook; + private NotebookRepo notebookRepo; private DependencyResolver depResolver; + private SchedulerFactory schedulerFactory; + @Mock + private JobListenerFactory jobListenerFactory; @Before public void setUp() throws Exception { tmpDir = new File(System.getProperty("java.io.tmpdir")+"/ZeppelinLTest_"+System.currentTimeMillis()); tmpDir.mkdirs(); new File(tmpDir, "conf").mkdirs(); + FileUtils.copyDirectory(new File("src/test/resources/interpreter"), new File(tmpDir, "interpreter")); Map<String, InterpreterProperty> propertiesMockInterpreter1 = new HashMap<String, InterpreterProperty>(); propertiesMockInterpreter1.put("PROPERTY_1", new InterpreterProperty("PROPERTY_1", "", "VALUE_1", "desc")); @@ -62,11 +78,22 @@ public class InterpreterFactoryTest { MockInterpreter2.register("mock2", "org.apache.zeppelin.interpreter.mock.MockInterpreter2"); System.setProperty(ConfVars.ZEPPELIN_HOME.getVarName(), tmpDir.getAbsolutePath()); - System.setProperty(ConfVars.ZEPPELIN_INTERPRETERS.getVarName(), "org.apache.zeppelin.interpreter.mock.MockInterpreter1,org.apache.zeppelin.interpreter.mock.MockInterpreter2"); + System.setProperty(ConfVars.ZEPPELIN_INTERPRETERS.getVarName(), + "org.apache.zeppelin.interpreter.mock.MockInterpreter1," + + "org.apache.zeppelin.interpreter.mock.MockInterpreter2," + + "org.apache.zeppelin.interpreter.mock.MockInterpreter11"); + System.setProperty(ConfVars.ZEPPELIN_INTERPRETER_GROUP_ORDER.getVarName(), + "mock1,mock2,mock11,dev"); conf = new ZeppelinConfiguration(); + schedulerFactory = new SchedulerFactory(); depResolver = new DependencyResolver(tmpDir.getAbsolutePath() + "/local-repo"); factory = new InterpreterFactory(conf, new InterpreterOption(false), null, null, null, depResolver); context = new InterpreterContext("note", "id", "title", "text", null, null, null, null, null, null, null); + + SearchService search = mock(SearchService.class); + notebookRepo = new VFSNotebookRepo(conf); + notebook = new Notebook(conf, notebookRepo, schedulerFactory, factory, jobListenerFactory, search, + null, null); } @After @@ -128,7 +155,7 @@ public class InterpreterFactoryTest { public void testFactoryDefaultList() throws IOException, RepositoryException { // get default settings List<String> all = factory.getDefaultInterpreterSettingList(); - assertTrue(factory.getRegisteredInterpreterList().size() >= all.size()); + assertTrue(factory.get().size() >= all.size()); } @Test @@ -166,8 +193,8 @@ public class InterpreterFactoryTest { @Test public void testInterpreterAliases() throws IOException, RepositoryException { factory = new InterpreterFactory(conf, null, null, null, depResolver); - final InterpreterInfo info1 = new InterpreterInfo("className1", "name1", true); - final InterpreterInfo info2 = new InterpreterInfo("className2", "name1", true); + final InterpreterInfo info1 = new InterpreterInfo("className1", "name1", true, null); + final InterpreterInfo info2 = new InterpreterInfo("className2", "name1", true, null); factory.add("group1", new ArrayList<InterpreterInfo>(){{ add(info1); }}, new ArrayList<Dependency>(), new InterpreterOption(true), new Properties(), "/path1"); @@ -196,4 +223,29 @@ public class InterpreterFactoryTest { assertEquals("'.' is invalid for InterpreterSetting name.", e.getMessage()); } } + + + @Test + public void getEditorSetting() throws IOException, RepositoryException, SchedulerException { + List<String> intpIds = new ArrayList<>(); + for(InterpreterSetting intpSetting: factory.get()) { + if (intpSetting.getName().startsWith("mock1")) { + intpIds.add(intpSetting.getId()); + } + } + Note note = notebook.createNote(intpIds, null); + + // get editor setting from interpreter-setting.json + Map<String, Object> editor = factory.getEditorSetting(note.getId(), "mock11"); + assertEquals("java", editor.get("language")); + + // when interpreter is not loaded via interpreter-setting.json + // or editor setting doesn't exit + editor = factory.getEditorSetting(note.getId(), "mock1"); + assertEquals(null, editor.get("language")); + + // when interpreter is not bound to note + editor = factory.getEditorSetting(note.getId(), "mock2"); + assertEquals("text", editor.get("language")); + } } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java index 98e301a..a44bfad 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NoteTest.java @@ -19,7 +19,6 @@ package org.apache.zeppelin.notebook; import org.apache.zeppelin.interpreter.Interpreter; import org.apache.zeppelin.interpreter.InterpreterFactory; -import org.apache.zeppelin.interpreter.InterpreterSetting; import org.apache.zeppelin.notebook.repo.NotebookRepo; import org.apache.zeppelin.scheduler.Scheduler; import org.apache.zeppelin.search.SearchService; @@ -28,7 +27,6 @@ import org.junit.Test; import org.junit.runner.RunWith; import org.mockito.ArgumentCaptor; import org.mockito.Mock; -import org.mockito.Mockito; import org.mockito.runners.MockitoJUnitRunner; import static org.junit.Assert.*; http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java index 0ec8e7c..18d343c 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java @@ -75,6 +75,7 @@ public class NotebookTest implements JobListenerFactory{ notebookDir = new File(tmpDir + "/notebook"); notebookDir.mkdirs(); + System.setProperty(ConfVars.ZEPPELIN_CONF_DIR.getVarName(), tmpDir.toString() + "/conf"); System.setProperty(ConfVars.ZEPPELIN_HOME.getVarName(), tmpDir.getAbsolutePath()); System.setProperty(ConfVars.ZEPPELIN_NOTEBOOK_DIR.getVarName(), notebookDir.getAbsolutePath()); System.setProperty(ConfVars.ZEPPELIN_INTERPRETERS.getVarName(), "org.apache.zeppelin.interpreter.mock.MockInterpreter1,org.apache.zeppelin.interpreter.mock.MockInterpreter2"); @@ -95,8 +96,7 @@ public class NotebookTest implements JobListenerFactory{ credentials = new Credentials(conf.credentialsPersist(), conf.getCredentialsPath()); notebook = new Notebook(conf, notebookRepo, schedulerFactory, factory, this, search, - notebookAuthorization, credentials); - + notebookAuthorization, credentials); } @After @@ -109,7 +109,7 @@ public class NotebookTest implements JobListenerFactory{ Note note = notebook.createNote(null); factory.setInterpreters(note.getId(), factory.getDefaultInterpreterSettingList()); - // run with defatul repl + // run with default repl Paragraph p1 = note.addParagraph(); Map config = p1.getConfig(); config.put("enabled", true); @@ -232,7 +232,7 @@ public class NotebookTest implements JobListenerFactory{ p1.setText("hello world"); note.run(p1.getId()); - while(p1.isTerminated()==false || p1.getResult()==null) Thread.yield(); + while(p1.isTerminated() == false || p1.getResult() == null) Thread.yield(); assertEquals("repl1: hello world", p1.getResult().message()); // clear paragraph output/result @@ -267,7 +267,7 @@ public class NotebookTest implements JobListenerFactory{ note.runAll(); // wait for finish - while(p3.isTerminated()==false) { + while(p3.isTerminated() == false) { Thread.yield(); } @@ -279,7 +279,7 @@ public class NotebookTest implements JobListenerFactory{ } @Test - public void testSchedule() throws InterruptedException, IOException{ + public void testSchedule() throws InterruptedException, IOException { // create a note and a paragraph Note note = notebook.createNote(null); factory.setInterpreters(note.getId(), factory.getDefaultInterpreterSettingList()); @@ -297,7 +297,7 @@ public class NotebookTest implements JobListenerFactory{ config.put("cron", "* * * * * ?"); note.setConfig(config); notebook.refreshCron(note.getId()); - Thread.sleep(1*1000); + Thread.sleep(1 * 1000); // remove cron scheduler. config.put("cron", null); @@ -306,7 +306,7 @@ public class NotebookTest implements JobListenerFactory{ Thread.sleep(1000); dateFinished = p.getDateFinished(); assertNotNull(dateFinished); - Thread.sleep(1*1000); + Thread.sleep(1 * 1000); assertEquals(dateFinished, p.getDateFinished()); } @@ -476,8 +476,8 @@ public class NotebookTest implements JobListenerFactory{ p2.setText("%mock2 world"); note.runAll(); - while(p1.isTerminated()==false || p1.getResult()==null) Thread.yield(); - while(p2.isTerminated()==false || p2.getResult()==null) Thread.yield(); + while (p1.isTerminated() == false || p1.getResult() == null) Thread.yield(); + while (p2.isTerminated() == false || p2.getResult() == null) Thread.yield(); assertEquals(2, ResourcePoolUtils.getAllResources().size()); @@ -604,26 +604,26 @@ public class NotebookTest implements JobListenerFactory{ new HashSet<String>(Arrays.asList("user1"))); assertEquals(notebookAuthorization.isOwner(note.getId(), - new HashSet<String>(Arrays.asList("user2"))), false); + new HashSet<String>(Arrays.asList("user2"))), false); assertEquals(notebookAuthorization.isOwner(note.getId(), new HashSet<String>(Arrays.asList("user1"))), true); assertEquals(notebookAuthorization.isReader(note.getId(), - new HashSet<String>(Arrays.asList("user3"))), false); + new HashSet<String>(Arrays.asList("user3"))), false); assertEquals(notebookAuthorization.isReader(note.getId(), - new HashSet<String>(Arrays.asList("user2"))), true); + new HashSet<String>(Arrays.asList("user2"))), true); assertEquals(notebookAuthorization.isWriter(note.getId(), - new HashSet<String>(Arrays.asList("user2"))), false); + new HashSet<String>(Arrays.asList("user2"))), false); assertEquals(notebookAuthorization.isWriter(note.getId(), - new HashSet<String>(Arrays.asList("user1"))), true); + new HashSet<String>(Arrays.asList("user1"))), true); // Test clearing of permssions notebookAuthorization.setReaders(note.getId(), Sets.<String>newHashSet()); assertEquals(notebookAuthorization.isReader(note.getId(), - new HashSet<String>(Arrays.asList("user2"))), true); + new HashSet<String>(Arrays.asList("user2"))), true); assertEquals(notebookAuthorization.isReader(note.getId(), - new HashSet<String>(Arrays.asList("user3"))), true); + new HashSet<String>(Arrays.asList("user3"))), true); notebook.removeNote(note.getId(), null); } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/a3ca8003/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json ---------------------------------------------------------------------- diff --git a/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json b/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json new file mode 100644 index 0000000..65568ef --- /dev/null +++ b/zeppelin-zengine/src/test/resources/interpreter/mock/interpreter-setting.json @@ -0,0 +1,12 @@ +[ + { + "group": "mock11", + "name": "mock11", + "className": "org.apache.zeppelin.interpreter.mock.MockInterpreter11", + "properties": { + }, + "editor": { + "language": "java" + } + } +]
![https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png"](https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png"){kind=link}
![https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png"](https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png"){kind=link}
![https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png"](https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png"){kind=link}
![https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png"](https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png"){kind=link}