[ZEPPELIN-2008] Introduce Spell ### What is this PR for?
Implemented **Spell** as one of Helium categories. *Technically, it's the frontend interpreter* runs on browser not backend. Spell can provide many benefits. 1. Anyone can install, remove easily using helium package registry by #1936 2. Implementing spell is extremely easier rather than adding backend interpreter 3. Can use existing javsacript libraries. (e.g [flowchart.js](http://flowchart.js.org/), [sequence diagram js](https://github.com/bramp/js-sequence-diagrams), ...). This enable us to add many visualization tools. Imagine that you can implement some custom interpreters with few lines of code like [flowchart-spell-example](https://github.com/apache/zeppelin/compare/master...1ambda:ZEPPELIN-2008/introduce-spell?expand=1#diff-364845b20d68e4d94688e44fef03da98) 4. The most important thing is, spell is not only interpreter but also display system. Because it runs on browser. So we can use spell display system with another spell **Display System with Spell** (see the screenshot section below) **In future**, we will be able to combine existing backend interpreters with spell like (**not supported in this PR cause we need to modify backend code a lot**) ``` // if we have markdown spell, we can use `%markdown` display in the spark interpreter %spark val calculated = doSomething() println(s"%markdown _${calculated}) ``` I added some examples. Checkout `echo`, `markdown`, `translator`, `flowchart` spells. ### What type of PR is it? [Feature] ### Todos * [x] - Add `SPELL` as one of Helium categories. * [x] - Implement framework code (`zeppelin-spell`) * [x] - Make some examples (flowchart, google translator, markdown, echo) * [x] - Support custom display system * [x] - Fix some bugs in `HeliumBundleFactory` * [x] - Save spell rendering result into `note.json` while broadcasting to other websocket clients * [x] - Fix `renderText` for stream output ### What is the Jira issue? [ZEPPELIN-2008](https://issues.apache.org/jira/browse/ZEPPELIN-2008) ### How should this be tested? - Build `mvn clean package -Phelium-dev -Pexamples -DskipTests;` - Go to helium page `http://localhost:8080/#/helium` - Enable all spells - Go to a notebook and refresh - Follow actions in the screenshots below. ### Screenshots (if appropriate) #### Flowchart Spell (Sample)  #### Google Translator Spell (Sample)  #### Display System with Spell  ### Questions: * Does the licenses files need update - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - YES, but framework can be enhanced so i would like to defer to write document right now. Author: 1ambda <[email protected]> Closes #1940 from 1ambda/ZEPPELIN-2008/introduce-spell and squashes the following commits: c1b5356 [1ambda] fix: RAT issues e07ecd3 [1ambda] fix: Set width for spell usage 6c91892 [1ambda] feat: Display magic, usage for spell 5be2890 [1ambda] feat: Support spell info 822a1d8 [1ambda] style: Remove useless func wrap for helium 35d0fcc [1ambda] fix: Update desc for spell examples 49e03fc [1ambda] fix: List visualziation bundles only in order 4029c02 [1ambda] fix: ParagraphIT, parameterizedQueryForm 08eba10 [1ambda] refactor: renderGraph in result.controller.js 69ce880 [1ambda] fix: Resolve append (stream) output 0f2d8b6 [1ambda] fix: Resolve output issue fc4389e [1ambda] fix: Resolve RAT issues c8c8f0e [1ambda] fix: Add setErrorMessage method to Job 4fec44c [1ambda] refactor: NotebookServer.java 1227d7d [1ambda] refactor: result controller retry 9fb7438 [1ambda] feat: Save spell result and propagate 3cdf2da [1ambda] fix: NPM installation error 72aadbf [1ambda] feat: Enhance translator spell bd2b3ef [1ambda] style: Rename generator -> data cac0667 [1ambda] style: Rename to Spell e81cb03 [1ambda] example: Add echo, markdown 0fa7eda [1ambda] feat: Support custom display c906da6 [1ambda] feat: Update examples to use single FrontIntpRes 5c49e6e [1ambda] feat: Automated display type checking in result 5810bf1 [1ambda] feat: Apply frontend interpreter to paragraph a163044 [1ambda] feat: Add flowchart, translator examples 247d00f [1ambda] feat: Add frontend interpreter framework e925967 [1ambda] feat: Support FRONTEND_INTERPRETER type in frontend c02d00a [1ambda] feat: Support FRONTEND_INTERPRETER type in backend Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/0589e27e Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/0589e27e Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/0589e27e Branch: refs/heads/master Commit: 0589e27e7bb84ec81e1438bcbf3f2fd80ee5a963 Parents: 019df1f Author: 1ambda <[email protected]> Authored: Mon Jan 30 12:44:55 2017 +0900 Committer: Lee moon soo <[email protected]> Committed: Thu Feb 2 08:33:48 2017 +0900 ---------------------------------------------------------------------- pom.xml | 1 + .../src/assemble/distribution.xml | 4 + zeppelin-examples/pom.xml | 4 + .../zeppelin-example-spell-echo/index.js | 32 ++ .../zeppelin-example-spell-echo/package.json | 15 + .../zeppelin-example-spell-echo/pom.xml | 116 ++++ .../zeppelin-example-spell-echo.json | 28 + .../zeppelin-example-spell-flowchart/index.js | 108 ++++ .../package.json | 17 + .../zeppelin-example-spell-flowchart/pom.xml | 116 ++++ .../zeppelin-example-spell-flowchart.json | 28 + .../zeppelin-example-spell-markdown/index.js | 42 ++ .../package.json | 16 + .../zeppelin-example-spell-markdown/pom.xml | 116 ++++ .../zeppelin-example-spell-markdown.json | 28 + .../zeppelin-example-spell-translator/index.js | 93 ++++ .../package.json | 16 + .../zeppelin-example-spell-translator/pom.xml | 116 ++++ .../zeppelin-example-spell-translator.json | 28 + .../zeppelin/helium/ApplicationLoader.java | 2 +- .../apache/zeppelin/helium/HeliumPackage.java | 25 +- .../org/apache/zeppelin/helium/HeliumType.java | 29 + .../zeppelin/helium/SpellPackageInfo.java | 34 ++ .../java/org/apache/zeppelin/scheduler/Job.java | 12 +- .../zeppelin/helium/ApplicationLoaderTest.java | 4 +- .../zeppelin/helium/HeliumPackageTest.java | 48 ++ .../org/apache/zeppelin/rest/HeliumRestApi.java | 21 +- .../apache/zeppelin/server/ZeppelinServer.java | 24 +- .../apache/zeppelin/socket/NotebookServer.java | 260 ++++++--- .../integration/ParagraphActionsIT.java | 3 +- zeppelin-web/package.json | 2 +- .../src/app/helium/helium.controller.js | 379 +++++++------ zeppelin-web/src/app/helium/helium.css | 34 +- zeppelin-web/src/app/helium/helium.html | 32 +- .../src/app/notebook/notebook.controller.js | 6 +- .../notebook/paragraph/paragraph-control.html | 2 +- .../paragraph-parameterizedQueryForm.html | 4 +- .../notebook/paragraph/paragraph.controller.js | 367 +++++++++---- .../src/app/notebook/paragraph/paragraph.html | 4 +- .../paragraph/result/result.controller.js | 534 +++++++++++-------- .../app/notebook/paragraph/result/result.html | 17 +- zeppelin-web/src/app/spell/.npmignore | 1 + zeppelin-web/src/app/spell/index.js | 25 + zeppelin-web/src/app/spell/package.json | 13 + zeppelin-web/src/app/spell/spell-base.js | 48 ++ zeppelin-web/src/app/spell/spell-result.js | 275 ++++++++++ .../src/components/helium/helium-type.js | 18 + .../src/components/helium/helium.service.js | 115 ++-- .../websocketEvents/websocketEvents.factory.js | 2 + .../websocketEvents/websocketMsg.service.js | 25 + zeppelin-web/test/spec/controllers/paragraph.js | 4 - zeppelin-web/webpack.config.js | 2 +- .../java/org/apache/zeppelin/helium/Helium.java | 73 +-- .../zeppelin/helium/HeliumBundleFactory.java | 415 ++++++++++++++ .../org/apache/zeppelin/helium/HeliumConf.java | 12 +- .../helium/HeliumVisualizationFactory.java | 376 ------------- .../zeppelin/notebook/socket/Message.java | 4 +- .../src/main/resources/helium/package.json | 2 +- .../src/main/resources/helium/webpack.config.js | 2 +- .../helium/HeliumApplicationFactoryTest.java | 8 +- .../helium/HeliumBundleFactoryTest.java | 197 +++++++ .../helium/HeliumLocalRegistryTest.java | 2 +- .../org/apache/zeppelin/helium/HeliumTest.java | 5 +- .../helium/HeliumVisualizationFactoryTest.java | 193 ------- 64 files changed, 3243 insertions(+), 1341 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 7772369..a677443 100644 --- a/pom.xml +++ b/pom.xml @@ -893,6 +893,7 @@ <exclude>conf/notebook-authorization.json</exclude> <exclude>conf/credentials.json</exclude> <exclude>conf/zeppelin-env.sh</exclude> + <exclude>conf/helium.json</exclude> <exclude>spark-*-bin*/**</exclude> <exclude>.spark-dist/**</exclude> <exclude>**/interpreter-setting.json</exclude> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-distribution/src/assemble/distribution.xml ---------------------------------------------------------------------- diff --git a/zeppelin-distribution/src/assemble/distribution.xml b/zeppelin-distribution/src/assemble/distribution.xml index e8188e8..5c369e2 100644 --- a/zeppelin-distribution/src/assemble/distribution.xml +++ b/zeppelin-distribution/src/assemble/distribution.xml @@ -103,5 +103,9 @@ <outputDirectory>/lib/node_modules/zeppelin-tabledata</outputDirectory> <directory>../zeppelin-web/src/app/tabledata</directory> </fileSet> + <fileSet> + <outputDirectory>/lib/node_modules/zeppelin-spell</outputDirectory> + <directory>../zeppelin-web/src/app/spell</directory> + </fileSet> </fileSets> </assembly> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-examples/pom.xml b/zeppelin-examples/pom.xml index 300ba57..e9f0473 100644 --- a/zeppelin-examples/pom.xml +++ b/zeppelin-examples/pom.xml @@ -36,6 +36,10 @@ <modules> <module>zeppelin-example-clock</module> <module>zeppelin-example-horizontalbar</module> + <module>zeppelin-example-spell-flowchart</module> + <module>zeppelin-example-spell-translator</module> + <module>zeppelin-example-spell-markdown</module> + <module>zeppelin-example-spell-echo</module> </modules> <build> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-echo/index.js ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-echo/index.js b/zeppelin-examples/zeppelin-example-spell-echo/index.js new file mode 100644 index 0000000..955178e --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-echo/index.js @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + SpellBase, + SpellResult, + DefaultDisplayType, +} from 'zeppelin-spell'; + +export default class EchoSpell extends SpellBase { + constructor() { + super("%echo"); + } + + interpret(paragraphText) { + return new SpellResult(paragraphText); + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-echo/package.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-echo/package.json b/zeppelin-examples/zeppelin-example-spell-echo/package.json new file mode 100644 index 0000000..2d9710e --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-echo/package.json @@ -0,0 +1,15 @@ +{ + "name": "echo-spell", + "description" : "Return just what receive (example)", + "version": "1.0.0", + "main": "index", + "author": "", + "license": "Apache-2.0", + "dependencies": { + "zeppelin-spell": "*" + }, + "spell": { + "magic": "%echo", + "usage": "%echo <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-echo/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-echo/pom.xml b/zeppelin-examples/zeppelin-example-spell-echo/pom.xml new file mode 100644 index 0000000..348abd2 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-echo/pom.xml @@ -0,0 +1,116 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one or more + ~ contributor license agreements. See the NOTICE file distributed with + ~ this work for additional information regarding copyright ownership. + ~ The ASF licenses this file to You under the Apache License, Version 2.0 + ~ (the "License"); you may not use this file except in compliance with + ~ the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, software + ~ distributed under the License is distributed on an "AS IS" BASIS, + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + ~ See the License for the specific language governing permissions and + ~ limitations under the License. + --> + +<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <artifactId>zeppelin-examples</artifactId> + <groupId>org.apache.zeppelin</groupId> + <version>0.8.0-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <groupId>org.apache.zeppelin</groupId> + <artifactId>zeppelin-example-spell-echo</artifactId> + <packaging>jar</packaging> + <version>0.8.0-SNAPSHOT</version> + <name>Zeppelin: Example Spell - Echo</name> + + <dependencies> + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>zeppelin-interpreter</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>helium-dev</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-api</artifactId> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-log4j12</artifactId> + </dependency> + + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <scope>test</scope> + </dependency> + </dependencies> + + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-deploy-plugin</artifactId> + <version>2.7</version> + <configuration> + <skip>true</skip> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-clean-plugin</artifactId> + <configuration> + <filesets> + <fileset> + <directory>${project.basedir}/../../helium</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </fileset> + </filesets> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-resources-plugin</artifactId> + <version>2.7</version> + <executions> + <execution> + <phase>generate-resources</phase> + <goals> + <goal>copy-resources</goal> + </goals> + + <configuration> + <outputDirectory>${project.basedir}/../../helium/</outputDirectory> + <resources> + <resource> + <directory>${project.basedir}</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </resource> + </resources> + </configuration> + </execution> + </executions> + </plugin> + </plugins> + </build> +</project> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json b/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json new file mode 100644 index 0000000..f267b97 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-echo/zeppelin-example-spell-echo.json @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +{ + "type" : "SPELL", + "name" : "echo-spell", + "description" : "Return just what receive (example)", + "artifact" : "./zeppelin-examples/zeppelin-example-spell-echo", + "license" : "Apache-2.0", + "icon" : "<i class='fa fa-repeat'></i>", + "spell": { + "magic": "%echo", + "usage": "%echo <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-flowchart/index.js ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-flowchart/index.js b/zeppelin-examples/zeppelin-example-spell-flowchart/index.js new file mode 100644 index 0000000..655814a --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-flowchart/index.js @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + SpellBase, + SpellResult, + DefaultDisplayType, +} from 'zeppelin-spell'; + +import flowchart from 'flowchart.js'; + +export default class FlowchartSpell extends SpellBase { + constructor() { + super("%flowchart"); + } + + interpret(paragraphText) { + /** + * `flowchart` library requires an existing DOM to render. + * but the DOM is not created yet when `interpret` is called. + * so Zeppelin allows to return callback function which accept a DOM element id. + * the callback function will executed when the DOM is ready. + */ + const callback = (targetElemId) => { + let diagram = flowchart.parse(paragraphText); + diagram.drawSVG(targetElemId, this.getOption()); + }; + + /** + * `interpret` method can return multiple results using `add()` + * but now, we return just 1 result + */ + return new SpellResult( + callback + ); + } + + getOption() { + return { + 'x': 0, + 'y': 0, + 'line-width': 3, + 'line-length': 50, + 'text-margin': 10, + 'font-size': 14, + 'font-color': 'black', + 'line-color': 'black', + 'element-color': 'black', + 'fill': 'white', + 'yes-text': 'yes', + 'no-text': 'no', + 'arrow-end': 'block', + 'scale': 1, + // style symbol types + 'symbols': { + 'start': { + 'font-color': 'red', + 'element-color': 'green', + 'fill': 'yellow' + }, + 'end':{ + 'class': 'end-element' + } + }, + // even flowstate support ;-) + 'flowstate' : { + 'past' : { 'fill' : '#CCCCCC', 'font-size' : 12}, + 'current' : {'fill' : 'yellow', 'font-color' : 'red', 'font-weight' : 'bold'}, + 'future' : { 'fill' : '#FFFF99'}, + 'request' : { 'fill' : 'blue'}, + 'invalid': {'fill' : '#444444'}, + 'approved' : { 'fill' : '#58C4A3', 'font-size' : 12, 'yes-text' : 'APPROVED', 'no-text' : 'n/a' }, + 'rejected' : { 'fill' : '#C45879', 'font-size' : 12, 'yes-text' : 'n/a', 'no-text' : 'REJECTED' } + } + } + } +} + + + + + + + + + + + + + + + + + http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-flowchart/package.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-flowchart/package.json b/zeppelin-examples/zeppelin-example-spell-flowchart/package.json new file mode 100644 index 0000000..24be73b --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-flowchart/package.json @@ -0,0 +1,17 @@ +{ + "name": "flowchart-spell", + "description" : "Draw flowchart using http://flowchart.js.org (example)", + "version": "1.0.0", + "main": "index", + "author": "", + "license": "Apache-2.0", + "dependencies": { + "raphael": "2.2.0", + "flowchart.js": "^1.6.5", + "zeppelin-spell": "*" + }, + "spell": { + "magic": "%flowchart", + "usage": "%flowchart <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-flowchart/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-flowchart/pom.xml b/zeppelin-examples/zeppelin-example-spell-flowchart/pom.xml new file mode 100644 index 0000000..b3575c9 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-flowchart/pom.xml @@ -0,0 +1,116 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one or more + ~ contributor license agreements. See the NOTICE file distributed with + ~ this work for additional information regarding copyright ownership. + ~ The ASF licenses this file to You under the Apache License, Version 2.0 + ~ (the "License"); you may not use this file except in compliance with + ~ the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, software + ~ distributed under the License is distributed on an "AS IS" BASIS, + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + ~ See the License for the specific language governing permissions and + ~ limitations under the License. + --> + +<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <artifactId>zeppelin-examples</artifactId> + <groupId>org.apache.zeppelin</groupId> + <version>0.8.0-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <groupId>org.apache.zeppelin</groupId> + <artifactId>zeppelin-example-spell-flowchart</artifactId> + <packaging>jar</packaging> + <version>0.8.0-SNAPSHOT</version> + <name>Zeppelin: Example Spell - Flowchart</name> + + <dependencies> + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>zeppelin-interpreter</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>helium-dev</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-api</artifactId> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-log4j12</artifactId> + </dependency> + + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <scope>test</scope> + </dependency> + </dependencies> + + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-deploy-plugin</artifactId> + <version>2.7</version> + <configuration> + <skip>true</skip> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-clean-plugin</artifactId> + <configuration> + <filesets> + <fileset> + <directory>${project.basedir}/../../helium</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </fileset> + </filesets> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-resources-plugin</artifactId> + <version>2.7</version> + <executions> + <execution> + <phase>generate-resources</phase> + <goals> + <goal>copy-resources</goal> + </goals> + + <configuration> + <outputDirectory>${project.basedir}/../../helium/</outputDirectory> + <resources> + <resource> + <directory>${project.basedir}</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </resource> + </resources> + </configuration> + </execution> + </executions> + </plugin> + </plugins> + </build> +</project> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-flowchart/zeppelin-example-spell-flowchart.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-flowchart/zeppelin-example-spell-flowchart.json b/zeppelin-examples/zeppelin-example-spell-flowchart/zeppelin-example-spell-flowchart.json new file mode 100644 index 0000000..0ea6e41 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-flowchart/zeppelin-example-spell-flowchart.json @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +{ + "type" : "SPELL", + "name" : "flowchart-spell", + "description" : "Draw flowchart using http://flowchart.js.org (example)", + "artifact" : "./zeppelin-examples/zeppelin-example-spell-flowchart", + "license" : "Apache-2.0", + "icon" : "<i class='fa fa-random'></i>", + "spell": { + "magic": "%flowchart", + "usage": "%flowchart <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-markdown/index.js ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-markdown/index.js b/zeppelin-examples/zeppelin-example-spell-markdown/index.js new file mode 100644 index 0000000..db7959f --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-markdown/index.js @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + SpellBase, + SpellResult, + DefaultDisplayType, +} from 'zeppelin-spell'; + +import md from 'markdown'; + +const markdown = md.markdown; + +export default class MarkdownSpell extends SpellBase { + constructor() { + super("%markdown"); + } + + interpret(paragraphText) { + const parsed = markdown.toHTML(paragraphText); + + /** + * specify `DefaultDisplayType.HTML` since `parsed` will contain DOM + * otherwise it will be rendered as `DefaultDisplayType.TEXT` (default) + */ + return new SpellResult(parsed, DefaultDisplayType.HTML); + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-markdown/package.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-markdown/package.json b/zeppelin-examples/zeppelin-example-spell-markdown/package.json new file mode 100644 index 0000000..997a2a2 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-markdown/package.json @@ -0,0 +1,16 @@ +{ + "name": "markdown-spell", + "description" : "Parse markdown using https://github.com/evilstreak/markdown-js (example)", + "version": "1.0.0", + "main": "index", + "author": "", + "license": "Apache-2.0", + "dependencies": { + "markdown": "0.5.0", + "zeppelin-spell": "*" + }, + "spell": { + "magic": "%markdown", + "usage": "%markdown <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-markdown/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-markdown/pom.xml b/zeppelin-examples/zeppelin-example-spell-markdown/pom.xml new file mode 100644 index 0000000..b615ead --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-markdown/pom.xml @@ -0,0 +1,116 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one or more + ~ contributor license agreements. See the NOTICE file distributed with + ~ this work for additional information regarding copyright ownership. + ~ The ASF licenses this file to You under the Apache License, Version 2.0 + ~ (the "License"); you may not use this file except in compliance with + ~ the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, software + ~ distributed under the License is distributed on an "AS IS" BASIS, + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + ~ See the License for the specific language governing permissions and + ~ limitations under the License. + --> + +<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <artifactId>zeppelin-examples</artifactId> + <groupId>org.apache.zeppelin</groupId> + <version>0.8.0-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <groupId>org.apache.zeppelin</groupId> + <artifactId>zeppelin-example-spell-markdown</artifactId> + <packaging>jar</packaging> + <version>0.8.0-SNAPSHOT</version> + <name>Zeppelin: Example Spell - Markdown</name> + + <dependencies> + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>zeppelin-interpreter</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>helium-dev</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-api</artifactId> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-log4j12</artifactId> + </dependency> + + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <scope>test</scope> + </dependency> + </dependencies> + + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-deploy-plugin</artifactId> + <version>2.7</version> + <configuration> + <skip>true</skip> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-clean-plugin</artifactId> + <configuration> + <filesets> + <fileset> + <directory>${project.basedir}/../../helium</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </fileset> + </filesets> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-resources-plugin</artifactId> + <version>2.7</version> + <executions> + <execution> + <phase>generate-resources</phase> + <goals> + <goal>copy-resources</goal> + </goals> + + <configuration> + <outputDirectory>${project.basedir}/../../helium/</outputDirectory> + <resources> + <resource> + <directory>${project.basedir}</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </resource> + </resources> + </configuration> + </execution> + </executions> + </plugin> + </plugins> + </build> +</project> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-markdown/zeppelin-example-spell-markdown.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-markdown/zeppelin-example-spell-markdown.json b/zeppelin-examples/zeppelin-example-spell-markdown/zeppelin-example-spell-markdown.json new file mode 100644 index 0000000..48ad246 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-markdown/zeppelin-example-spell-markdown.json @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +{ + "type" : "SPELL", + "name" : "markdown-spell", + "description" : "Parse markdown using https://github.com/evilstreak/markdown-js (example)", + "artifact" : "./zeppelin-examples/zeppelin-example-spell-markdown", + "license" : "Apache-2.0", + "icon" : "<i class='fa fa-bold'></i>", + "spell": { + "magic": "%markdown", + "usage": "%markdown <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-translator/index.js ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-translator/index.js b/zeppelin-examples/zeppelin-example-spell-translator/index.js new file mode 100644 index 0000000..834e707 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-translator/index.js @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { + SpellBase, + SpellResult, + DefaultDisplayType, +} from 'zeppelin-spell'; + +import 'whatwg-fetch'; + +export default class TranslatorSpell extends SpellBase { + constructor() { + super("%translator"); + } + + interpret(paragraphText) { + const parsed = this.parseConfig(paragraphText); + const source = parsed.source; + const target = parsed.target; + const auth = parsed.auth; + const text = parsed.text; + + /** + * SpellResult.add() + * - accepts not only `string` but also `promise` as a parameter + * - allows to add multiple output using the `add()` function + */ + const result = new SpellResult() + .add('<h4>Translation Result</h4>', DefaultDisplayType.HTML) + // or use display system implicitly like + // .add('%html <h4>Translation From English To Korean</h4>') + .add(this.translate(source, target, auth, text)); + return result; + } + + parseConfig(text) { + const pattern = /^\s*(\S+)-(\S+)\s*(\S+)([\S\s]*)/g; + const match = pattern.exec(text); + + if (!match) { + throw new Error(`Failed to parse configuration. See README`); + } + + return { + source: match[1], + target: match[2], + auth: match[3], + text: match[4], + } + } + + translate(source, target, auth, text) { + return fetch('https://translation.googleapis.com/language/translate/v2', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'Authorization': `Bearer ${auth}`, + }, + body: JSON.stringify({ + 'q': text, + 'source': source, + 'target': target, + 'format': 'text' + }) + }).then(response => { + if (response.status === 200) { + return response.json() + } + throw new Error(`https://translation.googleapis.com/language/translate/v2 ${response.status} (${response.statusText})`); + }).then((json) => { + const extracted = json.data.translations.map(t => { + return t.translatedText; + }); + return extracted.join('\n'); + }); + } +} + http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-translator/package.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-translator/package.json b/zeppelin-examples/zeppelin-example-spell-translator/package.json new file mode 100644 index 0000000..90624f8 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-translator/package.json @@ -0,0 +1,16 @@ +{ + "name": "translator-spell", + "description" : "Translate langauges using Google API (examaple)", + "version": "1.0.0", + "main": "index", + "author": "", + "license": "Apache-2.0", + "dependencies": { + "whatwg-fetch": "^2.0.1", + "zeppelin-spell": "*" + }, + "spell": { + "magic": "%translator", + "usage": "%translator <source>-<target> <access-key> <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-translator/pom.xml ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-translator/pom.xml b/zeppelin-examples/zeppelin-example-spell-translator/pom.xml new file mode 100644 index 0000000..09e6daa --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-translator/pom.xml @@ -0,0 +1,116 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- + ~ Licensed to the Apache Software Foundation (ASF) under one or more + ~ contributor license agreements. See the NOTICE file distributed with + ~ this work for additional information regarding copyright ownership. + ~ The ASF licenses this file to You under the Apache License, Version 2.0 + ~ (the "License"); you may not use this file except in compliance with + ~ the License. You may obtain a copy of the License at + ~ + ~ http://www.apache.org/licenses/LICENSE-2.0 + ~ + ~ Unless required by applicable law or agreed to in writing, software + ~ distributed under the License is distributed on an "AS IS" BASIS, + ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + ~ See the License for the specific language governing permissions and + ~ limitations under the License. + --> + +<project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd";> + <modelVersion>4.0.0</modelVersion> + + <parent> + <artifactId>zeppelin-examples</artifactId> + <groupId>org.apache.zeppelin</groupId> + <version>0.8.0-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <groupId>org.apache.zeppelin</groupId> + <artifactId>zeppelin-example-spell-translator</artifactId> + <packaging>jar</packaging> + <version>0.8.0-SNAPSHOT</version> + <name>Zeppelin: Example Spell - Translator</name> + + <dependencies> + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>zeppelin-interpreter</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>${project.groupId}</groupId> + <artifactId>helium-dev</artifactId> + <version>${project.version}</version> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-api</artifactId> + </dependency> + + <dependency> + <groupId>org.slf4j</groupId> + <artifactId>slf4j-log4j12</artifactId> + </dependency> + + <dependency> + <groupId>junit</groupId> + <artifactId>junit</artifactId> + <scope>test</scope> + </dependency> + </dependencies> + + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-deploy-plugin</artifactId> + <version>2.7</version> + <configuration> + <skip>true</skip> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-clean-plugin</artifactId> + <configuration> + <filesets> + <fileset> + <directory>${project.basedir}/../../helium</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </fileset> + </filesets> + </configuration> + </plugin> + + <plugin> + <artifactId>maven-resources-plugin</artifactId> + <version>2.7</version> + <executions> + <execution> + <phase>generate-resources</phase> + <goals> + <goal>copy-resources</goal> + </goals> + + <configuration> + <outputDirectory>${project.basedir}/../../helium/</outputDirectory> + <resources> + <resource> + <directory>${project.basedir}</directory> + <includes> + <include>${project.artifactId}.json</include> + </includes> + </resource> + </resources> + </configuration> + </execution> + </executions> + </plugin> + </plugins> + </build> +</project> http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json ---------------------------------------------------------------------- diff --git a/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json b/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json new file mode 100644 index 0000000..8f99783 --- /dev/null +++ b/zeppelin-examples/zeppelin-example-spell-translator/zeppelin-example-spell-translator.json @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +{ + "type" : "SPELL", + "name" : "translator-spell", + "description" : "Translate langauges using Google API (examaple)", + "artifact" : "./zeppelin-examples/zeppelin-example-spell-translator", + "license" : "Apache-2.0", + "icon" : "<i class='fa fa-globe '></i>", + "spell": { + "magic": "%translator", + "usage": "%translator <source>-<target> <access-key> <TEXT>" + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/ApplicationLoader.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/ApplicationLoader.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/ApplicationLoader.java index eacef51..ddd061c 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/ApplicationLoader.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/ApplicationLoader.java @@ -102,7 +102,7 @@ public class ApplicationLoader { */ public Application load(HeliumPackage packageInfo, ApplicationContext context) throws Exception { - if (packageInfo.getType() != HeliumPackage.Type.APPLICATION) { + if (packageInfo.getType() != HeliumType.APPLICATION) { throw new ApplicationException( "Can't instantiate " + packageInfo.getType() + " package using ApplicationLoader"); } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java index 84a2ab3..e8e6b7c 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java @@ -23,7 +23,7 @@ import org.apache.zeppelin.annotation.Experimental; */ @Experimental public class HeliumPackage { - private Type type; + private HeliumType type; private String name; // user friendly name of this application private String description; // description private String artifact; // artifact name e.g) groupId:artifactId:versionId @@ -33,17 +33,9 @@ public class HeliumPackage { private String license; private String icon; - /** - * Type of package - */ - public static enum Type { - INTERPRETER, - NOTEBOOK_REPO, - APPLICATION, - VISUALIZATION - } + public SpellPackageInfo spell; - public HeliumPackage(Type type, + public HeliumPackage(HeliumType type, String name, String description, String artifact, @@ -76,10 +68,15 @@ public class HeliumPackage { return type == info.type && artifact.equals(info.artifact) && className.equals(info.className); } - public Type getType() { + public HeliumType getType() { return type; } + public static boolean isBundleType(HeliumType type) { + return (type == HeliumType.VISUALIZATION || + type == HeliumType.SPELL); + } + public String getName() { return name; } @@ -106,4 +103,8 @@ public class HeliumPackage { public String getIcon() { return icon; } + + public SpellPackageInfo getSpellInfo() { + return spell; + } } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumType.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumType.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumType.java new file mode 100644 index 0000000..53360a0 --- /dev/null +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumType.java @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.helium; + +/** + * Type of Helium Package + */ +public enum HeliumType { + INTERPRETER, + NOTEBOOK_REPO, + APPLICATION, + VISUALIZATION, + SPELL +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/SpellPackageInfo.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/SpellPackageInfo.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/SpellPackageInfo.java new file mode 100644 index 0000000..519d09d --- /dev/null +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/SpellPackageInfo.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.helium; + +/** + * Info for Helium Spell Package. + */ +public class SpellPackageInfo { + private String magic; + private String usage; + + public String getMagic() { + return magic; + } + + public String getUsage() { + return usage; + } +} http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java index a690bef..76d90b9 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java @@ -64,7 +64,6 @@ public abstract class Job { } } - private String jobName; String id; @@ -135,6 +134,13 @@ public abstract class Job { return status; } + /** + * just set status without notifying to listeners for spell. + */ + public void setStatusWithoutNotification(Status status) { + this.status = status; + } + public void setStatus(Status status) { if (this.status == status) { return; @@ -257,4 +263,8 @@ public abstract class Job { } public abstract void setResult(Object results); + + public void setErrorMessage(String errorMessage) { + this.errorMessage = errorMessage; + } } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/ApplicationLoaderTest.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/ApplicationLoaderTest.java b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/ApplicationLoaderTest.java index 3924e28..acb4d7f 100644 --- a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/ApplicationLoaderTest.java +++ b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/ApplicationLoaderTest.java @@ -20,8 +20,6 @@ package org.apache.zeppelin.helium; import org.apache.commons.io.FileUtils; import org.apache.zeppelin.dep.DependencyResolver; import org.apache.zeppelin.interpreter.InterpreterOutput; -import org.apache.zeppelin.interpreter.InterpreterOutputListener; -import org.apache.zeppelin.interpreter.InterpreterResultMessageOutput; import org.apache.zeppelin.resource.LocalResourcePool; import org.junit.After; import org.junit.Before; @@ -74,7 +72,7 @@ public class ApplicationLoaderTest { public HeliumPackage createPackageInfo(String className, String artifact) { HeliumPackage app1 = new HeliumPackage( - HeliumPackage.Type.APPLICATION, + HeliumType.APPLICATION, "name1", "desc1", artifact, http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java ---------------------------------------------------------------------- diff --git a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java new file mode 100644 index 0000000..aadae41 --- /dev/null +++ b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/helium/HeliumPackageTest.java @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.helium; + +import com.google.gson.Gson; +import org.junit.Test; + +import static org.junit.Assert.*; + +public class HeliumPackageTest { + + private Gson gson = new Gson(); + + @Test + public void parseSpellPackageInfo() { + String exampleSpell = "{\n" + + " \"type\" : \"SPELL\",\n" + + " \"name\" : \"echo-spell\",\n" + + " \"description\" : \"'%echo' - return just what receive (example)\",\n" + + " \"artifact\" : \"./zeppelin-examples/zeppelin-example-spell-echo\",\n" + + " \"license\" : \"Apache-2.0\",\n" + + " \"icon\" : \"<i class='fa fa-repeat'></i>\",\n" + + " \"spell\": {\n" + + " \"magic\": \"%echo\",\n" + + " \"usage\": \"%echo <TEXT>\"\n" + + " }\n" + + "}"; + + HeliumPackage p = gson.fromJson(exampleSpell, HeliumPackage.class); + assertEquals(p.getSpellInfo().getMagic(), "%echo"); + assertEquals(p.getSpellInfo().getUsage(), "%echo <TEXT>"); + } +} \ No newline at end of file http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java b/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java index e5cf70d..c318be5 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/rest/HeliumRestApi.java @@ -17,7 +17,6 @@ package org.apache.zeppelin.rest; -import com.github.eirslett.maven.plugins.frontend.lib.TaskRunnerException; import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; import org.apache.commons.io.FileUtils; @@ -107,22 +106,22 @@ public class HeliumRestApi { } @GET - @Path("visualizations/load") + @Path("bundle/load") @Produces("text/javascript") - public Response visualizationLoad(@QueryParam("refresh") String refresh) { + public Response bundleLoad(@QueryParam("refresh") String refresh) { try { File bundle; if (refresh != null && refresh.equals("true")) { - bundle = helium.recreateVisualizationBundle(); + bundle = helium.recreateBundle(); } else { - bundle = helium.getVisualizationFactory().getCurrentBundle(); + bundle = helium.getBundleFactory().getCurrentCacheBundle(); } if (bundle == null) { return Response.ok().build(); } else { - String visBundle = FileUtils.readFileToString(bundle); - return Response.ok(visBundle).build(); + String stringifiedBundle = FileUtils.readFileToString(bundle); + return Response.ok(stringifiedBundle).build(); } } catch (Exception e) { logger.error(e.getMessage(), e); @@ -160,15 +159,15 @@ public class HeliumRestApi { } @GET - @Path("visualizationOrder") + @Path("order/visualization") public Response getVisualizationPackageOrder() { - List<String> order = helium.getVisualizationPackageOrder(); + List<String> order = helium.setVisualizationPackageOrder(); return new JsonResponse(Response.Status.OK, order).build(); } @POST - @Path("visualizationOrder") - public Response setVisualizationPackageOrder(String orderedPackageNameList) { + @Path("order/visualization") + public Response getVisualizationPackageOrder(String orderedPackageNameList) { List<String> orderedList = gson.fromJson( orderedPackageNameList, new TypeToken<List<String>>(){}.getType()); http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java index 6c4fcd8..371d0a1 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java @@ -35,7 +35,7 @@ import org.apache.zeppelin.conf.ZeppelinConfiguration.ConfVars; import org.apache.zeppelin.dep.DependencyResolver; import org.apache.zeppelin.helium.Helium; import org.apache.zeppelin.helium.HeliumApplicationFactory; -import org.apache.zeppelin.helium.HeliumVisualizationFactory; +import org.apache.zeppelin.helium.HeliumBundleFactory; import org.apache.zeppelin.interpreter.InterpreterFactory; import org.apache.zeppelin.interpreter.InterpreterOutput; import org.apache.zeppelin.notebook.Notebook; @@ -102,7 +102,7 @@ public class ZeppelinServer extends Application { InterpreterOutput.limit = conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_OUTPUT_LIMIT); HeliumApplicationFactory heliumApplicationFactory = new HeliumApplicationFactory(); - HeliumVisualizationFactory heliumVisualizationFactory; + HeliumBundleFactory heliumBundleFactory; if (isBinaryPackage(conf)) { /* In binary package, zeppelin-web/src/app/visualization and zeppelin-web/src/app/tabledata @@ -110,28 +110,30 @@ public class ZeppelinServer extends Application { * Check zeppelin/zeppelin-distribution/src/assemble/distribution.xml to see how they're * packaged into binary package. */ - heliumVisualizationFactory = new HeliumVisualizationFactory( + heliumBundleFactory = new HeliumBundleFactory( new File(conf.getRelativeDir(ConfVars.ZEPPELIN_DEP_LOCALREPO)), new File(conf.getRelativeDir("lib/node_modules/zeppelin-tabledata")), - new File(conf.getRelativeDir("lib/node_modules/zeppelin-vis"))); + new File(conf.getRelativeDir("lib/node_modules/zeppelin-vis")), + new File(conf.getRelativeDir("lib/node_modules/zeppelin-spell"))); } else { - heliumVisualizationFactory = new HeliumVisualizationFactory( + heliumBundleFactory = new HeliumBundleFactory( new File(conf.getRelativeDir(ConfVars.ZEPPELIN_DEP_LOCALREPO)), new File(conf.getRelativeDir("zeppelin-web/src/app/tabledata")), - new File(conf.getRelativeDir("zeppelin-web/src/app/visualization"))); + new File(conf.getRelativeDir("zeppelin-web/src/app/visualization")), + new File(conf.getRelativeDir("zeppelin-web/src/app/spell"))); } this.helium = new Helium( conf.getHeliumConfPath(), conf.getHeliumRegistry(), - new File( - conf.getRelativeDir(ConfVars.ZEPPELIN_DEP_LOCALREPO), "helium_registry_cache"), - heliumVisualizationFactory, + new File(conf.getRelativeDir(ConfVars.ZEPPELIN_DEP_LOCALREPO), + "helium-registry-cache"), + heliumBundleFactory, heliumApplicationFactory); - // create visualization bundle + // create bundle try { - heliumVisualizationFactory.bundle(helium.getVisualizationPackagesToBundle()); + heliumBundleFactory.buildBundle(helium.getBundlePackagesToBundle()); } catch (Exception e) { LOG.error(e.getMessage(), e); } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java index 6e58e3d..68b015d 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java @@ -262,6 +262,9 @@ public class NotebookServer extends WebSocketServlet case RUN_PARAGRAPH: runParagraph(conn, userAndRoles, notebook, messagereceived); break; + case PARAGRAPH_EXECUTED_BY_SPELL: + broadcastSpellExecution(conn, userAndRoles, notebook, messagereceived); + break; case RUN_ALL_PARAGRAPHS: runAllParagraphs(conn, userAndRoles, notebook, messagereceived); break; @@ -698,6 +701,63 @@ public class NotebookServer extends WebSocketServlet .toString()))); } + /** + * @return false if user doesn't have reader permission for this paragraph + */ + private boolean hasParagraphReaderPermission(NotebookSocket conn, + Notebook notebook, String noteId, + HashSet<String> userAndRoles, + String principal, String op) + throws IOException { + + NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); + if (!notebookAuthorization.isReader(noteId, userAndRoles)) { + permissionError(conn, op, principal, userAndRoles, + notebookAuthorization.getOwners(noteId)); + return false; + } + + return true; + } + + /** + * @return false if user doesn't have writer permission for this paragraph + */ + private boolean hasParagraphWriterPermission(NotebookSocket conn, + Notebook notebook, String noteId, + HashSet<String> userAndRoles, + String principal, String op) + throws IOException { + + NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); + if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { + permissionError(conn, op, principal, userAndRoles, + notebookAuthorization.getOwners(noteId)); + return false; + } + + return true; + } + + /** + * @return false if user doesn't have owner permission for this paragraph + */ + private boolean hasParagraphOwnerPermission(NotebookSocket conn, + Notebook notebook, String noteId, + HashSet<String> userAndRoles, + String principal, String op) + throws IOException { + + NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); + if (!notebookAuthorization.isOwner(noteId, userAndRoles)) { + permissionError(conn, op, principal, userAndRoles, + notebookAuthorization.getOwners(noteId)); + return false; + } + + return true; + } + private void sendNote(NotebookSocket conn, HashSet<String> userAndRoles, Notebook notebook, Message fromMessage) throws IOException { @@ -713,13 +773,13 @@ public class NotebookServer extends WebSocketServlet String user = fromMessage.principal; Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); if (note != null) { - if (!notebookAuthorization.isReader(noteId, userAndRoles)) { - permissionError(conn, "read", fromMessage.principal, userAndRoles, - notebookAuthorization.getReaders(noteId)); + + if (!hasParagraphReaderPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "read")) { return; } + addConnectionToNote(note.getId(), conn); if (note.isPersonalizedMode()) { @@ -743,12 +803,11 @@ public class NotebookServer extends WebSocketServlet } if (note != null) { - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isReader(noteId, userAndRoles)) { - permissionError(conn, "read", fromMessage.principal, userAndRoles, - notebookAuthorization.getReaders(noteId)); + if (!hasParagraphReaderPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "read")) { return; } + addConnectionToNote(note.getId(), conn); conn.send(serializeMessage(new Message(OP.NOTE).put("note", note))); sendAllAngularObjects(note, user, conn); @@ -770,10 +829,8 @@ public class NotebookServer extends WebSocketServlet return; } - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "update", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "update")) { return; } @@ -804,10 +861,8 @@ public class NotebookServer extends WebSocketServlet return; } - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isOwner(noteId, userAndRoles)) { - permissionError(conn, "persoanlized ", fromMessage.principal, userAndRoles, - notebookAuthorization.getOwners(noteId)); + if (!hasParagraphOwnerPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "persoanlized")) { return; } @@ -836,10 +891,8 @@ public class NotebookServer extends WebSocketServlet return; } - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isOwner(noteId, userAndRoles)) { - permissionError(conn, "rename", fromMessage.principal, userAndRoles, - notebookAuthorization.getOwners(noteId)); + if (!hasParagraphOwnerPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "rename")) { return; } @@ -870,12 +923,10 @@ public class NotebookServer extends WebSocketServlet return; } - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); for (Note note : notebook.getNotesUnderFolder(oldFolderId)) { String noteId = note.getId(); - if (!notebookAuthorization.isOwner(noteId, userAndRoles)) { - permissionError(conn, op + " folder of '" + note.getName() + "'", fromMessage.principal, - userAndRoles, notebookAuthorization.getOwners(noteId)); + if (!hasParagraphOwnerPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, op + " folder of '" + note.getName() + "'")) { return; } } @@ -960,11 +1011,8 @@ public class NotebookServer extends WebSocketServlet return; } - Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isOwner(noteId, userAndRoles)) { - permissionError(conn, "remove", fromMessage.principal, userAndRoles, - notebookAuthorization.getOwners(noteId)); + if (!hasParagraphOwnerPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "remove")) { return; } @@ -982,13 +1030,12 @@ public class NotebookServer extends WebSocketServlet return; } - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); List<Note> notes = notebook.getNotesUnderFolder(folderId); for (Note note : notes) { String noteId = note.getId(); - if (!notebookAuthorization.isOwner(noteId, userAndRoles)) { - permissionError(conn, "remove folder of '" + note.getName() + "'", fromMessage.principal, - userAndRoles, notebookAuthorization.getOwners(noteId)); + + if (!hasParagraphOwnerPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "remove folder of '" + note.getName() + "'")) { return; } } @@ -1107,17 +1154,16 @@ public class NotebookServer extends WebSocketServlet Map<String, Object> params = (Map<String, Object>) fromMessage.get("params"); Map<String, Object> config = (Map<String, Object>) fromMessage.get("config"); String noteId = getOpenNoteId(conn); - final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return; } + final Note note = notebook.getNote(noteId); Paragraph p = note.getParagraph(paragraphId); + AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); if (note.isPersonalizedMode()) { p = p.getUserParagraphMap().get(subject.getUser()); } @@ -1154,14 +1200,13 @@ public class NotebookServer extends WebSocketServlet if (StringUtils.isBlank(noteId)) { return; } - Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "clear output", fromMessage.principal, userAndRoles, - notebookAuthorization.getOwners(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "clear output")) { return; } + Note note = notebook.getNote(noteId); note.clearAllParagraphOutput(); broadcastNote(note); } @@ -1193,17 +1238,16 @@ public class NotebookServer extends WebSocketServlet return; } String noteId = getOpenNoteId(conn); - final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return; } /** We dont want to remove the last paragraph */ + final Note note = notebook.getNote(noteId); if (!note.isLastParagraph(paragraphId)) { + AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); Paragraph para = note.removeParagraph(subject.getUser(), paragraphId); note.persist(subject); if (para != null) { @@ -1219,14 +1263,14 @@ public class NotebookServer extends WebSocketServlet if (paragraphId == null) { return; } + String noteId = getOpenNoteId(conn); - final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return; } + + final Note note = notebook.getNote(noteId); note.clearParagraphOutput(paragraphId); Paragraph paragraph = note.getParagraph(paragraphId); broadcastParagraph(note, paragraph); @@ -1470,14 +1514,13 @@ public class NotebookServer extends WebSocketServlet final int newIndex = (int) Double.parseDouble(fromMessage.get("index").toString()); String noteId = getOpenNoteId(conn); final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return; } + AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); note.moveParagraph(paragraphId, newIndex); note.persist(subject); broadcast(note.getId(), @@ -1489,11 +1532,10 @@ public class NotebookServer extends WebSocketServlet final int index = (int) Double.parseDouble(fromMessage.get("index").toString()); String noteId = getOpenNoteId(conn); final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return null; } @@ -1524,14 +1566,13 @@ public class NotebookServer extends WebSocketServlet } String noteId = getOpenNoteId(conn); - final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return; } + final Note note = notebook.getNote(noteId); Paragraph p = note.getParagraph(paragraphId); p.abort(); } @@ -1544,11 +1585,8 @@ public class NotebookServer extends WebSocketServlet return; } - Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "run all paragraphs", fromMessage.principal, userAndRoles, - notebookAuthorization.getOwners(noteId)); + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "run all paragraphs")) { return; } @@ -1567,6 +1605,7 @@ public class NotebookServer extends WebSocketServlet Map<String, Object> params = (Map<String, Object>) raw.get("params"); Map<String, Object> config = (Map<String, Object>) raw.get("config"); + Note note = notebook.getNote(noteId); Paragraph p = setParagraphUsingMessage(note, fromMessage, paragraphId, text, title, params, config); @@ -1574,6 +1613,45 @@ public class NotebookServer extends WebSocketServlet } } + private void broadcastSpellExecution(NotebookSocket conn, HashSet<String> userAndRoles, + Notebook notebook, Message fromMessage) + throws IOException { + + final String paragraphId = (String) fromMessage.get("id"); + if (paragraphId == null) { + return; + } + + String noteId = getOpenNoteId(conn); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { + return; + } + + String text = (String) fromMessage.get("paragraph"); + String title = (String) fromMessage.get("title"); + Status status = Status.valueOf((String) fromMessage.get("status")); + Map<String, Object> params = (Map<String, Object>) fromMessage.get("params"); + Map<String, Object> config = (Map<String, Object>) fromMessage.get("config"); + + final Note note = notebook.getNote(noteId); + Paragraph p = setParagraphUsingMessage(note, fromMessage, paragraphId, + text, title, params, config); + p.setResult(fromMessage.get("results")); + p.setErrorMessage((String) fromMessage.get("errorMessage")); + p.setStatusWithoutNotification(status); + + addNewParagraphIfLastParagraphIsExecuted(note, p); + if (!persistNoteWithAuthInfo(conn, note, p)) { + return; + } + + // broadcast to other clients only + broadcastExcept(note.getId(), + new Message(OP.RUN_PARAGRAPH_USING_SPELL).put("paragraph", p), conn); + } + private void runParagraph(NotebookSocket conn, HashSet<String> userAndRoles, Notebook notebook, Message fromMessage) throws IOException { final String paragraphId = (String) fromMessage.get("id"); @@ -1582,11 +1660,9 @@ public class NotebookServer extends WebSocketServlet } String noteId = getOpenNoteId(conn); - final Note note = notebook.getNote(noteId); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "write", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "write")) { return; } @@ -1594,14 +1670,15 @@ public class NotebookServer extends WebSocketServlet String title = (String) fromMessage.get("title"); Map<String, Object> params = (Map<String, Object>) fromMessage.get("params"); Map<String, Object> config = (Map<String, Object>) fromMessage.get("config"); + + final Note note = notebook.getNote(noteId); Paragraph p = setParagraphUsingMessage(note, fromMessage, paragraphId, text, title, params, config); persistAndExecuteSingleParagraph(conn, note, p); } - private void persistAndExecuteSingleParagraph(NotebookSocket conn, - Note note, Paragraph p) throws IOException { + private void addNewParagraphIfLastParagraphIsExecuted(Note note, Paragraph p) { // if it's the last paragraph and empty, let's add a new one boolean isTheLastParagraph = note.isLastParagraph(p.getId()); if (!(p.getText().trim().equals(p.getMagic()) || @@ -1610,15 +1687,30 @@ public class NotebookServer extends WebSocketServlet Paragraph newPara = note.addParagraph(p.getAuthenticationInfo()); broadcastNewParagraph(note, newPara); } + } + /** + * @return false if failed to save a note + */ + private boolean persistNoteWithAuthInfo(NotebookSocket conn, + Note note, Paragraph p) throws IOException { try { note.persist(p.getAuthenticationInfo()); + return true; } catch (FileSystemException ex) { LOG.error("Exception from run", ex); conn.send(serializeMessage(new Message(OP.ERROR_INFO).put("info", "Oops! There is something wrong with the notebook file system. " + "Please check the logs for more details."))); // don't run the paragraph when there is error on persisting the note information + return false; + } + } + + private void persistAndExecuteSingleParagraph(NotebookSocket conn, + Note note, Paragraph p) throws IOException { + addNewParagraphIfLastParagraphIsExecuted(note, p); + if (!persistNoteWithAuthInfo(conn, note, p)) { return; } @@ -1701,10 +1793,8 @@ public class NotebookServer extends WebSocketServlet String revisionId = (String) fromMessage.get("revisionId"); AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal); - NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization(); - if (!notebookAuthorization.isWriter(noteId, userAndRoles)) { - permissionError(conn, "update", fromMessage.principal, userAndRoles, - notebookAuthorization.getWriters(noteId)); + if (!hasParagraphWriterPermission(conn, notebook, noteId, + userAndRoles, fromMessage.principal, "update")) { return; } http://git-wip-us.apache.org/repos/asf/zeppelin/blob/0589e27e/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java ---------------------------------------------------------------------- diff --git a/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java b/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java index feade7f..458b8d4 100644 --- a/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java +++ b/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java @@ -270,7 +270,7 @@ public class ParagraphActionsIT extends AbstractZeppelinIT { collector.checkThat("Before Run Output field contains ", driver.findElements(By.xpath(xpathToOutputField)).size(), CoreMatchers.equalTo(0)); - driver.findElement(By.xpath(getParagraphXPath(1) + "//span[@ng-click='runParagraph(getEditorValue())']")).click(); + runParagraph(1); waitForParagraph(1, "FINISHED"); collector.checkThat("After Run Output field contains ", driver.findElement(By.xpath(xpathToOutputField)).getText(), @@ -286,7 +286,6 @@ public class ParagraphActionsIT extends AbstractZeppelinIT { } catch (Exception e) { handleException("Exception in ParagraphActionsIT while testClearOutputButton ", e); } - } @Test
