Modified: zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json URL: http://svn.apache.org/viewvc/zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json?rev=1766246&r1=1766245&r2=1766246&view=diff ============================================================================== --- zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json (original) +++ zeppelin/site/docs/0.7.0-SNAPSHOT/search_data.json Sun Oct 23 04:33:46 2016 @@ -16,7 +16,7 @@ "/development/howtocontributewebsite.html": { "title": "Contributing to Apache Zeppelin (Website)", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Contributing to Apache Zeppelin ( Website )This page will give you an overview of how to build and contribute to the documentation of Apache Zeppelin.The online documentation at zeppelin.apache.org is also generated from the files found here.NOTE : Apache Zeppelin is an Apache2 License Software.Any contributions to Zeppelin (Source code, Documents, Image, Website) means you agree with license all your contributions as Apache2 Licen se.Getting the source codeFirst of all, you need Zeppelin source code. The official location of Zeppelin is http://git.apache.org/zeppelin.git.Documentation website is hosted in &#39;master&#39; branch under /docs/ dir.git accessFirst of all, you need the website source code. The official location of mirror for Zeppelin is http://git.apache.org/zeppelin.git.Get the source code on your development machine using git.git clone git://git.apache.org/zeppelin.gitcd docsApache Zeppelin follows Fork &amp; Pull as a source control workflow.If you want to not only build Zeppelin but also make any changes, then you need to fork Zeppelin github mirror repository and make a pull request.BuildYou&#39;ll need to install some prerequisites to build the code. Please check Build documentation section in docs/README.md.Run website in development modeWhile you&#39;re modifying website, you might want to see preview of it. Please check Run website section in docs/README.md.Then you&a mp;#39;ll be able to access it on http://localhost:4000 with your web browser.Making a Pull RequestWhen you are ready, just make a pull-request.Alternative wayYou can directly edit .md files in /docs/ directory at the web interface of github and make pull-request immediatly.Stay involvedContributors should join the Zeppelin mailing [email protected] is for people who want to contribute code to Zeppelin. subscribe, unsubscribe, archivesIf you have any issues, create a ticket in JIRA.", + "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Contributing to Apache Zeppelin ( Website )This page will give you an overview of how to build and contribute to the documentation of Apache Zeppelin.The online documentation at zeppelin.apache.org is also generated from the files found here.NOTE : Apache Zeppelin is an Apache2 License Software.Any contributions to Zeppelin (Source code, Documents, Image, Website) means you agree with license all your contributions as Apache2 Licen se.Getting the source codeFirst of all, you need Zeppelin source code. The official location of Zeppelin is http://git.apache.org/zeppelin.git.Documentation website is hosted in &#39;master&#39; branch under /docs/ dir.git accessFirst of all, you need the website source code. The official location of mirror for Zeppelin is http://git.apache.org/zeppelin.git.Get the source code on your development machine using git.git clone git://git.apache.org/zeppelin.gitcd docsApache Zeppelin follows Fork &amp; Pull as a source control workflow.If you want to not only build Zeppelin but also make any changes, then you need to fork Zeppelin github mirror repository and make a pull request.BuildYou&#39;ll need to install some prerequisites to build the code. Please check Build documentation section in docs/README.md.Run website in development modeWhile you&#39;re modifying website, you might want to see preview of it. Please check Run website section in docs/README.md.Then you&a mp;#39;ll be able to access it on http://localhost:4000 with your web browser.Making a Pull RequestWhen you are ready, just make a pull-request.Alternative wayYou can directly edit .md files in /docs/ directory at the web interface of github and make pull-request immediately.Stay involvedContributors should join the Zeppelin mailing [email protected] is for people who want to contribute code to Zeppelin. subscribe, unsubscribe, archivesIf you have any issues, create a ticket in JIRA.", "url": " /development/howtocontributewebsite.html", "group": "development", "excerpt": "How can you contribute to Apache Zeppelin project website? This document covers from building Zeppelin documentation site to making a pull request on Github." @@ -94,7 +94,7 @@ "/install/install.html": { "title": "Quick Start", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Quick StartWelcome to Apache Zeppelin! On this page are instructions to help you get started.InstallationApache Zeppelin officially supports and is tested on the following environments: Name Value Oracle JDK 1.7 (set JAVA_HOME) OS Mac OSX Ubuntu 14.X CentOS 6.X Windows 7 Pro SP1 To install Apache Zeppelin, you have two options:You can download pre-built binary packages from the archive. This is usua lly easier than building from source, and you can download the latest stable version (or older versions, if necessary).You can also build from source. This gives you a development version of Zeppelin, which is more unstable but has new features.Downloading Binary PackageStable binary packages are available on the Apache Zeppelin Download Page. You can download a default package with all interpreters, or you can download the net-install package, which lets you choose which interpreters to install.If you downloaded the default package, just unpack it in a directory of your choice and you&#39;re ready to go. If you downloaded the net-install package, you should manually install additional interpreters first. You can also install everything by running ./bin/install-interpreter.sh --all.After unpacking, jump to the Starting Apache Zeppelin with Command Line.Building from SourceIf you want to build from source, you must first install the following dependencies: Name Value Git (Any Version) Maven 3.1.x or higher If you haven&#39;t installed Git and Maven yet, check the Before Build section and follow the step by step instructions from there.1. Clone the Apache Zeppelin repositorygit clone https://github.com/apache/zeppelin.git2. Build source with optionsEach interpreter requires different build options. For more information about build options, please see the Build section.mvn clean package -DskipTests [Options]Here are some examples with several options:# build with spark-2.0, scala-2.11./dev/change_scala_version.sh 2.11mvn clean package -Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pscala-2.11# build with spark-1.6, scala-2.10mvn clean package -Pspark-1.6 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr# spark-cassandra integrationmvn clean package -Pcassandra-spark-1.5 -Dhadoop.version=2.6.0 -Phadoop-2.6 -DskipTests# with CDHmvn clean package -Pspark-1.5 -Dhadoop.version=2.6.0-cdh5.5.0 -Phadoop-2.6 -Pvendor-repo -DskipTests# with M apRmvn clean package -Pspark-1.5 -Pmapr50 -DskipTestsFor further information about building from source, please see README.md in the Zeppelin repository.Starting Apache Zeppelin from the Command LineStarting Apache ZeppelinOn all platforms except for Windows:bin/zeppelin-daemon.sh startIf you are using Windows:binzeppelin.cmdAfter Zeppelin has started successfully, go to http://localhost:8080 with your web browser.Stopping Zeppelinbin/zeppelin-daemon.sh stop(Optional) Start Apache Zeppelin with a service managerNote : The below description was written based on Ubuntu Linux.Apache Zeppelin can be auto-started as a service with an init script, using a service manager like upstart.This is an example upstart script saved as /etc/init/zeppelin.confThis allows the service to be managed with commands such assudo service zeppelin start sudo service zeppelin stop sudo service zeppelin restartOther service managers could use a similar approach with the upstart argument passed to the zeppeli n-daemon.sh script.bin/zeppelin-daemon.sh upstartzeppelin.confdescription &quot;zeppelin&quot;start on (local-filesystems and net-device-up IFACE!=lo)stop on shutdown# Respawn the process on unexpected terminationrespawn# respawn the job up to 7 times within a 5 second period.# If the job exceeds these values, it will be stopped and marked as failed.respawn limit 7 5# zeppelin was installed in /usr/share/zeppelin in this examplechdir /usr/share/zeppelinexec bin/zeppelin-daemon.sh upstartNext Steps:Congratulations, you have successfully installed Apache Zeppelin! Here are two next steps you might find useful:If you are new to Apache Zeppelin...For an in-depth overview of the Apache Zeppelin UI, head to Explore Apache Zeppelin UI.After getting familiar with the Apache Zeppelin UI, have fun with a short walk-through Tutorial that uses the Apache Spark backend.If you need more configuration for Apache Zeppelin, jump to the next section: Apache Zeppelin Configuration.If you need more information about Spark or JDBC interpreter settings...Apache Zeppelin provides deep integration with Apache Spark. For more informtation, see Spark Interpreter for Apache Zeppelin. You can also use generic JDBC connections in Apache Zeppelin. Go to Generic JDBC Interpreter for Apache Zeppelin.If you are in a multi-user environment...You can set permissions for your notebooks and secure data resource in a multi-user environment. Go to More -&gt; Security section.Apache Zeppelin ConfigurationYou can configure Apache Zeppelin with either environment variables in conf/zeppelin-env.sh (confzeppelin-env.cmd for Windows) or Java properties in conf/zeppelin-site.xml. If both are defined, then the environment variables will take priority. zeppelin-env.sh zeppelin-site.xml Default value Description ZEPPELIN_PORT zeppelin.server.port 8080 Zeppelin server port ZEPPELIN_SSL_PORT zeppelin.server.ssl.port 8443 Zeppelin Server ssl port (used w hen ssl environment/property is set to true) ZEPPELIN_MEM N/A -Xmx1024m -XX:MaxPermSize=512m JVM mem options ZEPPELIN_INTP_MEM N/A ZEPPELIN_MEM JVM mem options for interpreter process ZEPPELIN_JAVA_OPTS N/A JVM options ZEPPELIN_ALLOWED_ORIGINS zeppelin.server.allowed.origins * Enables a way to specify a ',' separated list of allowed origins for REST and websockets. i.e. http://localhost:8080 N/A zeppelin.anonymous.allowed true The anonymous user is allowed by default. ZEPPELIN_SERVER_CONTEXT_PATH zeppelin.server.context.path / Context path of the web application ZEPPELIN_SSL zeppelin.ssl false ZEPPELIN_SSL_CLIENT_AUTH zeppelin.ssl.client.auth false ZEPPELIN_SSL_KEYSTORE_PATH zeppelin.ssl.keystore.path keystore ZEPPELIN_SSL_KEYSTORE_TYPE zeppelin.ssl.keystore.type JKS ZEPPELIN_SSL_KEYSTORE_PASSWORD z eppelin.ssl.keystore.password ZEPPELIN_SSL_KEY_MANAGER_PASSWORD zeppelin.ssl.key.manager.password ZEPPELIN_SSL_TRUSTSTORE_PATH zeppelin.ssl.truststore.path ZEPPELIN_SSL_TRUSTSTORE_TYPE zeppelin.ssl.truststore.type ZEPPELIN_SSL_TRUSTSTORE_PASSWORD zeppelin.ssl.truststore.password ZEPPELIN_NOTEBOOK_HOMESCREEN zeppelin.notebook.homescreen Display notebook IDs on the Apache Zeppelin homescreen i.e. 2A94M5J1Z ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE zeppelin.notebook.homescreen.hide false Hide the notebook ID set by ZEPPELIN_NOTEBOOK_HOMESCREEN on the Apache Zeppelin homescreen. For the further information, please read Customize your Zeppelin homepage. ZEPPELIN_WAR_TEMPDIR zeppelin.war.tempdir webapps Location of the jetty temporary directory ZEPPELIN_NOTEBOOK_DIR zeppelin.notebook.dir notebook The root directory where notebook directories are saved ZEPPELIN_NOTEBOOK_S3_BUCKET zeppelin.notebook.s3.bucket zeppelin S3 Bucket where notebook files will be saved ZEPPELIN_NOTEBOOK_S3_USER zeppelin.notebook.s3.user user User name of an S3 bucketi.e. bucket/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_S3_ENDPOINT zeppelin.notebook.s3.endpoint s3.amazonaws.com Endpoint for the bucket ZEPPELIN_NOTEBOOK_S3_KMS_KEY_ID zeppelin.notebook.s3.kmsKeyID AWS KMS Key ID to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_S3_EMP zeppelin.notebook.s3.encryptionMaterialsProvider Class name of a custom S3 encryption materials provider implementation to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_AZURE_CONNECTION_STRING zeppelin.notebook.azure.connectionString The Azure storage account connection stringi.e. DefaultEndpointsProtocol=https;AccountName=&lt;accountName&gt;;AccountKey=&lt;accountKey&gt; ZEPP ELIN_NOTEBOOK_AZURE_SHARE zeppelin.notebook.azure.share zeppelin Azure Share where the notebook files will be saved ZEPPELIN_NOTEBOOK_AZURE_USER zeppelin.notebook.azure.user user Optional user name of an Azure file sharei.e. share/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_STORAGE zeppelin.notebook.storage org.apache.zeppelin.notebook.repo.VFSNotebookRepo Comma separated list of notebook storage locations ZEPPELIN_NOTEBOOK_ONE_WAY_SYNC zeppelin.notebook.one.way.sync false If there are multiple notebook storage locations, should we treat the first one as the only source of truth? ZEPPELIN_INTERPRETERS zeppelin.interpreters org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.shell.ShellInterpreter, ... Comma sep arated interpreter configurations [Class] NOTE: This property is deprecated since Zeppelin-0.6.0 and will not be supported from Zeppelin-0.7.0 on. ZEPPELIN_INTERPRETER_DIR zeppelin.interpreter.dir interpreter Interpreter directory ZEPPELIN_WEBSOCKET_MAX_TEXT_MESSAGE_SIZE zeppelin.websocket.max.text.message.size 1024000 Size (in characters) of the maximum text message that can be received by websocket. ", + "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Quick StartWelcome to Apache Zeppelin! On this page are instructions to help you get started.InstallationApache Zeppelin officially supports and is tested on the following environments: Name Value Oracle JDK 1.7 (set JAVA_HOME) OS Mac OSX Ubuntu 14.X CentOS 6.X Windows 7 Pro SP1 To install Apache Zeppelin, you have two options:You can download pre-built binary packages from the archive. This is usua lly easier than building from source, and you can download the latest stable version (or older versions, if necessary).You can also build from source. This gives you a development version of Zeppelin, which is more unstable but has new features.Downloading Binary PackageStable binary packages are available on the Apache Zeppelin Download Page. You can download a default package with all interpreters, or you can download the net-install package, which lets you choose which interpreters to install.If you downloaded the default package, just unpack it in a directory of your choice and you&#39;re ready to go. If you downloaded the net-install package, you should manually install additional interpreters first. You can also install everything by running ./bin/install-interpreter.sh --all.After unpacking, jump to the Starting Apache Zeppelin with Command Line.Building from SourceIf you want to build from source, you must first install the following dependencies: Name Value Git (Any Version) Maven 3.1.x or higher If you haven&#39;t installed Git and Maven yet, check the Before Build section and follow the step by step instructions from there.1. Clone the Apache Zeppelin repositorygit clone https://github.com/apache/zeppelin.git2. Build source with optionsEach interpreter requires different build options. For more information about build options, please see the Build section.mvn clean package -DskipTests [Options]Here are some examples with several options:# build with spark-2.0, scala-2.11./dev/change_scala_version.sh 2.11mvn clean package -Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pscala-2.11# build with spark-1.6, scala-2.10mvn clean package -Pspark-1.6 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr# spark-cassandra integrationmvn clean package -Pcassandra-spark-1.5 -Dhadoop.version=2.6.0 -Phadoop-2.6 -DskipTests# with CDHmvn clean package -Pspark-1.5 -Dhadoop.version=2.6.0-cdh5.5.0 -Phadoop-2.6 -Pvendor-repo -DskipTests# with M apRmvn clean package -Pspark-1.5 -Pmapr50 -DskipTestsFor further information about building from source, please see README.md in the Zeppelin repository.Starting Apache Zeppelin from the Command LineStarting Apache ZeppelinOn all platforms except for Windows:bin/zeppelin-daemon.sh startIf you are using Windows:binzeppelin.cmdAfter Zeppelin has started successfully, go to http://localhost:8080 with your web browser.Stopping Zeppelinbin/zeppelin-daemon.sh stop(Optional) Start Apache Zeppelin with a service managerNote : The below description was written based on Ubuntu Linux.Apache Zeppelin can be auto-started as a service with an init script, using a service manager like upstart.This is an example upstart script saved as /etc/init/zeppelin.confThis allows the service to be managed with commands such assudo service zeppelin start sudo service zeppelin stop sudo service zeppelin restartOther service managers could use a similar approach with the upstart argument passed to the zeppeli n-daemon.sh script.bin/zeppelin-daemon.sh upstartzeppelin.confdescription &quot;zeppelin&quot;start on (local-filesystems and net-device-up IFACE!=lo)stop on shutdown# Respawn the process on unexpected terminationrespawn# respawn the job up to 7 times within a 5 second period.# If the job exceeds these values, it will be stopped and marked as failed.respawn limit 7 5# zeppelin was installed in /usr/share/zeppelin in this examplechdir /usr/share/zeppelinexec bin/zeppelin-daemon.sh upstartNext Steps:Congratulations, you have successfully installed Apache Zeppelin! Here are two next steps you might find useful:If you are new to Apache Zeppelin...For an in-depth overview of the Apache Zeppelin UI, head to Explore Apache Zeppelin UI.After getting familiar with the Apache Zeppelin UI, have fun with a short walk-through Tutorial that uses the Apache Spark backend.If you need more configuration for Apache Zeppelin, jump to the next section: Apache Zeppelin Configuration.If you need more information about Spark or JDBC interpreter settings...Apache Zeppelin provides deep integration with Apache Spark. For more information, see Spark Interpreter for Apache Zeppelin.You can also use generic JDBC connections in Apache Zeppelin. Go to Generic JDBC Interpreter for Apache Zeppelin.If you are in a multi-user environment...You can set permissions for your notebooks and secure data resource in a multi-user environment. Go to More -&gt; Security section.Apache Zeppelin ConfigurationYou can configure Apache Zeppelin with either environment variables in conf/zeppelin-env.sh (confzeppelin-env.cmd for Windows) or Java properties in conf/zeppelin-site.xml. If both are defined, then the environment variables will take priority. zeppelin-env.sh zeppelin-site.xml Default value Description ZEPPELIN_PORT zeppelin.server.port 8080 Zeppelin server port ZEPPELIN_SSL_PORT zeppelin.server.ssl.port 8443 Zeppelin Server ssl port (used whe n ssl environment/property is set to true) ZEPPELIN_MEM N/A -Xmx1024m -XX:MaxPermSize=512m JVM mem options ZEPPELIN_INTP_MEM N/A ZEPPELIN_MEM JVM mem options for interpreter process ZEPPELIN_JAVA_OPTS N/A JVM options ZEPPELIN_ALLOWED_ORIGINS zeppelin.server.allowed.origins * Enables a way to specify a ',' separated list of allowed origins for REST and websockets. i.e. http://localhost:8080 N/A zeppelin.anonymous.allowed true The anonymous user is allowed by default. ZEPPELIN_SERVER_CONTEXT_PATH zeppelin.server.context.path / Context path of the web application ZEPPELIN_SSL zeppelin.ssl false ZEPPELIN_SSL_CLIENT_AUTH zeppelin.ssl.client.auth false ZEPPELIN_SSL_KEYSTORE_PATH zeppelin.ssl.keystore.path keystore ZEPPELIN_SSL_KEYSTORE_TYPE zeppelin.ssl.keystore.type JKS ZEPPELIN_SSL_KEYSTORE_PASSWORD zep pelin.ssl.keystore.password ZEPPELIN_SSL_KEY_MANAGER_PASSWORD zeppelin.ssl.key.manager.password ZEPPELIN_SSL_TRUSTSTORE_PATH zeppelin.ssl.truststore.path ZEPPELIN_SSL_TRUSTSTORE_TYPE zeppelin.ssl.truststore.type ZEPPELIN_SSL_TRUSTSTORE_PASSWORD zeppelin.ssl.truststore.password ZEPPELIN_NOTEBOOK_HOMESCREEN zeppelin.notebook.homescreen Display notebook IDs on the Apache Zeppelin homescreen i.e. 2A94M5J1Z ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE zeppelin.notebook.homescreen.hide false Hide the notebook ID set by ZEPPELIN_NOTEBOOK_HOMESCREEN on the Apache Zeppelin homescreen. For the further information, please read Customize your Zeppelin homepage. ZEPPELIN_WAR_TEMPDIR zeppelin.war.tempdir webapps Location of the jetty temporary directory ZEPPELIN_NOTEBOOK_DIR zeppelin.notebook.dir notebook The root directory where notebook directories are saved ZEPPELIN_NOTEBOOK_S3_BUCKET zeppelin.notebook.s3.bucket zeppelin S3 Bucket where notebook files will be saved ZEPPELIN_NOTEBOOK_S3_USER zeppelin.notebook.s3.user user User name of an S3 bucketi.e. bucket/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_S3_ENDPOINT zeppelin.notebook.s3.endpoint s3.amazonaws.com Endpoint for the bucket ZEPPELIN_NOTEBOOK_S3_KMS_KEY_ID zeppelin.notebook.s3.kmsKeyID AWS KMS Key ID to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_S3_EMP zeppelin.notebook.s3.encryptionMaterialsProvider Class name of a custom S3 encryption materials provider implementation to use for encrypting data in S3 (optional) ZEPPELIN_NOTEBOOK_AZURE_CONNECTION_STRING zeppelin.notebook.azure.connectionString The Azure storage account connection stringi.e. DefaultEndpointsProtocol=https;AccountName=&lt;accountName&gt;;AccountKey=&lt;accountKey&gt; ZEPPEL IN_NOTEBOOK_AZURE_SHARE zeppelin.notebook.azure.share zeppelin Azure Share where the notebook files will be saved ZEPPELIN_NOTEBOOK_AZURE_USER zeppelin.notebook.azure.user user Optional user name of an Azure file sharei.e. share/user/notebook/2A94M5J1Z/note.json ZEPPELIN_NOTEBOOK_STORAGE zeppelin.notebook.storage org.apache.zeppelin.notebook.repo.VFSNotebookRepo Comma separated list of notebook storage locations ZEPPELIN_NOTEBOOK_ONE_WAY_SYNC zeppelin.notebook.one.way.sync false If there are multiple notebook storage locations, should we treat the first one as the only source of truth? ZEPPELIN_INTERPRETERS zeppelin.interpreters org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.shell.ShellInterpreter, ... Comma separ ated interpreter configurations [Class] NOTE: This property is deprecated since Zeppelin-0.6.0 and will not be supported from Zeppelin-0.7.0 on. ZEPPELIN_INTERPRETER_DIR zeppelin.interpreter.dir interpreter Interpreter directory ZEPPELIN_WEBSOCKET_MAX_TEXT_MESSAGE_SIZE zeppelin.websocket.max.text.message.size 1024000 Size (in characters) of the maximum text message that can be received by websocket. ", "url": " /install/install.html", "group": "install", "excerpt": "This page will help you get started and will guide you through installing Apache Zeppelin, running it in the command line and configuring options." @@ -127,7 +127,7 @@ "/install/virtual_machine.html": { "title": "Apache Zeppelin on Vagrant Virtual Machine", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Apache Zeppelin on Vagrant Virtual MachineOverviewApache Zeppelin distribution includes a script directoryscripts/vagrant/zeppelin-devThis script creates a virtual machine that launches a repeatable, known set of core dependencies required for developing Zeppelin. It can also be used to run an existing Zeppelin build if you don&#39;t plan to build from source.For PySpark users, this script includes several helpful Python Librar ies.For SparkR users, this script includes several helpful R Libraries.PrerequisitesThis script requires three applications, Ansible, Vagrant and Virtual Box. All of these applications are freely available as Open Source projects and extremely easy to set up on most operating systems.Create a Zeppelin Ready VMIf you are running Windows and don&#39;t yet have python installed, install Python 2.7.x first.Download and Install Vagrant: Vagrant DownloadsInstall Ansible: Ansible Python pip installsudo easy_install pipsudo pip install ansibleansible --versionAfter then, please check whether it reports ansible version 1.9.2 or higher.Install Virtual Box: Virtual Box DownloadsType vagrant up from within the /scripts/vagrant/zeppelin-dev directoryThats it ! You can now run vagrant ssh and this will place you into the guest machines terminal prompt.If you don&#39;t wish to build Zeppelin from scratch, run the z-manager installer script while running in the guest VM:curl -fsSL https ://raw.githubusercontent.com/NFLabs/z-manager/master/zeppelin-installer.sh | bashBuilding ZeppelinYou can now git clone git://git.apache.org/zeppelin.gitinto a directory on your host machine, or directly in your virtual machine.Cloning Zeppelin into the /scripts/vagrant/zeppelin-dev directory from the host, will allow the directory to be shared between your host and the guest machine.Cloning the project again may seem counter intuitive, since this script likley originated from the project repository. Consider copying just the vagrant/zeppelin-dev script from the Zeppelin project as a stand alone directory, then once again clone the specific branch you wish to build.Synced folders enable Vagrant to sync a folder on the host machine to the guest machine, allowing you to continue working on your project&#39;s files on your host machine, but use the resources in the guest machine to compile or run your project. (1) Synced Folder Description from Vagrant UpBy default, Vagrant will s hare your project directory (the directory with the Vagrantfile) to /vagrant. Which means you should be able to build within the guest machine after youcd /vagrant/zeppelinWhat&#39;s in this VM?Running the following commands in the guest machine should display these expected versions:node --version should report v0.12.7mvn --version should report Apache Maven 3.3.9 and Java version: 1.7.0_85The virtual machine consists of:Ubuntu Server 14.04 LTSNode.js 0.12.7npm 2.11.3ruby 1.9.3 + rake, make and bundler (only required if building jekyll documentation)Maven 3.3.9GitUnziplibfontconfig to avoid phatomJs missing dependency issuesopenjdk-7-jdkPython addons: pip, matplotlib, scipy, numpy, pandasR and R Packages required to run the R Interpreter and the related R tutorial notebook, including: Knitr, devtools, repr, rCharts, ggplot2, googleVis, mplot, htmltools, base64enc, data.tableHow to build &amp; run ZeppelinThis assumes you&#39;ve already cloned the project either on the host machine in the zeppelin-dev directory (to be shared with the guest machine) or cloned directly into a directory while running inside the guest machine. The following build steps will also include Python and R support via PySpark and SparkR:cd /zeppelinmvn clean package -Pspark-1.6 -Ppyspark -Phadoop-2.4 -Psparkr -DskipTests./bin/zeppelin-daemon.sh startOn your host machine browse to http://localhost:8080/If you turned off port forwarding in the Vagrantfile browse to http://192.168.51.52:8080Tweaking the Virtual MachineIf you plan to run this virtual machine along side other Vagrant images, you may wish to bind the virtual machine to a specific IP address, and not use port fowarding from your local host.Comment out the forward_port line, and uncomment the private_network line in Vagrantfile. The subnet that works best for your local network will vary so adjust 192.168.*.* accordingly.#config.vm.network &quot;forwarded_port&quot;, guest: 8080, host: 8080config.vm.netwo rk &quot;private_network&quot;, ip: &quot;192.168.51.52&quot;vagrant halt followed by vagrant up will restart the guest machine bound to the IP address of 192.168.51.52.This approach usually is typically required if running other virtual machines that discover each other directly by IP address, such as Spark Masters and Slaves as well as Cassandra Nodes, Elasticsearch Nodes, and other Spark data sources. You may wish to launch nodes in virtual machines with IP addresses in a subnet that works for your local network, such as: 192.168.51.53, 192.168.51.54, 192.168.51.53, etc..ExtrasPython ExtrasWith Zeppelin running, Numpy, SciPy, Pandas and Matplotlib will be available. Create a pyspark notebook, and try the below code.%pysparkimport numpyimport scipyimport pandasimport matplotlibprint &quot;numpy &quot; + numpy.__version__print &quot;scipy &quot; + scipy.__version__print &quot;pandas &quot; + pandas.__version__print &quot;matplotlib & ;quot; + matplotlib.__version__To Test plotting using Matplotlib into a rendered %html SVG image, try%pysparkimport matplotlibmatplotlib.use(&#39;Agg&#39;) # turn off interactive charting so this works for server side SVG renderingimport matplotlib.pyplot as pltimport numpy as npimport StringIO# clear out any previous plots on this notebookplt.clf()def show(p): img = StringIO.StringIO() p.savefig(img, format=&#39;svg&#39;) img.seek(0) print &quot;%html &lt;div style=&#39;width:600px&#39;&gt;&quot; + img.buf + &quot;&lt;/div&gt;&quot;# Example datapeople = (&#39;Tom&#39;, &#39;Dick&#39;, &#39;Harry&#39;, &#39;Slim&#39;, &#39;Jim&#39;)y_pos = np.arange(len(people))performance = 3 + 10 * np.random.rand(len(people))error = np.random.rand(len(people))plt.barh(y_pos, performance, xerr=error, align=&#39;center&#39;, alpha=0.4)plt.yticks(y_pos, people)plt.xlabel(&#39;Per formance&#39;)plt.title(&#39;How fast do you want to go today?&#39;)show(plt)R ExtrasWith zeppelin running, an R Tutorial notebook will be available. The R packages required to run the examples and graphs in this tutorial notebook were installed by this virtual machine.The installed R Packages include: Knitr, devtools, repr, rCharts, ggplot2, googleVis, mplot, htmltools, base64enc, data.table", + "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Apache Zeppelin on Vagrant Virtual MachineOverviewApache Zeppelin distribution includes a script directoryscripts/vagrant/zeppelin-devThis script creates a virtual machine that launches a repeatable, known set of core dependencies required for developing Zeppelin. It can also be used to run an existing Zeppelin build if you don&#39;t plan to build from source.For PySpark users, this script includes several helpful Python Librar ies.For SparkR users, this script includes several helpful R Libraries.PrerequisitesThis script requires three applications, Ansible, Vagrant and Virtual Box. All of these applications are freely available as Open Source projects and extremely easy to set up on most operating systems.Create a Zeppelin Ready VMIf you are running Windows and don&#39;t yet have python installed, install Python 2.7.x first.Download and Install Vagrant: Vagrant DownloadsInstall Ansible: Ansible Python pip installsudo easy_install pipsudo pip install ansibleansible --versionAfter then, please check whether it reports ansible version 1.9.2 or higher.Install Virtual Box: Virtual Box DownloadsType vagrant up from within the /scripts/vagrant/zeppelin-dev directoryThats it ! You can now run vagrant ssh and this will place you into the guest machines terminal prompt.If you don&#39;t wish to build Zeppelin from scratch, run the z-manager installer script while running in the guest VM:curl -fsSL https ://raw.githubusercontent.com/NFLabs/z-manager/master/zeppelin-installer.sh | bashBuilding ZeppelinYou can now git clone git://git.apache.org/zeppelin.gitinto a directory on your host machine, or directly in your virtual machine.Cloning Zeppelin into the /scripts/vagrant/zeppelin-dev directory from the host, will allow the directory to be shared between your host and the guest machine.Cloning the project again may seem counter intuitive, since this script likely originated from the project repository. Consider copying just the vagrant/zeppelin-dev script from the Zeppelin project as a stand alone directory, then once again clone the specific branch you wish to build.Synced folders enable Vagrant to sync a folder on the host machine to the guest machine, allowing you to continue working on your project&#39;s files on your host machine, but use the resources in the guest machine to compile or run your project. (1) Synced Folder Description from Vagrant UpBy default, Vagrant will s hare your project directory (the directory with the Vagrantfile) to /vagrant. Which means you should be able to build within the guest machine after youcd /vagrant/zeppelinWhat&#39;s in this VM?Running the following commands in the guest machine should display these expected versions:node --version should report v0.12.7mvn --version should report Apache Maven 3.3.9 and Java version: 1.7.0_85The virtual machine consists of:Ubuntu Server 14.04 LTSNode.js 0.12.7npm 2.11.3ruby 1.9.3 + rake, make and bundler (only required if building jekyll documentation)Maven 3.3.9GitUnziplibfontconfig to avoid phatomJs missing dependency issuesopenjdk-7-jdkPython addons: pip, matplotlib, scipy, numpy, pandasR and R Packages required to run the R Interpreter and the related R tutorial notebook, including: Knitr, devtools, repr, rCharts, ggplot2, googleVis, mplot, htmltools, base64enc, data.tableHow to build &amp; run ZeppelinThis assumes you&#39;ve already cloned the project either on the host machine in the zeppelin-dev directory (to be shared with the guest machine) or cloned directly into a directory while running inside the guest machine. The following build steps will also include Python and R support via PySpark and SparkR:cd /zeppelinmvn clean package -Pspark-1.6 -Ppyspark -Phadoop-2.4 -Psparkr -DskipTests./bin/zeppelin-daemon.sh startOn your host machine browse to http://localhost:8080/If you turned off port forwarding in the Vagrantfile browse to http://192.168.51.52:8080Tweaking the Virtual MachineIf you plan to run this virtual machine along side other Vagrant images, you may wish to bind the virtual machine to a specific IP address, and not use port fowarding from your local host.Comment out the forward_port line, and uncomment the private_network line in Vagrantfile. The subnet that works best for your local network will vary so adjust 192.168.*.* accordingly.#config.vm.network &quot;forwarded_port&quot;, guest: 8080, host: 8080config.vm.netwo rk &quot;private_network&quot;, ip: &quot;192.168.51.52&quot;vagrant halt followed by vagrant up will restart the guest machine bound to the IP address of 192.168.51.52.This approach usually is typically required if running other virtual machines that discover each other directly by IP address, such as Spark Masters and Slaves as well as Cassandra Nodes, Elasticsearch Nodes, and other Spark data sources. You may wish to launch nodes in virtual machines with IP addresses in a subnet that works for your local network, such as: 192.168.51.53, 192.168.51.54, 192.168.51.53, etc..ExtrasPython ExtrasWith Zeppelin running, Numpy, SciPy, Pandas and Matplotlib will be available. Create a pyspark notebook, and try the below code.%pysparkimport numpyimport scipyimport pandasimport matplotlibprint &quot;numpy &quot; + numpy.__version__print &quot;scipy &quot; + scipy.__version__print &quot;pandas &quot; + pandas.__version__print &quot;matplotlib & ;quot; + matplotlib.__version__To Test plotting using Matplotlib into a rendered %html SVG image, try%pysparkimport matplotlibmatplotlib.use(&#39;Agg&#39;) # turn off interactive charting so this works for server side SVG renderingimport matplotlib.pyplot as pltimport numpy as npimport StringIO# clear out any previous plots on this notebookplt.clf()def show(p): img = StringIO.StringIO() p.savefig(img, format=&#39;svg&#39;) img.seek(0) print &quot;%html &lt;div style=&#39;width:600px&#39;&gt;&quot; + img.buf + &quot;&lt;/div&gt;&quot;# Example datapeople = (&#39;Tom&#39;, &#39;Dick&#39;, &#39;Harry&#39;, &#39;Slim&#39;, &#39;Jim&#39;)y_pos = np.arange(len(people))performance = 3 + 10 * np.random.rand(len(people))error = np.random.rand(len(people))plt.barh(y_pos, performance, xerr=error, align=&#39;center&#39;, alpha=0.4)plt.yticks(y_pos, people)plt.xlabel(&#39;Per formance&#39;)plt.title(&#39;How fast do you want to go today?&#39;)show(plt)R ExtrasWith zeppelin running, an R Tutorial notebook will be available. The R packages required to run the examples and graphs in this tutorial notebook were installed by this virtual machine.The installed R Packages include: Knitr, devtools, repr, rCharts, ggplot2, googleVis, mplot, htmltools, base64enc, data.table", "url": " /install/virtual_machine.html", "group": "install", "excerpt": "Apache Zeppelin provides a script for running a virtual machine for development through Vagrant. The script will create a virtual machine with core dependencies pre-installed, required for developing Apache Zeppelin." @@ -424,7 +424,7 @@ "/manual/interpreterinstallation.html": { "title": "Interpreter Installation in Netinst Binary Package", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Interpreter InstallationApache Zeppelin provides Interpreter Installation mechanism for whom downloaded Zeppelin netinst binary package, or just want to install another 3rd party interpreters. Community managed interpretersApache Zeppelin provides several interpreters as community managed interpreters. If you downloaded netinst binary package, you need to install by using below commands.Install all community managed interpreters./b in/install-interpreter.sh --allInstall specific interpreters./bin/install-interpreter.sh --name md,shell,jdbc,pythonYou can get full list of community managed interpreters by running./bin/install-interpreter.sh --listInstall interpreter built with Scala 2.10From version 0.6.1, Zeppelin support both Scala 2.10 and 2.11 for several interpreters as below: Name Maven Artifact for Scala 2.10 Maven Artifact for Scala 2.11 cassandra org.apache.zeppelin:zeppelin-cassandra_2.10:0.6.1 org.apache.zeppelin:zeppelin-cassandra_2.11:0.6.1 flink org.apache.zeppelin:zeppelin-flink_2.10:0.6.1 org.apache.zeppelin:zeppelin-flink_2.11:0.6.1 ignite org.apache.zeppelin:zeppelin-ignite_2.10:0.6.1 org.apache.zeppelin:zeppelin-ignite_2.11:0.6.1 flink org.apache.zeppelin:zeppelin-spark_2.10:0.6.1 org.apache.zeppelin:zeppelin-spark_2.11:0.6.1 If you install one of these interpreters only with --name option, installer will download interpreter buil t with Scala 2.11 by default. If you want to specify Scala version, you will need to add --artifact option. Here is the example of installing flink interpreter built with Scala 2.10../bin/install-interpreter.sh --name flink --artifact org.apache.zeppelin:zeppelin-flink_2.10:0.6.1Install Spark interpreter built with Scala 2.10Spark distribution package has been built with Scala 2.10 until 1.6.2. If you have SPARK_HOME set pointing to Spark version ealier than 2.0.0, you need to download Spark interpreter packaged with Scala 2.10. To do so, use follow command:rm -rf ./interpreter/spark./bin/install-interpreter.sh --name spark --artifact org.apache.zeppelin:zeppelin-spark_2.10:0.6.1Once you have installed interpreters, you need to restart Zeppelin. And then create interpreter setting and bind it with your notebook.3rd party interpretersYou can also install 3rd party interpreters located in the maven repository by using below commands.Install 3rd party interpreters./bin/install-interpre ter.sh --name interpreter1 --artifact groupId1:artifact1:version1The above command will download maven artifact groupId1:artifact1:version1 and all of it&#39;s transitive dependencies into interpreter/interpreter1 directory.Once you have installed interpreters, you&#39;ll need to add interpreter class name into zeppelin.interpreters property in configuration.And then restart Zeppelin, create interpreter setting and bind it with your notebook.Install multiple 3rd party interpreters at once./bin/install-interpreter.sh --name interpreter1,interpreter2 --artifact groupId1:artifact1:version1,groupId2:artifact2:version2--name and --artifact arguments will recieve comma separated list.Available community managed interpretersYou can also find the below community managed interpreter list in conf/interpreter-list file. Name Maven Artifact Description alluxio org.apache.zeppelin:zeppelin-alluxio:0.6.1 Alluxio interpreter angular org.apache.zeppelin:zep pelin-angular:0.6.1 HTML and AngularJS view rendering bigquery org.apache.zeppelin:zeppelin-bigquery:0.6.1 BigQuery interpreter cassandra org.apache.zeppelin:zeppelin-cassandra_2.11:0.6.1 Cassandra interpreter built with Scala 2.11 elasticsearch org.apache.zeppelin:zeppelin-elasticsearch:0.6.1 Elasticsearch interpreter file org.apache.zeppelin:zeppelin-file:0.6.1 HDFS file interpreter flink org.apache.zeppelin:zeppelin-flink_2.11:0.6.1 Flink interpreter built with Scala 2.11 hbase org.apache.zeppelin:zeppelin-hbase:0.6.1 Hbase interpreter ignite org.apache.zeppelin:zeppelin-ignite_2.11:0.6.1 Ignite interpreter built with Scala 2.11 jdbc org.apache.zeppelin:zeppelin-jdbc:0.6.1 Jdbc interpreter kylin org.apache.zeppelin:zeppelin-kylin:0.6.1 Kylin interpreter lens org.apache.zeppelin:zeppelin-lens:0.6.1 Lens interpreter livy org.apache.zeppelin: zeppelin-livy:0.6.1 Livy interpreter md org.apache.zeppelin:zeppelin-markdown:0.6.1 Markdown support postgresql org.apache.zeppelin:zeppelin-postgresql:0.6.1 Postgresql interpreter python org.apache.zeppelin:zeppelin-python:0.6.1 Python interpreter shell org.apache.zeppelin:zeppelin-shell:0.6.1 Shell command ", + "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Interpreter InstallationApache Zeppelin provides Interpreter Installation mechanism for whom downloaded Zeppelin netinst binary package, or just want to install another 3rd party interpreters. Community managed interpretersApache Zeppelin provides several interpreters as community managed interpreters. If you downloaded netinst binary package, you need to install by using below commands.Install all community managed interpreters./b in/install-interpreter.sh --allInstall specific interpreters./bin/install-interpreter.sh --name md,shell,jdbc,pythonYou can get full list of community managed interpreters by running./bin/install-interpreter.sh --listInstall interpreter built with Scala 2.10From version 0.6.1, Zeppelin support both Scala 2.10 and 2.11 for several interpreters as below: Name Maven Artifact for Scala 2.10 Maven Artifact for Scala 2.11 cassandra org.apache.zeppelin:zeppelin-cassandra_2.10:0.6.1 org.apache.zeppelin:zeppelin-cassandra_2.11:0.6.1 flink org.apache.zeppelin:zeppelin-flink_2.10:0.6.1 org.apache.zeppelin:zeppelin-flink_2.11:0.6.1 ignite org.apache.zeppelin:zeppelin-ignite_2.10:0.6.1 org.apache.zeppelin:zeppelin-ignite_2.11:0.6.1 flink org.apache.zeppelin:zeppelin-spark_2.10:0.6.1 org.apache.zeppelin:zeppelin-spark_2.11:0.6.1 If you install one of these interpreters only with --name option, installer will download interpreter buil t with Scala 2.11 by default. If you want to specify Scala version, you will need to add --artifact option. Here is the example of installing flink interpreter built with Scala 2.10../bin/install-interpreter.sh --name flink --artifact org.apache.zeppelin:zeppelin-flink_2.10:0.6.1Install Spark interpreter built with Scala 2.10Spark distribution package has been built with Scala 2.10 until 1.6.2. If you have SPARK_HOME set pointing to Spark version earlier than 2.0.0, you need to download Spark interpreter packaged with Scala 2.10. To do so, use follow command:rm -rf ./interpreter/spark./bin/install-interpreter.sh --name spark --artifact org.apache.zeppelin:zeppelin-spark_2.10:0.6.1Once you have installed interpreters, you need to restart Zeppelin. And then create interpreter setting and bind it with your notebook.3rd party interpretersYou can also install 3rd party interpreters located in the maven repository by using below commands.Install 3rd party interpreters./bin/install-interpr eter.sh --name interpreter1 --artifact groupId1:artifact1:version1The above command will download maven artifact groupId1:artifact1:version1 and all of it&#39;s transitive dependencies into interpreter/interpreter1 directory.Once you have installed interpreters, you&#39;ll need to add interpreter class name into zeppelin.interpreters property in configuration.And then restart Zeppelin, create interpreter setting and bind it with your notebook.Install multiple 3rd party interpreters at once./bin/install-interpreter.sh --name interpreter1,interpreter2 --artifact groupId1:artifact1:version1,groupId2:artifact2:version2--name and --artifact arguments will recieve comma separated list.Available community managed interpretersYou can also find the below community managed interpreter list in conf/interpreter-list file. Name Maven Artifact Description alluxio org.apache.zeppelin:zeppelin-alluxio:0.6.1 Alluxio interpreter angular org.apache.zeppelin:ze ppelin-angular:0.6.1 HTML and AngularJS view rendering bigquery org.apache.zeppelin:zeppelin-bigquery:0.6.1 BigQuery interpreter cassandra org.apache.zeppelin:zeppelin-cassandra_2.11:0.6.1 Cassandra interpreter built with Scala 2.11 elasticsearch org.apache.zeppelin:zeppelin-elasticsearch:0.6.1 Elasticsearch interpreter file org.apache.zeppelin:zeppelin-file:0.6.1 HDFS file interpreter flink org.apache.zeppelin:zeppelin-flink_2.11:0.6.1 Flink interpreter built with Scala 2.11 hbase org.apache.zeppelin:zeppelin-hbase:0.6.1 Hbase interpreter ignite org.apache.zeppelin:zeppelin-ignite_2.11:0.6.1 Ignite interpreter built with Scala 2.11 jdbc org.apache.zeppelin:zeppelin-jdbc:0.6.1 Jdbc interpreter kylin org.apache.zeppelin:zeppelin-kylin:0.6.1 Kylin interpreter lens org.apache.zeppelin:zeppelin-lens:0.6.1 Lens interpreter livy org.apache.zeppelin :zeppelin-livy:0.6.1 Livy interpreter md org.apache.zeppelin:zeppelin-markdown:0.6.1 Markdown support postgresql org.apache.zeppelin:zeppelin-postgresql:0.6.1 Postgresql interpreter python org.apache.zeppelin:zeppelin-python:0.6.1 Python interpreter shell org.apache.zeppelin:zeppelin-shell:0.6.1 Shell command ", "url": " /manual/interpreterinstallation.html", "group": "manual", "excerpt": "Apache Zeppelin provides Interpreter Installation mechanism for whom downloaded Zeppelin netinst binary package, or just want to install another 3rd party interpreters." @@ -435,7 +435,7 @@ "/manual/interpreters.html": { "title": "Interpreters in Apache Zeppelin", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Interpreters in Apache ZeppelinOverviewIn this section, we will explain about the role of interpreters, interpreters group and interpreter settings in Zeppelin.The concept of Zeppelin interpreter allows any language/data-processing-backend to be plugged into Zeppelin.Currently, Zeppelin supports many interpreters such as Scala ( with Apache Spark ), Python ( with Apache Spark ), Spark SQL, JDBC, Markdown, Shell and so on.What is Ze ppelin interpreter?Zeppelin Interpreter is a plug-in which enables Zeppelin users to use a specific language/data-processing-backend. For example, to use Scala code in Zeppelin, you need %spark interpreter.When you click the +Create button in the interpreter page, the interpreter drop-down list box will show all the available interpreters on your server.What is interpreter setting?Zeppelin interpreter setting is the configuration of a given interpreter on Zeppelin server. For example, the properties are required for hive JDBC interpreter to connect to the Hive server.Properties are exported as environment variable when property name is consisted of upper characters, numbers and underscore ([A-Z_0-9]). Otherwise set properties as JVM property.Each notebook can be bound to multiple Interpreter Settings using setting icon on upper right corner of the notebook.What is interpreter group?Every Interpreter is belonged to an Interpreter Group. Interpreter Group is a unit of start/stop inter preter.By default, every interpreter is belonged to a single group, but the group might contain more interpreters. For example, Spark interpreter group is including Spark support, pySpark, Spark SQL and the dependency loader.Technically, Zeppelin interpreters from the same group are running in the same JVM. For more information about this, please checkout here.Each interpreters is belonged to a single group and registered together. All of their properties are listed in the interpreter setting like below image.Interpreter binding modeEach Interpreter Setting can choose one of &#39;shared&#39;, &#39;scoped&#39;, &#39;isolated&#39; interpreter binding mode.In &#39;shared&#39; mode, every notebook bound to the Interpreter Setting will share the single Interpreter instance. In &#39;scoped&#39; mode, each notebook will create new Interpreter instance in the same interpreter process. In &#39;isolated&#39; mode, each notebook will create new I nterpreter process.Connecting to the existing remote interpreterZeppelin users can start interpreter thread embedded in their service. This will provide flexibility to user to start interpreter on remote host. To start interpreter along with your service you have to create an instance of RemoteInterpreterServer and start it as follows:RemoteInterpreterServer interpreter=new RemoteInterpreterServer(3678); // Here, 3678 is the port on which interpreter will listen. interpreter.start() The above code will start interpreter thread inside your process. Once the interpreter is started you can configure zeppelin to connect to RemoteInterpreter by checking Connect to existing process checkbox and then provide Host and Port on which interpreter porocess is listening as shown in the image below:", + "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Interpreters in Apache ZeppelinOverviewIn this section, we will explain about the role of interpreters, interpreters group and interpreter settings in Zeppelin.The concept of Zeppelin interpreter allows any language/data-processing-backend to be plugged into Zeppelin.Currently, Zeppelin supports many interpreters such as Scala ( with Apache Spark ), Python ( with Apache Spark ), Spark SQL, JDBC, Markdown, Shell and so on.What is Ze ppelin interpreter?Zeppelin Interpreter is a plug-in which enables Zeppelin users to use a specific language/data-processing-backend. For example, to use Scala code in Zeppelin, you need %spark interpreter.When you click the +Create button in the interpreter page, the interpreter drop-down list box will show all the available interpreters on your server.What is interpreter setting?Zeppelin interpreter setting is the configuration of a given interpreter on Zeppelin server. For example, the properties are required for hive JDBC interpreter to connect to the Hive server.Properties are exported as environment variable when property name is consisted of upper characters, numbers and underscore ([A-Z_0-9]). Otherwise set properties as JVM property.Each notebook can be bound to multiple Interpreter Settings using setting icon on upper right corner of the notebook.What is interpreter group?Every Interpreter is belonged to an Interpreter Group. Interpreter Group is a unit of start/stop inter preter.By default, every interpreter is belonged to a single group, but the group might contain more interpreters. For example, Spark interpreter group is including Spark support, pySpark, Spark SQL and the dependency loader.Technically, Zeppelin interpreters from the same group are running in the same JVM. For more information about this, please checkout here.Each interpreters is belonged to a single group and registered together. All of their properties are listed in the interpreter setting like below image.Interpreter binding modeEach Interpreter Setting can choose one of &#39;shared&#39;, &#39;scoped&#39;, &#39;isolated&#39; interpreter binding mode.In &#39;shared&#39; mode, every notebook bound to the Interpreter Setting will share the single Interpreter instance. In &#39;scoped&#39; mode, each notebook will create new Interpreter instance in the same interpreter process. In &#39;isolated&#39; mode, each notebook will create new I nterpreter process.Connecting to the existing remote interpreterZeppelin users can start interpreter thread embedded in their service. This will provide flexibility to user to start interpreter on remote host. To start interpreter along with your service you have to create an instance of RemoteInterpreterServer and start it as follows:RemoteInterpreterServer interpreter=new RemoteInterpreterServer(3678); // Here, 3678 is the port on which interpreter will listen. interpreter.start() The above code will start interpreter thread inside your process. Once the interpreter is started you can configure zeppelin to connect to RemoteInterpreter by checking Connect to existing process checkbox and then provide Host and Port on which interpreter process is listening as shown in the image below:(Experimental) Interpreter Execution HooksZeppelin allows for users to specify additional code to be executed by an interpreter at pre and post-paragraph code execution. This is primarily useful if you need to run the same set of code for all of the paragraphs within your notebook at specific times. Currently, this feature is only available for the spark and pyspark interpreters. To specify your hook code, you may use &#39;z.registerHook(). For example, enter the following into one paragraph:%pysparkz.registerHook(&quot;post_exec&quot;, &quot;print &#39;This code should be executed before the parapgraph code!&#39;&quot;)z.registerHook(&quot;pre_exec&quot;, &quot;print &#39;This code should be executed after the paragraph code!&#39;&quot;)These calls will not take into effect until the next time you run a paragraph. In another paragraph, enterpython%pysparkprint &quot;This code should be entered into the paragraph by the user!&quot;The output should be:This code should be executed before the paragraph code!This code should be entered into the paragraph by the user!This code should be executed after the paragraph code!I f you ever need to know the hook code, use z.getHook():python%pysparkprint z.getHook(&quot;post_exec&quot;)print &#39;This code should be executed after the paragraph code!&#39;Any call to z.registerHook() will automatically overwrite what was previously registered. To completely unregister a hook event, use z.unregisterHook(eventCode). Currently only &quot;post_exec&quot; and &quot;pre_exec&quot; are valid event codes for the Zeppelin Hook Registry system.Finally, the hook registry is internally shared by other interpreters in the same group. This would allow for hook code for one interpreter REPL to be set by another as follows:%sparkz.unregisterHook(&quot;post_exec&quot;, &quot;pyspark&quot;)The API is identical for both the spark (scala) and pyspark (python) implementations.CaveatsCalls to z.registerHook(&quot;pre_exec&quot;, ...) should be made with care. If there are errors in your specified hook code, this will cause the interpreter REPL to become unable to execute any code pass the pre-execute stage making it impossible for direct calls to z.unregisterHook() to take into effect. Current workarounds include calling z.unregisterHook() from a different interpreter REPL in the same interpreter group (see above) or manually restarting the interpreter group in the UI. ", "url": " /manual/interpreters.html", "group": "manual", "excerpt": "This document explains about the role of interpreters, interpreters group and interpreter settings in Apache Zeppelin. The concept of Zeppelin interpreter allows any language/data-processing-backend to be plugged into Zeppelin." @@ -446,7 +446,7 @@ "/manual/notebookashomepage.html": { "title": "Customize Apache Zeppelin homepage", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Customize Apache Zeppelin homepageApache Zeppelin allows you to use one of the notebooks you create as your Zeppelin Homepage.With that you can brand your Zeppelin installation, adjust the instruction to your users needs and even translate to other languages.How to set a notebook as your Zeppelin homepageThe process for creating your homepage is very simple as shown below:Create a notebook using ZeppelinSet the notebook id in the c onfig fileRestart ZeppelinCreate a notebook using ZeppelinCreate a new notebook using Zeppelin,you can use %md interpreter for markdown content or any other interpreter you like.You can also use the display system to generate text, html, table orAngular (backend API, frontend API).Run (shift+Enter) the notebook and see the output. Optionally, change the notebook view to report to hidethe code sections.Set the notebook id in the config fileTo set the notebook id in the config file, you should copy it from the last word in the notebook url.For example,Set the notebook id to the ZEPPELIN_NOTEBOOK_HOMESCREEN environment variableor zeppelin.notebook.homescreen property.You can also set the ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE environment variableor zeppelin.notebook.homescreen.hide property to hide the new notebook from the notebook list.Restart ZeppelinRestart your Zeppelin server./bin/zeppelin-deamon stop./bin/zeppelin-deamon startThat&#39;s it! Open your browser and navigate to Apach e Zeppelin and see your customized homepage.Show notebooks list in your custom homepageIf you want to display the list of notebooks on your custom Apache Zeppelin homepage allyou need to do is use our %angular support.Add the following code to a paragraph in you home page and run it... walla! you have your notebooks list.println(&quot;&quot;&quot;%angular &lt;div class=&quot;col-md-4&quot; ng-controller=&quot;HomeCtrl as home&quot;&gt; &lt;h4&gt;Notebooks&lt;/h4&gt; &lt;div&gt; &lt;h5&gt;&lt;a href=&quot;&quot; data-toggle=&quot;modal&quot; data-target=&quot;#noteNameModal&quot; style=&quot;text-decoration: none;&quot;&gt; &lt;i style=&quot;font-size: 15px;&quot; class=&quot;icon-notebook&quot;&gt;&lt;/i&gt; Create new note&lt;/a&gt;&lt;/h5&gt; &lt;ul style=&quot;list-style-type: none;&quot;& amp;gt; &lt;li ng-repeat=&quot;note in home.notes.list track by $index&quot;&gt;&lt;i style=&quot;font-size: 10px;&quot; class=&quot;icon-doc&quot;&gt;&lt;/i&gt; &lt;a style=&quot;text-decoration: none;&quot; href=&quot;#/notebook/&quot;&gt;&lt;/a&gt; &lt;/li&gt; &lt;/ul&gt; &lt;/div&gt; &lt;/div&gt;&quot;&quot;&quot;)After running the notebook you will see output similar to this one:The main trick here relays in linking the &lt;div&gt; to the controller:&lt;div class=&quot;col-md-4&quot; ng-controller=&quot;HomeCtrl as home&quot;&gt;Once we have home as our controller variable in our &lt;div&gt;&lt;/div&gt; we can use home.notes.list to get access to the notebook list.", + "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->Customize Apache Zeppelin homepageApache Zeppelin allows you to use one of the notebooks you create as your Zeppelin Homepage.With that you can brand your Zeppelin installation, adjust the instruction to your users needs and even translate to other languages.How to set a notebook as your Zeppelin homepageThe process for creating your homepage is very simple as shown below:Create a notebook using ZeppelinSet the notebook id in the c onfig fileRestart ZeppelinCreate a notebook using ZeppelinCreate a new notebook using Zeppelin,you can use %md interpreter for markdown content or any other interpreter you like.You can also use the display system to generate text, html, table orAngular (backend API, frontend API).Run (shift+Enter) the notebook and see the output. Optionally, change the notebook view to report to hidethe code sections.Set the notebook id in the config fileTo set the notebook id in the config file, you should copy it from the last word in the notebook url.For example,Set the notebook id to the ZEPPELIN_NOTEBOOK_HOMESCREEN environment variableor zeppelin.notebook.homescreen property.You can also set the ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE environment variableor zeppelin.notebook.homescreen.hide property to hide the new notebook from the notebook list.Restart ZeppelinRestart your Zeppelin server./bin/zeppelin-daemon stop./bin/zeppelin-daemon startThat&#39;s it! Open your browser and navigate to Apach e Zeppelin and see your customized homepage.Show notes list in your custom homepageIf you want to display the list of notes on your custom Apache Zeppelin homepage allyou need to do is use our %angular support.Add the following code to a paragraph in you home page and run it... Voila! You have your notes list.println(&quot;&quot;&quot;%angular &lt;div class=&quot;col-md-4&quot; ng-controller=&quot;HomeCtrl as home&quot;&gt; &lt;h4&gt;Notebooks&lt;/h4&gt; &lt;div&gt; &lt;h5&gt;&lt;a href=&quot;&quot; data-toggle=&quot;modal&quot; data-target=&quot;#noteNameModal&quot; style=&quot;text-decoration: none;&quot;&gt; &lt;i style=&quot;font-size: 15px;&quot; class=&quot;icon-notebook&quot;&gt;&lt;/i&gt; Create new note&lt;/a&gt;&lt;/h5&gt; &lt;ul style=&quot;list-style-type: none;&quot;&gt; &lt;li ng-repeat=&quot;note in home.notes.list track by $index&quot;&gt;&lt;i style=&quot;font-size: 10px;&quot; class=&quot;icon-doc&quot;&gt;&lt;/i&gt; &lt;a style=&quot;text-decoration: none;&quot; href=&quot;#/notebook/&quot;&gt;&lt;/a&gt; &lt;/li&gt; &lt;/ul&gt; &lt;/div&gt; &lt;/div&gt;&quot;&quot;&quot;)After running the notebook you will see output similar to this one:The main trick here relays in linking the &lt;div&gt; to the controller:&lt;div class=&quot;col-md-4&quot; ng-controller=&quot;HomeCtrl as home&quot;&gt;Once we have home as our controller variable in our &lt;div&gt;&lt;/div&gt; we can use home.notes.list to get access to the notebook list.", "url": " /manual/notebookashomepage.html", "group": "manual", "excerpt": "Apache Zeppelin allows you to use one of the notebooks you create as your Zeppelin Homepage. With that you can brand your Zeppelin installation, adjust the instruction to your users needs and even translate to other languages." @@ -480,7 +480,7 @@ "/quickstart/install_with_flink_and_spark_cluster.html": { "title": "Install Zeppelin with Flink and Spark in cluster mode", - "content" : "<!--Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.-->This tutorial is extremely entry-level. It assumes no prior knowledge of Linux, git, or other tools. If you carefully type what I tell you when I tell you, you should be able to get Zeppelin running.Installing Zeppelin with Flink and Spark in cluster modeThis tutorial assumes the user has a machine (real or virtual with a fresh, minimal installation of Ubuntu 14.04.3 Server.Note: On the size requirements of the Virtual Machine, som e users reported trouble when using the default virtual machine sizes, specifically that the hard drive needed to be at least 16GB- other users did not have this issue.There are many good tutorials on how to install Ubuntu Server on a virtual box, here is one of themRequired ProgramsAssuming the minimal install, there are several programs that we will need to install before Zeppelin, Flink, and Spark.gitopenssh-serverOpenJDK 7Maven 3.1+For git, openssh-server, and OpenJDK 7 we will be using the apt package manager.gitFrom the command prompt:sudo apt-get install gitopenssh-serversudo apt-get install openssh-serverOpenJDK 7sudo apt-get install openjdk-7-jdk openjdk-7-jre-libA note for those using Ubuntu 16.04: To install openjdk-7 on Ubuntu 16.04, one must add a repository. Sourcesudo add-apt-repository ppa:openjdk-r/ppasudo apt-get updatesudo apt-get install openjdk-7-jdk openjdk-7-jre-libMaven 3.1+Zeppelin requires maven version 3.x. The version available in the repositories at th e time of writing is 2.x, so maven must be installed manually.Purge any existing versions of maven.sudo apt-get purge maven maven2Download the maven 3.3.9 binary.wget &quot;http://www.us.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz&quot;Unarchive the binary and move to the /usr/local directory.tar -zxvf apache-maven-3.3.9-bin.tar.gzsudo mv ./apache-maven-3.3.9 /usr/localCreate symbolic links in /usr/bin.sudo ln -s /usr/local/apache-maven-3.3.9/bin/mvn /usr/bin/mvnInstalling ZeppelinThis provides a quick overview of Zeppelin installation from source, however the reader is encouraged to review the Zeppelin Installation GuideFrom the command prompt:Clone Zeppelin.git clone https://github.com/apache/zeppelin.gitEnter the Zeppelin root directory.cd zeppelinPackage Zeppelin.mvn clean package -DskipTests -Pspark-1.6 -Dflink.version=1.1.2-DskipTests skips build tests- you&#39;re not developing (yet), so you don&#39;t need to do tests, the clone version should build.-Pspark-1.6 tells maven to build a Zeppelin with Spark 1.6. This is important because Zeppelin has its own Spark interpreter and the versions must be the same.-Dflink.version=1.1.2 tells maven specifically to build Zeppelin with Flink version 1.1.2.Note: You may wish to include additional build flags such as -Ppyspark or -Psparkr. See the build section of github for more details.Note: You can build against any version of Spark that has a Zeppelin build profile available. The key is to make sure you check out the matching version of Spark to build. At the time of this writing, Spark 1.6 was the most recent Spark version available.Note: On build failures. Having installed Zeppelin close to 30 times now, I will tell you that sometimes the build fails for seemingly no reason.As long as you didn&#39;t edit any code, it is unlikely the build is failing because of something you did. What does tend to happen, is some dependency that maven is trying to download is unreachable. If your build fails on this step here are some tips:- Don&#39;t get discouraged.- Scroll up and read through the logs. There will be clues there.- Retry (that is, run the mvn clean package -DskipTests -Pspark-1.6 again)- If there were clues that a dependency couldn&#39;t be downloaded wait a few hours or even days and retry again. Open source software when compiling is trying to download all of the dependencies it needs, if a server is off-line there is nothing you can do but wait for it to come back.- Make sure you followed all of the steps carefully.- Ask the community to help you. Go here and join the user mailing list. People are there to help you. Make sure to copy and paste the build output (everything that happened in the console) and include that in your message.Start the Zeppelin daemon.bin/zeppelin-daemon.sh startUse ifconfig to determine the host machine&#39;s IP address. If you are not familiar with how to do this, a fairly comprehensive post ca n be found here.Open a web-browser on a machine connected to the same network as the host (or in the host operating system if using a virtual machine). Navigate to http://yourip:8080, where yourip is the IP address you found in ifconfig.See the Zeppelin tutorial for basic Zeppelin usage. It is also advised that you take a moment to check out the tutorial notebook that is included with each Zeppelin install, and to familiarize yourself with basic notebook functionality.Flink TestCreate a new notebook named &quot;Flink Test&quot; and copy and paste the following code.%flink // let Zeppelin know what interpreter to use.val text = env.fromElements(&quot;In the time of chimpanzees, I was a monkey&quot;, // some lines of text to analyze&quot;Butane in my veins and I&#39;m out to cut the junkie&quot;,&quot;With the plastic eyeballs, spray paint the vegetables&quot;,&quot;Dog food stalls with the beefcake pantyhose&quot;,&quot;Kill the hea dlights and put it in neutral&quot;,&quot;Stock car flamin&#39; with a loser in the cruise control&quot;,&quot;Baby&#39;s in Reno with the Vitamin D&quot;,&quot;Got a couple of couches, sleep on the love seat&quot;,&quot;Someone came in sayin&#39; I&#39;m insane to complain&quot;,&quot;About a shotgun wedding and a stain on my shirt&quot;,&quot;Don&#39;t believe everything that you breathe&quot;,&quot;You get a parking violation and a maggot on your sleeve&quot;,&quot;So shave your face with some mace in the dark&quot;,&quot;Savin&#39; all your food stamps and burnin&#39; down the trailer park&quot;,&quot;Yo, cut it&quot;)/* The meat and potatoes: this tells Flink to iterate through the elements, in this case strings, transform the string to lower case and split the string at white space into individual words then finally aggregate the occurrence of e ach word. This creates the count variable which is a list of tuples of the form (word, occurances)counts.collect().foreach(println(_)) // execute the script and print each element in the counts list*/val counts = text.flatMap{ _.toLowerCase.split(&quot;W+&quot;) }.map { (_,1) }.groupBy(0).sum(1)counts.collect().foreach(println(_)) // execute the script and print each element in the counts listRun the code to make sure the built-in Zeppelin Flink interpreter is working properly.Spark TestCreate a new notebook named &quot;Spark Test&quot; and copy and paste the following code.%spark // let Zeppelin know what interpreter to use.val text = sc.parallelize(List(&quot;In the time of chimpanzees, I was a monkey&quot;, // some lines of text to analyze&quot;Butane in my veins and I&#39;m out to cut the junkie&quot;,&quot;With the plastic eyeballs, spray paint the vegetables&quot;,&quot;Dog food stalls with the beefcake pantyhose&qu ot;,&quot;Kill the headlights and put it in neutral&quot;,&quot;Stock car flamin&#39; with a loser in the cruise control&quot;,&quot;Baby&#39;s in Reno with the Vitamin D&quot;,&quot;Got a couple of couches, sleep on the love seat&quot;,&quot;Someone came in sayin&#39; I&#39;m insane to complain&quot;,&quot;About a shotgun wedding and a stain on my shirt&quot;,&quot;Don&#39;t believe everything that you breathe&quot;,&quot;You get a parking violation and a maggot on your sleeve&quot;,&quot;So shave your face with some mace in the dark&quot;,&quot;Savin&#39; all your food stamps and burnin&#39; down the trailer park&quot;,&quot;Yo, cut it&quot;))/* The meat and potatoes: this tells spark to iterate through the elements, in this case strings, transform the string to lower case and split the string at white space into individual words then finally ag gregate the occurrence of each word. This creates the count variable which is a list of tuples of the form (word, occurances)*/val counts = text.flatMap { _.toLowerCase.split(&quot;W+&quot;) } .map { (_,1) } .reduceByKey(_ + _)counts.collect().foreach(println(_)) // execute the script and print each element in the counts listRun the code to make sure the built-in Zeppelin Flink interpreter is working properly.Finally, stop the Zeppelin daemon. From the command prompt run:bin/zeppelin-daemon.sh stopInstalling ClustersFlink ClusterDownload BinariesBuilding from source is recommended where possible, for simplicity in this tutorial we will download Flink and Spark Binaries.To download the Flink Binary use wgetwget &quot;http://mirror.cogentco.com/pub/apache/flink/flink-1.0.3/flink-1.0.3-bin-hadoop24-scala_2.10.tgz&quot;tar -xzvf flink-1.0.3-bin-hadoop24-scala_2.10.tgzThis will download Flink 1.0.3, compatible with Hadoop 2.4. Yo u do not have to install Hadoop for this binary to work, but if you are using Hadoop, please change 24 to your appropriate version.Start the Flink Cluster.flink-1.0.3/bin/start-cluster.shBuilding From sourceIf you wish to build Flink from source, the following will be instructive. Note that if you have downloaded and used the binary version this should be skipped. The changing nature of build tools and versions across platforms makes this section somewhat precarious. For example, Java8 and Maven 3.0.3 are recommended for building Flink, which are not recommended for Zeppelin at the time of writing. If the user wishes to attempt to build from source, this section will provide some reference. If errors are encountered, please contact the Apache Flink community.See the Flink Installation guide for more detailed instructions.Return to the directory where you have been downloading, this tutorial assumes that is $HOME. Clone Flink, check out release-1.0, and build.cd $HOMEgit clone https://github.com/apache/flink.gitcd flinkgit checkout release-1.0mvn clean install -DskipTestsStart the Flink Cluster in stand-alone modebuild-target/bin/start-cluster.shEnsure the cluster is upIn a browser, navigate to http://yourip:8082 to see the Flink Web-UI. Click on &#39;Task Managers&#39; in the left navigation bar. Ensure there is at least one Task Manager present.If no task managers are present, restart the Flink cluster with the following commands:(if binaries)flink-1.0.3/bin/stop-cluster.shflink-1.0.3/bin/start-cluster.sh(if built from source)build-target/bin/stop-cluster.shbuild-target/bin/start-cluster.shSpark 1.6 ClusterDownload BinariesBuilding from source is recommended where possible, for simplicity in this tutorial we will download Flink and Spark Binaries.Using binaries is alsoTo download the Spark Binary use wgetwget &quot;http://mirrors.koehn.com/apache/spark/spark-1.6.1/spark-1.6.1-bin-hadoop2.4.tgz&quot;tar -xzvf spark-1.6.1-bin-hadoop2.4.t gzmv spark-1.6.1-bin-hadoop4.4 sparkThis will download Spark 1.6.1, compatible with Hadoop 2.4. You do not have to install Hadoop for this binary to work, but if you are using Hadoop, please change 2.4 to your appropriate version.Building From sourceSpark is an extraordinarily large project, which takes considerable time to download and build. It is also prone to build failures for similar reasons listed in the Flink section. If the user wishes to attempt to build from source, this section will provide some reference. If errors are encountered, please contact the Apache Spark community.See the Spark Installation guide for more detailed instructions.Return to the directory where you have been downloading, this tutorial assumes that is $HOME. Clone Spark, check out branch-1.6, and build.Note: Recall, we&#39;re only checking out 1.6 because it is the most recent Spark for which a Zeppelin profile exists at the time of writing. You are free to check out other version, just make sure you build Zeppelin against the correct version of Spark.cd $HOMEClone, check out, and build Spark version 1.6.x.git clone https://github.com/apache/spark.gitcd sparkgit checkout branch-1.6mvn clean package -DskipTestsStart the Spark clusterReturn to the $HOME directory.cd $HOMEStart the Spark cluster in stand alone mode, specifying the webui-port as some port other than 8080 (the webui-port of Zeppelin).spark/sbin/start-master.sh --webui-port 8082Note: Why --webui-port 8082? There is a digression toward the end of this document that explains this.Open a browser and navigate to http://yourip:8082 to ensure the Spark master is running.Toward the top of the page there will be a URL: spark://yourhost:7077. Note this URL, the Spark Master URI, it will be needed in subsequent steps.Start the slave using the URI from the Spark master WebUI:spark/sbin/start-slave.sh spark://yourhostname:7077Return to the root directory and start the Zeppelin daemon.cd $HOMEzeppelin/bin/zeppelin-daemon .sh startConfigure InterpretersOpen a web browser and go to the Zeppelin web-ui at http://yourip:8080.Now go back to the Zeppelin web-ui at http://yourip:8080 and this time click on anonymous at the top right, which will open a drop-down menu, select Interpreters to enter interpreter configuration.In the Spark section, click the edit button in the top right corner to make the property values editable (looks like a pencil).The only field that needs to be edited in the Spark interpreter is the master field. Change this value from local[*] to the URL you used to start the slave, mine was spark://ubuntu:7077.Click Save to update the parameters, and click OK when it asks you about restarting the interpreter.Now scroll down to the Flink section. Click the edit button and change the value of host from local to localhost. Click Save again.Reopen the examples and execute them again (I.e. you need to click the play button at the top of the screen, or the button on the paragraph .You should be able check the Flink and Spark webuis (at something like http://yourip:8081, http://yourip:8082, http://yourip:8083) and see that jobs have been run against the clusters.Digression Sorry to be vague and use terms such as &#39;something like&#39;, but exactly what web-ui is at what port is going to depend on what order you started things. What is really going on here is you are pointing your browser at specific ports, namely 8081, 8082, and 8083. Flink and Spark all want to put their web-ui on port 8080, but are well behaved and will take the next port available. Since Zeppelin started first, it will get port 8080. When Flink starts (assuming you started Flink first), it will try to bind to port 8080, see that it is already taken, and go to the next one available, hopefully 8081. Spark has a webui for the master and the slave, so when they start they will try to bind to 8080 already taken by Zeppelin), then 8081 (already taken by Flink&#39;s webui), then 8082. If ev erything goes smoothy and you followed the directions precisely, the webuis should be 8081 and 8082. It is possible to specify the port you want the webui to bind to (at the command line by passing the --webui-port &lt;port&gt; flag when you start the Flink and Spark, where &lt;port&gt; is the port you want to see that webui on. You can also set the default webui port of Spark and Flink (and Zeppelin) in the configuration files, but this is a tutorial for novices and slightly out of scope.Next StepsCheck out the tutorial for more cool things you can do with your new toy!Join the community, ask questions and contribute! Every little bit helps.",
[... 5 lines stripped ...]
