http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/9fe018b6/install/install-vagrant/index.html ---------------------------------------------------------------------- diff --git a/install/install-vagrant/index.html b/install/install-vagrant/index.html index b679ac7..f648ea3 100644 --- a/install/install-vagrant/index.html +++ b/install/install-vagrant/index.html @@ -5,7 +5,7 @@ <span class="gp">$ </span>git checkout master </pre></td></tr></tbody></table> </div> <p>Inside the directory <code>PredictionIO-Vagrant/</code>, you will find a file named <code>Vagrantfile</code> which is the configuration file used by Vagrant to setup the VM. You may modify this file if you want to change the VM configuration.</p><p>For example, if you want to change the memory of the VM, you can locate the following line in the <code>Vagrantfile</code> and change the value passed to the <code>memory</code> parameter (default is 2048MB):</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre> v.customize <span class="o">[</span><span class="s2">"modifyvm"</span>, :id, <span class="s2">"--cpuexecutioncap"</span>, <span class="s2">"90"</span>, <span class="s2">"--memory"</span>, <span class="s2">"2048"</span><span class="o">]</span> </pre></td></tr></tbody></table> </div> <p>In the directory <code>PredictionIO-Vagrant/</code>, bring up PredictionIO VM by running:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre><span class="gp">$ </span>vagrant up -</pre></td></tr></tbody></table> </div> <div class="alert-message info"><p>When you run <code>vagrant up</code> for the first time, it will download the base box ubuntu/trusty64 if you don't have it. Then it will also install all necessary libraries and setup PredictionIO in the virtual machine.</p></div><p>When it finishes successfully, you should see somthing like the following:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td></tr></tbody></table> </div> <div class="alert-message info"><p>When you run <code>vagrant up</code> for the first time, it will download the base box ubuntu/trusty64 if you don't have it. Then it will also install all necessary libraries and setup PredictionIO in the virtual machine.</p></div><p>When it finishes successfully, you should see something like the following:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -29,7 +29,7 @@ </pre></td></tr></tbody></table> </div> <p>That's it! Now you have a PredictionIO VM running!</p><p>Please see the following notes regarding how to use PredictionIO VM with vagrant.</p><h2 id='using-the-predictionio-vm' class='header-anchors'>Using the PredictionIO VM</h2><h3 id='login-to-the-vm' class='header-anchors'>Login to the VM</h3><p>You could ssh to the VM by running the following from your host machine in the same directory where you run <code>vagrant up</code> (i.e. PredictionIO-Vagrant/)</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre><span class="gp">$ </span>vagrant ssh </pre></td></tr></tbody></table> </div> <p>Then your console prompt becomes something like the following, which means you have logged into the VM:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>vagrant@vagrant-ubuntu-trusty-64:~<span class="err">$</span> </pre></td></tr></tbody></table> </div> <p>One you've logged into the VM, you can proceed to <a href="/start/download">Choosing an Engine Template</a> or continue the QuickStart of the Engine template you have chosen.</p><h3 id='shutdown-and-bring-up-predictionio-vm-again' class='header-anchors'>Shutdown and bring up PredictionIO VM again</h3><p>When you are not using PredictionIO VM, you should shut down VM properly, by running the following <strong>in the host machine</strong> (not inside VM):</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre><span class="gp">$ </span>vagrant halt -</pre></td></tr></tbody></table> </div> <div class="alert-message warning"><p>If you didn't shut down VM properly or you ran <code>vagrant suspend</code>, the VM may go to suspend state. HBase may not be running propoerly next time when you run <code>vagrant up.</code> In this case, you can always run <code>vagrant halt</code> to do a clean shutdown first before run <code>vagrant up</code> again.</p></div><p>Then you can run <code>vagrant up</code> again later to bring up the PredicitonIO VM again.</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre><span class="gp">$ </span>vagrant up +</pre></td></tr></tbody></table> </div> <div class="alert-message warning"><p>If you didn't shut down VM properly or you ran <code>vagrant suspend</code>, the VM may go to suspend state. HBase may not be running properly next time when you run <code>vagrant up.</code> In this case, you can always run <code>vagrant halt</code> to do a clean shutdown first before run <code>vagrant up</code> again.</p></div><p>Then you can run <code>vagrant up</code> again later to bring up the PredicitonIO VM again.</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre><span class="gp">$ </span>vagrant up </pre></td></tr></tbody></table> </div> <p>When it's ready, you should see the following:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3
http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/9fe018b6/install/launch-aws/index.html ---------------------------------------------------------------------- diff --git a/install/launch-aws/index.html b/install/launch-aws/index.html index 7ba88b4..3d9248b 100644 --- a/install/launch-aws/index.html +++ b/install/launch-aws/index.html @@ -1,4 +1,4 @@ -<!DOCTYPE html><html><head><title>Launching PredictionIO on AWS</title><meta charset="utf-8"/><meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta class="swiftype" name="title" data-type="string" content="Launching PredictionIO on AWS"/><link rel="canonical" href="https://predictionio.apache.org/install/launch-aws/"/><link href="/images/favicon/normal-b330020a.png" rel="shortcut icon"/><link href="/images/favicon/apple-c0febcf2.png" rel="apple-touch-icon"/><link href="//fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800" rel="stylesheet"/><link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/><link href="/stylesheets/application-eccfc6cb.css" rel="stylesheet" type="text/css"/><script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.2/html5shiv.min.js"></script><script src="//cdn.mathjax.or g/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script><script src="//use.typekit.net/pqo0itb.js"></script><script>try{Typekit.load({ async: true });}catch(e){}</script></head><body><div id="global"><header><div class="container" id="header-wrapper"><div class="row"><div class="col-sm-12"><div id="logo-wrapper"><span id="drawer-toggle"></span><a href="#"></a><a href="http://predictionio.apache.org/";><img alt="Apache PredictionIO" id="logo" src="/images/logos/logo-ee2b9bb3.png"/></a><span>®</span></div><div id="menu-wrapper"><div id="pill-wrapper"><a class="pill left" href="/gallery/template-gallery">TEMPLATES</a> <a class="pill right" href="//github.com/apache/predictionio/">OPEN SOURCE</a></div></div><img class="mobile-search-bar-toggler hidden-md hidden-lg" src="/images/icons/search-glass-704bd4ff.png"/></div></div></div></header><div id="search-bar-row-wrapper"><div class="container-fluid" id="search-bar-row"><div class="row"><div class="col-md-9 col-sm-11 col-xs-11" ><div class="hidden-md hidden-lg" >id="mobile-page-heading-wrapper"><p>PredictionIO Docs</p><h4>Launching >PredictionIO on AWS</h4></div><h4 class="hidden-sm hidden-xs">PredictionIO >Docs</h4></div><div class="col-md-3 col-sm-1 col-xs-1 hidden-md >hidden-lg"><img id="left-menu-indicator" >src="/images/icons/down-arrow-dfe9f7fe.png"/></div><div class="col-md-3 >col-sm-12 col-xs-12 swiftype-wrapper"><div class="swiftype"><form >class="search-form"><img class="search-box-toggler hidden-xs hidden-sm" >src="/images/icons/search-glass-704bd4ff.png"/><div class="search-box"><img >src="/images/icons/search-glass-704bd4ff.png"/><input type="text" >id="st-search-input" class="st-search-input" placeholder="Search >Doc..."/></div><img class="swiftype-row-hider hidden-md hidden-lg" >src="/images/icons/drawer-toggle-active-fcbef12a.png"/></form></div></div><div > class="mobile-left-menu-toggler hidden-md >hidden-lg"></div></div></div></div><div id="page" >class="container-fluid"><div class="row"><div id="left-me nu-wrapper" class="col-md-3"><nav id="nav-main"><ul><li class="level-1"><a class="expandible" href="/"><span>Apache PredictionIO® Documentation</span></a><ul><li class="level-2"><a class="final" href="/"><span>Welcome to Apache PredictionIO®</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Started</span></a><ul><li class="level-2"><a class="final" href="/start/"><span>A Quick Intro</span></a></li><li class="level-2"><a class="final" href="/install/"><span>Installing Apache PredictionIO</span></a></li><li class="level-2"><a class="final" href="/start/download/"><span>Downloading an Engine Template</span></a></li><li class="level-2"><a class="final" href="/start/deploy/"><span>Deploying Your First Engine</span></a></li><li class="level-2"><a class="final" href="/start/customize/"><span>Customizing the Engine</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Integrating with Your App</span></a><ul><li class="lev el-2"><a class="final" href="/appintegration/"><span>App Integration Overview</span></a></li><li class="level-2"><a class="expandible" href="/sdk/"><span>List of SDKs</span></a><ul><li class="level-3"><a class="final" href="/sdk/java/"><span>Java & Android SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/php/"><span>PHP SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/python/"><span>Python SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/ruby/"><span>Ruby SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/community/"><span>Community Powered SDKs</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Deploying an Engine</span></a><ul><li class="level-2"><a class="final" href="/deploy/"><span>Deploying as a Web Service</span></a></li><li class="level-2"><a class="final" href="/batchpredict/"><span>Batch Predictions</span></a></li><li class="level-2"><a class="final" href="/deploy /monitoring/"><span>Monitoring Engine</span></a></li><li class="level-2"><a class="final" href="/deploy/engineparams/"><span>Setting Engine Parameters</span></a></li><li class="level-2"><a class="final" href="/deploy/enginevariants/"><span>Deploying Multiple Engine Variants</span></a></li><li class="level-2"><a class="final" href="/deploy/plugin/"><span>Engine Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Customizing an Engine</span></a><ul><li class="level-2"><a class="final" href="/customize/"><span>Learning DASE</span></a></li><li class="level-2"><a class="final" href="/customize/dase/"><span>Implement DASE</span></a></li><li class="level-2"><a class="final" href="/customize/troubleshooting/"><span>Troubleshooting Engine Development</span></a></li><li class="level-2"><a class="final" href="/api/current/#package"><span>Engine Scala APIs</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Collecting and Analyzing Data</span></a><ul><li class="level-2"><a class="final" href="/datacollection/"><span>Event Server Overview</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventapi/"><span>Collecting Data with REST/SDKs</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventmodel/"><span>Events Modeling</span></a></li><li class="level-2"><a class="final" href="/datacollection/webhooks/"><span>Unifying Multichannel Data with Webhooks</span></a></li><li class="level-2"><a class="final" href="/datacollection/channel/"><span>Channel</span></a></li><li class="level-2"><a class="final" href="/datacollection/batchimport/"><span>Importing Data in Batch</span></a></li><li class="level-2"><a class="final" href="/datacollection/analytics/"><span>Using Analytics Tools</span></a></li><li class="level-2"><a class="final" href="/datacollection/plugin/"><span>Event Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"> <span>Choosing an Algorithm</span></a><ul><li class="level-2"><a class="final" href="/algorithm/"><span>Built-in Algorithm Libraries</span></a></li><li class="level-2"><a class="final" href="/algorithm/switch/"><span>Switching to Another Algorithm</span></a></li><li class="level-2"><a class="final" href="/algorithm/multiple/"><span>Combining Multiple Algorithms</span></a></li><li class="level-2"><a class="final" href="/algorithm/custom/"><span>Adding Your Own Algorithms</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Tuning and Evaluation</span></a><ul><li class="level-2"><a class="final" href="/evaluation/"><span>Overview</span></a></li><li class="level-2"><a class="final" href="/evaluation/paramtuning/"><span>Hyperparameter Tuning</span></a></li><li class="level-2"><a class="final" href="/evaluation/evaluationdashboard/"><span>Evaluation Dashboard</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricchoose/"><span>Choosing E valuation Metrics</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricbuild/"><span>Building Evaluation Metrics</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>System Architecture</span></a><ul><li class="level-2"><a class="final" href="/system/"><span>Architecture Overview</span></a></li><li class="level-2"><a class="final" href="/system/anotherdatastore/"><span>Using Another Data Store</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>PredictionIO® Official Templates</span></a><ul><li class="level-2"><a class="final" href="/templates/"><span>Intro</span></a></li><li class="level-2"><a class="expandible" href="#"><span>Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/recommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/dase/"><span>DASE</span></a></li><li class="level-3"><a class="fi nal" href="/templates/recommendation/evaluation/"><span>Evaluation Explained</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/reading-custom-events/"><span>Read Custom Events</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-data-prep/"><span>Customize Data Preparator</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-serving/"><span>Customize Serving</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/training-with-implicit-preference/"><span>Train with Implicit Preference</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/blacklist-items/"><span>Filter Recommended Items by Blacklist in Query</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/batch-evaluator/">< span>Batch Persistable Evaluator</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>E-Commerce Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/adjust-score/"><span>Adjust Score</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Similar Product</span></a><ul><li class="level-3"><a class="final" href="/templates/similarproduct/quickstart/"><span>Quick Start</span></a></li><li clas s="level-3"><a class="final" href="/templates/similarproduct/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/multi-events-multi-algos/"><span>Multiple Events and Multiple Algorithms</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/return-item-properties/"><span>Returns Item Properties</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/rid-user-set-event/"><span>Get Rid of Events for Users</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/recommended-user/"><span>Recommend Users</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Classification</span></a><ul><li c lass="level-3"><a class="final" href="/templates/classification/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/classification/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/classification/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/classification/add-algorithm/"><span>Use Alternative Algorithm</span></a></li><li class="level-3"><a class="final" href="/templates/classification/reading-custom-properties/"><span>Read Custom Properties</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Engine Template Gallery</span></a><ul><li class="level-2"><a class="final" href="/gallery/template-gallery/"><span>Browse</span></a></li><li class="level-2"><a class="final" href="/community/submit-template/"><span>Submit your Engine as a Template</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span >Demo Tutorials</span></a><ul><li class="level-2"><a class="final" >href="/demo/tapster/"><span>Comics Recommendation Demo</span></a></li><li >class="level-2"><a class="final" href="/demo/community/"><span>Community >Contributed Demo</span></a></li><li class="level-2"><a class="final" >href="/demo/textclassification/"><span>Text Classification Engine >Tutorial</span></a></li></ul></li><li class="level-1"><a class="expandible" >href="/community/"><span>Getting Involved</span></a><ul><li >class="level-2"><a class="final" >href="/community/contribute-code/"><span>Contribute Code</span></a></li><li >class="level-2"><a class="final" >href="/community/contribute-documentation/"><span>Contribute >Documentation</span></a></li><li class="level-2"><a class="final" >href="/community/contribute-sdk/"><span>Contribute a SDK</span></a></li><li >class="level-2"><a class="final" >href="/community/contribute-webhook/"><span>Contribute a >Webhook</span></a></li><li class="level-2"><a class="final" href="/community/ projects/"><span>Community Projects</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Help</span></a><ul><li class="level-2"><a class="final" href="/resources/faq/"><span>FAQs</span></a></li><li class="level-2"><a class="final" href="/support/"><span>Support</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Resources</span></a><ul><li class="level-2"><a class="final" href="/cli/"><span>Command-line Interface</span></a></li><li class="level-2"><a class="final" href="/resources/release/"><span>Release Cadence</span></a></li><li class="level-2"><a class="final" href="/resources/intellij/"><span>Developing Engines with IntelliJ IDEA</span></a></li><li class="level-2"><a class="final" href="/resources/upgrade/"><span>Upgrade Instructions</span></a></li><li class="level-2"><a class="final" href="/resources/glossary/"><span>Glossary</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Apach e Software Foundation</span></a><ul><li class="level-2"><a class="final" href="https://www.apache.org/";><span>Apache Homepage</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/licenses/";><span>License</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/sponsorship.html";><span>Sponsorship</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/thanks.html";><span>Thanks</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/security/";><span>Security</span></a></li></ul></li></ul></nav></div><div class="col-md-9 col-sm-12"><div class="content-header hidden-md hidden-lg"><div id="page-title"><h1>Launching PredictionIO on AWS</h1></div></div><div id="table-of-content-wrapper"><h5>On this page</h5><aside id="table-of-contents"><ul> <li> <a href="#prerequisites">Prerequisites</a> </li> <li> <a href="#access-aws-marketplace">Access AWS Marketplace</a> </li> <l i> <a href="#using-1-click-launch">Using 1-Click Launch</a> </li> <li> <a href="#setting-security-group">Setting Security Group</a> </li> <li> <a href="#start-using-predictionio">Start Using PredictionIO</a> </li> </ul> </aside><hr/><a id="edit-page-link" href="https://github.com/apache/predictionio/tree/livedoc/docs/manual/source/install/launch-aws.html.md.erb";><img src="/images/icons/edit-pencil-d6c1bb3d.png"/>Edit this page</a></div><div class="content-header hidden-sm hidden-xs"><div id="page-title"><h1>Launching PredictionIO on AWS</h1></div></div><div class="content"> <p>Deploying PredictionIO on Amazon Web Services is extremely easy thanks to AWS Marketplace. As long as you have access to AWS, you can launch a ready-to-use PredictionIO Amazon EC2 instance with a single click.</p><h2 id='prerequisites' class='header-anchors'>Prerequisites</h2> <ul> <li>Amazon Web Services account</li> <li>Amazon EC2</li> </ul> <h2 id='access-aws-marketplace' class='header-anchors'>Access AWS M arketplace</h2><p>Visit <a href="https://aws.amazon.com/marketplace/pp/B00RPIFSYS/";>PredictionIO product's page on AWS Marketplace</a> and sign in with your AWS account.</p><h2 id='using-1-click-launch' class='header-anchors'>Using 1-Click Launch</h2><p>You should see the following screen after you have logged in.</p><p><img alt="alt text" src="/images/awsm-product-1be37f9d.png"/></p><p>Under the big yellow "Continue" botton, select the region where you want to launch the PredictionIO EC2 instance, then click "Continue".</p><p><img alt="alt text" src="/images/awsm-1click-9d9ac1c4.png"/></p><p>Review your instance's settings before launching. For quick prototyping work, we recommend using the "memory optimized" instances for the cheapest memory configurations at least the "Memory Optimized R3 (r3.large)" or for larger datasets the "(r3.xlarge)".</p><h2 id='setting-security-group' class='header-anchors'>Setting Security Group</ h2><p>The default security group, marked by "AutogenByAWSMP", has the following ports opened to public:</p> <ul> <li>22 (SSH)</li> <li>7070 (PredictionIO Event Server)</li> <li>8000 (PredictionIO Server)</li> <li>8080 (Spark Master)</li> <li>9200 (Elasticsearch)</li> </ul> <h2 id='start-using-predictionio' class='header-anchors'>Start Using PredictionIO</h2><p>It may take a few minutes after the EC2 instance has launched for all PredictionIO components to become ready. When they are ready, you may connect to your instance, see <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-connect-to-instance-linux.html";>AWS documentation</a> for more details.</p><p>Once you connect to your instance, you can find PredictionIO at <code>/opt/PredictionIO</code> and the binary command path is <code>/opt/PredictionIO/bin</code>.</p> <p>You can proceed to <a href="/start/download">Choosing an Engine Template</a>, or continue the QuickStart guide of the Engine template if you ha ve already chosen one.</p> <div class="alert-message note"><p>The AWS instance will have all PredictionIO components automatically started for you, so you could safely skip the <strong>pio-start-all</strong> command as described in QuickStart.</p></div></div></div></div></div><footer><div class="container"><div class="seperator"></div><div class="row"><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Community</h4><ul><li><a href="//predictionio.apache.org/install/" target="blank">Download</a></li><li><a href="//predictionio.apache.org/" target="blank">Docs</a></li><li><a href="//github.com/apache/predictionio" target="blank">GitHub</a></li><li><a href="mailto:user-subscr...@predictionio.apache.org"; target="blank">Subscribe to User Mailing List</a></li><li><a href="//stackoverflow.com/questions/tagged/predictionio" target="blank">Stackoverflow</a></li></ul></div></div><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Cont ribute</h4><ul><li><a href="//predictionio.apache.org/community/contribute-code/" target="blank">Contribute</a></li><li><a href="//github.com/apache/predictionio" target="blank">Source Code</a></li><li><a href="//issues.apache.org/jira/browse/PIO" target="blank">Bug Tracker</a></li><li><a href="mailto:dev-subscr...@predictionio.apache.org"; target="blank">Subscribe to Development Mailing List</a></li></ul></div></div></div><div class="row"><div class="col-md-12 footer-link-column"><p>Apache PredictionIO, PredictionIO, Apache, the Apache feather logo, and the Apache PredictionIO project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p><p>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p></div></div></div><div id="footer-bottom"><div class="container"><div class="row"><div class="col-md-12"><div id="footer-logo-wrapper"><img alt="PredictionIO" src="/images /logos/logo-white-d1e9c6e6.png"/><span>®</span></div><div id="social-icons-wrapper"><a class="github-button" href="https://github.com/apache/predictionio"; data-icon="octicon-star" data-show-count="true" aria-label="Star apache/predictionio on GitHub">Star</a> <a class="github-button" href="https://github.com/apache/predictionio/fork"; data-icon="octicon-repo-forked" data-show-count="true" aria-label="Fork apache/predictionio on GitHub">Fork</a> <script id="github-bjs" async="" defer="" src="https://buttons.github.io/buttons.js";></script><a href="https://twitter.com/predictionio"; target="blank"><img alt="PredictionIO on Twitter" src="/images/icons/twitter-ea9dc152.png"/></a> <a href="https://www.facebook.com/predictionio"; target="blank"><img alt="PredictionIO on Facebook" src="/images/icons/facebook-5c57939c.png"/></a> </div></div></div></div></div></footer></div><script>(function(w,d,t,u,n,s,e){w['SwiftypeObject']=n;w[n]=w[n]||function(){ +<!DOCTYPE html><html><head><title>Launching PredictionIO on AWS</title><meta charset="utf-8"/><meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta class="swiftype" name="title" data-type="string" content="Launching PredictionIO on AWS"/><link rel="canonical" href="https://predictionio.apache.org/install/launch-aws/"/><link href="/images/favicon/normal-b330020a.png" rel="shortcut icon"/><link href="/images/favicon/apple-c0febcf2.png" rel="apple-touch-icon"/><link href="//fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800" rel="stylesheet"/><link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/><link href="/stylesheets/application-eccfc6cb.css" rel="stylesheet" type="text/css"/><script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.2/html5shiv.min.js"></script><script src="//cdn.mathjax.or g/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script><script src="//use.typekit.net/pqo0itb.js"></script><script>try{Typekit.load({ async: true });}catch(e){}</script></head><body><div id="global"><header><div class="container" id="header-wrapper"><div class="row"><div class="col-sm-12"><div id="logo-wrapper"><span id="drawer-toggle"></span><a href="#"></a><a href="http://predictionio.apache.org/";><img alt="Apache PredictionIO" id="logo" src="/images/logos/logo-ee2b9bb3.png"/></a><span>®</span></div><div id="menu-wrapper"><div id="pill-wrapper"><a class="pill left" href="/gallery/template-gallery">TEMPLATES</a> <a class="pill right" href="//github.com/apache/predictionio/">OPEN SOURCE</a></div></div><img class="mobile-search-bar-toggler hidden-md hidden-lg" src="/images/icons/search-glass-704bd4ff.png"/></div></div></div></header><div id="search-bar-row-wrapper"><div class="container-fluid" id="search-bar-row"><div class="row"><div class="col-md-9 col-sm-11 col-xs-11" ><div class="hidden-md hidden-lg" >id="mobile-page-heading-wrapper"><p>PredictionIO Docs</p><h4>Launching >PredictionIO on AWS</h4></div><h4 class="hidden-sm hidden-xs">PredictionIO >Docs</h4></div><div class="col-md-3 col-sm-1 col-xs-1 hidden-md >hidden-lg"><img id="left-menu-indicator" >src="/images/icons/down-arrow-dfe9f7fe.png"/></div><div class="col-md-3 >col-sm-12 col-xs-12 swiftype-wrapper"><div class="swiftype"><form >class="search-form"><img class="search-box-toggler hidden-xs hidden-sm" >src="/images/icons/search-glass-704bd4ff.png"/><div class="search-box"><img >src="/images/icons/search-glass-704bd4ff.png"/><input type="text" >id="st-search-input" class="st-search-input" placeholder="Search >Doc..."/></div><img class="swiftype-row-hider hidden-md hidden-lg" >src="/images/icons/drawer-toggle-active-fcbef12a.png"/></form></div></div><div > class="mobile-left-menu-toggler hidden-md >hidden-lg"></div></div></div></div><div id="page" >class="container-fluid"><div class="row"><div id="left-me nu-wrapper" class="col-md-3"><nav id="nav-main"><ul><li class="level-1"><a class="expandible" href="/"><span>Apache PredictionIO® Documentation</span></a><ul><li class="level-2"><a class="final" href="/"><span>Welcome to Apache PredictionIO®</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Started</span></a><ul><li class="level-2"><a class="final" href="/start/"><span>A Quick Intro</span></a></li><li class="level-2"><a class="final" href="/install/"><span>Installing Apache PredictionIO</span></a></li><li class="level-2"><a class="final" href="/start/download/"><span>Downloading an Engine Template</span></a></li><li class="level-2"><a class="final" href="/start/deploy/"><span>Deploying Your First Engine</span></a></li><li class="level-2"><a class="final" href="/start/customize/"><span>Customizing the Engine</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Integrating with Your App</span></a><ul><li class="lev el-2"><a class="final" href="/appintegration/"><span>App Integration Overview</span></a></li><li class="level-2"><a class="expandible" href="/sdk/"><span>List of SDKs</span></a><ul><li class="level-3"><a class="final" href="/sdk/java/"><span>Java & Android SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/php/"><span>PHP SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/python/"><span>Python SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/ruby/"><span>Ruby SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/community/"><span>Community Powered SDKs</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Deploying an Engine</span></a><ul><li class="level-2"><a class="final" href="/deploy/"><span>Deploying as a Web Service</span></a></li><li class="level-2"><a class="final" href="/batchpredict/"><span>Batch Predictions</span></a></li><li class="level-2"><a class="final" href="/deploy /monitoring/"><span>Monitoring Engine</span></a></li><li class="level-2"><a class="final" href="/deploy/engineparams/"><span>Setting Engine Parameters</span></a></li><li class="level-2"><a class="final" href="/deploy/enginevariants/"><span>Deploying Multiple Engine Variants</span></a></li><li class="level-2"><a class="final" href="/deploy/plugin/"><span>Engine Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Customizing an Engine</span></a><ul><li class="level-2"><a class="final" href="/customize/"><span>Learning DASE</span></a></li><li class="level-2"><a class="final" href="/customize/dase/"><span>Implement DASE</span></a></li><li class="level-2"><a class="final" href="/customize/troubleshooting/"><span>Troubleshooting Engine Development</span></a></li><li class="level-2"><a class="final" href="/api/current/#package"><span>Engine Scala APIs</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Collecting and Analyzing Data</span></a><ul><li class="level-2"><a class="final" href="/datacollection/"><span>Event Server Overview</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventapi/"><span>Collecting Data with REST/SDKs</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventmodel/"><span>Events Modeling</span></a></li><li class="level-2"><a class="final" href="/datacollection/webhooks/"><span>Unifying Multichannel Data with Webhooks</span></a></li><li class="level-2"><a class="final" href="/datacollection/channel/"><span>Channel</span></a></li><li class="level-2"><a class="final" href="/datacollection/batchimport/"><span>Importing Data in Batch</span></a></li><li class="level-2"><a class="final" href="/datacollection/analytics/"><span>Using Analytics Tools</span></a></li><li class="level-2"><a class="final" href="/datacollection/plugin/"><span>Event Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"> <span>Choosing an Algorithm</span></a><ul><li class="level-2"><a class="final" href="/algorithm/"><span>Built-in Algorithm Libraries</span></a></li><li class="level-2"><a class="final" href="/algorithm/switch/"><span>Switching to Another Algorithm</span></a></li><li class="level-2"><a class="final" href="/algorithm/multiple/"><span>Combining Multiple Algorithms</span></a></li><li class="level-2"><a class="final" href="/algorithm/custom/"><span>Adding Your Own Algorithms</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Tuning and Evaluation</span></a><ul><li class="level-2"><a class="final" href="/evaluation/"><span>Overview</span></a></li><li class="level-2"><a class="final" href="/evaluation/paramtuning/"><span>Hyperparameter Tuning</span></a></li><li class="level-2"><a class="final" href="/evaluation/evaluationdashboard/"><span>Evaluation Dashboard</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricchoose/"><span>Choosing E valuation Metrics</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricbuild/"><span>Building Evaluation Metrics</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>System Architecture</span></a><ul><li class="level-2"><a class="final" href="/system/"><span>Architecture Overview</span></a></li><li class="level-2"><a class="final" href="/system/anotherdatastore/"><span>Using Another Data Store</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>PredictionIO® Official Templates</span></a><ul><li class="level-2"><a class="final" href="/templates/"><span>Intro</span></a></li><li class="level-2"><a class="expandible" href="#"><span>Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/recommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/dase/"><span>DASE</span></a></li><li class="level-3"><a class="fi nal" href="/templates/recommendation/evaluation/"><span>Evaluation Explained</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/reading-custom-events/"><span>Read Custom Events</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-data-prep/"><span>Customize Data Preparator</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-serving/"><span>Customize Serving</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/training-with-implicit-preference/"><span>Train with Implicit Preference</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/blacklist-items/"><span>Filter Recommended Items by Blacklist in Query</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/batch-evaluator/">< span>Batch Persistable Evaluator</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>E-Commerce Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/adjust-score/"><span>Adjust Score</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Similar Product</span></a><ul><li class="level-3"><a class="final" href="/templates/similarproduct/quickstart/"><span>Quick Start</span></a></li><li clas s="level-3"><a class="final" href="/templates/similarproduct/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/multi-events-multi-algos/"><span>Multiple Events and Multiple Algorithms</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/return-item-properties/"><span>Returns Item Properties</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/rid-user-set-event/"><span>Get Rid of Events for Users</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/recommended-user/"><span>Recommend Users</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Classification</span></a><ul><li c lass="level-3"><a class="final" href="/templates/classification/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/classification/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/classification/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/classification/add-algorithm/"><span>Use Alternative Algorithm</span></a></li><li class="level-3"><a class="final" href="/templates/classification/reading-custom-properties/"><span>Read Custom Properties</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Engine Template Gallery</span></a><ul><li class="level-2"><a class="final" href="/gallery/template-gallery/"><span>Browse</span></a></li><li class="level-2"><a class="final" href="/community/submit-template/"><span>Submit your Engine as a Template</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span >Demo Tutorials</span></a><ul><li class="level-2"><a class="final" >href="/demo/tapster/"><span>Comics Recommendation Demo</span></a></li><li >class="level-2"><a class="final" href="/demo/community/"><span>Community >Contributed Demo</span></a></li><li class="level-2"><a class="final" >href="/demo/textclassification/"><span>Text Classification Engine >Tutorial</span></a></li></ul></li><li class="level-1"><a class="expandible" >href="/community/"><span>Getting Involved</span></a><ul><li >class="level-2"><a class="final" >href="/community/contribute-code/"><span>Contribute Code</span></a></li><li >class="level-2"><a class="final" >href="/community/contribute-documentation/"><span>Contribute >Documentation</span></a></li><li class="level-2"><a class="final" >href="/community/contribute-sdk/"><span>Contribute a SDK</span></a></li><li >class="level-2"><a class="final" >href="/community/contribute-webhook/"><span>Contribute a >Webhook</span></a></li><li class="level-2"><a class="final" href="/community/ projects/"><span>Community Projects</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Help</span></a><ul><li class="level-2"><a class="final" href="/resources/faq/"><span>FAQs</span></a></li><li class="level-2"><a class="final" href="/support/"><span>Support</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Resources</span></a><ul><li class="level-2"><a class="final" href="/cli/"><span>Command-line Interface</span></a></li><li class="level-2"><a class="final" href="/resources/release/"><span>Release Cadence</span></a></li><li class="level-2"><a class="final" href="/resources/intellij/"><span>Developing Engines with IntelliJ IDEA</span></a></li><li class="level-2"><a class="final" href="/resources/upgrade/"><span>Upgrade Instructions</span></a></li><li class="level-2"><a class="final" href="/resources/glossary/"><span>Glossary</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Apach e Software Foundation</span></a><ul><li class="level-2"><a class="final" href="https://www.apache.org/";><span>Apache Homepage</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/licenses/";><span>License</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/sponsorship.html";><span>Sponsorship</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/thanks.html";><span>Thanks</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/security/";><span>Security</span></a></li></ul></li></ul></nav></div><div class="col-md-9 col-sm-12"><div class="content-header hidden-md hidden-lg"><div id="page-title"><h1>Launching PredictionIO on AWS</h1></div></div><div id="table-of-content-wrapper"><h5>On this page</h5><aside id="table-of-contents"><ul> <li> <a href="#prerequisites">Prerequisites</a> </li> <li> <a href="#access-aws-marketplace">Access AWS Marketplace</a> </li> <l i> <a href="#using-1-click-launch">Using 1-Click Launch</a> </li> <li> <a href="#setting-security-group">Setting Security Group</a> </li> <li> <a href="#start-using-predictionio">Start Using PredictionIO</a> </li> </ul> </aside><hr/><a id="edit-page-link" href="https://github.com/apache/predictionio/tree/livedoc/docs/manual/source/install/launch-aws.html.md.erb";><img src="/images/icons/edit-pencil-d6c1bb3d.png"/>Edit this page</a></div><div class="content-header hidden-sm hidden-xs"><div id="page-title"><h1>Launching PredictionIO on AWS</h1></div></div><div class="content"> <p>Deploying PredictionIO on Amazon Web Services is extremely easy thanks to AWS Marketplace. As long as you have access to AWS, you can launch a ready-to-use PredictionIO Amazon EC2 instance with a single click.</p><h2 id='prerequisites' class='header-anchors'>Prerequisites</h2> <ul> <li>Amazon Web Services account</li> <li>Amazon EC2</li> </ul> <h2 id='access-aws-marketplace' class='header-anchors'>Access AWS M arketplace</h2><p>Visit <a href="https://aws.amazon.com/marketplace/pp/B00RPIFSYS/";>PredictionIO product's page on AWS Marketplace</a> and sign in with your AWS account.</p><h2 id='using-1-click-launch' class='header-anchors'>Using 1-Click Launch</h2><p>You should see the following screen after you have logged in.</p><p><img alt="alt text" src="/images/awsm-product-1be37f9d.png"/></p><p>Under the big yellow "Continue" button, select the region where you want to launch the PredictionIO EC2 instance, then click "Continue".</p><p><img alt="alt text" src="/images/awsm-1click-9d9ac1c4.png"/></p><p>Review your instance's settings before launching. For quick prototyping work, we recommend using the "memory optimized" instances for the cheapest memory configurations at least the "Memory Optimized R3 (r3.large)" or for larger datasets the "(r3.xlarge)".</p><h2 id='setting-security-group' class='header-anchors'>Setting Security Group</ h2><p>The default security group, marked by "AutogenByAWSMP", has the following ports opened to public:</p> <ul> <li>22 (SSH)</li> <li>7070 (PredictionIO Event Server)</li> <li>8000 (PredictionIO Server)</li> <li>8080 (Spark Master)</li> <li>9200 (Elasticsearch)</li> </ul> <h2 id='start-using-predictionio' class='header-anchors'>Start Using PredictionIO</h2><p>It may take a few minutes after the EC2 instance has launched for all PredictionIO components to become ready. When they are ready, you may connect to your instance, see <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-connect-to-instance-linux.html";>AWS documentation</a> for more details.</p><p>Once you connect to your instance, you can find PredictionIO at <code>/opt/PredictionIO</code> and the binary command path is <code>/opt/PredictionIO/bin</code>.</p> <p>You can proceed to <a href="/start/download">Choosing an Engine Template</a>, or continue the QuickStart guide of the Engine template if you ha ve already chosen one.</p> <div class="alert-message note"><p>The AWS instance will have all PredictionIO components automatically started for you, so you could safely skip the <strong>pio-start-all</strong> command as described in QuickStart.</p></div></div></div></div></div><footer><div class="container"><div class="seperator"></div><div class="row"><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Community</h4><ul><li><a href="//predictionio.apache.org/install/" target="blank">Download</a></li><li><a href="//predictionio.apache.org/" target="blank">Docs</a></li><li><a href="//github.com/apache/predictionio" target="blank">GitHub</a></li><li><a href="mailto:user-subscr...@predictionio.apache.org"; target="blank">Subscribe to User Mailing List</a></li><li><a href="//stackoverflow.com/questions/tagged/predictionio" target="blank">Stackoverflow</a></li></ul></div></div><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Cont ribute</h4><ul><li><a href="//predictionio.apache.org/community/contribute-code/" target="blank">Contribute</a></li><li><a href="//github.com/apache/predictionio" target="blank">Source Code</a></li><li><a href="//issues.apache.org/jira/browse/PIO" target="blank">Bug Tracker</a></li><li><a href="mailto:dev-subscr...@predictionio.apache.org"; target="blank">Subscribe to Development Mailing List</a></li></ul></div></div></div><div class="row"><div class="col-md-12 footer-link-column"><p>Apache PredictionIO, PredictionIO, Apache, the Apache feather logo, and the Apache PredictionIO project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p><p>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p></div></div></div><div id="footer-bottom"><div class="container"><div class="row"><div class="col-md-12"><div id="footer-logo-wrapper"><img alt="PredictionIO" src="/images /logos/logo-white-d1e9c6e6.png"/><span>®</span></div><div id="social-icons-wrapper"><a class="github-button" href="https://github.com/apache/predictionio"; data-icon="octicon-star" data-show-count="true" aria-label="Star apache/predictionio on GitHub">Star</a> <a class="github-button" href="https://github.com/apache/predictionio/fork"; data-icon="octicon-repo-forked" data-show-count="true" aria-label="Fork apache/predictionio on GitHub">Fork</a> <script id="github-bjs" async="" defer="" src="https://buttons.github.io/buttons.js";></script><a href="https://twitter.com/predictionio"; target="blank"><img alt="PredictionIO on Twitter" src="/images/icons/twitter-ea9dc152.png"/></a> <a href="https://www.facebook.com/predictionio"; target="blank"><img alt="PredictionIO on Facebook" src="/images/icons/facebook-5c57939c.png"/></a> </div></div></div></div></div></footer></div><script>(function(w,d,t,u,n,s,e){w['SwiftypeObject']=n;w[n]=w[n]||function(){ (w[n].q=w[n].q||[]).push(arguments);};s=d.createElement(t); e=d.getElementsByTagName(t)[0];s.async=1;s.src=u;e.parentNode.insertBefore(s,e); })(window,document,'script','//s.swiftypecdn.com/install/v1/st.js','_st'); http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/9fe018b6/machinelearning/dimensionalityreduction/index.html ---------------------------------------------------------------------- diff --git a/machinelearning/dimensionalityreduction/index.html b/machinelearning/dimensionalityreduction/index.html index e63b0ac..7ada946 100644 --- a/machinelearning/dimensionalityreduction/index.html +++ b/machinelearning/dimensionalityreduction/index.html @@ -126,7 +126,7 @@ Import digit recognition data. <span class="o">[</span>INFO] <span class="o">[</span>Remoting] Remoting started; listening on addresses :[akka.tcp://sparkDriver@10.0.0.30:65523] <span class="o">[</span>INFO] <span class="o">[</span>FileToEvents<span class="nv">$]</span> Events are imported. <span class="o">[</span>INFO] <span class="o">[</span>FileToEvents<span class="nv">$]</span> Done. -</pre></td></tr></tbody></table> </div> <p>The data is now in the event server.</p><h2 id='principal-component-analysis' class='header-anchors'>Principal Component Analysis</h2><p>PCA begins with the data matrix \(\bf X\) whose rows are feature vectors corresponding to a set of observations. In our case, each row represents the pixel information of the corresponding hand-written numerc digit image. The model then computes the <a href="https://en.wikipedia.org/wiki/Covariance_matrix";>covariance matrix</a> estimated from the data matrix \(\bf X\). The algorithm then takes the covariance matrix and computes the <a href="https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors";>eigenvectors</a> that correspond to its \(k\) (some integer) largest <a href="https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors";>eigenvalues</a>. The data matrix is then mapped to the space generated by these \(k\) vectors, which are called the \(k\) <strong>ptincipal components</strong> of \(\bf X\). What this is doing is mapping the data observations into a lower-dimensional space that explains the largest variability in the data (contains the most information). The algorithm for implementing PCA is listed as follows:</p><h3 id='pca-algorithm' class='header-anchors'>PCA Algorithm</h3><p><strong>Input:</strong> \(N \times p\) data matrix \(\bf X\); \(k \leq p\), the number of desired features.</p><p><strong>1.</strong> For each column in the data matrix: compute the average of all the entries contained in the column, and then subtract this average from each of the column entries.</p><p><strong>2.</strong> Compute the \(k\) eigenvectors corresponding to the \(k\) largest eigenvalues of the matrix obtained in the first step.</p><p><strong>Output:</strong> \(p \times k\) matrix \(P\) whose \(k\) rows are the eigenvectors computed in the second step.</p><p>Now, to transform a \(p \times 1\) feature vector \(\bf {x}\), you multiply by the matrix \(P^T\). Now, the vector \(P^T {\bf x} \) is a feature vector with only \(k\) components, which has accomplished the desired dimensionality reduction. Also, as a side note, the first step in the algorithm reduces the covariance matrix computation to that of only performing <a href="https://spark.apache.org/docs/1.3.1/mllib-dimensionality-reduction.html#singular-value-decomposition-svd";>SVD</a> on matrix obtained from step 1, which is numerically preferred, and necessary to extract the required eigenvectors.</p><h2 id='modifying-the-engine-template' class='header-anchors'>Modifying the Engine Template</h2><p>We will be modifying the engine template by first re-defining our <code>Query</code> class located in the <code>Engine.scala</code> script as follows:</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td></tr></tbody></table> </div> <p>The data is now in the event server.</p><h2 id='principal-component-analysis' class='header-anchors'>Principal Component Analysis</h2><p>PCA begins with the data matrix \(\bf X\) whose rows are feature vectors corresponding to a set of observations. In our case, each row represents the pixel information of the corresponding hand-written numeric digit image. The model then computes the <a href="https://en.wikipedia.org/wiki/Covariance_matrix";>covariance matrix</a> estimated from the data matrix \(\bf X\). The algorithm then takes the covariance matrix and computes the <a href="https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors";>eigenvectors</a> that correspond to its \(k\) (some integer) largest <a href="https://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors";>eigenvalues</a>. The data matrix is then mapped to the space generated by these \(k\) vectors, which are called the \(k\) <strong>principal components</strong> of \(\bf X\). What this is doing is mapping the data observations into a lower-dimensional space that explains the largest variability in the data (contains the most information). The algorithm for implementing PCA is listed as follows:</p><h3 id='pca-algorithm' class='header-anchors'>PCA Algorithm</h3><p><strong>Input:</strong> \(N \times p\) data matrix \(\bf X\); \(k \leq p\), the number of desired features.</p><p><strong>1.</strong> For each column in the data matrix: compute the average of all the entries contained in the column, and then subtract this average from each of the column entries.</p><p><strong>2.</strong> Compute the \(k\) eigenvectors corresponding to the \(k\) largest eigenvalues of the matrix obtained in the first step.</p><p><strong>Output:</strong> \(p \times k\) matrix \(P\) whose \(k\) rows are the eigenvectors computed in the second step.</p><p>Now, to transform a \(p \times 1\) feature vector \(\bf {x}\), you multiply by the matrix \(P^T\). Now, the vector \(P^T {\bf x }\) is a feature vector with only \(k\) components, which has accomplished the desired dimensionality reduction. Also, as a side note, the first step in the algorithm reduces the covariance matrix computation to that of only performing <a href="https://spark.apache.org/docs/1.3.1/mllib-dimensionality-reduction.html#singular-value-decomposition-svd";>SVD</a> on matrix obtained from step 1, which is numerically preferred, and necessary to extract the required eigenvectors.</p><h2 id='modifying-the-engine-template' class='header-anchors'>Modifying the Engine Template</h2><p>We will be modifying the engine template by first re-defining our <code>Query</code> class located in the <code>Engine.scala</code> script as follows:</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3</pre></td><td class="code"><pre><span class="k">class</span> <span class="nc">Query</span><span class="o">(</span> <span class="k">val</span> <span class="n">features</span> <span class="k">:</span> <span class="kt">String</span> http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/9fe018b6/resources/faq/index.html ---------------------------------------------------------------------- diff --git a/resources/faq/index.html b/resources/faq/index.html index 467e05f..0458adf 100644 --- a/resources/faq/index.html +++ b/resources/faq/index.html @@ -74,7 +74,7 @@ Unable to connect to all storage backend<span class="o">(</span>s<span class="o" 2015-02-03 18:40:04,812 ERROR zookeeper.ZooKeeperWatcher - hconnection-0x1e4075ce, <span class="nv">quorum</span><span class="o">=</span>localhost:2181, <span class="nv">baseZNode</span><span class="o">=</span>/hbase Received unexpected KeeperException, re-throwing exception org.apache.zookeeper.KeeperException<span class="nv">$ConnectionLossException</span>: KeeperErrorCode <span class="o">=</span> ConnectionLoss <span class="k">for</span> /hbase/hbaseid ... -2015-02-03 18:40:07,021 ERROR hbase.StorageClient - Failed to connect to HBase. Plase check <span class="k">if </span>HBase is running properly. +2015-02-03 18:40:07,021 ERROR hbase.StorageClient - Failed to connect to HBase. Please check <span class="k">if </span>HBase is running properly. 2015-02-03 18:40:07,026 ERROR storage.Storage<span class="nv">$ </span>- Error initializing storage client <span class="k">for </span><span class="nb">source </span>HBASE 2015-02-03 18:40:07,027 ERROR storage.Storage<span class="nv">$ </span>- Can<span class="s1">'t connect to ZooKeeper java.util.NoSuchElementException: None.get
