Documentation based on apache/predictionio#54415e1066ae2d646eef62ebfdf801ace1de2097
Project: http://git-wip-us.apache.org/repos/asf/predictionio-site/repo Commit: http://git-wip-us.apache.org/repos/asf/predictionio-site/commit/c17b9607 Tree: http://git-wip-us.apache.org/repos/asf/predictionio-site/tree/c17b9607 Diff: http://git-wip-us.apache.org/repos/asf/predictionio-site/diff/c17b9607 Branch: refs/heads/asf-site Commit: c17b96077035bc02afee626e1a403b6aa67f4086 Parents: 2e455bb Author: jenkins <[email protected]> Authored: Sat Aug 11 00:23:55 2018 +0000 Committer: jenkins <[email protected]> Committed: Sat Aug 11 00:23:55 2018 +0000 ---------------------------------------------------------------------- datacollection/batchimport/index.html | 4 +- datacollection/channel/index.html | 6 +- datacollection/eventapi/index.html | 20 +- deploy/index.html | 2 +- evaluation/metricbuild/index.html | 4 +- evaluation/metricchoose/index.html | 2 +- evaluation/paramtuning/index.html | 6 +- gallery/template-gallery/index.html | 2 +- gallery/templates.yaml | 4 +- resources/faq/index.html | 2 +- samples/tabs/index.html | 18 +- sitemap.xml | 260 +++++++++---------- templates/classification/quickstart/index.html | 30 +-- templates/complementarypurchase/dase/index.html | 4 +- .../complementarypurchase/quickstart/index.html | 20 +- .../ecommercerecommendation/dase/index.html | 2 +- .../quickstart/index.html | 60 ++--- .../quickstart/index.html | 60 ++--- templates/leadscoring/dase/index.html | 2 +- templates/leadscoring/quickstart/index.html | 30 +-- templates/productranking/quickstart/index.html | 40 +-- .../recommendation/batch-evaluator/index.html | 2 +- templates/recommendation/evaluation/index.html | 2 +- templates/recommendation/quickstart/index.html | 32 +-- templates/similarproduct/dase/index.html | 2 +- .../multi-events-multi-algos/index.html | 4 +- templates/similarproduct/quickstart/index.html | 40 +-- templates/vanilla/quickstart/index.html | 10 +- tryit/index.html | 2 +- 29 files changed, 336 insertions(+), 336 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/c17b9607/datacollection/batchimport/index.html ---------------------------------------------------------------------- diff --git a/datacollection/batchimport/index.html b/datacollection/batchimport/index.html index a979e5b..b4c83b7 100644 --- a/datacollection/batchimport/index.html +++ b/datacollection/batchimport/index.html @@ -7,7 +7,7 @@ </span><span class="p">{</span><span class="s2">"event"</span><span class="p">:</span><span class="s2">"rate"</span><span class="p">,</span><span class="s2">"entityType"</span><span class="p">:</span><span class="s2">"user"</span><span class="p">,</span><span class="s2">"entityId"</span><span class="p">:</span><span class="s2">"3"</span><span class="p">,</span><span class="s2">"targetEntityType"</span><span class="p">:</span><span class="s2">"item"</span><span class="p">,</span><span class="s2">"targetEntityId"</span><span class="p">:</span><span class="s2">"2"</span><span class="p">,</span><span class="s2">"properties"</span><span class="p">:{</span><span class="s2">"rating"</span><span class="p">:</span><span class="mf">1.0</span><span class="p">},</span><span class="s2">"eventTime"</span><span class="p">:</span><span class="s2">"2014-11-21T01:04:14.729Z"</span><span class="p">}</span><span class="w"> </span><span class="p">{</span><span class="s2">"event"</span><span class="p">:</span><span class="s2">"buy"</span><span class="p">,</span><span class="s2">"entityType"</span><span class="p">:</span><span class="s2">"user"</span><span class="p">,</span><span class="s2">"entityId"</span><span class="p">:</span><span class="s2">"3"</span><span class="p">,</span><span class="s2">"targetEntityType"</span><span class="p">:</span><span class="s2">"item"</span><span class="p">,</span><span class="s2">"targetEntityId"</span><span class="p">:</span><span class="s2">"7"</span><span class="p">,</span><span class="s2">"eventTime"</span><span class="p">:</span><span class="s2">"2014-11-21T01:04:14.735Z"</span><span class="p">}</span><span class="w"> </span><span class="p">{</span><span class="s2">"event"</span><span class="p">:</span><span class="s2">"buy"</span><span class="p">,</span><span class="s2">"entityType"</span><span class="p">:</span><span class="s2">"user"</span><span class="p">,</span><span class="s2">"entityId"</span><span class="p">:</span><span class="s2">"3"</span><span class="p">,</span><span class="s2">"targetEntityType"</span><span class="p">:</span><span class="s2">"item"</span><span class="p">,</span><span class="s2">"targetEntityId"</span><span class="p">:</span><span class="s2">"8"</span><span class="p">,</span><span class="s2">"eventTime"</span><span class="p">:</span><span class="s2">"2014-11-21T01:04:14.741Z"</span><span class="p">}</span><span class="w"> -</span></pre></td></tr></tbody></table> </div> <div class="alert-message warning"><p>Please make sure your import file does not contain any empty lines. Empty lines will be treated as a null object and will return an error during import.</p></div><h2 id='use-sdk-to-prepare-batch-input-file' class='header-anchors'>Use SDK to Prepare Batch Input File</h2><p>Some of the Apache PredictionIO SDKs also provides FileExporter client. You may use them to prepare the JSON file as described above. The FileExporter creates event in the same way as EventClient except that the events are written to a JSON file instead of being sent to EventSever. The written JSON file can then be used by batch import.</p><div class="tabs"> <ul class="control"> <li data-lang="php"><a href="#tab-7c604a57-12d0-4202-a23f-4298592bda68">PHP SDK</a></li> <li data-lang="python"><a href="#tab-82fa1414-2416-418f-8013-12f921c26b24">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-fcc14827-f55d-4a1f-b90b-31c344047e35"> Ruby SDK</a></li> <li data-lang="java"><a href="#tab-cbc5af80-c1b9-431a-a11d-a8cdbb7b2bb4">Java SDK</a></li> </ul> <div data-tab="PHP SDK" data-lang="php" id="tab-7c604a57-12d0-4202-a23f-4298592bda68"> (coming soon) </div> <div data-tab="Python SDK" data-lang="python" id="tab-82fa1414-2416-418f-8013-12f921c26b24"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</span></pre></td></tr></tbody></table> </div> <div class="alert-message warning"><p>Please make sure your import file does not contain any empty lines. Empty lines will be treated as a null object and will return an error during import.</p></div><h2 id='use-sdk-to-prepare-batch-input-file' class='header-anchors'>Use SDK to Prepare Batch Input File</h2><p>Some of the Apache PredictionIO SDKs also provides FileExporter client. You may use them to prepare the JSON file as described above. The FileExporter creates event in the same way as EventClient except that the events are written to a JSON file instead of being sent to EventSever. The written JSON file can then be used by batch import.</p><div class="tabs"> <ul class="control"> <li data-lang="php"><a href="#tab-df5ba779-acb2-4b89-a3ce-2334d8066514">PHP SDK</a></li> <li data-lang="python"><a href="#tab-b3a6cd3d-a8df-4bb8-94cc-f51f9c6ef8fc">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-580ef14a-aa9a-48f5-871c-c81aee6e8bf9"> Ruby SDK</a></li> <li data-lang="java"><a href="#tab-bc097fca-27c6-473f-8824-74d484efdbe7">Java SDK</a></li> </ul> <div data-tab="PHP SDK" data-lang="php" id="tab-df5ba779-acb2-4b89-a3ce-2334d8066514"> (coming soon) </div> <div data-tab="Python SDK" data-lang="python" id="tab-b3a6cd3d-a8df-4bb8-94cc-f51f9c6ef8fc"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -58,7 +58,7 @@ <span class="c"># close the FileExporter when finish writing all events</span> <span class="n">exporter</span><span class="o">.</span><span class="n">close</span><span class="p">()</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-fcc14827-f55d-4a1f-b90b-31c344047e35"> (coming soon) </div> <div data-tab="Java SDK" data-lang="java" id="tab-cbc5af80-c1b9-431a-a11d-a8cdbb7b2bb4"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-580ef14a-aa9a-48f5-871c-c81aee6e8bf9"> (coming soon) </div> <div data-tab="Java SDK" data-lang="java" id="tab-bc097fca-27c6-473f-8824-74d484efdbe7"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> </pre></td> </tr></tbody></table> </div> </div> </div> <h2 id='import-events-from-input-file' class='header-anchors'>Import Events from Input File</h2><p>Importing events from a file can be done easily using the command line interface. Assuming that <code>pio</code> be in your search path, your App ID be <code>123</code>, and the input file <code>my_events.json</code> be in your current working directory:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre><span class="gp">$ </span>pio import --appid 123 --input my_events.json </pre></td></tr></tbody></table> </div> <p>After a brief while, the tool should return to the console without any error. Congratulations! You have successfully imported your events.</p></div></div></div></div><footer><div class="container"><div class="seperator"></div><div class="row"><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Community</h4><ul><li><a href="//predictionio.apache.org/install/" target="blank">Download</a></li><li><a href="//predictionio.apache.org/" target="blank">Docs</a></li><li><a href="//github.com/apache/predictionio" target="blank">GitHub</a></li><li><a href="mailto:[email protected]"; target="blank">Subscribe to User Mailing List</a></li><li><a href="//stackoverflow.com/questions/tagged/predictionio" target="blank">Stackoverflow</a></li></ul></div></div><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Contribute</h4><ul><li><a href="//predictionio.apache.org/community/cont ribute-code/" target="blank">Contribute</a></li><li><a href="//github.com/apache/predictionio" target="blank">Source Code</a></li><li><a href="//issues.apache.org/jira/browse/PIO" target="blank">Bug Tracker</a></li><li><a href="mailto:[email protected]"; target="blank">Subscribe to Development Mailing List</a></li></ul></div></div></div><div class="row"><div class="col-md-12 footer-link-column"><p>Apache PredictionIO, PredictionIO, Apache, the Apache feather logo, and the Apache PredictionIO project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p><p>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p></div></div></div><div id="footer-bottom"><div class="container"><div class="row"><div class="col-md-12"><div id="footer-logo-wrapper"><img alt="PredictionIO" src="/images/logos/logo-white-d1e9c6e6.png"/><span>®</span></div><div id="socia l-icons-wrapper"><a class="github-button" href="https://github.com/apache/predictionio"; data-style="mega" data-count-href="/apache/predictionio/stargazers" data-count-api="/repos/apache/predictionio#stargazers_count" data-count-aria-label="# stargazers on GitHub" aria-label="Star apache/predictionio on GitHub">Star</a> <a class="github-button" href="https://github.com/apache/predictionio/fork"; data-icon="octicon-git-branch" data-style="mega" data-count-href="/apache/predictionio/network" data-count-api="/repos/apache/predictionio#forks_count" data-count-aria-label="# forks on GitHub" aria-label="Fork apache/predictionio on GitHub">Fork</a> <script id="github-bjs" async="" defer="" src="https://buttons.github.io/buttons.js";></script><a href="https://twitter.com/predictionio"; target="blank"><img alt="PredictionIO on Twitter" src="/images/icons/twitter-ea9dc152.png"/></a> <a href="https://www.facebook.com/predictionio"; target="blank"><img alt="PredictionIO on Facebook" src="/images/ico ns/facebook-5c57939c.png"/></a> </div></div></div></div></div></footer></div><script>(function(w,d,t,u,n,s,e){w['SwiftypeObject']=n;w[n]=w[n]||function(){ (w[n].q=w[n].q||[]).push(arguments);};s=d.createElement(t); http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/c17b9607/datacollection/channel/index.html ---------------------------------------------------------------------- diff --git a/datacollection/channel/index.html b/datacollection/channel/index.html index ffefe92..0237a54 100644 --- a/datacollection/channel/index.html +++ b/datacollection/channel/index.html @@ -12,7 +12,7 @@ [INFO] [App$] Channel Name: myChannel [INFO] [App$] Channel ID: 2 [INFO] [App$] App ID: 5 -</span></pre></td></tr></tbody></table> </div> <p>Now "myChannel" is created and ready for collecting data.</p><h2 id='collect-data-through-channel' class='header-anchors'>Collect data through Channel</h2><p>The Event API support optional <code>channel</code> query parameter. This allows you to import and query events of the specified channel. When the <code>channel</code> parameter is not specified, the data is collected through the default channel.</p><p>URL: <code>http://localhost:7070/events.json?accessKey=yourAccessKeyString&channel=yourChannelName</code></p><p>Query parameters:</p> <table><thead> <tr> <th style="text-align: left">Field</th> <th style="text-align: left">Type</th> <th style="text-align: left">Description</th> </tr> </thead><tbody> <tr> <td style="text-align: left"><code>accessKey</code></td> <td style="text-align: left">String</td> <td style="text-align: left">The Access Key for your App</td> </tr> <tr> <td style="text-align: left"><code>channel</c ode></td> <td style="text-align: left">String</td> <td style="text-align: left">The channel name (optional). Specify this to import data to this channel. <strong>NOTE: supported in PIO version >= 0.9.2</strong> only. Channel must be created first.</td> </tr> </tbody></table> <p>For SDK usage, one EventClient should be responsible for collecting data of one specific channel. The channel name is specified when the EventClient object is instantiated.</p><p>For example, the following code import event to "YOUR_CHANNEL" of the corresponding App.</p><div class="tabs"> <ul class="control"> <li data-lang="bash"><a href="#tab-411a5b2a-9d82-4116-8f9d-a7aae4e77134">Raw HTTP</a></li> <li data-lang="php"><a href="#tab-8069d3ae-78e1-482e-a1b0-9731853f6946">PHP SDK</a></li> <li data-lang="python"><a href="#tab-526f56d1-af1c-42cf-9abd-bc075b8c45b2">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-43fdbd30-3e6e-484c-a9e9-1dcebb6bdb0d">Ruby SDK</a></li> <li data-lang="java"><a href ="#tab-076d251f-8352-4354-a75d-39a40d83ee65">Java SDK</a></li> </ul> <div data-tab="Raw HTTP" data-lang="bash" id="tab-411a5b2a-9d82-4116-8f9d-a7aae4e77134"> <div class="highlight shell"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</span></pre></td></tr></tbody></table> </div> <p>Now "myChannel" is created and ready for collecting data.</p><h2 id='collect-data-through-channel' class='header-anchors'>Collect data through Channel</h2><p>The Event API support optional <code>channel</code> query parameter. This allows you to import and query events of the specified channel. When the <code>channel</code> parameter is not specified, the data is collected through the default channel.</p><p>URL: <code>http://localhost:7070/events.json?accessKey=yourAccessKeyString&channel=yourChannelName</code></p><p>Query parameters:</p> <table><thead> <tr> <th style="text-align: left">Field</th> <th style="text-align: left">Type</th> <th style="text-align: left">Description</th> </tr> </thead><tbody> <tr> <td style="text-align: left"><code>accessKey</code></td> <td style="text-align: left">String</td> <td style="text-align: left">The Access Key for your App</td> </tr> <tr> <td style="text-align: left"><code>channel</c ode></td> <td style="text-align: left">String</td> <td style="text-align: left">The channel name (optional). Specify this to import data to this channel. <strong>NOTE: supported in PIO version >= 0.9.2</strong> only. Channel must be created first.</td> </tr> </tbody></table> <p>For SDK usage, one EventClient should be responsible for collecting data of one specific channel. The channel name is specified when the EventClient object is instantiated.</p><p>For example, the following code import event to "YOUR_CHANNEL" of the corresponding App.</p><div class="tabs"> <ul class="control"> <li data-lang="bash"><a href="#tab-9686b748-ff2d-44b0-b95e-b5b5f4ccf3e4">Raw HTTP</a></li> <li data-lang="php"><a href="#tab-fab4217e-3082-40d9-a73f-c87271c80cd6">PHP SDK</a></li> <li data-lang="python"><a href="#tab-02b989ec-2bbe-480c-a9d1-b0e9f70986cb">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-385486bf-65da-4434-941e-1a11044e7812">Ruby SDK</a></li> <li data-lang="java"><a href ="#tab-73e02180-ece3-41a4-8ec9-a2daae110ab4">Java SDK</a></li> </ul> <div data-tab="Raw HTTP" data-lang="bash" id="tab-9686b748-ff2d-44b0-b95e-b5b5f4ccf3e4"> <div class="highlight shell"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -39,7 +39,7 @@ }, "eventTime" : "2004-12-13T21:39:45.618Z" }'</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="PHP SDK" data-lang="php" id="tab-8069d3ae-78e1-482e-a1b0-9731853f6946"> (TODO: update me) </div> <div data-tab="Python SDK" data-lang="python" id="tab-526f56d1-af1c-42cf-9abd-bc075b8c45b2"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="PHP SDK" data-lang="php" id="tab-fab4217e-3082-40d9-a73f-c87271c80cd6"> (TODO: update me) </div> <div data-tab="Python SDK" data-lang="python" id="tab-02b989ec-2bbe-480c-a9d1-b0e9f70986cb"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -80,7 +80,7 @@ <span class="n">properties</span><span class="o">=</span><span class="n">event_properties</span><span class="p">,</span> <span class="n">event_time</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2014</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">21</span><span class="p">,</span> <span class="mi">38</span><span class="p">,</span> <span class="mi">45</span><span class="p">,</span> <span class="mi">618000</span><span class="p">,</span> <span class="n">pytz</span><span class="o">.</span><span class="n">utc</span><span class="p">))</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-43fdbd30-3e6e-484c-a9e9-1dcebb6bdb0d"> (TODO: update me) </div> <div data-tab="Java SDK" data-lang="java" id="tab-076d251f-8352-4354-a75d-39a40d83ee65"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-385486bf-65da-4434-941e-1a11044e7812"> (TODO: update me) </div> <div data-tab="Java SDK" data-lang="java" id="tab-73e02180-ece3-41a4-8ec9-a2daae110ab4"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> </pre></td> </tr></tbody></table> </div> </div> </div> <p>You can also follow the EventAPI <a href="/datacollection/eventapi/#debugging-recipes">debug receipts</a> to query the events of specific channel by adding the <code>channel</code> query parameter in the URL.</p><h2 id='delete-a-channel-(including-all-imported-data)' class='header-anchors'>Delete a Channel (including all imported data)</h2><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>pio app channel-delete <app name> <channel name> </pre></td></tr></tbody></table> </div> <h2 id='delete-the-data-only-of-a-channel' class='header-anchors'>Delete the data-only of a Channel</h2><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>pio app data-delete <app name> --channel <channel name> </pre></td></tr></tbody></table> </div> <h2 id='accessing-channel-data-in-engine' class='header-anchors'>Accessing Channel Data in Engine</h2><p>To acccess channel data, simply specify the channel name when use the PEventStore or LEventStore API. Data is read from from the default channel if channelName is not specified.</p><p>For example, read data from default channel:</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/c17b9607/datacollection/eventapi/index.html ---------------------------------------------------------------------- diff --git a/datacollection/eventapi/index.html b/datacollection/eventapi/index.html index 11d06bb..a3d11db 100644 --- a/datacollection/eventapi/index.html +++ b/datacollection/eventapi/index.html @@ -22,7 +22,7 @@ Content-Length: 18 <span class="o">[</span>INFO] <span class="o">[</span>App<span class="nv">$]</span> Name: MyTestApp <span class="o">[</span>INFO] <span class="o">[</span>App<span class="nv">$]</span> ID: 6 <span class="o">[</span>INFO] <span class="o">[</span>App<span class="nv">$]</span> Access Key: WPgcXKd42FPQpZHVbVeMyqF4CQJUnXQmIMTHhX3ZUrSzvy1KXJjdFUrslifa9rnB -</pre></td></tr></tbody></table> </div> <h3 id='creating-your-first-event' class='header-anchors'>Creating Your First Event</h3><p>You may connect to the Event Server with HTTP request or by using one of many <strong>Apache PredictionIO SDKs</strong>.</p><p>For example, the following shows how one can create an event involving a single entity. Replace the value of <code>accessKey</code> by the <em>Access Key</em> generated for your App.</p><div class="tabs"> <ul class="control"> <li data-lang="bash"><a href="#tab-9254a132-3543-4182-b2ce-0096535740e2">Raw HTTP</a></li> <li data-lang="php"><a href="#tab-8175256a-bb2e-49a8-8b2d-088fcd4827b2">PHP SDK</a></li> <li data-lang="python"><a href="#tab-afb752a5-9798-4204-b82b-6f185e9edc74">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-d942a8bf-3f09-46a2-beb3-065169b33edb">Ruby SDK</a></li> <li data-lang="java"><a href="#tab-891e9834-c356-4b80-8dea-1cf78fac4fe1">Java SDK</a></li> </ul> <div data-tab="Raw HTTP" data-lang="bash" id="tab- 9254a132-3543-4182-b2ce-0096535740e2"> <div class="highlight shell"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td></tr></tbody></table> </div> <h3 id='creating-your-first-event' class='header-anchors'>Creating Your First Event</h3><p>You may connect to the Event Server with HTTP request or by using one of many <strong>Apache PredictionIO SDKs</strong>.</p><p>For example, the following shows how one can create an event involving a single entity. Replace the value of <code>accessKey</code> by the <em>Access Key</em> generated for your App.</p><div class="tabs"> <ul class="control"> <li data-lang="bash"><a href="#tab-194583ac-c4b7-4a90-9874-b9757cf1b09f">Raw HTTP</a></li> <li data-lang="php"><a href="#tab-889d5a76-588b-48bc-8824-1115e0ab575a">PHP SDK</a></li> <li data-lang="python"><a href="#tab-5b17f7fa-8344-420e-bfa9-9cc082871842">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-8ee34816-1104-4047-b8bb-149b0b37dfbf">Ruby SDK</a></li> <li data-lang="java"><a href="#tab-877e7d41-a0c2-40b9-9ae2-fca778e653ca">Java SDK</a></li> </ul> <div data-tab="Raw HTTP" data-lang="bash" id="tab- 194583ac-c4b7-4a90-9874-b9757cf1b09f"> <div class="highlight shell"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -53,7 +53,7 @@ Content-Length: 18 } "eventTime" : "2004-12-13T21:39:45.618-07:00" }'</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="PHP SDK" data-lang="php" id="tab-8175256a-bb2e-49a8-8b2d-088fcd4827b2"> <div class="highlight php"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="PHP SDK" data-lang="php" id="tab-889d5a76-588b-48bc-8824-1115e0ab575a"> <div class="highlight php"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -94,7 +94,7 @@ Content-Length: 18 <span class="s1">'eventTime'</span> <span class="o">=></span> <span class="s1">'2004-12-13T21:39:45.618-07:00'</span> <span class="p">));</span> <span class="cp">?></span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Python SDK" data-lang="python" id="tab-afb752a5-9798-4204-b82b-6f185e9edc74"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Python SDK" data-lang="python" id="tab-5b17f7fa-8344-420e-bfa9-9cc082871842"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -137,7 +137,7 @@ Content-Length: 18 <span class="n">properties</span><span class="o">=</span><span class="n">first_event_properties</span><span class="p">,</span> <span class="n">event_time</span><span class="o">=</span><span class="n">first_event_time</span><span class="p">,</span> <span class="p">)</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-d942a8bf-3f09-46a2-beb3-065169b33edb"> <div class="highlight ruby"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-8ee34816-1104-4047-b8bb-149b0b37dfbf"> <div class="highlight ruby"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -158,8 +158,8 @@ Content-Length: 18 <span class="s1">'prop4'</span> <span class="o">=></span> <span class="kp">true</span><span class="p">,</span> <span class="s1">'prop5'</span> <span class="o">=></span> <span class="sx">%w(a b c)</span><span class="p">,</span> <span class="s1">'prop6'</span> <span class="o">=></span> <span class="mi">4</span><span class="o">.</span><span class="mi">56</span> <span class="p">})</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Java SDK" data-lang="java" id="tab-891e9834-c356-4b80-8dea-1cf78fac4fe1"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> -</pre></td> </tr></tbody></table> </div> </div> </div> <p>For example, the following shows how one can create an event involving two entities (with <code>targetEntity</code>).</p><div class="tabs"> <ul class="control"> <li data-lang="bash"><a href="#tab-cab5f84e-f1ea-49d7-a2c2-8561c777c27f">Raw HTTP</a></li> <li data-lang="php"><a href="#tab-06a74d76-2bc5-4bc6-a7df-92b5454d48e1">PHP SDK</a></li> <li data-lang="python"><a href="#tab-62ea118f-0f4d-4b25-a2ff-e0e03743b506">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-7a78f99f-b9af-4bde-a93b-8c1763b8f515">Ruby SDK</a></li> <li data-lang="java"><a href="#tab-be138470-3758-45ed-b4fe-8de337fb4b51">Java SDK</a></li> </ul> <div data-tab="Raw HTTP" data-lang="bash" id="tab-cab5f84e-f1ea-49d7-a2c2-8561c777c27f"> <div class="highlight shell"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Java SDK" data-lang="java" id="tab-877e7d41-a0c2-40b9-9ae2-fca778e653ca"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> +</pre></td> </tr></tbody></table> </div> </div> </div> <p>For example, the following shows how one can create an event involving two entities (with <code>targetEntity</code>).</p><div class="tabs"> <ul class="control"> <li data-lang="bash"><a href="#tab-90f9e6a1-c4e5-4045-b1d4-054c0548ded5">Raw HTTP</a></li> <li data-lang="php"><a href="#tab-4f039be0-1b1a-4553-93db-b209d5aeb638">PHP SDK</a></li> <li data-lang="python"><a href="#tab-f86a98e9-1609-4305-90b6-2e02eca636d9">Python SDK</a></li> <li data-lang="ruby"><a href="#tab-e6ace4c5-8c1d-4fc1-a13d-05cc1189a850">Ruby SDK</a></li> <li data-lang="java"><a href="#tab-19761dc1-3e13-4f7d-9c36-2e3febda7dae">Java SDK</a></li> </ul> <div data-tab="Raw HTTP" data-lang="bash" id="tab-90f9e6a1-c4e5-4045-b1d4-054c0548ded5"> <div class="highlight shell"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -186,7 +186,7 @@ Content-Length: 18 }, "eventTime" : "2004-12-13T21:39:45.618Z" }'</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="PHP SDK" data-lang="php" id="tab-06a74d76-2bc5-4bc6-a7df-92b5454d48e1"> <div class="highlight php"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="PHP SDK" data-lang="php" id="tab-4f039be0-1b1a-4553-93db-b209d5aeb638"> <div class="highlight php"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -221,7 +221,7 @@ Content-Length: 18 <span class="s1">'eventTime'</span> <span class="o">=></span> <span class="s1">'2004-12-13T21:39:45.618Z'</span> <span class="p">));</span> <span class="cp">?></span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Python SDK" data-lang="python" id="tab-62ea118f-0f4d-4b25-a2ff-e0e03743b506"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Python SDK" data-lang="python" id="tab-f86a98e9-1609-4305-90b6-2e02eca636d9"> <div class="highlight python"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -246,7 +246,7 @@ Content-Length: 18 <span class="n">target_entity_id</span><span class="o">=</span><span class="s">"iid"</span><span class="p">,</span> <span class="n">properties</span><span class="o">=</span><span class="n">second_event_properties</span><span class="p">,</span> <span class="n">event_time</span><span class="o">=</span><span class="n">datetime</span><span class="p">(</span><span class="mi">2014</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">21</span><span class="p">,</span> <span class="mi">38</span><span class="p">,</span> <span class="mi">45</span><span class="p">,</span> <span class="mi">618000</span><span class="p">,</span> <span class="n">pytz</span><span class="o">.</span><span class="n">utc</span><span class="p">))</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-7a78f99f-b9af-4bde-a93b-8c1763b8f515"> <div class="highlight ruby"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Ruby SDK" data-lang="ruby" id="tab-e6ace4c5-8c1d-4fc1-a13d-05cc1189a850"> <div class="highlight ruby"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -263,7 +263,7 @@ Content-Length: 18 <span class="s1">'eventTime'</span> <span class="o">=></span> <span class="s1">'2004-12-13T21:39:45.618Z'</span><span class="p">,</span> <span class="s1">'properties'</span> <span class="o">=></span> <span class="p">{</span> <span class="s1">'someProperty'</span> <span class="o">=></span> <span class="s1">'value1'</span><span class="p">,</span> <span class="s1">'anotherProperty'</span> <span class="o">=></span> <span class="s1">'value2'</span> <span class="p">})</span> -</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Java SDK" data-lang="java" id="tab-be138470-3758-45ed-b4fe-8de337fb4b51"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> +</pre></td> </tr></tbody></table> </div> </div> <div data-tab="Java SDK" data-lang="java" id="tab-19761dc1-3e13-4f7d-9c36-2e3febda7dae"> <div class="highlight java"> <table style="border-spacing: 0"><tbody><tr> <td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td> <td class="code"><pre><span class="o">(</span><span class="n">coming</span> <span class="n">soon</span><span class="o">)</span> </pre></td> </tr></tbody></table> </div> </div> </div> <p>Sample response:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/c17b9607/deploy/index.html ---------------------------------------------------------------------- diff --git a/deploy/index.html b/deploy/index.html index 2514a17..fcc6ce1 100644 --- a/deploy/index.html +++ b/deploy/index.html @@ -4,7 +4,7 @@ </pre></td></tr></tbody></table> </div> <p>For example, if you want to re-train the model every day, you may add this to your <em>crontab</em>:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>0 0 <span class="k">*</span> <span class="k">*</span> <span class="k">*</span> <span class="nv">$PIO_HOME</span>/bin/pio train; <span class="nv">$PIO_HOME</span>/bin/pio deploy </pre></td></tr></tbody></table> </div> <p>where <em>$PIO_HOME</em> is the installation path of PredictionIO. See <a href="#retrain-and-deploy-script">Retrain and Deploy Script</a> below for a script ready for customization.</p><h2 id='specify-a-different-engine-port' class='header-anchors'>Specify a Different Engine Port</h2><p>By default, <code>pio deploy</code> deploys an engine on <strong>port 8000</strong>.</p><p>You can specify another port with an <em>--port</em> argument. For example, to deploy on port 8123</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>pio deploy --port 8123 </pre></td></tr></tbody></table> </div> <p>You can also specify the binding IP with <em>--ip</em>, which is set to <em>localhost</em> if not specified. For example:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>pio deploy --port 8123 --ip 1.2.3.4 -</pre></td></tr></tbody></table> </div> <h2 id='retrain-and-deploy-script' class='header-anchors'>Retrain and Deploy Script</h2><p>A retrain and deploy script is available <a href="https://github.com/apache/predictionio/tree/develop/examples/redeploy-script";>in the <em>examples/redeploy-script</em> directory</a>.</p><p>To use the script, copy <em>local.sh.template</em> as <em>local.sh</em>, <em>redeploy.sh</em> as (say) <em>MyEngine_Redeploy_(production).sh</em> (Name of the script will appear as title of email) and put both files under the <em>scripts/</em> directory of your engine. Then, modify the settings inside both file, filling in details like <code>PIO_HOME</code>, <code>LOG_DIR</code>, <code>TARGET_EMAIL</code>, <code>ENGINE_JSON</code> and others. You need to do <code>pio build</code> once before using this script. This script only trains and deploys. If <code>pio train</code> or <code>pio deploy</code> fails for some reason, the running engine stays put in most cases. If engine is retrained and deployed successfully, the email sent will have <em>Normal</em> in the title so you can set filtering rules.</p><p><code>mailutils</code> is used in this script. For Ubuntu, you can do <code>sudo update-alternatives --config mailx</code> and see if <code>/usr/bin/mail.mailutils</code> is selected. If you are using a server that blocks email, you will need to use services like SendGrid.</p><p>This script does not guarantee no down time since at some point during <code>pio deploy</code> the original engine is shut down. The down time is usually not more than a few seconds though it can be more.</p><p>The last thing to do is to add this to your <em>crontab</em>:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>0 0 <span class="k">*</span> <span class="k">*</span> <span class="k">*</span> /path/to/script >/dev/null 2>/dev/null < span class="c"># mute both stdout and stderr to supress email sent from cron</span> +</pre></td></tr></tbody></table> </div> <h2 id='retrain-and-deploy-script' class='header-anchors'>Retrain and Deploy Script</h2><p>A retrain and deploy script is available <a href="https://github.com/apache/predictionio/tree/develop/examples/redeploy-script";>in the <em>examples/redeploy-script</em> directory</a>.</p><p>To use the script, copy <em>local.sh.template</em> as <em>local.sh</em>, <em>redeploy.sh</em> as (say) <em>MyEngine_Redeploy_(production).sh</em> (Name of the script will appear as title of email) and put both files under the <em>scripts/</em> directory of your engine. Then, modify the settings inside both file, filling in details like <code>PIO_HOME</code>, <code>LOG_DIR</code>, <code>TARGET_EMAIL</code>, <code>ENGINE_JSON</code> and others. You need to do <code>pio build</code> once before using this script. This script only trains and deploys. If <code>pio train</code> or <code>pio deploy</code> fails for some reason, the running engine stays put in most cases. If engine is retrained and deployed successfully, the email sent will have <em>Normal</em> in the title so you can set filtering rules.</p><p><code>mailutils</code> is used in this script. For Ubuntu, you can do <code>sudo update-alternatives --config mailx</code> and see if <code>/usr/bin/mail.mailutils</code> is selected. If you are using a server that blocks email, you will need to use services like SendGrid.</p><p>This script does not guarantee no down time since at some point during <code>pio deploy</code> the original engine is shut down. The down time is usually not more than a few seconds though it can be more.</p><p>The last thing to do is to add this to your <em>crontab</em>:</p><div class="highlight shell"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1</pre></td><td class="code"><pre>0 0 <span class="k">*</span> <span class="k">*</span> <span class="k">*</span> /path/to/script >/dev/null 2>/dev/null < span class="c"># mute both stdout and stderr to suppress email sent from cron</span> </pre></td></tr></tbody></table> </div> </div></div></div></div><footer><div class="container"><div class="seperator"></div><div class="row"><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Community</h4><ul><li><a href="//predictionio.apache.org/install/" target="blank">Download</a></li><li><a href="//predictionio.apache.org/" target="blank">Docs</a></li><li><a href="//github.com/apache/predictionio" target="blank">GitHub</a></li><li><a href="mailto:[email protected]"; target="blank">Subscribe to User Mailing List</a></li><li><a href="//stackoverflow.com/questions/tagged/predictionio" target="blank">Stackoverflow</a></li></ul></div></div><div class="col-md-6 footer-link-column"><div class="footer-link-column-row"><h4>Contribute</h4><ul><li><a href="//predictionio.apache.org/community/contribute-code/" target="blank">Contribute</a></li><li><a href="//github.com/apache/predictionio" target="blank">Source Code</a></li><li><a href="// issues.apache.org/jira/browse/PIO" target="blank">Bug Tracker</a></li><li><a href="mailto:[email protected]"; target="blank">Subscribe to Development Mailing List</a></li></ul></div></div></div><div class="row"><div class="col-md-12 footer-link-column"><p>Apache PredictionIO, PredictionIO, Apache, the Apache feather logo, and the Apache PredictionIO project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p><p>All other marks mentioned may be trademarks or registered trademarks of their respective owners.</p></div></div></div><div id="footer-bottom"><div class="container"><div class="row"><div class="col-md-12"><div id="footer-logo-wrapper"><img alt="PredictionIO" src="/images/logos/logo-white-d1e9c6e6.png"/><span>®</span></div><div id="social-icons-wrapper"><a class="github-button" href="https://github.com/apache/predictionio"; data-style="mega" data-count-href="/apache/predictionio/s targazers" data-count-api="/repos/apache/predictionio#stargazers_count" data-count-aria-label="# stargazers on GitHub" aria-label="Star apache/predictionio on GitHub">Star</a> <a class="github-button" href="https://github.com/apache/predictionio/fork"; data-icon="octicon-git-branch" data-style="mega" data-count-href="/apache/predictionio/network" data-count-api="/repos/apache/predictionio#forks_count" data-count-aria-label="# forks on GitHub" aria-label="Fork apache/predictionio on GitHub">Fork</a> <script id="github-bjs" async="" defer="" src="https://buttons.github.io/buttons.js";></script><a href="https://twitter.com/predictionio"; target="blank"><img alt="PredictionIO on Twitter" src="/images/icons/twitter-ea9dc152.png"/></a> <a href="https://www.facebook.com/predictionio"; target="blank"><img alt="PredictionIO on Facebook" src="/images/icons/facebook-5c57939c.png"/></a> </div></div></div></div></div></footer></div><script>(function(w,d,t,u,n,s,e){w['SwiftypeObject']=n;w[n]=w[n]||fu nction(){ (w[n].q=w[n].q||[]).push(arguments);};s=d.createElement(t); e=d.getElementsByTagName(t)[0];s.async=1;s.src=u;e.parentNode.insertBefore(s,e); http://git-wip-us.apache.org/repos/asf/predictionio-site/blob/c17b9607/evaluation/metricbuild/index.html ---------------------------------------------------------------------- diff --git a/evaluation/metricbuild/index.html b/evaluation/metricbuild/index.html index 75a5bc9..f9fb74e 100644 --- a/evaluation/metricbuild/index.html +++ b/evaluation/metricbuild/index.html @@ -1,4 +1,4 @@ -<!DOCTYPE html><html><head><title>Building Evaluation Metrics</title><meta charset="utf-8"/><meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta class="swiftype" name="title" data-type="string" content="Building Evaluation Metrics"/><link rel="canonical" href="https://predictionio.apache.org/evaluation/metricbuild/"/><link href="/images/favicon/normal-b330020a.png" rel="shortcut icon"/><link href="/images/favicon/apple-c0febcf2.png" rel="apple-touch-icon"/><link href="//fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800" rel="stylesheet"/><link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/><link href="/stylesheets/application-eccfc6cb.css" rel="stylesheet" type="text/css"/><script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.2/html5shiv.min.js"></script><script src="//cdn.mathjax.or g/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script><script src="//use.typekit.net/pqo0itb.js"></script><script>try{Typekit.load({ async: true });}catch(e){}</script></head><body><div id="global"><header><div class="container" id="header-wrapper"><div class="row"><div class="col-sm-12"><div id="logo-wrapper"><span id="drawer-toggle"></span><a href="#"></a><a href="http://predictionio.apache.org/";><img alt="Apache PredictionIO" id="logo" src="/images/logos/logo-ee2b9bb3.png"/></a><span>®</span></div><div id="menu-wrapper"><div id="pill-wrapper"><a class="pill left" href="/gallery/template-gallery">TEMPLATES</a> <a class="pill right" href="//github.com/apache/predictionio/">OPEN SOURCE</a></div></div><img class="mobile-search-bar-toggler hidden-md hidden-lg" src="/images/icons/search-glass-704bd4ff.png"/></div></div></div></header><div id="search-bar-row-wrapper"><div class="container-fluid" id="search-bar-row"><div class="row"><div class="col-md-9 col-sm-11 col-xs-11" ><div class="hidden-md hidden-lg" >id="mobile-page-heading-wrapper"><p>PredictionIO Docs</p><h4>Building >Evaluation Metrics</h4></div><h4 class="hidden-sm hidden-xs">PredictionIO >Docs</h4></div><div class="col-md-3 col-sm-1 col-xs-1 hidden-md >hidden-lg"><img id="left-menu-indicator" >src="/images/icons/down-arrow-dfe9f7fe.png"/></div><div class="col-md-3 >col-sm-12 col-xs-12 swiftype-wrapper"><div class="swiftype"><form >class="search-form"><img class="search-box-toggler hidden-xs hidden-sm" >src="/images/icons/search-glass-704bd4ff.png"/><div class="search-box"><img >src="/images/icons/search-glass-704bd4ff.png"/><input type="text" >id="st-search-input" class="st-search-input" placeholder="Search >Doc..."/></div><img class="swiftype-row-hider hidden-md hidden-lg" >src="/images/icons/drawer-toggle-active-fcbef12a.png"/></form></div></div><div > class="mobile-left-menu-toggler hidden-md >hidden-lg"></div></div></div></div><div id="page" >class="container-fluid"><div class="row"><div id="left-menu -wrapper" class="col-md-3"><nav id="nav-main"><ul><li class="level-1"><a class="expandible" href="/"><span>Apache PredictionIO® Documentation</span></a><ul><li class="level-2"><a class="final" href="/"><span>Welcome to Apache PredictionIO®</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Started</span></a><ul><li class="level-2"><a class="final" href="/start/"><span>A Quick Intro</span></a></li><li class="level-2"><a class="final" href="/install/"><span>Installing Apache PredictionIO</span></a></li><li class="level-2"><a class="final" href="/start/download/"><span>Downloading an Engine Template</span></a></li><li class="level-2"><a class="final" href="/start/deploy/"><span>Deploying Your First Engine</span></a></li><li class="level-2"><a class="final" href="/start/customize/"><span>Customizing the Engine</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Integrating with Your App</span></a><ul><li class="level -2"><a class="final" href="/appintegration/"><span>App Integration Overview</span></a></li><li class="level-2"><a class="expandible" href="/sdk/"><span>List of SDKs</span></a><ul><li class="level-3"><a class="final" href="/sdk/java/"><span>Java & Android SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/php/"><span>PHP SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/python/"><span>Python SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/ruby/"><span>Ruby SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/community/"><span>Community Powered SDKs</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Deploying an Engine</span></a><ul><li class="level-2"><a class="final" href="/deploy/"><span>Deploying as a Web Service</span></a></li><li class="level-2"><a class="final" href="/batchpredict/"><span>Batch Predictions</span></a></li><li class="level-2"><a class="final" href="/deploy/m onitoring/"><span>Monitoring Engine</span></a></li><li class="level-2"><a class="final" href="/deploy/engineparams/"><span>Setting Engine Parameters</span></a></li><li class="level-2"><a class="final" href="/deploy/enginevariants/"><span>Deploying Multiple Engine Variants</span></a></li><li class="level-2"><a class="final" href="/deploy/plugin/"><span>Engine Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Customizing an Engine</span></a><ul><li class="level-2"><a class="final" href="/customize/"><span>Learning DASE</span></a></li><li class="level-2"><a class="final" href="/customize/dase/"><span>Implement DASE</span></a></li><li class="level-2"><a class="final" href="/customize/troubleshooting/"><span>Troubleshooting Engine Development</span></a></li><li class="level-2"><a class="final" href="/api/current/#package"><span>Engine Scala APIs</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Collecting and A nalyzing Data</span></a><ul><li class="level-2"><a class="final" href="/datacollection/"><span>Event Server Overview</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventapi/"><span>Collecting Data with REST/SDKs</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventmodel/"><span>Events Modeling</span></a></li><li class="level-2"><a class="final" href="/datacollection/webhooks/"><span>Unifying Multichannel Data with Webhooks</span></a></li><li class="level-2"><a class="final" href="/datacollection/channel/"><span>Channel</span></a></li><li class="level-2"><a class="final" href="/datacollection/batchimport/"><span>Importing Data in Batch</span></a></li><li class="level-2"><a class="final" href="/datacollection/analytics/"><span>Using Analytics Tools</span></a></li><li class="level-2"><a class="final" href="/datacollection/plugin/"><span>Event Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><s pan>Choosing an Algorithm(s)</span></a><ul><li class="level-2"><a class="final" href="/algorithm/"><span>Built-in Algorithm Libraries</span></a></li><li class="level-2"><a class="final" href="/algorithm/switch/"><span>Switching to Another Algorithm</span></a></li><li class="level-2"><a class="final" href="/algorithm/multiple/"><span>Combining Multiple Algorithms</span></a></li><li class="level-2"><a class="final" href="/algorithm/custom/"><span>Adding Your Own Algorithms</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>ML Tuning and Evaluation</span></a><ul><li class="level-2"><a class="final" href="/evaluation/"><span>Overview</span></a></li><li class="level-2"><a class="final" href="/evaluation/paramtuning/"><span>Hyperparameter Tuning</span></a></li><li class="level-2"><a class="final" href="/evaluation/evaluationdashboard/"><span>Evaluation Dashboard</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricchoose/"><span>Choosi ng Evaluation Metrics</span></a></li><li class="level-2"><a class="final active" href="/evaluation/metricbuild/"><span>Building Evaluation Metrics</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>System Architecture</span></a><ul><li class="level-2"><a class="final" href="/system/"><span>Architecture Overview</span></a></li><li class="level-2"><a class="final" href="/system/anotherdatastore/"><span>Using Another Data Store</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>PredictionIO® Official Templates</span></a><ul><li class="level-2"><a class="final" href="/templates/"><span>Intro</span></a></li><li class="level-2"><a class="expandible" href="#"><span>Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/recommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/dase/"><span>DASE</span></a></li><li class="level-3">< a class="final" href="/templates/recommendation/evaluation/"><span>Evaluation Explained</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/reading-custom-events/"><span>Read Custom Events</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-data-prep/"><span>Customize Data Preparator</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-serving/"><span>Customize Serving</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/training-with-implicit-preference/"><span>Train with Implicit Preference</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/blacklist-items/"><span>Filter Recommended Items by Blacklist in Query</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/batch-ev aluator/"><span>Batch Persistable Evaluator</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>E-Commerce Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/adjust-score/"><span>Adjust Score</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Similar Product</span></a><ul><li class="level-3"><a class="final" href="/templates/similarproduct/quickstart/"><span>Quick Start</span></a></ li><li class="level-3"><a class="final" href="/templates/similarproduct/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/multi-events-multi-algos/"><span>Multiple Events and Multiple Algorithms</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/return-item-properties/"><span>Returns Item Properties</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/rid-user-set-event/"><span>Get Rid of Events for Users</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/recommended-user/"><span>Recommend Users</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Classification</span></ a><ul><li class="level-3"><a class="final" href="/templates/classification/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/classification/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/classification/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/classification/add-algorithm/"><span>Use Alternative Algorithm</span></a></li><li class="level-3"><a class="final" href="/templates/classification/reading-custom-properties/"><span>Read Custom Properties</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Engine Template Gallery</span></a><ul><li class="level-2"><a class="final" href="/gallery/template-gallery/"><span>Browse</span></a></li><li class="level-2"><a class="final" href="/community/submit-template/"><span>Submit your Engine as a Template</span></a></li></ul></li><li class="level-1"><a class="expandible" hre f="#"><span>Demo Tutorials</span></a><ul><li class="level-2"><a class="final" href="/demo/tapster/"><span>Comics Recommendation Demo</span></a></li><li class="level-2"><a class="final" href="/demo/community/"><span>Community Contributed Demo</span></a></li><li class="level-2"><a class="final" href="/demo/textclassification/"><span>Text Classification Engine Tutorial</span></a></li></ul></li><li class="level-1"><a class="expandible" href="/community/"><span>Getting Involved</span></a><ul><li class="level-2"><a class="final" href="/community/contribute-code/"><span>Contribute Code</span></a></li><li class="level-2"><a class="final" href="/community/contribute-documentation/"><span>Contribute Documentation</span></a></li><li class="level-2"><a class="final" href="/community/contribute-sdk/"><span>Contribute a SDK</span></a></li><li class="level-2"><a class="final" href="/community/contribute-webhook/"><span>Contribute a Webhook</span></a></li><li class="level-2"><a class="final" href=" /community/projects/"><span>Community Projects</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Help</span></a><ul><li class="level-2"><a class="final" href="/resources/faq/"><span>FAQs</span></a></li><li class="level-2"><a class="final" href="/support/"><span>Support</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Resources</span></a><ul><li class="level-2"><a class="final" href="/cli/"><span>Command-line Interface</span></a></li><li class="level-2"><a class="final" href="/resources/release/"><span>Release Cadence</span></a></li><li class="level-2"><a class="final" href="/resources/intellij/"><span>Developing Engines with IntelliJ IDEA</span></a></li><li class="level-2"><a class="final" href="/resources/upgrade/"><span>Upgrade Instructions</span></a></li><li class="level-2"><a class="final" href="/resources/glossary/"><span>Glossary</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"> <span>Apache Software Foundation</span></a><ul><li class="level-2"><a class="final" href="https://www.apache.org/";><span>Apache Homepage</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/licenses/";><span>License</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/sponsorship.html";><span>Sponsorship</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/thanks.html";><span>Thanks</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/security/";><span>Security</span></a></li></ul></li></ul></nav></div><div class="col-md-9 col-sm-12"><div class="content-header hidden-md hidden-lg"><div id="breadcrumbs" class="hidden-sm hidden xs"><ul><li><a href="#">ML Tuning and Evaluation</a><span class="spacer">></span></li><li><span class="last">Building Evaluation Metrics</span></li></ul></div><div id="page-title"><h1>Building Evaluation Metrics</h1></div></div><d iv id="table-of-content-wrapper"><h5>On this page</h5><aside id="table-of-contents"><ul> <li> <a href="#overview">Overview</a> </li> <li> <a href="#example-1-accuracy-metric">Example 1: Accuracy Metric</a> </li> <li> <a href="#example-2-precision-metric">Example 2: Precision Metric</a> </li> </ul> </aside><hr/><a id="edit-page-link" href="https://github.com/apache/predictionio/tree/livedoc/docs/manual/source/evaluation/metricbuild.html.md";><img src="/images/icons/edit-pencil-d6c1bb3d.png"/>Edit this page</a></div><div class="content-header hidden-sm hidden-xs"><div id="breadcrumbs" class="hidden-sm hidden xs"><ul><li><a href="#">ML Tuning and Evaluation</a><span class="spacer">></span></li><li><span class="last">Building Evaluation Metrics</span></li></ul></div><div id="page-title"><h1>Building Evaluation Metrics</h1></div></div><div class="content"> <p>PredictionIO enables developer to implement evaluation custom evaluation metric with just a few lines of code. We illustrate it with <a href="/templates/classification/quickstart/">the classification template</a>.</p><h2 id='overview' class='header-anchors'>Overview</h2><p>A simplistic form of metric is a function which takes a <code>(Query, PredictedResult, ActualResult)</code>-tuple (<em>QPA-tuple</em>) as input and return a score. Exploiting this properties allows us to implement custom metric with a single line of code (plus some boilerplates). We demonstate this with two metrics: accuracy and precision.</p> <h2 id='example-1:-accuracy-metric' class='header-anchors'>Example 1: Accuracy Metric</h2><p>Accuracy is a metric capturing the portion of correct prediction among all test data points. A way to model this is for each correct QPA-tuple, we give a score of 1.0 and otherwise 0.0, then we take an average of all tuple scores.</p><p>PredictionIO has a [[AverageMetric]] helper class which provides this feature. This class takes 4 type parameters, [[EvalInfo]], [[Query]], [[PredictedResult]], and [[ActualRe sult]], these types can be found from the engine's signature. Line 5 below is the custom calculation.</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 +<!DOCTYPE html><html><head><title>Building Evaluation Metrics</title><meta charset="utf-8"/><meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><meta class="swiftype" name="title" data-type="string" content="Building Evaluation Metrics"/><link rel="canonical" href="https://predictionio.apache.org/evaluation/metricbuild/"/><link href="/images/favicon/normal-b330020a.png" rel="shortcut icon"/><link href="/images/favicon/apple-c0febcf2.png" rel="apple-touch-icon"/><link href="//fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800" rel="stylesheet"/><link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet"/><link href="/stylesheets/application-eccfc6cb.css" rel="stylesheet" type="text/css"/><script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.2/html5shiv.min.js"></script><script src="//cdn.mathjax.or g/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script><script src="//use.typekit.net/pqo0itb.js"></script><script>try{Typekit.load({ async: true });}catch(e){}</script></head><body><div id="global"><header><div class="container" id="header-wrapper"><div class="row"><div class="col-sm-12"><div id="logo-wrapper"><span id="drawer-toggle"></span><a href="#"></a><a href="http://predictionio.apache.org/";><img alt="Apache PredictionIO" id="logo" src="/images/logos/logo-ee2b9bb3.png"/></a><span>®</span></div><div id="menu-wrapper"><div id="pill-wrapper"><a class="pill left" href="/gallery/template-gallery">TEMPLATES</a> <a class="pill right" href="//github.com/apache/predictionio/">OPEN SOURCE</a></div></div><img class="mobile-search-bar-toggler hidden-md hidden-lg" src="/images/icons/search-glass-704bd4ff.png"/></div></div></div></header><div id="search-bar-row-wrapper"><div class="container-fluid" id="search-bar-row"><div class="row"><div class="col-md-9 col-sm-11 col-xs-11" ><div class="hidden-md hidden-lg" >id="mobile-page-heading-wrapper"><p>PredictionIO Docs</p><h4>Building >Evaluation Metrics</h4></div><h4 class="hidden-sm hidden-xs">PredictionIO >Docs</h4></div><div class="col-md-3 col-sm-1 col-xs-1 hidden-md >hidden-lg"><img id="left-menu-indicator" >src="/images/icons/down-arrow-dfe9f7fe.png"/></div><div class="col-md-3 >col-sm-12 col-xs-12 swiftype-wrapper"><div class="swiftype"><form >class="search-form"><img class="search-box-toggler hidden-xs hidden-sm" >src="/images/icons/search-glass-704bd4ff.png"/><div class="search-box"><img >src="/images/icons/search-glass-704bd4ff.png"/><input type="text" >id="st-search-input" class="st-search-input" placeholder="Search >Doc..."/></div><img class="swiftype-row-hider hidden-md hidden-lg" >src="/images/icons/drawer-toggle-active-fcbef12a.png"/></form></div></div><div > class="mobile-left-menu-toggler hidden-md >hidden-lg"></div></div></div></div><div id="page" >class="container-fluid"><div class="row"><div id="left-menu -wrapper" class="col-md-3"><nav id="nav-main"><ul><li class="level-1"><a class="expandible" href="/"><span>Apache PredictionIO® Documentation</span></a><ul><li class="level-2"><a class="final" href="/"><span>Welcome to Apache PredictionIO®</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Started</span></a><ul><li class="level-2"><a class="final" href="/start/"><span>A Quick Intro</span></a></li><li class="level-2"><a class="final" href="/install/"><span>Installing Apache PredictionIO</span></a></li><li class="level-2"><a class="final" href="/start/download/"><span>Downloading an Engine Template</span></a></li><li class="level-2"><a class="final" href="/start/deploy/"><span>Deploying Your First Engine</span></a></li><li class="level-2"><a class="final" href="/start/customize/"><span>Customizing the Engine</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Integrating with Your App</span></a><ul><li class="level -2"><a class="final" href="/appintegration/"><span>App Integration Overview</span></a></li><li class="level-2"><a class="expandible" href="/sdk/"><span>List of SDKs</span></a><ul><li class="level-3"><a class="final" href="/sdk/java/"><span>Java & Android SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/php/"><span>PHP SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/python/"><span>Python SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/ruby/"><span>Ruby SDK</span></a></li><li class="level-3"><a class="final" href="/sdk/community/"><span>Community Powered SDKs</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Deploying an Engine</span></a><ul><li class="level-2"><a class="final" href="/deploy/"><span>Deploying as a Web Service</span></a></li><li class="level-2"><a class="final" href="/batchpredict/"><span>Batch Predictions</span></a></li><li class="level-2"><a class="final" href="/deploy/m onitoring/"><span>Monitoring Engine</span></a></li><li class="level-2"><a class="final" href="/deploy/engineparams/"><span>Setting Engine Parameters</span></a></li><li class="level-2"><a class="final" href="/deploy/enginevariants/"><span>Deploying Multiple Engine Variants</span></a></li><li class="level-2"><a class="final" href="/deploy/plugin/"><span>Engine Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Customizing an Engine</span></a><ul><li class="level-2"><a class="final" href="/customize/"><span>Learning DASE</span></a></li><li class="level-2"><a class="final" href="/customize/dase/"><span>Implement DASE</span></a></li><li class="level-2"><a class="final" href="/customize/troubleshooting/"><span>Troubleshooting Engine Development</span></a></li><li class="level-2"><a class="final" href="/api/current/#package"><span>Engine Scala APIs</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Collecting and A nalyzing Data</span></a><ul><li class="level-2"><a class="final" href="/datacollection/"><span>Event Server Overview</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventapi/"><span>Collecting Data with REST/SDKs</span></a></li><li class="level-2"><a class="final" href="/datacollection/eventmodel/"><span>Events Modeling</span></a></li><li class="level-2"><a class="final" href="/datacollection/webhooks/"><span>Unifying Multichannel Data with Webhooks</span></a></li><li class="level-2"><a class="final" href="/datacollection/channel/"><span>Channel</span></a></li><li class="level-2"><a class="final" href="/datacollection/batchimport/"><span>Importing Data in Batch</span></a></li><li class="level-2"><a class="final" href="/datacollection/analytics/"><span>Using Analytics Tools</span></a></li><li class="level-2"><a class="final" href="/datacollection/plugin/"><span>Event Server Plugin</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><s pan>Choosing an Algorithm(s)</span></a><ul><li class="level-2"><a class="final" href="/algorithm/"><span>Built-in Algorithm Libraries</span></a></li><li class="level-2"><a class="final" href="/algorithm/switch/"><span>Switching to Another Algorithm</span></a></li><li class="level-2"><a class="final" href="/algorithm/multiple/"><span>Combining Multiple Algorithms</span></a></li><li class="level-2"><a class="final" href="/algorithm/custom/"><span>Adding Your Own Algorithms</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>ML Tuning and Evaluation</span></a><ul><li class="level-2"><a class="final" href="/evaluation/"><span>Overview</span></a></li><li class="level-2"><a class="final" href="/evaluation/paramtuning/"><span>Hyperparameter Tuning</span></a></li><li class="level-2"><a class="final" href="/evaluation/evaluationdashboard/"><span>Evaluation Dashboard</span></a></li><li class="level-2"><a class="final" href="/evaluation/metricchoose/"><span>Choosi ng Evaluation Metrics</span></a></li><li class="level-2"><a class="final active" href="/evaluation/metricbuild/"><span>Building Evaluation Metrics</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>System Architecture</span></a><ul><li class="level-2"><a class="final" href="/system/"><span>Architecture Overview</span></a></li><li class="level-2"><a class="final" href="/system/anotherdatastore/"><span>Using Another Data Store</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>PredictionIO® Official Templates</span></a><ul><li class="level-2"><a class="final" href="/templates/"><span>Intro</span></a></li><li class="level-2"><a class="expandible" href="#"><span>Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/recommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/dase/"><span>DASE</span></a></li><li class="level-3">< a class="final" href="/templates/recommendation/evaluation/"><span>Evaluation Explained</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/reading-custom-events/"><span>Read Custom Events</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-data-prep/"><span>Customize Data Preparator</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/customize-serving/"><span>Customize Serving</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/training-with-implicit-preference/"><span>Train with Implicit Preference</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/blacklist-items/"><span>Filter Recommended Items by Blacklist in Query</span></a></li><li class="level-3"><a class="final" href="/templates/recommendation/batch-ev aluator/"><span>Batch Persistable Evaluator</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>E-Commerce Recommendation</span></a><ul><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/ecommercerecommendation/adjust-score/"><span>Adjust Score</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Similar Product</span></a><ul><li class="level-3"><a class="final" href="/templates/similarproduct/quickstart/"><span>Quick Start</span></a></ li><li class="level-3"><a class="final" href="/templates/similarproduct/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/multi-events-multi-algos/"><span>Multiple Events and Multiple Algorithms</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/return-item-properties/"><span>Returns Item Properties</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/train-with-rate-event/"><span>Train with Rate Event</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/rid-user-set-event/"><span>Get Rid of Events for Users</span></a></li><li class="level-3"><a class="final" href="/templates/similarproduct/recommended-user/"><span>Recommend Users</span></a></li></ul></li><li class="level-2"><a class="expandible" href="#"><span>Classification</span></ a><ul><li class="level-3"><a class="final" href="/templates/classification/quickstart/"><span>Quick Start</span></a></li><li class="level-3"><a class="final" href="/templates/classification/dase/"><span>DASE</span></a></li><li class="level-3"><a class="final" href="/templates/classification/how-to/"><span>How-To</span></a></li><li class="level-3"><a class="final" href="/templates/classification/add-algorithm/"><span>Use Alternative Algorithm</span></a></li><li class="level-3"><a class="final" href="/templates/classification/reading-custom-properties/"><span>Read Custom Properties</span></a></li></ul></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Engine Template Gallery</span></a><ul><li class="level-2"><a class="final" href="/gallery/template-gallery/"><span>Browse</span></a></li><li class="level-2"><a class="final" href="/community/submit-template/"><span>Submit your Engine as a Template</span></a></li></ul></li><li class="level-1"><a class="expandible" hre f="#"><span>Demo Tutorials</span></a><ul><li class="level-2"><a class="final" href="/demo/tapster/"><span>Comics Recommendation Demo</span></a></li><li class="level-2"><a class="final" href="/demo/community/"><span>Community Contributed Demo</span></a></li><li class="level-2"><a class="final" href="/demo/textclassification/"><span>Text Classification Engine Tutorial</span></a></li></ul></li><li class="level-1"><a class="expandible" href="/community/"><span>Getting Involved</span></a><ul><li class="level-2"><a class="final" href="/community/contribute-code/"><span>Contribute Code</span></a></li><li class="level-2"><a class="final" href="/community/contribute-documentation/"><span>Contribute Documentation</span></a></li><li class="level-2"><a class="final" href="/community/contribute-sdk/"><span>Contribute a SDK</span></a></li><li class="level-2"><a class="final" href="/community/contribute-webhook/"><span>Contribute a Webhook</span></a></li><li class="level-2"><a class="final" href=" /community/projects/"><span>Community Projects</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Getting Help</span></a><ul><li class="level-2"><a class="final" href="/resources/faq/"><span>FAQs</span></a></li><li class="level-2"><a class="final" href="/support/"><span>Support</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"><span>Resources</span></a><ul><li class="level-2"><a class="final" href="/cli/"><span>Command-line Interface</span></a></li><li class="level-2"><a class="final" href="/resources/release/"><span>Release Cadence</span></a></li><li class="level-2"><a class="final" href="/resources/intellij/"><span>Developing Engines with IntelliJ IDEA</span></a></li><li class="level-2"><a class="final" href="/resources/upgrade/"><span>Upgrade Instructions</span></a></li><li class="level-2"><a class="final" href="/resources/glossary/"><span>Glossary</span></a></li></ul></li><li class="level-1"><a class="expandible" href="#"> <span>Apache Software Foundation</span></a><ul><li class="level-2"><a class="final" href="https://www.apache.org/";><span>Apache Homepage</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/licenses/";><span>License</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/sponsorship.html";><span>Sponsorship</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/foundation/thanks.html";><span>Thanks</span></a></li><li class="level-2"><a class="final" href="https://www.apache.org/security/";><span>Security</span></a></li></ul></li></ul></nav></div><div class="col-md-9 col-sm-12"><div class="content-header hidden-md hidden-lg"><div id="breadcrumbs" class="hidden-sm hidden xs"><ul><li><a href="#">ML Tuning and Evaluation</a><span class="spacer">></span></li><li><span class="last">Building Evaluation Metrics</span></li></ul></div><div id="page-title"><h1>Building Evaluation Metrics</h1></div></div><d iv id="table-of-content-wrapper"><h5>On this page</h5><aside id="table-of-contents"><ul> <li> <a href="#overview">Overview</a> </li> <li> <a href="#example-1-accuracy-metric">Example 1: Accuracy Metric</a> </li> <li> <a href="#example-2-precision-metric">Example 2: Precision Metric</a> </li> </ul> </aside><hr/><a id="edit-page-link" href="https://github.com/apache/predictionio/tree/livedoc/docs/manual/source/evaluation/metricbuild.html.md";><img src="/images/icons/edit-pencil-d6c1bb3d.png"/>Edit this page</a></div><div class="content-header hidden-sm hidden-xs"><div id="breadcrumbs" class="hidden-sm hidden xs"><ul><li><a href="#">ML Tuning and Evaluation</a><span class="spacer">></span></li><li><span class="last">Building Evaluation Metrics</span></li></ul></div><div id="page-title"><h1>Building Evaluation Metrics</h1></div></div><div class="content"> <p>PredictionIO enables developer to implement evaluation custom evaluation metric with just a few lines of code. We illustrate it with <a href="/templates/classification/quickstart/">the classification template</a>.</p><h2 id='overview' class='header-anchors'>Overview</h2><p>A simplistic form of metric is a function which takes a <code>(Query, PredictedResult, ActualResult)</code>-tuple (<em>QPA-tuple</em>) as input and return a score. Exploiting this properties allows us to implement custom metric with a single line of code (plus some boilerplates). We demonstrate this with two metrics: accuracy and precision.</p> <h2 id='example-1:-accuracy-metric' class='header-anchors'>Example 1: Accuracy Metric</h2><p>Accuracy is a metric capturing the portion of correct prediction among all test data points. A way to model this is for each correct QPA-tuple, we give a score of 1.0 and otherwise 0.0, then we take an average of all tuple scores.</p><p>PredictionIO has a [[AverageMetric]] helper class which provides this feature. This class takes 4 type parameters, [[EvalInfo]], [[Query]], [[PredictedResult]], and [[ActualR esult]], these types can be found from the engine's signature. Line 5 below is the custom calculation.</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 @@ -16,7 +16,7 @@ ... <span class="gp">$ </span>pio <span class="nb">eval </span>org.example.classification.AccuracyEvaluation org.example.classification.EngineParamsList ... -</pre></td></tr></tbody></table> </div> <p>(See MyClassification/src/main/scala/<strong><em>Evaluation.scala</em></strong> for full usage.)</p><h2 id='example-2:-precision-metric' class='header-anchors'>Example 2: Precision Metric</h2><p>Precision is a metric for binary classifier capturing the portion of correction prediction among all <em>positive</em> predictions. We don't care about the cases where the QPA-tuple gives a negative prediction. (Recall that a binary classifier only provide two output values: <em>positive</em> and <em>negative</em>.) The following table illustrates all four cases:</p> <table><thead> <tr> <th style="text-align: center">PredictedResult</th> <th style="text-align: center">ActualResult</th> <th style="text-align: center">Value</th> </tr> </thead><tbody> <tr> <td style="text-align: center">Positive</td> <td style="text-align: center">Positive</td> <td style="text-align: center">1.0</td> </tr> <tr> <td style="text-align: center">Positive</td> <td style ="text-align: center">Negative</td> <td style="text-align: center">0.0</td> </tr> <tr> <td style="text-align: center">Negative</td> <td style="text-align: center">Positive</td> <td style="text-align: center">Don't care</td> </tr> <tr> <td style="text-align: center">Negative</td> <td style="text-align: center">Negative</td> <td style="text-align: center">Don't care</td> </tr> </tbody></table> <p>Calculating the precision metric is a slightly more involved procedure than calculating the accuracy metric as we have to specially handle the <em>don't care</em> negative cases.</p><p>PredictionIO provides a helper class <code>OptionAverageMetric</code> allows user to specify <em>don't care</em> values as <code>None</code>. It only aggregates the non-None values. Lines 3 to 4 is the method signature of <code>calcuate</code> method. The key difference is that the return value is a <code>Option[Double]</code>, in contrast to <code>Double</code> for <code>AverageMetric</code>. T his class only computes the average of <code>Some(.)</code> results. Lines 5 to 13 are the actual logic. The first <code>if</code> factors out the positively predicted case, and the computation is simliar to the accuracy metric. The negatively predicted case are the <em>don't cares</em>, which we return <code>None</code>.</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 +</pre></td></tr></tbody></table> </div> <p>(See MyClassification/src/main/scala/<strong><em>Evaluation.scala</em></strong> for full usage.)</p><h2 id='example-2:-precision-metric' class='header-anchors'>Example 2: Precision Metric</h2><p>Precision is a metric for binary classifier capturing the portion of correction prediction among all <em>positive</em> predictions. We don't care about the cases where the QPA-tuple gives a negative prediction. (Recall that a binary classifier only provide two output values: <em>positive</em> and <em>negative</em>.) The following table illustrates all four cases:</p> <table><thead> <tr> <th style="text-align: center">PredictedResult</th> <th style="text-align: center">ActualResult</th> <th style="text-align: center">Value</th> </tr> </thead><tbody> <tr> <td style="text-align: center">Positive</td> <td style="text-align: center">Positive</td> <td style="text-align: center">1.0</td> </tr> <tr> <td style="text-align: center">Positive</td> <td style ="text-align: center">Negative</td> <td style="text-align: center">0.0</td> </tr> <tr> <td style="text-align: center">Negative</td> <td style="text-align: center">Positive</td> <td style="text-align: center">Don't care</td> </tr> <tr> <td style="text-align: center">Negative</td> <td style="text-align: center">Negative</td> <td style="text-align: center">Don't care</td> </tr> </tbody></table> <p>Calculating the precision metric is a slightly more involved procedure than calculating the accuracy metric as we have to specially handle the <em>don't care</em> negative cases.</p><p>PredictionIO provides a helper class <code>OptionAverageMetric</code> allows user to specify <em>don't care</em> values as <code>None</code>. It only aggregates the non-None values. Lines 3 to 4 is the method signature of <code>calcuate</code> method. The key difference is that the return value is a <code>Option[Double]</code>, in contrast to <code>Double</code> for <code>AverageMetric</code>. T his class only computes the average of <code>Some(.)</code> results. Lines 5 to 13 are the actual logic. The first <code>if</code> factors out the positively predicted case, and the computation is similiar to the accuracy metric. The negatively predicted case are the <em>don't cares</em>, which we return <code>None</code>.</p><div class="highlight scala"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4
