Modified: tez/site/tez_ui_user_data.html URL: http://svn.apache.org/viewvc/tez/site/tez_ui_user_data.html?rev=1921903&r1=1921902&r2=1921903&view=diff ============================================================================== --- tez/site/tez_ui_user_data.html (original) +++ tez/site/tez_ui_user_data.html Fri Nov 15 15:42:22 2024 @@ -1,25 +1,20 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2019-03-29 - | Rendered using Apache Maven Fluido Skin 1.3.0 + | Generated by Apache Maven Doxia Site Renderer 1.11.1 from src/site/markdown/tez_ui_user_data.md at 2024-11-15 + | Rendered using Apache Maven Fluido Skin 1.9 --> -<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> +<html xmlns="http://www.w3.org/1999/xhtml"; lang="en"> <head> <meta charset="UTF-8" /> - <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20190329" /> - <meta http-equiv="Content-Language" content="en" /> + <meta name="viewport" content="width=device-width, initial-scale=1" /> + <meta name="generator" content="Apache Maven Doxia Site Renderer 1.11.1" /> <title>Apache Tez – Embedding Application Specific Data into Tez UI</title> - <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.9.min.css" /> <link rel="stylesheet" href="./css/site.css" /> <link rel="stylesheet" href="./css/print.css" media="print" /> - - - <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> - - - -<style>.well.sidebar-nav { + <script src="./js/apache-maven-fluido-1.9.min.js"></script> +<style> + .well.sidebar-nav { background-color: #fff; } a.externalLink[href^=http] { @@ -34,451 +29,152 @@ } .nav-list .active a:hover { background-color: #a0a0a0; - }</style> - - </head> - <body class="topBarEnabled"> - - - - - - <div id="topbar" class="navbar navbar-fixed-top "> + } + </style> + </head> + <body class="topBarEnabled"> + <header id="topbar" class="navbar navbar-fixed-top "> <div class="navbar-inner"> - <div class="container-fluid"> + <div class="container-fluid"> <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </a> - - <ul class="nav"> - <li class="dropdown"> + <ul class="nav"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Getting Started <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="index.html" title="Overview">Overview</a> -</li> - - <li> <a href="talks.html" title="Talks and Meetup Recordings">Talks and Meetup Recordings</a> -</li> - - <li> <a href="http://www.apache.org/licenses/LICENSE-2.0.txt"; title="Project License">Project License</a> -</li> - </ul> + <li><a href="index.html" title="Overview">Overview</a></li> + <li><a href="talks.html" title="Talks and Meetup Recordings">Talks and Meetup Recordings</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0.txt"; title="Project License">Project License</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="install.html" title="Install Guide">Install Guide</a> -</li> - - <li> <a href="localmode.html" title="Local Mode">Local Mode</a> -</li> - - <li> <a href="tez-ui.html" title="Tez UI">Tez UI</a> -</li> - - <li> <a href="shuffle-handler.html" title="Tez Shuffle Handler">Tez Shuffle Handler</a> -</li> - - <li> <a href="user_guides.html" title="User Guides">User Guides</a> -</li> - </ul> + <li><a href="install.html" title="Install Guide">Install Guide</a></li> + <li><a href="localmode.html" title="Local Mode">Local Mode</a></li> + <li><a href="tez-ui.html" title="Tez UI">Tez UI</a></li> + <li><a href="shuffle-handler.html" title="Tez Shuffle Handler">Tez Shuffle Handler</a></li> + <li><a href="user_guides.html" title="User Guides">User Guides</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> -</li> - - <li> <a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a> -</li> - - <li> <a href="team-list.html" title="Project Team">Project Team</a> -</li> - - <li> <a href="by-laws.html" title="Project By-Laws">Project By-Laws</a> -</li> - - <li> <a href="http://www.meetup.com/Apache-Tez-User-Group"; title="User Meetup Group">User Meetup Group</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/TEZ"; title="Tez Wiki">Tez Wiki</a> -</li> - </ul> + <li><a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a></li> + <li><a href="team-list.html" title="Project Team">Project Team</a></li> + <li><a href="by-laws.html" title="Project By-Laws">Project By-Laws</a></li> + <li><a href="https://www.meetup.com/Apache-Tez-User-Group"; title="User Meetup Group">User Meetup Group</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ"; title="Tez Wiki">Tez Wiki</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Download Apache TEZ® Releases <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://archive.apache.org/dist/incubator/tez/tez-0.4.1-incubating/"; title="0.4.1-incubating">0.4.1-incubating</a> -</li> - - <li> <a href="releases/apache-tez-0-5-4.html" title="0.5.4">0.5.4</a> -</li> - - <li> <a href="releases/apache-tez-0-6-2.html" title="0.6.2">0.6.2</a> -</li> - - <li> <a href="releases/apache-tez-0-7-1.html" title="0.7.1">0.7.1</a> -</li> - - <li> <a href="releases/apache-tez-0-8-5.html" title="0.8.5">0.8.5</a> -</li> - - <li> <a href="releases/apache-tez-0-9-0.html" title="0.9.0">0.9.0</a> -</li> - - <li> <a href="releases/apache-tez-0-9-1.html" title="0.9.1">0.9.1</a> -</li> - - <li> <a href="releases/apache-tez-0-9-2.html" title="0.9.2">0.9.2</a> -</li> - - <li> <a href="releases/index.html" title="All Releases">All Releases</a> -</li> - </ul> + <li><a href="releases/apache-tez-0-8-5.html" title="0.8.5">0.8.5</a></li> + <li><a href="releases/apache-tez-0-9-2.html" title="0.9.2">0.9.2</a></li> + <li><a href="releases/apache-tez-0-10-3.html" title="0.10.3">0.10.3</a></li> + <li><a href="releases/apache-tez-0-10-4.html" title="0.10.4">0.10.4</a></li> + <li><a href="releases/index.html" title="All Releases">All Releases</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contribute <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="https://issues.apache.org/jira/browse/TEZ"; title="JIRA">JIRA</a> -</li> - - <li> <a href="https://gitbox.apache.org/repos/asf/tez.git"; title="Git (Read/Write)">Git (Read/Write)</a> -</li> - - <li> <a href="git://git.apache.org/tez.git" title="Git (Read-only)">Git (Read-only)</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; title="How to Contribute">How to Contribute</a> -</li> - </ul> + <li><a href="https://issues.apache.org/jira/browse/TEZ"; title="JIRA">JIRA</a></li> + <li><a href="https://gitbox.apache.org/repos/asf/tez.git"; title="Git (Read/Write)">Git (Read/Write)</a></li> + <li><a href="git://git.apache.org/tez.git" title="Git (Read-only)">Git (Read-only)</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; title="How to Contribute">How to Contribute</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/licenses/"; title="License">License</a> -</li> - - <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Donate">Donate</a> -</li> - - <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> -</li> - - <li> <a href="http://www.apache.org/security/"; title="Security">Security</a> -</li> - </ul> + <li><a href="https://www.apache.org/licenses/"; title="License">License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship.html"; title="Donate">Donate</a></li> + <li><a href="https://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a></li> + <li><a href="https://www.apache.org/security/"; title="Security">Security</a></li> + </ul> </li> - </ul> - - - - - </div> - + </ul> </div> </div> - </div> - - <div class="container-fluid"> - <div id="banner"> - <div class="pull-left"> - <a href=".././" id="bannerLeft"> - <img src="../images/ApacheTezLogo_lowres.png" alt="Apache Tez" width="25%" height="25%"/> - </a> - </div> - <div class="pull-right" style="width: 25%; height: 25%"> <a href="http://www.apache.org/"; id="bannerRight"> - <img src="https://apache.org/foundation/press/kit/asf_logo_wide.png"; alt="Apache Software Foundation"/> - </a> - </div> - <div class="clear"><hr/></div> - </div> - - <div id="breadcrumbs"> - <ul class="breadcrumb"> - - - <li class=""> - <a href="./" title="Apache TEZ®"> - Apache TEZ®</a> - </li> - <li class="divider ">/</li> - <li class="">Embedding Application Specific Data into Tez UI</li> - - - - <li id="publishDate" class="pull-right">Last Published: 2019-03-29</li> - - </ul> - </div> + </header> + <div class="container-fluid"> + <header> + <div id="banner"> + <div class="pull-left"><a href="./" id="bannerLeft"><img src="images/ApacheTezLogo_lowres.png" alt="Apache Tez" width="25%" height="25%"/></a></div> + <div class="pull-right"><a href="https://www.apache.org/"; id="bannerRight"><img src="https://apache.org/foundation/press/kit/asf_logo_wide.png"; alt="Apache Software Foundation"/></a></div> + <div class="clear"><hr/></div> + </div> - + <div id="breadcrumbs"> + <ul class="breadcrumb"> + <li class=""><a href="./" title="Apache TEZ®">Apache TEZ®</a><span class="divider">/</span></li> + <li class="active ">Embedding Application Specific Data into Tez UI</li> + <li id="publishDate" class="pull-right">Last Published: 2024-11-15</li> + </ul> + </div> + </header> <div class="row-fluid"> - <div id="leftColumn" class="span2"> + <header id="leftColumn" class="span2"> + <nav class="well sidebar-nav"> + <ul class="nav nav-list"> + <li class="nav-header">Getting Started</li> + <li><a href="index.html" title="Overview"><span class="none"></span>Overview</a></li> + <li><a href="talks.html" title="Talks and Meetup Recordings"><span class="none"></span>Talks and Meetup Recordings</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0.txt"; class="externalLink" title="Project License"><span class="none"></span>Project License</a></li> + <li class="nav-header">Documentation</li> + <li><a href="install.html" title="Install Guide"><span class="none"></span>Install Guide</a></li> + <li><a href="localmode.html" title="Local Mode"><span class="none"></span>Local Mode</a></li> + <li><a href="tez-ui.html" title="Tez UI"><span class="none"></span>Tez UI</a></li> + <li><a href="shuffle-handler.html" title="Tez Shuffle Handler"><span class="none"></span>Tez Shuffle Handler</a></li> + <li><a href="user_guides.html" title="User Guides"><span class="none"></span>User Guides</a></li> + <li class="nav-header">Community</li> + <li><a href="mail-lists.html" title="Mailing Lists"><span class="none"></span>Mailing Lists</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking"><span class="none"></span>Issue Tracking</a></li> + <li><a href="team-list.html" title="Project Team"><span class="none"></span>Project Team</a></li> + <li><a href="by-laws.html" title="Project By-Laws"><span class="none"></span>Project By-Laws</a></li> + <li><a href="https://www.meetup.com/Apache-Tez-User-Group"; class="externalLink" title="User Meetup Group"><span class="none"></span>User Meetup Group</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ"; class="externalLink" title="Tez Wiki"><span class="none"></span>Tez Wiki</a></li> + <li class="nav-header">Download Apache TEZ® Releases</li> + <li><a href="releases/apache-tez-0-8-5.html" title="0.8.5"><span class="none"></span>0.8.5</a></li> + <li><a href="releases/apache-tez-0-9-2.html" title="0.9.2"><span class="none"></span>0.9.2</a></li> + <li><a href="releases/apache-tez-0-10-3.html" title="0.10.3"><span class="none"></span>0.10.3</a></li> + <li><a href="releases/apache-tez-0-10-4.html" title="0.10.4"><span class="none"></span>0.10.4</a></li> + <li><a href="releases/index.html" title="All Releases"><span class="none"></span>All Releases</a></li> + <li class="nav-header">Contribute</li> + <li><a href="https://issues.apache.org/jira/browse/TEZ"; class="externalLink" title="JIRA"><span class="none"></span>JIRA</a></li> + <li><a href="https://gitbox.apache.org/repos/asf/tez.git"; class="externalLink" title="Git (Read/Write)"><span class="none"></span>Git (Read/Write)</a></li> + <li><a href="git://git.apache.org/tez.git" class="externalLink" title="Git (Read-only)"><span class="none"></span>Git (Read-only)</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; class="externalLink" title="How to Contribute"><span class="none"></span>How to Contribute</a></li> + <li class="nav-header">ASF</li> + <li><a href="https://www.apache.org/licenses/"; class="externalLink" title="License"><span class="none"></span>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Donate"><span class="none"></span>Donate</a></li> + <li><a href="https://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"><span class="none"></span>Thanks</a></li> + <li><a href="https://www.apache.org/security/"; class="externalLink" title="Security"><span class="none"></span>Security</a></li> + </ul> + </nav> <div class="well sidebar-nav"> - - - <ul class="nav nav-list"> - <li class="nav-header">Getting Started</li> - - <li> - - <a href="index.html" title="Overview"> - <i class="none"></i> - Overview</a> - </li> - - <li> - - <a href="talks.html" title="Talks and Meetup Recordings"> - <i class="none"></i> - Talks and Meetup Recordings</a> - </li> - - <li> - - <a href="http://www.apache.org/licenses/LICENSE-2.0.txt"; class="externalLink" title="Project License"> - <i class="none"></i> - Project License</a> - </li> - <li class="nav-header">Documentation</li> - - <li> - - <a href="install.html" title="Install Guide"> - <i class="none"></i> - Install Guide</a> - </li> - - <li> - - <a href="localmode.html" title="Local Mode"> - <i class="none"></i> - Local Mode</a> - </li> - - <li> - - <a href="tez-ui.html" title="Tez UI"> - <i class="none"></i> - Tez UI</a> - </li> - - <li> - - <a href="shuffle-handler.html" title="Tez Shuffle Handler"> - <i class="none"></i> - Tez Shuffle Handler</a> - </li> - - <li> - - <a href="user_guides.html" title="User Guides"> - <i class="none"></i> - User Guides</a> - </li> - <li class="nav-header">Community</li> - - <li> - - <a href="mail-lists.html" title="Mailing Lists"> - <i class="none"></i> - Mailing Lists</a> - </li> - - <li> - - <a href="issue-tracking.html" title="Issue Tracking"> - <i class="none"></i> - Issue Tracking</a> - </li> - - <li> - - <a href="team-list.html" title="Project Team"> - <i class="none"></i> - Project Team</a> - </li> - - <li> - - <a href="by-laws.html" title="Project By-Laws"> - <i class="none"></i> - Project By-Laws</a> - </li> - - <li> - - <a href="http://www.meetup.com/Apache-Tez-User-Group"; class="externalLink" title="User Meetup Group"> - <i class="none"></i> - User Meetup Group</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/TEZ"; class="externalLink" title="Tez Wiki"> - <i class="none"></i> - Tez Wiki</a> - </li> - <li class="nav-header">Download Apache TEZ® Releases</li> - - <li> - - <a href="http://archive.apache.org/dist/incubator/tez/tez-0.4.1-incubating/"; class="externalLink" title="0.4.1-incubating"> - <i class="none"></i> - 0.4.1-incubating</a> - </li> - - <li> - - <a href="releases/apache-tez-0-5-4.html" title="0.5.4"> - <i class="none"></i> - 0.5.4</a> - </li> - - <li> - - <a href="releases/apache-tez-0-6-2.html" title="0.6.2"> - <i class="none"></i> - 0.6.2</a> - </li> - - <li> - - <a href="releases/apache-tez-0-7-1.html" title="0.7.1"> - <i class="none"></i> - 0.7.1</a> - </li> - - <li> - - <a href="releases/apache-tez-0-8-5.html" title="0.8.5"> - <i class="none"></i> - 0.8.5</a> - </li> - - <li> - - <a href="releases/apache-tez-0-9-0.html" title="0.9.0"> - <i class="none"></i> - 0.9.0</a> - </li> - - <li> - - <a href="releases/apache-tez-0-9-1.html" title="0.9.1"> - <i class="none"></i> - 0.9.1</a> - </li> - - <li> - - <a href="releases/apache-tez-0-9-2.html" title="0.9.2"> - <i class="none"></i> - 0.9.2</a> - </li> - - <li> - - <a href="releases/index.html" title="All Releases"> - <i class="none"></i> - All Releases</a> - </li> - <li class="nav-header">Contribute</li> - - <li> - - <a href="https://issues.apache.org/jira/browse/TEZ"; class="externalLink" title="JIRA"> - <i class="none"></i> - JIRA</a> - </li> - - <li> - - <a href="https://gitbox.apache.org/repos/asf/tez.git"; class="externalLink" title="Git (Read/Write)"> - <i class="none"></i> - Git (Read/Write)</a> - </li> - - <li> - - <a href="git://git.apache.org/tez.git" class="externalLink" title="Git (Read-only)"> - <i class="none"></i> - Git (Read-only)</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; class="externalLink" title="How to Contribute"> - <i class="none"></i> - How to Contribute</a> - </li> - <li class="nav-header">ASF</li> - - <li> - - <a href="http://www.apache.org/licenses/"; class="externalLink" title="License"> - <i class="none"></i> - License</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Donate"> - <i class="none"></i> - Donate</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"> - <i class="none"></i> - Thanks</a> - </li> - - <li> - - <a href="http://www.apache.org/security/"; class="externalLink" title="Security"> - <i class="none"></i> - Security</a> - </li> - </ul> - - - - <hr class="divider" /> - - <div id="poweredBy"> - <div class="clear"></div> - <div class="clear"></div> - - - - <div id="twitter"> - + <hr /> + <div id="poweredBy"> + <div class="clear"></div> + <div id="twitter" style="border:none; margin-top: 10px"> <a href="https://twitter.com/ApacheTez"; class="twitter-follow-button" data-show-count="false" data-align="left" data-size="medium" data-show-screen-name="true" data-lang="en">Follow ApacheTez</a> - <script type="text/javascript">!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> - - </div> - <div class="clear"></div> - <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> - <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> - </a> - </div> + <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> + </div> + <div class="clear"></div> + <div class="clear"></div> +<a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a> + </div> </div> - </div> - - - <div id="bodyColumn" class="span10" > - - <!-- Licensed to the Apache Software Foundation (ASF) under one or more + </header> + <main id="bodyColumn" class="span10" > +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 @@ -491,66 +187,55 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and - limitations under the License. --><p></p> + limitations under the License. +--> + <h1>Embedding Application Specific Data into Tez UI</h1> -<p>The Tez UI is built mainly off data stored in <a href="./tez_yarn_timeline.html">YARN Timeline</a>. The Tez API, currently, provides some minimal support for an application to inject application-specific data into the same storage layer. Using a general standard guideline by following a well-defined format, this data can also be displayed in the Tez UI. </p> -<div class="section"> +<p>The Tez UI is built mainly off data stored in <a href="./tez_yarn_timeline.html">YARN Timeline</a>. The Tez API, currently, provides some minimal support for an application to inject application-specific data into the same storage layer. Using a general standard guideline by following a well-defined format, this data can also be displayed in the Tez UI.</p><section> <h2><a name="Setting_DAG-level_Information"></a>Setting DAG-level Information</h2> -<p>To set DAG level information, the API to use is DAG::setDAGInfo. ( Please refer to the Javadocs for more detailed and up-to-date information )</p> +<p>To set DAG level information, the API to use is DAG::setDAGInfo. ( Please refer to the Javadocs for more detailed and up-to-date information )</p> <p>The DAG::setDAGInfo() API expects a to be String passed to it. This string is recommended to be a json-encoded value with the following keys recognized keys by the UI:</p> - <ul> - + <li>“context”: The application context in which this DAG is being used. For example, this could be set to “Hive” or “Pig” if this is being run as part of a Hive or Pig script.</li> - <li>“description”: General description on what this DAG is going to do. In the case of Hive, this could be the SQL query text.</li> -</ul></div> -<div class="section"> -<h2><a name="Setting_Information_for_each_InputOutputProcessor"></a>Setting Information for each Input/Output/Processor</h2> -<p>Each Input/Output/Processor specified in the DAG Plan is specified via a TezEntityDescriptor. Applications specify a user payload that is used to initialize/configure the instance as needed. From a Tez UI point of view, users are usually keen to understand what “work” the particular Input/Output/Processor is doing in addition to any additional configuration information on how the object was initialized/configured. Keeping that in mind, each TezEntityDescriptor supports an api for application developers to specify this information when creating the DAG plan. The API to use for this is setHistoryText(). </p> +</ul></section><section> +<h2><a name="Setting_Information_for_each_Input.2FOutput.2FProcessor"></a>Setting Information for each Input/Output/Processor</h2> +<p>Each Input/Output/Processor specified in the DAG Plan is specified via a TezEntityDescriptor. Applications specify a user payload that is used to initialize/configure the instance as needed. From a Tez UI point of view, users are usually keen to understand what “work” the particular Input/Output/Processor is doing in addition to any additional configuration information on how the object was initialized/configured. Keeping that in mind, each TezEntityDescriptor supports an api for application developers to specify this information when creating the DAG plan. The API to use for this is setHistoryText().</p> <p>The setHistoryText() API expects a String to be passed to it. This string is recommended to be a json-encoded value with the following keys recognized keys by the UI:</p> - <ul> - + <li>“desc” : A simple string describing for the object in question. For example, for a particular Hive Processor, this could be a description of what that particular processor is doing.</li> - <li>“config” : A map of key-value pairs representing the configuration/payload used to initialize the object in question.</li> </ul> <p>By default, the Inputs/Outputs/Processors that are part of the tez-runtime-library do not publish their configuration information via the setHistoryText() API. To enable this, the following property needs to be enabled:</p> - <blockquote> -<p><property><br />    <name>tez.runtime.convert.user-payload.to.history-text</name><br />    <value>true</value><br /> </property><br /></p> + +<p><property><br /> +   <name>tez.runtime.convert.user-payload.to.history-text</name><br /> +   <value>true</value><br /> +</property><br /></p> </blockquote> -<p>Please note that this is a data-heavy operation and adds a lot of load on the TimelineServer (due to the additional amount of configuration data generated) and therefore should be disabled for most jobs unless there is a real debugging need.</p></div> -<div class="section"> +<p>Please note that this is a data-heavy operation and adds a lot of load on the TimelineServer (due to the additional amount of configuration data generated) and therefore should be disabled for most jobs unless there is a real debugging need.</p></section><section> <h2><a name="Use_of_DAG_Info_and_History_Text_in_the_Tez_UI"></a>Use of DAG Info and History Text in the Tez UI</h2> -<p>If the data setup in the DAG Info and History Text conforms to the format expected by the UI, it will be displayed in the Tez UI in an easy to consume manner. In cases where this is not possible, the UI may fall back to either not displaying the data at all or displaying the string as is in a safe manner. </p></div> - </div> - </div> - </div> - +<p>If the data setup in the DAG Info and History Text conforms to the format expected by the UI, it will be displayed in the Tez UI in an easy to consume manner. In cases where this is not possible, the UI may fall back to either not displaying the data at all or displaying the string as is in a safe manner.</p></section> + </main> + </div> + </div> <hr/> - <footer> - <div class="container-fluid"> - <div class="row span12">Copyright © 2019 - <a href="http://www.apache.org/";>Apache Software Foundation</a>. - All Rights Reserved. - - </div> - - <?xml version="1.0" encoding="UTF-8"?> -<div class="row span12">Apache Tez, Apache, the Apache feather logo, and the Apache Tez project logos are trademarks of The Apache Software Foundation. - All other marks mentioned may be trademarks or registered trademarks of their respective owners.</div> - <?xml version="1.0" encoding="UTF-8"?> -<br/> - <?xml version="1.0" encoding="UTF-8"?> + <div class="container-fluid"> + <div class="row-fluid"> <div class="row span12"> - <a href="privacy-policy.html">Privacy Policy</a> -</div> - - - </div> + Apache Tez, Apache, the Apache feather logo, and the Apache Tez project logos are trademarks of The Apache Software Foundation. + All other marks mentioned may be trademarks or registered trademarks of their respective owners. + </div> + <br/> + <div class="row span12"> + <a href="privacy-policy.html">Privacy Policy</a> + </div> + </div> + </div> </footer> </body> </html>
Modified: tez/site/tez_yarn_timeline.html URL: http://svn.apache.org/viewvc/tez/site/tez_yarn_timeline.html?rev=1921903&r1=1921902&r2=1921903&view=diff ============================================================================== --- tez/site/tez_yarn_timeline.html (original) +++ tez/site/tez_yarn_timeline.html Fri Nov 15 15:42:22 2024 @@ -1,25 +1,20 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia at 2019-03-29 - | Rendered using Apache Maven Fluido Skin 1.3.0 + | Generated by Apache Maven Doxia Site Renderer 1.11.1 from src/site/markdown/tez_yarn_timeline.md at 2024-11-15 + | Rendered using Apache Maven Fluido Skin 1.9 --> -<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> +<html xmlns="http://www.w3.org/1999/xhtml"; lang="en"> <head> <meta charset="UTF-8" /> - <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20190329" /> - <meta http-equiv="Content-Language" content="en" /> + <meta name="viewport" content="width=device-width, initial-scale=1" /> + <meta name="generator" content="Apache Maven Doxia Site Renderer 1.11.1" /> <title>Apache Tez – Using YARN Timeline with Tez for History</title> - <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.9.min.css" /> <link rel="stylesheet" href="./css/site.css" /> <link rel="stylesheet" href="./css/print.css" media="print" /> - - - <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> - - - -<style>.well.sidebar-nav { + <script src="./js/apache-maven-fluido-1.9.min.js"></script> +<style> + .well.sidebar-nav { background-color: #fff; } a.externalLink[href^=http] { @@ -34,451 +29,152 @@ } .nav-list .active a:hover { background-color: #a0a0a0; - }</style> - - </head> - <body class="topBarEnabled"> - - - - - - <div id="topbar" class="navbar navbar-fixed-top "> + } + </style> + </head> + <body class="topBarEnabled"> + <header id="topbar" class="navbar navbar-fixed-top "> <div class="navbar-inner"> - <div class="container-fluid"> + <div class="container-fluid"> <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar"> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </a> - - <ul class="nav"> - <li class="dropdown"> + <ul class="nav"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Getting Started <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="index.html" title="Overview">Overview</a> -</li> - - <li> <a href="talks.html" title="Talks and Meetup Recordings">Talks and Meetup Recordings</a> -</li> - - <li> <a href="http://www.apache.org/licenses/LICENSE-2.0.txt"; title="Project License">Project License</a> -</li> - </ul> + <li><a href="index.html" title="Overview">Overview</a></li> + <li><a href="talks.html" title="Talks and Meetup Recordings">Talks and Meetup Recordings</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0.txt"; title="Project License">Project License</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="install.html" title="Install Guide">Install Guide</a> -</li> - - <li> <a href="localmode.html" title="Local Mode">Local Mode</a> -</li> - - <li> <a href="tez-ui.html" title="Tez UI">Tez UI</a> -</li> - - <li> <a href="shuffle-handler.html" title="Tez Shuffle Handler">Tez Shuffle Handler</a> -</li> - - <li> <a href="user_guides.html" title="User Guides">User Guides</a> -</li> - </ul> + <li><a href="install.html" title="Install Guide">Install Guide</a></li> + <li><a href="localmode.html" title="Local Mode">Local Mode</a></li> + <li><a href="tez-ui.html" title="Tez UI">Tez UI</a></li> + <li><a href="shuffle-handler.html" title="Tez Shuffle Handler">Tez Shuffle Handler</a></li> + <li><a href="user_guides.html" title="User Guides">User Guides</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a> -</li> - - <li> <a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a> -</li> - - <li> <a href="team-list.html" title="Project Team">Project Team</a> -</li> - - <li> <a href="by-laws.html" title="Project By-Laws">Project By-Laws</a> -</li> - - <li> <a href="http://www.meetup.com/Apache-Tez-User-Group"; title="User Meetup Group">User Meetup Group</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/TEZ"; title="Tez Wiki">Tez Wiki</a> -</li> - </ul> + <li><a href="mail-lists.html" title="Mailing Lists">Mailing Lists</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking">Issue Tracking</a></li> + <li><a href="team-list.html" title="Project Team">Project Team</a></li> + <li><a href="by-laws.html" title="Project By-Laws">Project By-Laws</a></li> + <li><a href="https://www.meetup.com/Apache-Tez-User-Group"; title="User Meetup Group">User Meetup Group</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ"; title="Tez Wiki">Tez Wiki</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Download Apache TEZ® Releases <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://archive.apache.org/dist/incubator/tez/tez-0.4.1-incubating/"; title="0.4.1-incubating">0.4.1-incubating</a> -</li> - - <li> <a href="releases/apache-tez-0-5-4.html" title="0.5.4">0.5.4</a> -</li> - - <li> <a href="releases/apache-tez-0-6-2.html" title="0.6.2">0.6.2</a> -</li> - - <li> <a href="releases/apache-tez-0-7-1.html" title="0.7.1">0.7.1</a> -</li> - - <li> <a href="releases/apache-tez-0-8-5.html" title="0.8.5">0.8.5</a> -</li> - - <li> <a href="releases/apache-tez-0-9-0.html" title="0.9.0">0.9.0</a> -</li> - - <li> <a href="releases/apache-tez-0-9-1.html" title="0.9.1">0.9.1</a> -</li> - - <li> <a href="releases/apache-tez-0-9-2.html" title="0.9.2">0.9.2</a> -</li> - - <li> <a href="releases/index.html" title="All Releases">All Releases</a> -</li> - </ul> + <li><a href="releases/apache-tez-0-8-5.html" title="0.8.5">0.8.5</a></li> + <li><a href="releases/apache-tez-0-9-2.html" title="0.9.2">0.9.2</a></li> + <li><a href="releases/apache-tez-0-10-3.html" title="0.10.3">0.10.3</a></li> + <li><a href="releases/apache-tez-0-10-4.html" title="0.10.4">0.10.4</a></li> + <li><a href="releases/index.html" title="All Releases">All Releases</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contribute <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="https://issues.apache.org/jira/browse/TEZ"; title="JIRA">JIRA</a> -</li> - - <li> <a href="https://gitbox.apache.org/repos/asf/tez.git"; title="Git (Read/Write)">Git (Read/Write)</a> -</li> - - <li> <a href="git://git.apache.org/tez.git" title="Git (Read-only)">Git (Read-only)</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; title="How to Contribute">How to Contribute</a> -</li> - </ul> + <li><a href="https://issues.apache.org/jira/browse/TEZ"; title="JIRA">JIRA</a></li> + <li><a href="https://gitbox.apache.org/repos/asf/tez.git"; title="Git (Read/Write)">Git (Read/Write)</a></li> + <li><a href="git://git.apache.org/tez.git" title="Git (Read-only)">Git (Read-only)</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; title="How to Contribute">How to Contribute</a></li> + </ul> </li> - <li class="dropdown"> + <li class="dropdown"> <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/licenses/"; title="License">License</a> -</li> - - <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Donate">Donate</a> -</li> - - <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> -</li> - - <li> <a href="http://www.apache.org/security/"; title="Security">Security</a> -</li> - </ul> + <li><a href="https://www.apache.org/licenses/"; title="License">License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship.html"; title="Donate">Donate</a></li> + <li><a href="https://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a></li> + <li><a href="https://www.apache.org/security/"; title="Security">Security</a></li> + </ul> </li> - </ul> - - - - - </div> - + </ul> </div> </div> - </div> - - <div class="container-fluid"> - <div id="banner"> - <div class="pull-left"> - <a href="./" id="bannerLeft"> - <img src="images/ApacheTezLogo_lowres.png" alt="Apache Tez" width="25%" height="25%"/> - </a> - </div> - <div class="pull-right" style="width: 25%; height: 25%"> <a href="http://www.apache.org/"; id="bannerRight"> - <img src="https://apache.org/foundation/press/kit/asf_logo_wide.png"; alt="Apache Software Foundation"/> - </a> - </div> - <div class="clear"><hr/></div> - </div> - - <div id="breadcrumbs"> - <ul class="breadcrumb"> - - - <li class=""> - <a href="./" title="Apache TEZ®"> - Apache TEZ®</a> - </li> - <li class="divider ">/</li> - <li class="">Using YARN Timeline with Tez for History</li> - - - - <li id="publishDate" class="pull-right">Last Published: 2019-03-29</li> - - </ul> - </div> + </header> + <div class="container-fluid"> + <header> + <div id="banner"> + <div class="pull-left"><a href="./" id="bannerLeft"><img src="images/ApacheTezLogo_lowres.png" alt="Apache Tez" width="25%" height="25%"/></a></div> + <div class="pull-right"><a href="https://www.apache.org/"; id="bannerRight"><img src="https://apache.org/foundation/press/kit/asf_logo_wide.png"; alt="Apache Software Foundation"/></a></div> + <div class="clear"><hr/></div> + </div> - + <div id="breadcrumbs"> + <ul class="breadcrumb"> + <li class=""><a href="./" title="Apache TEZ®">Apache TEZ®</a><span class="divider">/</span></li> + <li class="active ">Using YARN Timeline with Tez for History</li> + <li id="publishDate" class="pull-right">Last Published: 2024-11-15</li> + </ul> + </div> + </header> <div class="row-fluid"> - <div id="leftColumn" class="span2"> + <header id="leftColumn" class="span2"> + <nav class="well sidebar-nav"> + <ul class="nav nav-list"> + <li class="nav-header">Getting Started</li> + <li><a href="index.html" title="Overview"><span class="none"></span>Overview</a></li> + <li><a href="talks.html" title="Talks and Meetup Recordings"><span class="none"></span>Talks and Meetup Recordings</a></li> + <li><a href="https://www.apache.org/licenses/LICENSE-2.0.txt"; class="externalLink" title="Project License"><span class="none"></span>Project License</a></li> + <li class="nav-header">Documentation</li> + <li><a href="install.html" title="Install Guide"><span class="none"></span>Install Guide</a></li> + <li><a href="localmode.html" title="Local Mode"><span class="none"></span>Local Mode</a></li> + <li><a href="tez-ui.html" title="Tez UI"><span class="none"></span>Tez UI</a></li> + <li><a href="shuffle-handler.html" title="Tez Shuffle Handler"><span class="none"></span>Tez Shuffle Handler</a></li> + <li><a href="user_guides.html" title="User Guides"><span class="none"></span>User Guides</a></li> + <li class="nav-header">Community</li> + <li><a href="mail-lists.html" title="Mailing Lists"><span class="none"></span>Mailing Lists</a></li> + <li><a href="issue-tracking.html" title="Issue Tracking"><span class="none"></span>Issue Tracking</a></li> + <li><a href="team-list.html" title="Project Team"><span class="none"></span>Project Team</a></li> + <li><a href="by-laws.html" title="Project By-Laws"><span class="none"></span>Project By-Laws</a></li> + <li><a href="https://www.meetup.com/Apache-Tez-User-Group"; class="externalLink" title="User Meetup Group"><span class="none"></span>User Meetup Group</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ"; class="externalLink" title="Tez Wiki"><span class="none"></span>Tez Wiki</a></li> + <li class="nav-header">Download Apache TEZ® Releases</li> + <li><a href="releases/apache-tez-0-8-5.html" title="0.8.5"><span class="none"></span>0.8.5</a></li> + <li><a href="releases/apache-tez-0-9-2.html" title="0.9.2"><span class="none"></span>0.9.2</a></li> + <li><a href="releases/apache-tez-0-10-3.html" title="0.10.3"><span class="none"></span>0.10.3</a></li> + <li><a href="releases/apache-tez-0-10-4.html" title="0.10.4"><span class="none"></span>0.10.4</a></li> + <li><a href="releases/index.html" title="All Releases"><span class="none"></span>All Releases</a></li> + <li class="nav-header">Contribute</li> + <li><a href="https://issues.apache.org/jira/browse/TEZ"; class="externalLink" title="JIRA"><span class="none"></span>JIRA</a></li> + <li><a href="https://gitbox.apache.org/repos/asf/tez.git"; class="externalLink" title="Git (Read/Write)"><span class="none"></span>Git (Read/Write)</a></li> + <li><a href="git://git.apache.org/tez.git" class="externalLink" title="Git (Read-only)"><span class="none"></span>Git (Read-only)</a></li> + <li><a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; class="externalLink" title="How to Contribute"><span class="none"></span>How to Contribute</a></li> + <li class="nav-header">ASF</li> + <li><a href="https://www.apache.org/licenses/"; class="externalLink" title="License"><span class="none"></span>License</a></li> + <li><a href="https://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Donate"><span class="none"></span>Donate</a></li> + <li><a href="https://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"><span class="none"></span>Thanks</a></li> + <li><a href="https://www.apache.org/security/"; class="externalLink" title="Security"><span class="none"></span>Security</a></li> + </ul> + </nav> <div class="well sidebar-nav"> - - - <ul class="nav nav-list"> - <li class="nav-header">Getting Started</li> - - <li> - - <a href="index.html" title="Overview"> - <i class="none"></i> - Overview</a> - </li> - - <li> - - <a href="talks.html" title="Talks and Meetup Recordings"> - <i class="none"></i> - Talks and Meetup Recordings</a> - </li> - - <li> - - <a href="http://www.apache.org/licenses/LICENSE-2.0.txt"; class="externalLink" title="Project License"> - <i class="none"></i> - Project License</a> - </li> - <li class="nav-header">Documentation</li> - - <li> - - <a href="install.html" title="Install Guide"> - <i class="none"></i> - Install Guide</a> - </li> - - <li> - - <a href="localmode.html" title="Local Mode"> - <i class="none"></i> - Local Mode</a> - </li> - - <li> - - <a href="tez-ui.html" title="Tez UI"> - <i class="none"></i> - Tez UI</a> - </li> - - <li> - - <a href="shuffle-handler.html" title="Tez Shuffle Handler"> - <i class="none"></i> - Tez Shuffle Handler</a> - </li> - - <li> - - <a href="user_guides.html" title="User Guides"> - <i class="none"></i> - User Guides</a> - </li> - <li class="nav-header">Community</li> - - <li> - - <a href="mail-lists.html" title="Mailing Lists"> - <i class="none"></i> - Mailing Lists</a> - </li> - - <li> - - <a href="issue-tracking.html" title="Issue Tracking"> - <i class="none"></i> - Issue Tracking</a> - </li> - - <li> - - <a href="team-list.html" title="Project Team"> - <i class="none"></i> - Project Team</a> - </li> - - <li> - - <a href="by-laws.html" title="Project By-Laws"> - <i class="none"></i> - Project By-Laws</a> - </li> - - <li> - - <a href="http://www.meetup.com/Apache-Tez-User-Group"; class="externalLink" title="User Meetup Group"> - <i class="none"></i> - User Meetup Group</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/TEZ"; class="externalLink" title="Tez Wiki"> - <i class="none"></i> - Tez Wiki</a> - </li> - <li class="nav-header">Download Apache TEZ® Releases</li> - - <li> - - <a href="http://archive.apache.org/dist/incubator/tez/tez-0.4.1-incubating/"; class="externalLink" title="0.4.1-incubating"> - <i class="none"></i> - 0.4.1-incubating</a> - </li> - - <li> - - <a href="releases/apache-tez-0-5-4.html" title="0.5.4"> - <i class="none"></i> - 0.5.4</a> - </li> - - <li> - - <a href="releases/apache-tez-0-6-2.html" title="0.6.2"> - <i class="none"></i> - 0.6.2</a> - </li> - - <li> - - <a href="releases/apache-tez-0-7-1.html" title="0.7.1"> - <i class="none"></i> - 0.7.1</a> - </li> - - <li> - - <a href="releases/apache-tez-0-8-5.html" title="0.8.5"> - <i class="none"></i> - 0.8.5</a> - </li> - - <li> - - <a href="releases/apache-tez-0-9-0.html" title="0.9.0"> - <i class="none"></i> - 0.9.0</a> - </li> - - <li> - - <a href="releases/apache-tez-0-9-1.html" title="0.9.1"> - <i class="none"></i> - 0.9.1</a> - </li> - - <li> - - <a href="releases/apache-tez-0-9-2.html" title="0.9.2"> - <i class="none"></i> - 0.9.2</a> - </li> - - <li> - - <a href="releases/index.html" title="All Releases"> - <i class="none"></i> - All Releases</a> - </li> - <li class="nav-header">Contribute</li> - - <li> - - <a href="https://issues.apache.org/jira/browse/TEZ"; class="externalLink" title="JIRA"> - <i class="none"></i> - JIRA</a> - </li> - - <li> - - <a href="https://gitbox.apache.org/repos/asf/tez.git"; class="externalLink" title="Git (Read/Write)"> - <i class="none"></i> - Git (Read/Write)</a> - </li> - - <li> - - <a href="git://git.apache.org/tez.git" class="externalLink" title="Git (Read-only)"> - <i class="none"></i> - Git (Read-only)</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez"; class="externalLink" title="How to Contribute"> - <i class="none"></i> - How to Contribute</a> - </li> - <li class="nav-header">ASF</li> - - <li> - - <a href="http://www.apache.org/licenses/"; class="externalLink" title="License"> - <i class="none"></i> - License</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Donate"> - <i class="none"></i> - Donate</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"> - <i class="none"></i> - Thanks</a> - </li> - - <li> - - <a href="http://www.apache.org/security/"; class="externalLink" title="Security"> - <i class="none"></i> - Security</a> - </li> - </ul> - - - - <hr class="divider" /> - - <div id="poweredBy"> - <div class="clear"></div> - <div class="clear"></div> - - - - <div id="twitter"> - + <hr /> + <div id="poweredBy"> + <div class="clear"></div> + <div id="twitter" style="border:none; margin-top: 10px"> <a href="https://twitter.com/ApacheTez"; class="twitter-follow-button" data-show-count="false" data-align="left" data-size="medium" data-show-screen-name="true" data-lang="en">Follow ApacheTez</a> - <script type="text/javascript">!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> - - </div> - <div class="clear"></div> - <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> - <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> - </a> - </div> + <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0];if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src="//platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> + </div> + <div class="clear"></div> + <div class="clear"></div> +<a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /></a> + </div> </div> - </div> - - - <div id="bodyColumn" class="span10" > - - <!-- Licensed to the Apache Software Foundation (ASF) under one or more + </header> + <main id="bodyColumn" class="span10" > +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 @@ -491,100 +187,101 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and - limitations under the License. --><p></p> -<div class="section"> + limitations under the License. +--> + +<section> <h2><a name="YARN_Timeline_Background"></a>YARN Timeline Background</h2> -<p>Initial support for <a class="externalLink" href="http://hadoop.apache.org/docs/r2.4.0/hadoop-yarn/hadoop-yarn-site/TimelineServer.html";>YARN Timeline</a> was introduced in Apache Hadoop 2.4.0. Support for ACLs in Timeline was introduced in Apache Hadoop 2.6.0. Support for Timeline was introduced in Tez in 0.5.x ( with some experimental support in 0.4.x ). However, Tez ACLs integration with Timeline is only available from Tez 0.6.0 onwards.</p></div> -<div class="section"> +<p>Initial support for <a class="externalLink" href="https://hadoop.apache.org/docs/r2.4.0/hadoop-yarn/hadoop-yarn-site/TimelineServer.html";>YARN Timeline</a> was introduced in Apache Hadoop 2.4.0. Support for ACLs in Timeline was introduced in Apache Hadoop 2.6.0. Support for Timeline was introduced in Tez in 0.5.x ( with some experimental support in 0.4.x ). However, Tez ACLs integration with Timeline is only available from Tez 0.6.0 onwards.</p></section><section> <h2><a name="How_Tez_Uses_YARN_Timeline"></a>How Tez Uses YARN Timeline</h2> -<p>Tez uses YARN Timeline as its application history store. Tez stores most of its lifecycle information into this history store such as: - DAG information such as: - DAG Plan - DAG Submission, Start and End times - DAG Counters - Final status of the DAG and additional diagnostics - Vertex, Task and Task Attempt Information - Start and End times - Counters - Diagnostics</p> -<p>Using the above information, a user can analyze a Tez DAG while it is running and after it has completed.</p></div> -<div class="section"> +<p>Tez uses YARN Timeline as its application history store. Tez stores most of its lifecycle information into this history store such as:</p> +<ul> + +<li>DAG information such as: +<ul> + +<li>DAG Plan</li> +<li>DAG Submission, Start and End times</li> +<li>DAG Counters</li> +<li>Final status of the DAG and additional diagnostics</li> +</ul> +</li> +<li>Vertex, Task and Task Attempt Information +<ul> + +<li>Start and End times</li> +<li>Counters</li> +<li>Diagnostics</li> +</ul> +</li> +</ul> +<p>Using the above information, a user can analyze a Tez DAG while it is running and after it has completed.</p></section><section> <h2><a name="YARN_Timeline_and_Hadoop_Versions"></a>YARN Timeline and Hadoop Versions</h2> <p>Given that the support for YARN Timeline with full security was only realized in Apache Hadoop 2.6.0, some features may or may not be supported depending on which version of Apache Hadoop is used.</p> - <table border="0" class="table table-striped"> - <thead> - +<thead> + <tr class="a"> - <th> </th> - -<th>Hadoop 2.2.x, 2.3.x </th> - -<th>Hadoop 2.4.x, 2.5.x </th> - -<th>Hadoop 2.6.x and higher </th> - </tr> - </thead> - <tbody> - +<th>Hadoop 2.2.x, 2.3.x</th> +<th>Hadoop 2.4.x, 2.5.x</th> +<th>Hadoop 2.6.x and higher</th></tr> +</thead><tbody> + <tr class="b"> - -<td>Timeline Support </td> - -<td>No </td> - -<td>Yes </td> - -<td>Yes </td> - </tr> - +<td align="left">Timeline Support</td> +<td>No</td> +<td>Yes</td> +<td>Yes</td></tr> <tr class="a"> - -<td>Timeline with ACLs Support </td> - -<td>No </td> - -<td>No </td> - -<td>Yes </td> - </tr> - </tbody> -</table></div> -<div class="section"> +<td align="left">Timeline with ACLs Support</td> +<td>No</td> +<td>No</td> +<td>Yes</td></tr> +</tbody> +</table></section><section> <h2><a name="Configuring_Tez_to_use_YARN_Timeline"></a>Configuring Tez to use YARN Timeline</h2> <p>By default, Tez writes its history data into a file on HDFS. To use Timeline, add the following property into your tez-site.xml:</p> - <blockquote> -<p><property><br />    <name>tez.history.logging.service.class</name><br />    <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value><br /> </property><br /></p> + +<p><property><br /> +   <name>tez.history.logging.service.class</name><br /> +   <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value><br /> +</property><br /></p> </blockquote> <p>For Tez 0.4.x, the above property is not respected. For 0.4.x, please set the following property:</p> - <blockquote> -<p><property><br />    <name>tez.yarn.ats.enabled</name><br />    <value>true</value><br /> </property><br /></p> + +<p><property><br /> +   <name>tez.yarn.ats.enabled</name><br /> +   <value>true</value><br /> +</property><br /></p> </blockquote> <p>When using Tez with Apache Hadoop 2.4.x or 2.5.x, given that these versions are not fully secure, the following property also needs to be enabled:</p> - <blockquote> -<p><property><br />    <name>tez.allow.disabled.timeline-domains</name><br />    <value>true</value><br /> </property><br /></p> -</blockquote></div> - </div> - </div> - </div> +<p><property><br /> +   <name>tez.allow.disabled.timeline-domains</name><br /> +   <value>true</value><br /> +</property><br /></p> +</blockquote></section> + </main> + </div> + </div> <hr/> - <footer> - <div class="container-fluid"> - <div class="row span12">Copyright © 2019 - <a href="http://www.apache.org/";>Apache Software Foundation</a>. - All Rights Reserved. - - </div> - - <?xml version="1.0" encoding="UTF-8"?> -<div class="row span12">Apache Tez, Apache, the Apache feather logo, and the Apache Tez project logos are trademarks of The Apache Software Foundation. - All other marks mentioned may be trademarks or registered trademarks of their respective owners.</div> - <?xml version="1.0" encoding="UTF-8"?> -<br/> - <?xml version="1.0" encoding="UTF-8"?> + <div class="container-fluid"> + <div class="row-fluid"> <div class="row span12"> - <a href="privacy-policy.html">Privacy Policy</a> -</div> - - - </div> + Apache Tez, Apache, the Apache feather logo, and the Apache Tez project logos are trademarks of The Apache Software Foundation. + All other marks mentioned may be trademarks or registered trademarks of their respective owners. + </div> + <br/> + <div class="row span12"> + <a href="privacy-policy.html">Privacy Policy</a> + </div> + </div> + </div> </footer> </body> </html>
![https://apache.org/foundation/press/kit/asf_logo_wide.png"](https://apache.org/foundation/press/kit/asf_logo_wide.png"){kind=link}