http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/contribute/index.html ---------------------------------------------------------------------- diff --git a/contribute/index.html b/contribute/index.html new file mode 100644 index 0000000..0781f8e --- /dev/null +++ b/contribute/index.html @@ -0,0 +1,179 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="how-to-contribute">How To Contribute</h1> + +<h2 id="submitting-changes">Submitting Changes</h2> + +<p>We use git as our version control system. To streamline the process of giving proper credit to the contributors when committing patches, we encourage contributors to submit patches generated using git format-patch. This has many benefits:</p> + +<ul> +<li>Committers can’t forget to attribute proper credit to the contributor</li> +<li>The contributors name and email address shows up in git log</li> +<li>When viewing Yetus’s source code on <a href="https://github.com/apache/yetus";>https://github.com/apache/yetus</a> , the commits from the contributor are linked to their github.com account if it’s linked to the same email address they used when generating the git format-patch</li> +</ul> + +<p>Long story short, it makes both the contributors’ and committers’ lives easier, so please generate your patches using git format-patch.</p> + +<p>Here are some instructions on how to generate patches:</p> + +<ul> +<li>Ensure that you have all of your change as 1 commit which has the correct commit message - something like <code>YETUS-1. Update shellcheck plug-in to support bats files</code></li> +<li>Then run a command like: <code>git format-patch HEAD^..HEAD --stdout > YETUS-1.00.patch</code></li> +<li>Upload the YETUS-1.00.patch file to the aformenentioned JIRA</li> +</ul> + +<p>The naming of the patch should be in (JIRA).(patch number).patch or, if it needs to apply to a specific branch, (JIRA).(branch name).(patch number).patch format. For example, YETUS-9.00.patch, YETUS-500.02.patch, or YETUS-23.cmake.11.patch. This way, if you need to upload another version of the patch, you should keep the file name the same and JIRA will sort them according to date/time if multiple files have the same name. This feature is also useful to traceback the history of a patch and roll-back to an earlier version if needed.</p> + +<h2 id="website">Website</h2> + +<p>We use middleman to generate the website content from markdown and other +dynamic templates. The following steps assume you have a working +ruby 2.x environment setup:</p> +<pre class="highlight shell"><code>gem install bundler +bundle install +</code></pre> + +<h3 id="make-changes-in-asf-site-src-source">Make changes in asf-site-src/source</h3> + +<p>Make any changes in the source directory:</p> +<pre class="highlight shell"><code><span class="nb">cd </span>asf-site-src/source +vi contribute.html.md +</code></pre> + +<h3 id="generating-the-website">Generating the website</h3> + +<p>To generate the static wesbite for Apache Yetus run the following commands at the root asf-site-src directory:</p> +<pre class="highlight shell"><code>bundle <span class="nb">exec </span>middleman build +</code></pre> + +<h3 id="live-development">Live Development</h3> + +<p>Live development of the site enables automatic reload when changes are saved. +To enable run the following command and then open a browser and navigate to +<a href="http://localhost:4567/";>http://localhost:4567</a></p> +<pre class="highlight plaintext"><code>bundle exec middleman +</code></pre> + +<h3 id="publishing-the-site">Publishing the Site</h3> + +<p>Commit the publish directory to the asf-site branch.</p> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html>
http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/documentation/latest/index.html ---------------------------------------------------------------------- diff --git a/documentation/latest/index.html b/documentation/latest/index.html new file mode 100644 index 0000000..40a4d7e --- /dev/null +++ b/documentation/latest/index.html @@ -0,0 +1,139 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="yetus-precommit">Yetus Precommit</h1> + +<p>The Yetus Precommit Patch Tester allows projects to codify their patch acceptance criteria and then evaluate incoming contributions prior to review by a committer.</p> + +<ul> +<li>Take a quick look at <a href="precommit-glossary">our glossary of terms</a> to ensure you are familiar with the ASF and Maven jargon we’ll use as terminology specific to this project.</li> +<li>For an overview of Yetus’ philosophy on testing contributions and how evaluation is performed, see our <a href="precommit-architecture">overview</a>.</li> +<li>To get started on your project, including an explanation of what we’ll expect in a runtime environment and what optional utilities we’ll leverage, read through the <a href="precommit-basic">basic usage guide</a>.</li> +<li>If your project has advanced requirements such as module relationships not expressed in Maven, special profiles, or a need on os-specific prerequisites not managed by Maven then you’ll need to use our <a href="precommit-advanced">advanced usage guide</a>.</li> +</ul> + +<p>There is also documentation on <a href="precommit-buildtools">build systems</a> and <a href="precommit-bugsystems">bug systems</a></p> + +<h1 id="yetus-release-doc-maker">Yetus Release Doc Maker</h1> + +<p>The Release Documentation Maker allows projects to generate nicely formated Markdown Changelogs and Release Notes based upon JIRA. You can view that +documenation <a href="releasedocmaker">here</a>.</p> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/documentation/latest/precommit-advanced/index.html ---------------------------------------------------------------------- diff --git a/documentation/latest/precommit-advanced/index.html b/documentation/latest/precommit-advanced/index.html new file mode 100644 index 0000000..44fe94e --- /dev/null +++ b/documentation/latest/precommit-advanced/index.html @@ -0,0 +1,314 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="test-patch">test-patch</h1> + +<ul> +<li><a href="#Docker_Support">Docker Support</a></li> +<li><a href="#Plug-ins">Plug-ins</a></li> +<li><a href="#Configuring_for_Other_Projects">Configuring for Other Projects</a></li> +<li><a href="#Important_Variables">Important Variables</a></li> +</ul> + +<h1 id="docker-support">Docker Support</h1> + +<p>By default, test-patch runs in the same shell where it was launched. It can alternatively use Docker to launch itself into a container. This is particularly useful if running under a QA environment that does not provide all the necessary binaries. For example, if the patch requires a newer version of Java.</p> + +<p>The <code>--docker</code> parameter tells test-patch to run in Docker mode. The <code>--dockerfile</code> parameter allows one to provide a custom Dockerfile. The Dockerfile should contain all of the necessary binaries and tooling needed to run the test. However be aware that test-patch will copy this file and append its necessary hooks to re-launch itself prior to executing docker.</p> + +<p>NOTE: If you are using Boot2Docker, you must use directories under /Users (OSX) or C:\Users (Windows) as the base and patchprocess directories (specified by the –basedir and –patch-dir options respectively), because automatically mountable directories are limited to them. See <a href="https://docs.docker.com/userguide/dockervolumes/#mount-a-host-directory-as-a-data-volume";>the Docker documentation</a>.</p> + +<p>Dockerfile images will be named with a test-patch prefix and suffix with either a date or a git commit hash. By using this information, test-patch will automatically manage broken/stale container images that are hanging around if it is run in –jenkins mode. In this way, if Docker fails to build the image, the disk space should eventually be cleaned and returned back to the system.</p> + +<h1 id="plug-ins">Plug-ins</h1> + +<p>test-patch allows one to add to its basic feature set via plug-ins. There is a directory called test-patch.d off of the directory where test-patch.sh lives. Inside this directory one may place some bash shell fragments that, if setup with proper functions, will allow for test-patch to call it as necessary. Different plug-ins have specific functions for that particular functionality. In this document, the common functions as well as test functions are covered. See other documentat for pertinent information for the other plug-in types.</p> + +<h2 id="common-plug-in-functions">Common Plug-in Functions</h2> + +<p>Every plug-in must have one line in order to be recognized, usually an ‘add’ statement. Test plug-ins, for example, have this statement:</p> +<pre class="highlight shell"><code>add_plugin <pluginname> +</code></pre> + +<p>This function call registers the <code>pluginname</code> so that test-patch knows that it exists. The <code>pluginname</code> also acts as the key to the custom functions that you can define. For example:</p> +<pre class="highlight shell"><code><span class="k">function </span>pluginname_filefilter +</code></pre> + +<p>defines the filefilter for the <code>pluginname</code> plug-in.</p> + +<p>Similarly, there are other functions that may be defined during the test-patch run:</p> +<pre class="highlight plaintext"><code>HINT: It is recommended to make the pluginname relatively small, 10 characters at the most. Otherwise, the ASCII output table may be skewed. +</code></pre> + +<ul> +<li><p>pluginname_initialize</p> + +<ul> +<li>After argument parsing and prior to any other work, the initialize step allows a plug-in to do any precursor work, set internal defaults, etc.</li> +</ul></li> +<li><p>pluginname_usage</p> + +<ul> +<li>executed when the help message is displayed. This is used to display the plug-in specific options for the user.</li> +</ul></li> +<li><p>pluginname_parse_args</p> + +<ul> +<li>executed prior to any other above functions except for pluginname_usage. This is useful for parsing the arguments passed from the user and setting up the execution environment.</li> +</ul></li> +<li><p>pluginname_precheck</p> + +<ul> +<li>executed prior to the patch being applied but after the git repository is setup. Returning a fail status here will exit test-patch.</li> +</ul></li> +<li><p>pluginname_precompile</p> + +<ul> +<li>executed prior to the compilation part of the lifecycle. This is useful for doing setup work required by the compilation process.</li> +</ul></li> +<li><p>pluginname_compile</p> + +<ul> +<li>executed immediately after the actual compilation. This is step is intended to be used to verify the results and add extra checking of the compile phase and it’s stdout/stderr.</li> +</ul></li> +<li><p>pluginname_postcompile</p> + +<ul> +<li>This step happens after the compile phase.</li> +</ul></li> +<li><p>pluginname_rebuild</p> + +<ul> +<li>Any non-unit tests that require the source to be rebuilt in a destructive way should be run here.</li> +</ul></li> +</ul> + +<h1 id="test-plug-ins">Test Plug-ins</h1> + +<p>Plugins geared towards independent tests are registed via:</p> +<pre class="highlight shell"><code>add_plugin <pluginname> +</code></pre> + +<ul> +<li><p>pluginname_filefilter</p> + +<ul> +<li>executed while determine which files trigger which tests. This function should use 'add_test pluginname’ to add the plug-in to the test list.</li> +</ul></li> +<li><p>pluginname_tests</p> + +<ul> +<li>executed after the unit tests have completed.</li> +</ul></li> +</ul> + +<h1 id="configuring-for-other-projects">Configuring for Other Projects</h1> + +<p>It is impossible for any general framework to be predictive about what types of special rules any given project may have, especially when it comes to ordering and Maven profiles. In order to direct test-patch to do the correct action, a project <code>personality</code> should be added that enacts these custom rules.</p> + +<p>A personality consists of two functions. One that determines which test types to run and another that allows a project to dictate ordering rules, flags, and profiles on a per-module, per-test run.</p> + +<p>There can be only <strong>one</strong> of each personality function defined.</p> + +<h2 id="test-determination">Test Determination</h2> + +<p>The <code>personality_file_tests</code> function determines which tests to turn on based upon the file name. It is relatively simple. For example, to turn on a full suite of tests for Java files:</p> +<pre class="highlight shell"><code><span class="k">function </span>personality_file_tests +<span class="o">{</span> + <span class="nb">local </span><span class="nv">filename</span><span class="o">=</span><span class="nv">$1</span> + + <span class="k">if</span> <span class="o">[[</span> <span class="k">${</span><span class="nv">filename</span><span class="k">}</span> <span class="o">=</span>~ <span class="se">\.</span>java<span class="nv">$ </span><span class="o">]]</span>; <span class="k">then + </span>add_test findbugs + add_test javac + add_test javadoc + add_test mvninstall + add_test unit + <span class="k">fi</span> + +<span class="o">}</span> +</code></pre> + +<p>The <code>add_test</code> function is used to activate the standard tests. Additional plug-ins (such as checkstyle), will get queried on their own.</p> + +<h2 id="module-amp-profile-determination">Module & Profile Determination</h2> + +<p>Once the tests are determined, it is now time to pick which <a href="precommit-glossary.md#genericoutside-definitions">modules</a> should get used. That’s the job of the <code>personality_modules</code> function.</p> +<pre class="highlight shell"><code><span class="k">function </span>personality_modules +<span class="o">{</span> + + clear_personality_queue + +... + + personality_enqueue_module <module> <flags> + +<span class="o">}</span> +</code></pre> + +<p>It takes exactly two parameters <code>repostatus</code> and <code>testtype</code>.</p> + +<p>The <code>repostatus</code> parameter tells the <code>personality</code> function exactly what state the source repository is in. It can only be in one of two states: <code>branch</code> or <code>patch</code>. <code>branch</code> means the patch has not been applied. The <code>patch</code> state is after the patch has been applied.</p> + +<p>The <code>testtype</code> state tells the personality exactly which test is about to be executed.</p> + +<p>In order to communicate back to test-patch, there are two functions for the personality to use.</p> + +<p>The first is <code>clear_personality_queue</code>. This removes the previous test’s configuration so that a new module queue may be built. Custom personality_modules will almost always want to do this as the first action.</p> + +<p>The second is <code>personality_enqueue_module</code>. This function takes two parameters. The first parameter is the name of the module to add to this test’s queue. The second parameter is an option list of additional flags to pass to Maven when processing it. <code>personality_enqueue_module</code> may be called as many times as necessary for your project.</p> +<pre class="highlight plaintext"><code>NOTE: A module name of . signifies the root of the repository. +</code></pre> + +<p>For example, let’s say your project uses a special configuration to skip unit tests (-DskipTests). Running unit tests during a javadoc build isn’t very useful and wastes a lot of time. We can write a simple personality check to disable the unit tests:</p> +<pre class="highlight shell"><code><span class="k">function </span>personality_modules +<span class="o">{</span> + <span class="nb">local </span><span class="nv">repostatus</span><span class="o">=</span><span class="nv">$1</span> + <span class="nb">local </span><span class="nv">testtype</span><span class="o">=</span><span class="nv">$2</span> + + <span class="k">if</span> <span class="o">[[</span> <span class="k">${</span><span class="nv">testtype</span><span class="k">}</span> <span class="o">==</span> <span class="s1">'javadoc'</span> <span class="o">]]</span>; <span class="k">then + </span>personality_enqueue_module . -DskipTests + <span class="k">return + fi</span> + ... + +</code></pre> + +<p>This function will tell test-patch that when the javadoc test is being run, do the documentation build at the base of the source repository and make sure the -DskipTests flag is passed to our build tool.</p> + +<h1 id="important-variables">Important Variables</h1> + +<p>There are a handful of extremely important system variables that make life easier for personality and plug-in writers. Other variables may be provided by individual plug-ins. Check their development documentation for more information.</p> + +<ul> +<li><p>BUILD_NATIVE will be set to true if the system has requested that non-JVM-based code be built (e.g., JNI or other compiled C code). Under Jenkins, this is always true.</p></li> +<li><p>BUILDTOOL specifies which tool is currently being used to drive compilation. Additionally, many build tools define xyz_ARGS to pass on to the build tool command line. (e.g., MAVEN_ARGS if maven is in use). Projects may set this in their personality. NOTE: today, only one build tool at a time is supported. This may change in the future.</p></li> +<li><p>CHANGED_FILES is a list of all files that appear to be added, deleted, or modified in the patch.</p></li> +<li><p>CHANGED_UNFILTERED_MODULES is a list of all modules that house all of the CHANGED_FILES. Be aware that the root of the source tree is reported as ’.’.</p></li> +<li><p>CHANGED_MODULES reports which modules that appear to have source code in them.</p></li> +<li><p>GITHUB_REPO is to help test-patch when talking to Github. If test-patch is given just a number on the command line, it will default to using this repo to determine the pull request.</p></li> +<li><p>HOW_TO_CONTRIBUTE should be a URL that points to a project’s on-boarding documentation for new users. Currently, it is used to suggest a review of patch naming guidelines. Since this should be project specific information, it is useful to set in a project’s personality.</p></li> +<li><p>JIRA_ISSUE_RE is to help test-patch when talking to JIRA. It helps determine if the given project is appropriate for the given JIRA issue.</p></li> +<li><p>MODULE and other MODULE_* are arrays that contain which modules, the status, etc, to be operated upon. These should be treated as read-only by plug-ins.</p></li> +<li><p>PATCH_BRANCH_DEFAULT is the name of the branch in the git repo that is considered the master. This is useful to set in personalities.</p></li> +<li><p>PATCH_DIR is the name of the temporary directory that houses test-patch artifacts (such as logs and the patch file itself)</p></li> +<li><p>TEST_PARALLEL if parallel unit tests have been requested. Project personalities are responsible for actually enabling or ignoring the request. TEST_THREADS is the number of threads that have been requested to run in parallel.</p></li> +</ul> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/documentation/latest/precommit-architecture/index.html ---------------------------------------------------------------------- diff --git a/documentation/latest/precommit-architecture/index.html b/documentation/latest/precommit-architecture/index.html new file mode 100644 index 0000000..96add38 --- /dev/null +++ b/documentation/latest/precommit-architecture/index.html @@ -0,0 +1,215 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="some-philosophy">Some Philosophy</h1> + +<ul> +<li><p>Everyone’s time is valuable. The quicker contributors can get feedback and iterate, the more likely and faster their contribution will get checked in. A committer should be able to focus on the core issues of a contribution rather than details that can be determined automatically.</p></li> +<li><p>Precommit checks should be fast. There is no value in testing parts of the source tree that are not immediately impacted by a change. Unit testing is the target. They are not a replacement for full builds or integration tests.</p></li> +<li><p>Many open source projects have a desire to have this capability. Why not generalize a solution?</p></li> +<li><p>In many build systems (especially with maven), a modular design has been picked. Why not leverage that design to make checks faster?</p></li> +<li><p>Projects that use the same language will, with a high degree of certainty, benefit from the same types of checks.</p></li> +<li><p>Portability matters. Tooling should be as operating system and language agnostic as possible.</p></li> +</ul> + +<h1 id="phases">Phases</h1> + +<p>test-patch works effectively under several different phases:</p> + +<h2 id="initialize">Initialize</h2> + +<p>This is where test-patch configures and validates the environment. Some things done in this phase:</p> + +<ul> +<li>Defaults</li> +<li>Parameter handling</li> +<li>Importing plug-ins and personalities</li> +<li>Docker container launching</li> +<li>Re-exec support</li> +<li>Patch file downloading</li> +<li>git repository management (fresh pull, branch switching, etc)</li> +</ul> + +<h2 id="precheck">Precheck</h2> + +<p>Checks done here are <em>fatal</em>.</p> + +<p>This acts as a verification of all of the setup parts and is the final place to short-cut the full test cycle. The most significant built-in check done here is verifying the patch file is a valid.</p> + +<h2 id="patch-file-tests">Patch File Tests</h2> + +<p>Tests that only require the patch file are run. Note that the repository is still from the initial checkout!</p> + +<h2 id="compile-cycle-branch">Compile Cycle (Branch)</h2> + +<p>When compilation must be done, we follow these five steps:</p> + +<ul> +<li>The list of modules that require analysis is built.</li> +<li>A precompile step to set things up for the actual compile</li> +<li>The actual compile</li> +<li>A postcompile to do analysis on the output of that compile phase</li> +<li>A rebuild phase to run tests that require recompiles</li> +</ul> + +<p>The first time this is done is with the pristine checkout. This is called the “branch compile”. For this pass, this is where the ‘before’ work is handled. Some things that typically get checked in this phase:</p> + +<ul> +<li>The first pass of files and modules that will get patched</li> +<li>Validation and information gathering of the source tree pre-patch</li> +<li>javadoc, scaladoc, etc</li> +</ul> + +<h2 id="distribution-clean">Distribution Clean</h2> + +<p>This step is to wipe the repository clean back to a pristine state such that the previous cycle will not impact the next cycle.</p> + +<h2 id="patch-application">Patch Application</h2> + +<p>The patch gets applied.</p> + +<h2 id="compile-cycle-patch">Compile Cycle (Patch)</h2> + +<p>Now that the patch has been applied the steps to compile we outlined in the compilation (branch) phase are repeated but with the patch applied. This is where a lot of 'after’ checks are performed.</p> + +<h2 id="unit-tests">Unit Tests</h2> + +<p>Since unit tests are generally the slowest part of the precommit process, they are run last. At this point, all the prerequisites to running them should be in place and ready to go.</p> + +<h2 id="reporting">Reporting</h2> + +<p>Finally, the results are reported to the screen and, optionally, to JIRA and/or whatever bug system has been configured.</p> + +<h1 id="test-flow">Test Flow</h1> + +<p>The basic workflow for many of the sub-items in individual phases are:</p> + +<ol> +<li>print a header, so the end user knows that something is happening</li> +<li>verify if the test is needed. If so, continue on. Otherwise, return success and let the next part of the phase execute.</li> +<li>Ask the personality about what modules and what flags should get used</li> +<li>Execute maven (or some other build tool) in the given modules with the given flags. Log the output and record the time and result code.</li> +<li>Do any extra work as appropriate (diffs, counts, etc) and either accept the status and message given by the maven run or change the vote, message, log file, etc, based upon this extra work.</li> +<li>Add the outcome(s) to the report generator</li> +</ol> + +<p>As one can see, the modules list is one of the key inputs into what actually gets executed. As a result, projects must full flexibility in either adding, modifying, or even removing modules from the test list. If a personality removes the entire list of modules, then that test should just be ignored.</p> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/documentation/latest/precommit-basic/index.html ---------------------------------------------------------------------- diff --git a/documentation/latest/precommit-basic/index.html b/documentation/latest/precommit-basic/index.html new file mode 100644 index 0000000..9b7a3d7 --- /dev/null +++ b/documentation/latest/precommit-basic/index.html @@ -0,0 +1,303 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="test-patch">test-patch</h1> + +<ul> +<li><a href="#Purpose">Purpose</a></li> +<li><a href="#Pre-requisites">Pre-requisites</a></li> +<li><a href="#Basic_Usage">Basic Usage</a></li> +<li><a href="#Build_Tool">Build Tool</a></li> +<li><a href="#Providing_Patch_Files">Providing Patch Files</a></li> +<li><a href="#Project-Specific_Capabilities">Project-Specific Capabilities</a></li> +<li><a href="#MultiJDK">MultiJDK</a></li> +<li><a href="#Docker">Docker</a></li> +</ul> + +<h1 id="purpose">Purpose</h1> + +<p>As part of Hadoop’s commit process, all patches to the source base go through a precommit test that does some (relatively) light checking to make sure the proposed change does not break unit tests and/or passes some other prerequisites such as code formatting guidelines. This is meant as a preliminary check for committers so that the basic patch is in a known state and for contributors to know if they have followed the project’s guidelines. This check, called test-patch, may also be used by individual developers to verify a patch prior to sending to the Hadoop QA systems.</p> + +<p>Other projects have adopted a similar methodology after seeing great success in the Hadoop model. Some have even gone as far as forking Hadoop’s precommit code and modifying it to meet their project’s needs.</p> + +<p>This is a modification to Hadoop’s version of test-patch so that we may bring together all of these forks under a common code base to help the community as a whole.</p> + +<h1 id="pre-requisites">Pre-requisites</h1> + +<p>test-patch has the following requirements:</p> + +<ul> +<li>A project with a supported build tool (ant, gradle, maven, …)</li> +<li>git-based project (and git 1.7.3 or higher installed)</li> +<li>bash v3.2 or higher</li> +<li>findbugs 3.x installed</li> +<li>shellcheck installed, preferably 0.3.6 or higher</li> +<li>pylint installed</li> +<li>GNU diff</li> +<li>GNU patch</li> +<li>POSIX awk</li> +<li>POSIX grep</li> +<li>POSIX sed</li> +<li>curl</li> +<li>file command</li> +<li>smart-apply-patch.sh (included!)</li> +</ul> + +<p>Maven plug-ins requirements:</p> + +<ul> +<li>Apache RAT</li> +<li>Apache FindBugs</li> +</ul> + +<p>Optional:</p> + +<ul> +<li>JIRA-based issue tracking</li> +<li>GitHub-based issue tracking</li> +<li>Some other supported bug system</li> +</ul> + +<p>The locations of these files are (mostly) assumed to be in the file path, but may be overridden via command line options. For Solaris and Solaris-like operating systems, the default location for the POSIX binaries is in /usr/xpg4/bin and the default location for the GNU binaries is /usr/gnu/bin.</p> + +<h1 id="basic-usage">Basic Usage</h1> + +<p>This command will execute basic patch testing against a patch file stored in “filename”:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">cd</span> <your repo> +<span class="gp">$ </span>dev-support/test-patch.sh --dirty-workspace --project<span class="o">=</span>projectname <filename> +</code></pre> + +<p>The <code>--dirty-workspace</code> flag tells test-patch that the repository is not clean and it is ok to continue. By default, unit tests are not run since they may take a significant amount of time.</p> + +<p>To do turn them on, we need to provide the –run-tests option:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">cd</span> <your repo> +<span class="gp">$ </span>dev-support/test-patch.sh --dirty-workspace --run-tests <filename> +</code></pre> + +<p>This is the same command, but now runs the unit tests.</p> + +<p>A typical configuration is to have two repositories. One with the code you are working on and another, clean repository. This means you can:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">cd</span> <workrepo> +<span class="gp">$ </span>git diff master > /tmp/patchfile +<span class="gp">$ </span><span class="nb">cd</span> ../<testrepo> +<span class="gp">$ </span><workrepo>/dev-support/test-patch.sh --basedir<span class="o">=</span><testrepo> --resetrepo /tmp/patchfile +</code></pre> + +<p>We used two new options here. –basedir sets the location of the repository to use for testing. –resetrepo tells test patch that it can go into <strong>destructive</strong> mode. Destructive mode will wipe out any changes made to that repository, so use it with care!</p> + +<p>After the tests have run, there is a directory that contains all of the test-patch related artifacts. This is generally referred to as the patchprocess directory. By default, test-patch tries to make something off of /tmp to contain this content. Using the <code>--patch-dir</code> option, one can specify exactly which directory to use. This is helpful for automated precommit testing so that Jenkins or other automated workflow system knows where to look to gather up the output.</p> + +<p>For example:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh --jenkins --patch-dir<span class="o">=</span><span class="k">${</span><span class="nv">WORKSPACE</span><span class="k">}</span>/patchprocess --basedir<span class="o">=</span><span class="k">${</span><span class="nv">WORKSPACE</span><span class="k">}</span>/source <span class="k">${</span><span class="nv">WORKSPACE</span><span class="k">}</span>/patchfile +</code></pre> + +<p>… will trigger test-patch to run in fully automated Jenkins mode, using ${WORKSPACE}/patchprocess as its scratch space, ${WORKSPACE}/source as the source repository, and ${WORKSPACE}/patchfile as the name of the patch to test against.</p> + +<h1 id="build-tool">Build Tool</h1> + +<p>Out of the box, test-patch is built to use maven. But what if the project is built using something else, such as ant?</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> --build-tool<span class="o">=</span>ant +</code></pre> + +<p>will tell test-patch to use ant instead of maven to drive the project.</p> + +<h1 id="providing-patch-files">Providing Patch Files</h1> + +<h2 id="jira">JIRA</h2> + +<p>It is a fairly common practice within the Apache community to use Apache’s JIRA instance to store potential patches. As a result, test-patch supports providing just a JIRA issue number. test-patch will find the <em>last</em> attachment, download it, then process it.</p> + +<p>For example:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> HADOOP-9905 +</code></pre> + +<p>… will process the patch file associated with this JIRA issue.</p> + +<p>If the Apache JIRA system is not in use, then override options may be provided on the command line to point to a different JIRA instance.</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh --jira-issue-re<span class="o">=</span><span class="s1">'^PROJECT-[0-9]+$'</span> --jira-base-url<span class="o">=</span><span class="s1">'https://example.com/jira'</span> PROJECT-90 +</code></pre> + +<p>… will process the patch file attached to PROJECT-90 on the JIRA instance located on the example.com server.</p> + +<h2 id="github">GITHUB</h2> + +<p>test-patch has some basic support for Github. test-patch supports many forms of providing pull requests to work on:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh --github-repo<span class="o">=</span>apache/pig 99 +</code></pre> + +<p>or</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh https://github.com/apache/pig/pulls/99 +</code></pre> + +<p>or</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh https://github.com/apache/pig/pulls/99.patch +</code></pre> + +<p>… will process PR #99 on the apache/pig repo.</p> + +<h2 id="generic-urls">Generic URLs</h2> + +<p>Luckily, test-patch supports provide ways to provide unified diffs via URLs.</p> + +<p>For example:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> https://example.com/webserver/file.patch +</code></pre> + +<p>… will download and process the file.patch from the example.com webserver.</p> + +<h1 id="project-specific-capabilities">Project-specific Capabilities</h1> + +<p>Due to the extensible nature of the system, test-patch allows for projects to define project-specific rules which we call personalities. (How to build those rules is covered elsewhere.) There are two ways to specify which personality to use:</p> + +<h2 id="direct-method">Direct Method</h2> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> --personality<span class="o">=(</span>filename<span class="o">)</span> +</code></pre> + +<p>This tells test-patch to use the personality in the given file.</p> + +<h2 id="project-method">Project Method</h2> + +<p>However, test-patch can detect if it is a personality that is in its “personality” directory based upon the project name:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> --project<span class="o">=(</span>project<span class="o">)</span> +</code></pre> + +<h1 id="multijdk">MultiJDK</h1> + +<p>For many projects, it is useful to test Java code against multiple versions of JDKs at the same time. test-patch can do this with the –multijdkdirs option:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> --multijdkdirs<span class="o">=</span><span class="s2">"/j/d/k/1,/j/d/k/2"</span> +</code></pre> + +<p>Not all Java tests support this mode, but those that do will now run their tests with all of the given versions of Java consecutively (e.g., javac–the Java compliation test). Tests that do not support MultiJDK mode (e.g., checkstyle, mvn install) will use JAVA_HOME.</p> + +<p>NOTE: JAVA_HOME is always appended to the list of JDKs in MultiJDK mode. If JAVA_HOME is in the list, it will be moved to the end.</p> + +<h1 id="docker">Docker</h1> + +<p>test-patch also has a mode to utilize Docker:</p> +<pre class="highlight shell"><code><span class="gp">$ </span><span class="nb">test</span>-patch.sh <span class="o">(</span>other options<span class="o">)</span> --docker +</code></pre> + +<p>This will do some preliminary setup and then re-execute itself inside a Docker container. For more information on how to provide a custom Dockerfile, see the advanced guide.</p> + +<h2 id="in-closing">In Closing</h2> + +<p>test-patch has many other features and command line options for the basic user. Many of these are self-explanatory. To see the list of options, run test-patch.sh without any options or with –help.</p> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/documentation/latest/precommit-bugsystems/index.html ---------------------------------------------------------------------- diff --git a/documentation/latest/precommit-bugsystems/index.html b/documentation/latest/precommit-bugsystems/index.html new file mode 100644 index 0000000..587c203 --- /dev/null +++ b/documentation/latest/precommit-bugsystems/index.html @@ -0,0 +1,160 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="bug-system-support">Bug System Support</h1> + +<p>test-patch has the ability to support multiple bug systems. Bug tools have some extra hooks to fetch patches, line-level reporting, and posting a final report. Every bug system plug-in must have one line in order to be recognized:</p> +<pre class="highlight shell"><code>add_bugsystem <pluginname> +</code></pre> + +<ul> +<li><p>pluginname_locate_patch</p> + +<ul> +<li>Given input from the user, download the patch if possible.</li> +</ul></li> +<li><p>pluginname_determine_branch</p> + +<ul> +<li>Using any heuristics available, return the branch to process, if possible.</li> +</ul></li> +<li><p>pluginname_determine_issue</p> + +<ul> +<li>Using any heuristics available, set the issue, bug number, etc, for this bug system, if possible. This is typically used to fill in supplementary information in the final output table.</li> +</ul></li> +<li><p>pluginname_writecomment</p> + +<ul> +<li>Given text input, write this output to the bug system as a comment. NOTE: It is the bug system’s responsibility to format appropriately.</li> +</ul></li> +<li><p>pluginname_linecomments</p> + +<ul> +<li>This function allows for the system to write specific comments on specific lines if the bug system supports code review comments.</li> +</ul></li> +<li><p>pluginname_finalreport</p> + +<ul> +<li>Write the final result table to the bug system.</li> +</ul></li> +</ul> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html> http://git-wip-us.apache.org/repos/asf/yetus/blob/ed7cf3b0/documentation/latest/precommit-buildtools/index.html ---------------------------------------------------------------------- diff --git a/documentation/latest/precommit-buildtools/index.html b/documentation/latest/precommit-buildtools/index.html new file mode 100644 index 0000000..6c7a99d --- /dev/null +++ b/documentation/latest/precommit-buildtools/index.html @@ -0,0 +1,227 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<html> + <head> + <meta charset="utf-8"> + <title>Apache Yetus</title> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + <meta name="description" content=""> + <meta name="author" content=""> + + <link href="/assets/css/bootstrap.css" rel="stylesheet"> + <link href="/assets/css/bootstrap-theme.css" rel="stylesheet"> + <link href="/assets/css/font-awesome.css" rel="stylesheet"> + + <!-- JS --> + <script type="text/javascript" src="/assets/js/jquery-2.1.1.min.js"></script> + <script type="text/javascript" src="/assets/js/bootstrap.js"></script> + </head> + <body> + <div class="navbar navbar-custom navbar-static-top" role="navigation"> + <div class="container"> + <div class="navbar-header"> + <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="img-responsive pull-left" href="/"> + <img src="/assets/img/yetus_logo.png" alt="Apache Yetus logo" /> + </a> + </div> + <div class="navbar-collapse collapse"> + <ul class="nav navbar-nav"> + <li><a href="/documentation/latest/">Documentation</a> + </li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a> + <ul class="dropdown-menu" role="menu" aria-labelledby="drop1"> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="http://issues.apache.org/jira/browse/YETUS";><i class="fa fa-bug"></i> JIRA (Bugs)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://git-wip-us.apache.org/repos/asf?s=yetus";><i class="fa fa-code"></i> Source (Apache)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus";><i class="fa fa-github-alt"></i> Source (GitHub)</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute"><i class="fa fa-code-fork"></i> Contributing</a> + </li> + <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus";><i class="fa fa-twitter"></i> @ApacheYetus</a> + </li> + </ul> + </li> + <li> + <li class="dropdown"> + <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a> + <ul class="dropdown-menu" role="menu"> + <li><a href="http://www.apache.org";>Apache Homepage</a> + </li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a> + </li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a> + </li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a> + </li> + <li><a href="http://www.apache.org/security/";>Security</a> + </li> + </ul> + </li> + </li> + </ul> + </div> + <!--/.nav-collapse --> + </div> +</div> + + <div class="container"> + <!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +<h1 id="build-tool-support">Build Tool Support</h1> + +<p>test-patch has the ability to support multiple build tools. Build tool plug-ins have some extra hooks to do source and object maintenance at key points. Every build tool plug-in must have one line in order to be recognized:</p> +<pre class="highlight shell"><code>add_build_tool <pluginname> +</code></pre> + +<h1 id="global-variables">Global Variables</h1> + +<ul> +<li><p>BUILDTOOLCWD</p> + +<ul> +<li>If the build tool does not always run from the ${BASEDIR} and instead needs to change the current working directory to work on a specific module, then set this to true. The default is false.</li> +</ul></li> +<li><p>UNSUPPORTED_TEST</p> + +<ul> +<li>If pluginname_modules_worker is given a test type that is not supported by the build system, set UNSUPPORTED<em>TEST=true. If it is supported, set UNSUPPORTED</em>TEST=false.</li> +</ul></li> +</ul> + +<p>For example, the gradle build tool does not have a standard way to execute checkstyle. So when checkstyle is requested, gradle_modules_worker sets UNSUPPORTED_TEST to true and returns out of the routine.</p> + +<h1 id="required-functions">Required Functions</h1> + +<ul> +<li><p>pluginname_buildfile</p> + +<ul> +<li>This should be an echo of the file that controls the build system. This is used for module determination.</li> +</ul></li> +<li><p>pluginname_executor</p> + +<ul> +<li>This should be an echo of how to run the build tool, any extra arguments, etc.</li> +</ul></li> +<li><p>pluginname_modules_worker</p> + +<ul> +<li>Input is the branch and the test being run. This should call modules<em>workers with the generic parts to run that test on the build system. For example, if it is convention to use ‘test’ to trigger 'unit’ tests, then module</em>workers should be called with 'test’ appended onto its normal parameters.</li> +</ul></li> +<li><p>pluginname_builtin_personality_modules</p> + +<ul> +<li>Default method to determine how to enqueue modules for processing. Note that personalities may override this function.</li> +</ul></li> +<li><p>pluginname_builtin_personality_file_tests</p> + +<ul> +<li>Default method to determine which tests to trigger. Note that personalities may override this function.</li> +</ul></li> +</ul> + +<h1 id="optional-functions">Optional Functions</h1> + +<ul> +<li><p>pluginname_postapply_install</p> + +<ul> +<li>After the install step, this allows the build tool plug-in to do extra work.</li> +</ul></li> +<li><p>pluginname_parse_args</p> + +<ul> +<li>executed prior to any other above functions except for pluginname_usage. This is useful for parsing the arguments passed from the user and setting up the execution environment.</li> +</ul></li> +<li><p>pluginname_initialize</p> + +<ul> +<li>After argument parsing and prior to any other work, the initialize step allows a plug-in to do any precursor work, set internal defaults, etc.</li> +</ul></li> +<li><p>pluginname_count_(test)_probs</p> + +<ul> +<li>Certain language test plug-ins require assistance from the build tool to count problems in the compile log due to some tools having custom handling for those languages. The test plug-in name should be in the (test) part of the function name.</li> +</ul></li> +</ul> + +<h1 id="ant-specific">Ant Specific</h1> + +<h2 id="command-arguments">Command Arguments</h2> + +<p>test-patch always passes -noinput to Ant. This forces ant to be non-interactive.</p> + +<h1 id="gradle-specific">Gradle Specific</h1> + +<p>The gradle plug-in always rebuilds the gradlew file and uses gradlew as the method to execute commands.</p> + +<h1 id="maven-specific">Maven Specific</h1> + +<h2 id="command-arguments">Command Arguments</h2> + +<p>test-patch always passes –batch-mode to maven to force it into non-interactive mode. Additionally, some tests will also force -fae in order to get all of messages/errors during that mode. Some tests are executed with -DskipTests. Additional arguments should be handled via the personality.</p> + +<h2 id="test-profile">Test Profile</h2> + +<p>By default, test-patch will pass -Ptest-patch to Maven. This will allow you to configure special actions that should only happen when running underneath test-patch.</p> + +<h2 id="custom-maven-tests">Custom Maven Tests</h2> + +<ul> +<li>Maven will trigger a maven install as part of the precompile.</li> +<li>Maven will test eclipse integration as part of the postcompile.</li> +<li>If src/site is modified, maven site tests are executed.</li> +</ul> + + </div> + <div class="container"> + <hr> + <footer class="footer"> + <div class="row-fluid"> + <div class="span12 text-left"> + <div class="span12"> + Copyright 2008-2015 <a href="http://www.apache.org/";>Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/";>Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation. + </div> + </div> + + </div> + + </footer> +</div> + + </body> +</html>
