This is an automated email from the ASF dual-hosted git repository. mergebot-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 0ba31906a1a003c779dd968ecb69ae6c986d9011 Author: Mergebot <[email protected]> AuthorDate: Fri Apr 27 09:39:19 2018 -0700 Prepare repository for deployment. --- content/documentation/io/testing/index.html | 158 ++++++++++++++-------------- 1 file changed, 80 insertions(+), 78 deletions(-) diff --git a/content/documentation/io/testing/index.html b/content/documentation/io/testing/index.html index 20c9f13..daf6e44 100644 --- a/content/documentation/io/testing/index.html +++ b/content/documentation/io/testing/index.html @@ -373,15 +373,15 @@ <li>Have a running Kubernetes cluster you can connect to locally using kubectl</li> </ol> -<p>You won’t need to invoke PerfKit Benchmarker directly. Run mvn verify in the directory of the I/O module you’d like to test, with the parameter io-it-suite when running in jenkins CI or with a kubernetes cluster on the same network or io-it-suite-local when running on a local dev box accessing a kubernetes cluster on a remote network.</p> +<p>You won’t need to invoke PerfKit Benchmarker directly. Run <code class="highlighter-rouge">./gradlew performanceTest</code> in project’s root directory, passing appropriate kubernetes scripts depending on the network you’re using (local network or remote one).</p> <p>Example run with the direct runner:</p> -<div class="highlighter-rouge"><pre class="highlight"><code>mvn verify -Dio-it-suite-local -pl sdks/java/io/jdbc,sdks/java/io/jdbc -DpkbLocation="/Users/me/dev/PerfKitBenchmarker/pkb.py" -DforceDirectRunner -DintegrationTestPipelineOptions=["--myTestParam=val"] +<div class="highlighter-rouge"><pre class="highlight"><code>./gradlew performanceTest -DpkbLocation="/Users/me/PerfKitBenchmarker/pkb.py" -DintegrationTestPipelineOptions='["--numberOfRecords=1000"]' -DitModule=sdks/java/io/jdbc/ -DintegrationTest=org.apache.beam.sdk.io.jdbc.JdbcIOIT -DkubernetesScripts="/Users/me/beam/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml" -DbeamITOptions="/Users/me/beam/.test-infra/kubernetes/postgres/pkb-config-local.yml" -DintegrationTest [...] </code></pre> </div> <p>Example run with the Cloud Dataflow runner:</p> -<div class="highlighter-rouge"><pre class="highlight"><code>mvn verify -Dio-it-suite -pl sdks/java/io/jdbc -DintegrationTestPipelineOptions=["--project=PROJECT","--gcpTempLocation=GSBUCKET"] -DintegrationTestRunner=dataflow -DpkbLocation="/Users/me/dev/PerfKitBenchmarker/pkb.py" +<div class="highlighter-rouge"><pre class="highlight"><code>/gradlew performanceTest -DpkbLocation="/Users/me/PerfKitBenchmarker/pkb.py" -DintegrationTestPipelineOptions='["--numberOfRecords=1000", "--project=GOOGLE_CLOUD_PROJECT", "--tempRoot=GOOGLE_STORAGE_BUCKET"]' -DitModule=sdks/java/io/jdbc/ -DintegrationTest=org.apache.beam.sdk.io.jdbc.JdbcIOIT -DkubernetesScripts="/Users/me/beam/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml" -DbeamITOptions="/Users/me/beam/.t [...] </code></pre> </div> @@ -400,40 +400,46 @@ </thead> <tbody> <tr> - <td>-Dio-it-suite + <td>-DpkbLocation </td> - <td>Invokes the call to PerfKit Benchmarker when running in apache beam's jenkins instance or with a kubernetes cluster on the same network. + <td>Path to PerfKit Benchmarker project. </td> </tr> <tr> - <td>-Dio-it-suite-local + <td>-DintegrationTestPipelineOptions </td> - <td>io-it-suite-local when running on a local dev box accessing a kubernetes cluster on a remote network. May not be supported for all I/O transforms. + <td>Passes pipeline options directly to the test being run. </td> </tr> <tr> - <td>-pl sdks/java/io/jdbc + <td>-DitModule </td> - <td>Specifies the maven project of the I/O to test. + <td>Specifies the project submodule of the I/O to test. </td> </tr> <tr> - <td>-Dkubectl="path-to-kubectl" -Dkubeconfig="path-to-kubeconfig" + <td>-DintegrationTest </td> - <td>Options for specifying non-standard kubectl configurations. Optional. Defaults to "kubectl" and "~/.kube/config". + <td>Specifies the test to be run. </td> </tr> <tr> - <td>integrationTestPipelineOptions + <td>-DkubernetesScripts </td> - <td>Passes pipeline options directly to the test being run. + <td>Paths to scripts with necessary kubernetes infrastructure. </td> </tr> <tr> - <td>-DforceDirectRunner - </td> - <td>Runs the test with the direct runner. - </td> + <td>-DbeamITOptions + </td> + <td>Path to file with Benchmark configuration (static and dynamic pipeline options. See below for description). + </td> + </tr> + <tr> + <td>-DintegrationTestRunner + </td> + <td>Runner to be used for running the test. Currently possible options are: direct, dataflow. + </td> </tr> </tbody> </table> @@ -481,7 +487,7 @@ <ul> <li><strong>Test code</strong>: the code that does the actual testing: interacting with the I/O transform, reading and writing data, and verifying the data.</li> <li><strong>Kubernetes scripts</strong>: a Kubernetes script that sets up the data store that will be used by the test code.</li> - <li><strong>Integrate with PerfKit Benchmarker using io-it-suite</strong>: this allows users to easily invoke PerfKit Benchmarker, creating the Kubernetes resources and running the test code.</li> + <li><strong>Integrate with PerfKit Benchmarker</strong>: this allows users to easily invoke PerfKit Benchmarker, creating the Kubernetes resources and running the test code.</li> </ul> <p>These three pieces are discussed in detail below.</p> @@ -564,13 +570,7 @@ <h4 id="integrate-with-perfkit-benchmarker">Integrate with PerfKit Benchmarker</h4> -<p>To allow developers to easily invoke your I/O integration test, you must perform these two steps. The follow sections describe each step in more detail.</p> -<ol> - <li>Create a PerfKit Benchmarker benchmark configuration file for the data store. Each pipeline option needed by the integration test should have a configuration entry.</li> - <li>Modify the per-I/O Maven pom configuration so that PerfKit Benchmarker can be invoked from Maven.</li> -</ol> - -<p>The goal is that a checked in config has defaults such that other developers can run the test without changing the configuration.</p> +<p>To allow developers to easily invoke your I/O integration test, you should create a PerfKit Benchmarker benchmark configuration file for the data store. Each pipeline option needed by the integration test should have a configuration entry. This is to be passed to perfkit via “beamITOptions” option in “performanceTest” task (described above). The goal is that a checked in config has defaults such that other developers can run the test without changing the configuration.</p> <h4 id="defining-the-benchmark-configuration-file">Defining the benchmark configuration file</h4> @@ -657,19 +657,19 @@ dynamic_pipeline_options: <tr> <td>static_pipeline_options </td> - <td>The set of preconfigured mvn pipeline options. + <td>The set of preconfigured pipeline options. </td> </tr> <tr> <td>dynamic_pipeline_options </td> - <td>The set of mvn pipeline options that PerfKit Benchmarker will determine at runtime. + <td>The set of pipeline options that PerfKit Benchmarker will determine at runtime. </td> </tr> <tr> <td>dynamic_pipeline_options.name </td> - <td>The name of the parameter to be passed to mvn's invocation of the I/O integration test. + <td>The name of the parameter to be passed to gradle's invocation of the I/O integration test. </td> </tr> <tr> @@ -687,125 +687,127 @@ dynamic_pipeline_options: </tbody> </table> -<h4 id="per-i-o-mvn-pom-configuration">Per-I/O mvn pom configuration</h4> +<h4 id="customizing-perf-kit-benchmarker-behaviour">Customizing PerfKit Benchmarker behaviour</h4> -<p>Each I/O is responsible for adding a section to its pom with a profile that invokes PerfKit Benchmarker with the proper parameters during the verify phase. Below are the set of PerfKit Benchmarker parameters and how to configure them.</p> - -<p>The <a href="https://github.com/apache/beam/blob/master/sdks/java/io/jdbc/pom.xml";>JdbcIO pom</a> has an example of how to put these options together into a profile and invoke Python+PerfKit Benchmarker with them.</p> +<p>In most cases, to run the <em>performanceTest</em> task it is sufficient to pass the properties described above, which makes it easy to use. However, users can customize Perfkit Benchmarker’s behavior even more by pasing some extra Gradle properties:</p> <table class="table"> <thead> <tr> <td><strong>PerfKit Benchmarker Parameter</strong> </td> - <td><strong>Description</strong> + <td><strong>Corresponding Gradle property</strong> </td> - <td><strong>Example value</strong> + <td><strong>Default value</strong> + </td> + <td><strong>Description</strong> </td> </tr> </thead> <tbody> <tr> - <td>benchmarks + <td>dpb_log_level </td> - <td>Defines the PerfKit Benchmarker benchmark to run. This is same for all I/O integration tests. + <td>-DlogLevel </td> - <td>beam_integration_benchmark + <td>INFO </td> - </tr> - <tr> - <td>beam_location - </td> - <td>The location where PerfKit Benchmarker can find the Beam repository. - </td> - <td>${beamRootProjectDir} - this is a variable you'll need to define for each maven pom. See example pom for an example. + <td>Data Processing Backend's log level. </td> </tr> <tr> - <td>beam_prebuilt + <td>gradle_binary </td> - <td>Whether or not to rebuild the Beam repository before invoking the I/O integration test command. + <td>-DgradleBinary </td> - <td>true + <td>./gradlew </td> - </tr> - <tr> - <td>beam_sdk - </td> - <td>Whether PerfKit Benchmarker will run the Beam SDK for Java or Python. - </td> - <td>java + <td>Path to gradle binary. </td> </tr> <tr> - <td>beam_runner_profile + <td>official + </td> + <td>-Dofficial </td> - <td>Optional command line parameter used to override the runner, allowing us to use the direct runner. + <td>false </td> - <td>Always use the predefined variable instead of specifying this parameter ${pkbBeamRunnerProfile} + <td>If true, the benchmark results are marked as "official" and can be displayed on PerfKitExplorer dashboards. </td> </tr> <tr> - <td>beam_runner_option + <td>benchmarks </td> - <td>Optional command line parameter used to override the runner, allowing us to use the direct runner. + <td>-Dbenchmarks </td> - <td>Always use the predefined variable instead of specifying this parameter ${pkbBeamRunnerOption} + <td>beam_integration_benchmark + </td> + <td>Defines the PerfKit Benchmarker benchmark to run. This is same for all I/O integration tests. </td> </tr> <tr> - <td>beam_it_module + <td>beam_prebuilt </td> - <td>The path to the pom that contains the test (needed for invoking the test with PerfKit Benchmarker). + <td>-DbeamPrebuilt </td> - <td>sdks/java/io/jdbc + <td>true + </td> + <td>If false, PerfKit Benchmarker runs the build task before running the tests. </td> </tr> <tr> - <td>beam_it_class + <td>beam_sdk </td> - <td>The test to run. + <td>-DbeamSdk </td> - <td>org.apache.beam.sdk.io.jdbc.JdbcIOIT + <td>java + </td> + <td>Beam's sdk to be used by PerfKit Benchmarker. </td> </tr> <tr> - <td>beam_it_options + <td>beam_timeout + </td> + <td>-DitTimeout </td> - <td>Pipeline options for the beam job - meant to be a way to pass pipeline options the user specifies on the commandline when invoking io-it-suite + <td>1200 </td> - <td>Always use ${integrationTestPipelineOptions}, which allows the user to pass in parameters. + <td>Timeout (in seconds) after which PerfKit Benchmarker will stop executing the benchmark (and will fail). </td> </tr> <tr> <td>kubeconfig </td> - <td>The standard PerfKit Benchmarker parameter `kubeconfig`, which specifies where the Kubernetes config file lives. + <td>-Dkubeconfig </td> - <td>Always use ${kubeconfig} + <td>~/.kube/config + </td> + <td>Path to kubernetes configuration file. </td> </tr> <tr> <td>kubectl </td> - <td>The standard PerfKit Benchmarker parameter `kubectl`, which specifies where the kubectl binary lives. + <td>-Dkubectl + </td> + <td>kubectl </td> - <td>Always use ${kubectl} + <td>Path to kubernetes executable. </td> </tr> <tr> - <td>beam_kubernetes_scripts + <td>beam_extra_properties </td> - <td>The Kubernetes script files to create and teardown via create/delete. Specify absolute path. + <td>-DbeamExtraProperties </td> - <td>${beamRootProjectDir}/.test-infra/kubernetes/postgres/pkb-config.yml + <td>(empty string) + </td> + <td>Any additional properties to be appended to benchmark execution command. </td> </tr> </tbody> </table> -<p>There is also a set of Maven properties which are useful when invoking PerfKit Benchmarker. These properties are configured in the I/O parent pom, and some are only available when the io-it-suite profile is active in Maven.</p> - <h4 id="small-scale-and-large-scale-integration-tests">Small Scale and Large Scale Integration Tests</h4> <p>Apache Beam expects that it can run integration tests in multiple configurations:</p> -- To stop receiving notification emails like this one, please contact [email protected].
