A new script is proposed as part of Python integration testing to improve following aspects: - Support integration tests with Jenkins default configuration and customized configurations. - Many command line flags are provided to support more use cases. - Easy to integrate to Gradle. - Easy to add the support of running validates runner and validates container tests.
Based on the new script, I created a Gradle task `integrationTest` which hopefully can be an ideal entry point to run any integration test. This task automatically setup the vertualenv, generate tar ball and run `run_integration_test.sh` with provided args. By adding `integrationTest` task, Perfkit benchmark can use it to run integration tests with simpler setup. We can also get rid of [Beam virtualenv setup on Jenkins](https://github.com/apache/beam/blob/master/.test-infra/jenkins/CommonJobProperties.groovy#L343-L351) which currently fails [Python benchmark job](https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_Python/1433/). Corresponding changes to Perfkit is available [here](https://github.com/markflyhigh/PerfKitBenchmarker/pull/2/files). Command built from Perfkit could be as simple as: ``` ./gradlew beam-sdks-python:integrationTest \ -Pattr=IT \ -Ptests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it \ -PpipelineOptions=<pipeline options> ``` Future work includes move rest of features in `run_postcommits.sh` to `run_integration_test.sh`, use `integrationTest` task in Jenkins Postcommit and performance jobs. ------------------------ Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) ------------------------------------------------------------------------------------------------ Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) </br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | --- | --- | --- | --- [ Full content available at: https://github.com/apache/beam/pull/6392 ] This message was relayed via gitbox.apache.org for [email protected]
