[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90340&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90340
]
ASF GitHub Bot logged work on BEAM-3942:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Apr/18 09:09
Start Date: 12/Apr/18 09:09
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181013431
##########
File path: build_rules.gradle
##########
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// When applied in a module's build.gradle file, this closure provides set of
tasks
+// needed to run IOIT integration tests (manually, without perfkit).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+ def runner = configuration.runner
+ dependencies {
+ if (runner.contains('dataflow')) {
+ testCompile project(path: ":runners:google-cloud-dataflow-java",
configuration: 'shadowTest')
+ }
+
+ if (runner.contains('direct')) {
+ testCompile project(path: ":runners:direct-java", configuration:
'shadowTest')
+ }
+ }
+ }
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
Review comment:
I wouldn't call this a requirement - rather a good gradle practice to handle
things this way.
For invoking the integration test maven used the Failsafe plugin. It was
designed in a way to look for all test classes that name ends with "IT" and
execute them. On the contrary unit tests were run using Surefire plugin. They
both "connected" to mavens "verify" phase and this was a problem in maven:
normally when someone typed `mvn verify` all plugins were fired. This is why we
needed to "skipITs" or skip unit tests depending on which kind of tests we were
going to execute and introduce profiles like "io-it" for this job:
https://github.com/apache/beam/blob/b8cf07dc03643ded1d4e9d45caf4151316cf2c3e/sdks/java/io/pom.xml#L104
Gradle is built totally differently there and has no phases. It operates on
independent tasks each representing "an atomic piece of work". This is IMO a
perfect fit for our case: we define a Test type task called `integrationTest`
or `performanceTest` and invoke it on demand, not worrying that unit tests will
fire along with integration tests. Thanks to that we do not have to define a
whole series of additional profiles. It's sufficient to have just two tasks
defined in one place.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 90340)
Time Spent: 8h 50m (was: 8h 40m)
> Update performance testing framework to use Gradle.
> ---------------------------------------------------
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Chamikara Jayalath
> Assignee: Łukasz Gajowy
> Priority: Major
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
