[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91568&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91568 ]
ASF GitHub Bot logged work on BEAM-3942: ---------------------------------------- Author: ASF GitHub Bot Created on: 17/Apr/18 01:44 Start Date: 17/Apr/18 01:44 Worklog Time Spent: 10m Work Description: chamikaramj commented on a change in pull request #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#discussion_r181932094 ########## File path: build_rules.gradle ########## @@ -837,6 +837,179 @@ ext.applyJavaNature = { } } +// Reads and contains all necessary performance test parameters +class JavaPerformanceTestConfiguration { + + /* Optional properties (set only if needed in your case): */ + + // Path to PerfKitBenchmarker application (pkb.py). + // It is only required when running Performance Tests with PerfKitBenchmarker + String pkbLocation = System.getProperty('pkbLocation') + + // Data Processing Backend's log level. + String logLevel = System.getProperty('logLevel', 'INFO') + + // Path to gradle binary. + String gradleBinary = System.getProperty('gradleBinary', './gradlew') + + // If benchmark is official or not. + // Official benchmark results are meant to be displayed on PerfKitExplorer dashboards. + String isOfficial = System.getProperty('official', 'false') + + // Specifies names of benchmarks to be run by PerfKitBenchmarker. + String benchmarks = System.getProperty('benchmarks', 'beam_integration_benchmark') + + // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task before running the tests. + String beamPrebuilt = System.getProperty('beamPrebuilt', 'true') + + // Beam's sdk to be used by PerfKitBenchmarker. + String beamSdk = System.getProperty('beamSdk', 'java') + + // Timeout (in seconds) after which PerfKitBenchmarker will stop executing the benchmark (and will fail). + String timeout = System.getProperty('itTimeout', '1200') + + // Path to kubernetes configuration file. + String kubeconfig = System.getProperty('kubeconfig', System.getProperty('user.home') + '/.kube/config') + + // Path to kubernetes executable. + String kubectl = System.getProperty('kubectl', 'kubectl') + + // Paths to files with kubernetes infrastructure to setup before the test runs. + // PerfKitBenchmarker will have trouble reading 'null' path. It expects empty string if no scripts are expected. + String kubernetesScripts = System.getProperty('kubernetesScripts', '') + + // Path to file with 'dynamic' and 'static' pipeline options. + // that will be appended by PerfKitBenchmarker to the test running command. + // PerfKitBenchmarker will have trouble reading 'null' path. It expects empty string if no config file is expected. + String optionsConfigFile = System.getProperty('beamITOptions', '') + + // Any additional properties to be appended to benchmark execution command. + String extraProperties = System.getProperty('beamExtraProperties', '') + + // Runner which will be used for running the tests. Possible values: dataflow/direct. + // PerfKitBenchmarker will have trouble reading 'null' value. It expects empty string if no config file is expected. + String runner = System.getProperty('integrationTestRunner', '') + + // Filesystem which will be used for running the tests. Possible values: hdfs. + // if not specified runner's local filesystem will be used. + String filesystem = System.getProperty('filesystem') + + /* Always required properties: */ + + // Pipeline options to be used by the tested pipeline. + String integrationTestPipelineOptions = System.getProperty('integrationTestPipelineOptions') + + // Fully qualified name of the test to be run, eg: + // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'. + String integrationTest = System.getProperty('integrationTest') + + // Relative path to module where the test is, eg. 'sdks/java/io/jdbc. + String itModule = System.getProperty('itModule') +} + +// When applied in a module's build.gradle file, this closure provides task for running +// IO integration tests (manually, without PerfKitBenchmarker). +ext.enableJavaPerformanceTesting = { + println "enableJavaPerformanceTesting with ${it ? "$it" : "default configuration"} for project ${project.name}" + + // Use the implicit it parameter of the closure to handle zero argument or one argument map calls. + // See: http://groovy-lang.org/closures.html#implicit-it + JavaPerformanceTestConfiguration configuration = it ? it as JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration() + + // Task for running integration tests + task integrationTest(type: Test) { + include "**/*IT.class" + systemProperties.beamTestPipelineOptions = configuration.integrationTestPipelineOptions + } +} + +// When applied in a module's build.gradle file, this closure adds task providing +// additional dependencies that might be needed while running integration tests. +ext.provideIntegrationTestingDependencies = { + println "provideIntegrationTestingDependencies with ${it ? "$it" : "default configuration"} for project ${project.name}" + + // Use the implicit it parameter of the closure to handle zero argument or one argument map calls. + // See: http://groovy-lang.org/closures.html#implicit-it + JavaPerformanceTestConfiguration configuration = it ? it as JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration() + + // Add runners needed to run integration tests on + task packageIntegrationTests(type: Jar) { + def runner = configuration.runner + def filesystem = configuration.filesystem + + dependencies { + + /* include dependencies required by runners */ + if (runner?.contains('dataflow')) { + testCompile project(path: ":beam-runners-google-cloud-dataflow-java", configuration: 'shadowTest') + } + + if (runner?.contains('direct')) { + testCompile project(path: ":beam-runners-direct-java", configuration: 'shadowTest') + } + + /* include dependencies required by filesystems */ + if (filesystem?.contains('hdfs')) { + testCompile project(path: ":beam-sdks-java-io-hadoop-file-system", configuration: 'shadowTest') + shadowTest library.java.hadoop_client + } + } + } +} + +// When applied in a module's build gradle file, this closure provides a task +// that will involve PerfKitBenchmarker for running integrationTests. +ext.createPerformanceTestHarness = { + println "createPerformanceTestHarness with ${it ? "$it" : "default configuration"} for project ${project.name}" + + // Use the implicit it parameter of the closure to handle zero argument or one argument map calls. + // See: http://groovy-lang.org/closures.html#implicit-it + JavaPerformanceTestConfiguration configuration = it ? it as JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration() + + // This task runs PerfKitBenchmarker, which does benchmarking of the IO ITs. + // The arguments passed to it allows it to invoke gradle again with the desired benchmark. + // + // To invoke this, run: + // + // ./gradlew performanceTest \ + // -DpkbLocation="<path to pkb.py>" + // -DintegrationTestPipelineOptions='["--numberOfRecords=1000", "<more options>"]' \ + // -DintegrationTest=<io test, eg. org.apache.beam.sdk.io.text.TextIOIT> \ + // -DitModule=<directory containing desired test, eg. sdks/java/io/file-based-io-tests> \ + // -DintegrationTestRunner=direct/dataflow/<other> Review comment: <!--new_thread; commit:3165e6e7301ddfb1565b1de36397bfac474d253b; resolved:0--> What's the value in providing multiple values for 'runner' parameter. Don't we always run a single execution using a single runner ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 91568) Time Spent: 10h 50m (was: 10h 40m) > Update performance testing framework to use Gradle. > --------------------------------------------------- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing > Reporter: Chamikara Jayalath > Assignee: Ćukasz Gajowy > Priority: Major > Time Spent: 10h 50m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)