[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=117281=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-117281
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 29/Jun/18 08:37
Start Date: 29/Jun/18 08:37
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5812:
[BEAM-3942] Remove mvn leftovers from jenkins IOIT job definitions
URL: https://github.com/apache/beam/pull/5812#discussion_r199091638
##
File path: .test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
##
@@ -129,16 +129,15 @@ private void
create_filebasedio_performance_test_job(testConfiguration) {
}
def argMap = [
-benchmarks : 'beam_integration_benchmark',
-beam_it_timeout : '1200',
-beam_it_profile : 'io-it',
-beam_prebuilt: 'false',
-beam_sdk : 'java',
-beam_it_module : 'sdks/java/io/file-based-io-tests',
-beam_it_class: testConfiguration.itClass,
-beam_it_options :
common_job_properties.joinPipelineOptions(pipelineOptions),
-beam_extra_mvn_properties: '["filesystem=gcs"]',
-bigquery_table : testConfiguration.bqTable,
+benchmarks : 'beam_integration_benchmark',
Review comment:
It is automatic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 117281)
Time Spent: 12h 50m (was: 12h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Fix For: 2.5.0
>
> Time Spent: 12h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=117248=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-117248
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 29/Jun/18 07:31
Start Date: 29/Jun/18 07:31
Worklog Time Spent: 10m
Work Description: iemejia closed pull request #5812: [BEAM-3942] Remove
mvn leftovers from jenkins IOIT job definitions
URL: https://github.com/apache/beam/pull/5812
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.test-infra/jenkins/common_job_properties.groovy
b/.test-infra/jenkins/common_job_properties.groovy
index 5d9dad63e78..b30f0485c4c 100644
--- a/.test-infra/jenkins/common_job_properties.groovy
+++ b/.test-infra/jenkins/common_job_properties.groovy
@@ -265,7 +265,6 @@ class common_job_properties {
LinkedHashMap standardArgs = [
project: 'apache-beam-testing',
dpb_log_level: 'INFO',
- maven_binary: '/home/jenkins/tools/maven/latest/bin/mvn',
bigquery_table: 'beam_performance.pkb_results',
k8s_get_retry_count: 36, // wait up to 6 minutes for K8s LoadBalancer
k8s_get_wait_interval: 10,
diff --git a/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
b/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
index 5d95be3f51b..c9aca0b82f8 100644
--- a/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
+++ b/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
@@ -129,16 +129,15 @@ private void
create_filebasedio_performance_test_job(testConfiguration) {
}
def argMap = [
-benchmarks : 'beam_integration_benchmark',
-beam_it_timeout : '1200',
-beam_it_profile : 'io-it',
-beam_prebuilt: 'false',
-beam_sdk : 'java',
-beam_it_module : 'sdks/java/io/file-based-io-tests',
-beam_it_class: testConfiguration.itClass,
-beam_it_options :
common_job_properties.joinPipelineOptions(pipelineOptions),
-beam_extra_mvn_properties: '["filesystem=gcs"]',
-bigquery_table : testConfiguration.bqTable,
+benchmarks : 'beam_integration_benchmark',
+beam_it_timeout : '1200',
+beam_prebuilt: 'false',
+beam_sdk : 'java',
+beam_it_module : 'sdks/java/io/file-based-io-tests',
+beam_it_class: testConfiguration.itClass,
+beam_it_options :
common_job_properties.joinPipelineOptions(pipelineOptions),
+beam_extra_properties: '["filesystem=gcs"]',
+bigquery_table : testConfiguration.bqTable,
]
common_job_properties.buildPerformanceTest(delegate, argMap)
}
diff --git
a/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT_HDFS.groovy
b/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT_HDFS.groovy
index 26e65202d43..2cf331144ef 100644
--- a/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT_HDFS.groovy
+++ b/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT_HDFS.groovy
@@ -139,19 +139,18 @@ private void
create_filebasedio_performance_test_job(testConfiguration) {
String kubeconfig =
common_job_properties.getKubeconfigLocationForNamespace(namespace)
def argMap = [
-kubeconfig : kubeconfig,
-benchmarks : 'beam_integration_benchmark',
-beam_it_timeout : '1200',
-beam_it_profile : 'io-it',
-beam_prebuilt: 'false',
-beam_sdk : 'java',
-beam_it_module : 'sdks/java/io/file-based-io-tests',
-beam_it_class: testConfiguration.itClass,
-beam_it_options : pipelineArgsJoined,
-beam_extra_mvn_properties: '["filesystem=hdfs"]',
-bigquery_table : testConfiguration.bqTable,
-beam_options_config_file : makePathAbsolute('pkb-config.yml'),
-beam_kubernetes_scripts :
makePathAbsolute('hdfs-multi-datanode-cluster.yml')
+kubeconfig : kubeconfig,
+benchmarks : 'beam_integration_benchmark',
+beam_it_timeout : '1200',
+beam_prebuilt : 'false',
+beam_sdk
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=117247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-117247
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 29/Jun/18 07:30
Start Date: 29/Jun/18 07:30
Worklog Time Spent: 10m
Work Description: iemejia commented on a change in pull request #5812:
[BEAM-3942] Remove mvn leftovers from jenkins IOIT job definitions
URL: https://github.com/apache/beam/pull/5812#discussion_r199075858
##
File path: .test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy
##
@@ -129,16 +129,15 @@ private void
create_filebasedio_performance_test_job(testConfiguration) {
}
def argMap = [
-benchmarks : 'beam_integration_benchmark',
-beam_it_timeout : '1200',
-beam_it_profile : 'io-it',
-beam_prebuilt: 'false',
-beam_sdk : 'java',
-beam_it_module : 'sdks/java/io/file-based-io-tests',
-beam_it_class: testConfiguration.itClass,
-beam_it_options :
common_job_properties.joinPipelineOptions(pipelineOptions),
-beam_extra_mvn_properties: '["filesystem=gcs"]',
-bigquery_table : testConfiguration.bqTable,
+benchmarks : 'beam_integration_benchmark',
Review comment:
Minor question, is this reformatting automatic? I kind of dislike having
everything aligned to colons but it seems this is common in groovy.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 117247)
Time Spent: 12.5h (was: 12h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Fix For: 2.5.0
>
> Time Spent: 12.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91858=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91858 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 17/Apr/18 18:06 Start Date: 17/Apr/18 18:06 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-382088136 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 91858) Time Spent: 12h (was: 11h 50m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 12h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91859=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91859 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 17/Apr/18 18:06 Start Date: 17/Apr/18 18:06 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-382088178 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 91859) Time Spent: 12h 10m (was: 12h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 12h 10m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91765=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91765 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 17/Apr/18 14:28 Start Date: 17/Apr/18 14:28 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-382013189 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 91765) Time Spent: 11h 50m (was: 11h 40m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 11h 50m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91738=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91738 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 17/Apr/18 13:11 Start Date: 17/Apr/18 13:11 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-381986805 Done. Rebased to master and divided into two commits. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 91738) Time Spent: 11h 40m (was: 11.5h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 11h 40m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91708
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 17/Apr/18 10:28
Start Date: 17/Apr/18 10:28
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r182025407
##
File path: build_rules.gradle
##
@@ -837,6 +837,179 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests. Possible values:
dataflow/direct.
+ // PerfKitBenchmarker will have trouble reading 'null' value. It expects
empty string if no config file is expected.
+ String runner = System.getProperty('integrationTestRunner', '')
+
+ // Filesystem which will be used for running the tests. Possible values:
hdfs.
+ // if not specified runner's local filesystem will be used.
+ String filesystem = System.getProperty('filesystem')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+}
+
+// When applied in a module's build.gradle file, this closure provides task
for running
+// IO integration tests (manually, without PerfKitBenchmarker).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+// When applied in a module's build.gradle file, this
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91707
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 17/Apr/18 10:28
Start Date: 17/Apr/18 10:28
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r182025402
##
File path: build_rules.gradle
##
@@ -837,6 +837,179 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests. Possible values:
dataflow/direct.
+ // PerfKitBenchmarker will have trouble reading 'null' value. It expects
empty string if no config file is expected.
+ String runner = System.getProperty('integrationTestRunner', '')
+
+ // Filesystem which will be used for running the tests. Possible values:
hdfs.
+ // if not specified runner's local filesystem will be used.
+ String filesystem = System.getProperty('filesystem')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+}
+
+// When applied in a module's build.gradle file, this closure provides task
for running
+// IO integration tests (manually, without PerfKitBenchmarker).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+// When applied in a module's build.gradle file, this
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91706=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91706
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 17/Apr/18 10:27
Start Date: 17/Apr/18 10:27
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r182025388
##
File path: build_rules.gradle
##
@@ -837,6 +837,179 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests. Possible values:
dataflow/direct.
+ // PerfKitBenchmarker will have trouble reading 'null' value. It expects
empty string if no config file is expected.
+ String runner = System.getProperty('integrationTestRunner', '')
+
+ // Filesystem which will be used for running the tests. Possible values:
hdfs.
+ // if not specified runner's local filesystem will be used.
+ String filesystem = System.getProperty('filesystem')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+}
+
+// When applied in a module's build.gradle file, this closure provides task
for running
+// IO integration tests (manually, without PerfKitBenchmarker).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+// When applied in a module's build.gradle file, this
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91568
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 17/Apr/18 01:44
Start Date: 17/Apr/18 01:44
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181932094
##
File path: build_rules.gradle
##
@@ -837,6 +837,179 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests. Possible values:
dataflow/direct.
+ // PerfKitBenchmarker will have trouble reading 'null' value. It expects
empty string if no config file is expected.
+ String runner = System.getProperty('integrationTestRunner', '')
+
+ // Filesystem which will be used for running the tests. Possible values:
hdfs.
+ // if not specified runner's local filesystem will be used.
+ String filesystem = System.getProperty('filesystem')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+}
+
+// When applied in a module's build.gradle file, this closure provides task
for running
+// IO integration tests (manually, without PerfKitBenchmarker).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+// When applied in a module's build.gradle file,
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91569=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91569
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 17/Apr/18 01:44
Start Date: 17/Apr/18 01:44
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181932097
##
File path: build_rules.gradle
##
@@ -837,6 +837,179 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests. Possible values:
dataflow/direct.
+ // PerfKitBenchmarker will have trouble reading 'null' value. It expects
empty string if no config file is expected.
+ String runner = System.getProperty('integrationTestRunner', '')
+
+ // Filesystem which will be used for running the tests. Possible values:
hdfs.
+ // if not specified runner's local filesystem will be used.
+ String filesystem = System.getProperty('filesystem')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+}
+
+// When applied in a module's build.gradle file, this closure provides task
for running
+// IO integration tests (manually, without PerfKitBenchmarker).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+// When applied in a module's build.gradle file,
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=91340=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-91340 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 16/Apr/18 13:39 Start Date: 16/Apr/18 13:39 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-381602540 Things that changed (see the commit descriptions for more details): 1. Filesystem support was added (crucial for recently added hdfs IOITs) 2. The direct runner dependency was deleted from file-based-io-tests module (it is added either way by "packageIntegrationTests" task and caused problems (couldn't compile) 3. Moved _integrationTest_ and _packageIntegrationTests_ tasks to different closures to make integration test configuration more flexible and avoid cyclic dependencies in some cases. Due to some problems i had after merging master recently to this branch, I also made sure that all configurations needed by current Jenkins jobs work. I also updated the "[errata](https://docs.google.com/document/d/1CJZURnqCabc_GA-qgC3JYKQdhh00T2l5FzFS0DF12wU/edit)" document. Could you guys take a look again? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 91340) Time Spent: 10h 40m (was: 10.5h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 10h 40m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90856=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90856 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 13/Apr/18 15:52 Start Date: 13/Apr/18 15:52 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-381179356 Update: I was able to successfully run tests on hdfs filesystem too using "integrationTest" task but there are several issues due to previous merge commit that prevent me from running the "performanceTest" task. Working on demystifying and solving the issue. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90856) Time Spent: 10.5h (was: 10h 20m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 10.5h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90855=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90855 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 13/Apr/18 15:52 Start Date: 13/Apr/18 15:52 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-381179356 Update: I was able to successfully run tests on hdfs filesystem too using "integrationTest" task but there are several issues due to previous merge commit that prevent me from running the performanceTest task. Working on demystifying and solving the issue. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90855) Time Spent: 10h 20m (was: 10h 10m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 10h 20m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90443=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90443
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 15:22
Start Date: 12/Apr/18 15:22
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181119759
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. This property would then be always
required (same as "integrationTestPipelineOptions"). WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90443)
Time Spent: 10h 10m (was: 10h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90442=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90442
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 15:18
Start Date: 12/Apr/18 15:18
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181119759
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. This property would then be always
required (same as "integrationTestPipelineOptions"). WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90442)
Time Spent: 10h (was: 9h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 10h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90440=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90440
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 15:17
Start Date: 12/Apr/18 15:17
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181119759
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. This property would then be "always
required" WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90440)
Time Spent: 9h 40m (was: 9.5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90441=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90441
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 15:17
Start Date: 12/Apr/18 15:17
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181093624
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. This property would then be "always
required" WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90441)
Time Spent: 9h 50m (was: 9h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90422=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90422 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 12/Apr/18 14:18 Start Date: 12/Apr/18 14:18 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-380821527 Please don't merge this yet. We already started running tests on hdfs filesystem on Jenkins while this PR was being rewiewed. Therefore, support for filesystems needs to be done to be able to run all current Jenkins performance tests. Sorry for not noticing this earlier. I'll prepare a fix shortly. It will be very similar to runner support: dependencies needed by certain filesystems (eg. hdfs) can be included conditionally based on a "filesystem" property. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 90422) Time Spent: 9.5h (was: 9h 20m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 9.5h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90418=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90418
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 14:05
Start Date: 12/Apr/18 14:05
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181093624
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ /* Always required properties: */
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. This property would then be "always
required" WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90418)
Time Spent: 9h 20m (was: 9h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90417=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90417
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 14:04
Start Date: 12/Apr/18 14:04
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181092827
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90417)
Time Spent: 9h 10m (was: 9h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90416=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90416
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 14:03
Start Date: 12/Apr/18 14:03
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181092827
##
File path: build_rules.gradle
##
@@ -837,6 +837,154 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ /* Optional properties (set only if needed in your case): */
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ // It is only required when running Performance Tests with PerfKitBenchmarker
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by PerfKitBenchmarker.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt" then PerfKitBenchmarker runs the build task
before running the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by PerfKitBenchmarker.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKitBenchmarker will stop executing
the benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by PerfKitBenchmarker to the test running command.
+ // PerfKitBenchmarker will have trouble reading 'null' path. It expects
empty string if no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
Review comment:
I think, unfortunately I've made a mistake here: we shouldn't set default
value of this property. This will result in adding direct runner as a
dependency everytime building is run (even if we are not running the
integrationTest task). Sorry for that!
A solution would be to not set the default value - the dependency won't be
included when there is no runner to attach. WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90416)
Time Spent: 9h (was: 8h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 9h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90339=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90339
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 09:08
Start Date: 12/Apr/18 09:08
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181013383
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90339)
Time Spent: 8h 40m (was: 8.5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90338=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90338
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 09:08
Start Date: 12/Apr/18 09:08
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181013371
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90338)
Time Spent: 8.5h (was: 8h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 8.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90340=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90340
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 09:09
Start Date: 12/Apr/18 09:09
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181013431
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// When applied in a module's build.gradle file, this closure provides set of
tasks
+// needed to run IOIT integration tests (manually, without perfkit).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
+ testCompile project(path: ":runners:google-cloud-dataflow-java",
configuration: 'shadowTest')
+}
+
+if (runner.contains('direct')) {
+ testCompile project(path: ":runners:direct-java", configuration:
'shadowTest')
+}
+}
+ }
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
Review comment:
I wouldn't call this a requirement - rather a good gradle practice to handle
things this way.
For invoking the integration test maven used the Failsafe plugin. It was
designed in a way to look for all test classes that name ends with "IT" and
execute them. On the contrary unit tests were
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90337=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90337
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 09:08
Start Date: 12/Apr/18 09:08
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r181013356
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// When applied in a module's build.gradle file, this closure provides set of
tasks
+// needed to run IOIT integration tests (manually, without perfkit).
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90337)
Time Spent: 8h 20m (was: 8h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90248=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90248
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 01:08
Start Date: 12/Apr/18 01:08
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180940673
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
Review comment:
Nit: end quote after prebuilt.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90248)
Time Spent: 7h 40m (was: 7.5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90246
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 01:08
Start Date: 12/Apr/18 01:08
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180940672
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
Review comment:
Can we group required and optional properties separately ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90246)
Time Spent: 7h 20m (was: 7h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90249=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90249
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 01:08
Start Date: 12/Apr/18 01:08
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180940674
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// When applied in a module's build.gradle file, this closure provides set of
tasks
+// needed to run IOIT integration tests (manually, without perfkit).
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ // See: http://groovy-lang.org/closures.html#implicit-it
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
+ testCompile project(path: ":runners:google-cloud-dataflow-java",
configuration: 'shadowTest')
+}
+
+if (runner.contains('direct')) {
+ testCompile project(path: ":runners:direct-java", configuration:
'shadowTest')
+}
+}
+ }
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
Review comment:
Why do we have to define a new task for integration tests ? I think for
Maven we used a standard task (verify). Is this a requirement for Gradle ?
This is an automated message from the Apache Git Service.
To respond to the
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90247
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 01:08
Start Date: 12/Apr/18 01:08
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180940671
##
File path: build_rules.gradle
##
@@ -593,6 +593,149 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+
+ // Path to PerfKitBenchmarker application (pkb.py).
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ // Data Processing Backend's log level.
+ String logLevel = System.getProperty('logLevel', 'INFO')
+
+ // Path to gradle binary.
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+
+ // If benchmark is official or not.
+ // Official benchmark results are meant to be displayed on PerfKitExplorer
dashboards.
+ String isOfficial = System.getProperty('official', 'false')
+
+ // Specifies names of benchmarks to be run by perfkit.
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ // If beam is not "prebuilt then perfkit runs the build task before running
the tests.
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+
+ // Beam's sdk to be used by perfkit.
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ // Timeout (in seconds) after which PerfKit will stop executing the
benchmark (and will fail).
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ // Path to kubernetes configuration file.
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+
+ // Path to kubernetes executable.
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // Paths to files with kubernetes infrastructure to setup before the test
runs.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ // Pipeline options to be used by the tested pipeline.
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // Path to file with 'dynamic' and 'static' pipeline options.
+ // that will be appended by perfkit to the test running command.
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ // Fully qualified name of the test to be run, eg:
+ // 'org.apache.beam.sdks.java.io.jdbc.JdbcIOIT'.
+ String integrationTest = System.getProperty('integrationTest')
+
+ // Relative path to module where the test is, eg. 'sdks/java/io/jdbc.
+ String itModule = System.getProperty('itModule')
+
+ // Runner which will be used for running the tests.
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ // Any additional properties to be appended to benchmark execution command.
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// When applied in a module's build.gradle file, this closure provides set of
tasks
+// needed to run IOIT integration tests (manually, without perfkit).
Review comment:
s/perfkit/PerfKitBenchmarker (here and elsewhere).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90247)
Time Spent: 7.5h (was: 7h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90251=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90251
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 01:08
Start Date: 12/Apr/18 01:08
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180940679
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
Review comment:
> **lgajowy** wrote:
> This line effectively means that we should include the runner dependencies
only when the `integrationTest` task is run. So when user runs `./gradlew
integrationTest...` those dependencies are added while building the jar.
>
> Now I see that this is not enough due the following reason: there is no
way to "prebuild" tests - we cannot build with a separate build command and
then just run the tests. Such prebuilding is needed by perfkit so it's a
necessity I forgot to take into account here. "Prebuilding" is invoked when we
specify the "beam_prebuilt" flag to false, see more in [Perfkit's
code](https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/blob/a3e6c0382809cb68cd294f34cbd5b3b558e53f56/perfkitbenchmarker/beam_benchmark_helper.py#L147).
>
> I think we should remove this condition and include the runner dependency
only if the command is run with appropriate "integrationTestRunner" property .
I changed this in the next commit.
> So now we can prebuild in Perfkit using this command:
> `./gradlew clean build -xtest` -DintegrationTestRunner=dataflow
> and run the tests using `integrationTest` task, for example like this:
>
> and execute the tests using this one:
> `./gradlew clean integrationTest -p sdks/java/io/file-based-io-tests/
-DintegrationTestPipelineOptions='[]' --tests
org.apache.beam.sdk.io.text.TextIOIT -DintegrationTestRunner=dataflow`
>
> WDYT?
Makes sense. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90251)
Time Spent: 8h 10m (was: 8h)
> Update performance testing framework to use Gradle.
> ---
>
>
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=90250=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90250
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 12/Apr/18 01:08
Start Date: 12/Apr/18 01:08
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180940678
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
Review comment:
> **lgajowy** wrote:
> I think we can fix this on the perfkit side but I rather tend to think it
should be done in separate PR. I wanted to change the least possible amount of
code in perfkit to integrate with gradle and then improve. Do you agree with me?
Sounds good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 90250)
Time Spent: 8h (was: 7h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 8h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89466=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89466
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 16:32
Start Date: 10/Apr/18 16:32
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180486735
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
Review comment:
This is fine. I was asking about, if there is a more scalable way of adding
new runner dependencies here in the future. (As opposed to bunch if this runner
... lines.)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89466)
Time Spent: 7h (was: 6h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 7h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89467=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89467
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 16:33
Start Date: 10/Apr/18 16:33
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180487218
##
File path: sdks/java/io/cassandra/build.gradle
##
@@ -18,6 +18,7 @@
apply from: project(":").file("build_rules.gradle")
applyJavaNature(artifactId: "beam-sdks-java-io-cassandra")
+enableJavaPerformanceTesting()
Review comment:
The current approach seems better. Thank you for the explanation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89467)
Time Spent: 7h 10m (was: 7h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89435=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89435 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 10/Apr/18 14:48 Start Date: 10/Apr/18 14:48 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-380127156 @aaltay @chamikaramj Thank you guys for all the comments. I responded to them and created another commit with changes, mostly in documentation to ease your effort and make things more clear. @chamikaramj There already is a documentation page describing how [the framework](https://beam.apache.org/documentation/io/testing/#i-o-transform-integration-tests) works and it won't change much IMO. Part of the gradle migration task is to update it to use Gradle commands. So clear description will be available to community soon. Meanwhile I created a [Performance testing framework documentation errata](https://docs.google.com/document/d/1CJZURnqCabc_GA-qgC3JYKQdhh00T2l5FzFS0DF12wU/edit?usp=sharing) that provides updated commands and (possibly in the future) more things to be changed in the docs This will also make getting the documentation up to date easier. Of course feel free to add comments and suggestions. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89435) Time Spent: 6h 50m (was: 6h 40m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 6h 50m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89431=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89431 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 10/Apr/18 14:46 Start Date: 10/Apr/18 14:46 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-380127156 Thank you guys for all the comments. I responded to them and created another commit with changes, mostly in documentation to ease your effort and make things more clear. @chamikaramj There already is a documentation page describing how [the framework](https://beam.apache.org/documentation/io/testing/#i-o-transform-integration-tests) works and it won't change much IMO. Part of the gradle migration task is to update it to Gradle. So clear description will be available soon.Meanwhile I created a [Performance testing framework documentation errata](https://docs.google.com/document/d/1CJZURnqCabc_GA-qgC3JYKQdhh00T2l5FzFS0DF12wU/edit?usp=sharing) that provides updated commands and (possibly in the future) more things to be changed in the docs This will also make getting the documentation up to date easier. Of course feel free to add comments and suggestions. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89431) Time Spent: 6h 20m (was: 6h 10m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 6h 20m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89433=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89433 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 10/Apr/18 14:46 Start Date: 10/Apr/18 14:46 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-380127156 Thank you guys for all the comments. I responded to them and created another commit with changes, mostly in documentation to ease your effort and make things more clear. @chamikaramj There already is a documentation page describing how [the framework](https://beam.apache.org/documentation/io/testing/#i-o-transform-integration-tests) works and it won't change much IMO. Part of the gradle migration task is to update it to use Gradle commands. So clear description will be available to community soon.Meanwhile I created a [Performance testing framework documentation errata](https://docs.google.com/document/d/1CJZURnqCabc_GA-qgC3JYKQdhh00T2l5FzFS0DF12wU/edit?usp=sharing) that provides updated commands and (possibly in the future) more things to be changed in the docs This will also make getting the documentation up to date easier. Of course feel free to add comments and suggestions. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89433) Time Spent: 6.5h (was: 6h 20m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 6.5h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89434=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89434 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 10/Apr/18 14:46 Start Date: 10/Apr/18 14:46 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-380127156 Thank you guys for all the comments. I responded to them and created another commit with changes, mostly in documentation to ease your effort and make things more clear. @chamikaramj There already is a documentation page describing how [the framework](https://beam.apache.org/documentation/io/testing/#i-o-transform-integration-tests) works and it won't change much IMO. Part of the gradle migration task is to update it to use Gradle commands. So clear description will be available to community soon. Meanwhile I created a [Performance testing framework documentation errata](https://docs.google.com/document/d/1CJZURnqCabc_GA-qgC3JYKQdhh00T2l5FzFS0DF12wU/edit?usp=sharing) that provides updated commands and (possibly in the future) more things to be changed in the docs This will also make getting the documentation up to date easier. Of course feel free to add comments and suggestions. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89434) Time Spent: 6h 40m (was: 6.5h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 6h 40m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89429=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89429 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 10/Apr/18 14:44 Start Date: 10/Apr/18 14:44 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-380127156 Thank you guys for all the comments. I responded to them and created another commit with changes, mostly in documentation to ease your effort and make things more clear. @chamikaramj There already is a documentation page describing how [the framework](https://beam.apache.org/documentation/io/testing/#i-o-transform-integration-tests) works. Part of the gradle migration task is to update it to Gradle. So clear description will be available soon.Meanwhile I created a [Performance testing framework documentation errata](https://docs.google.com/document/d/1CJZURnqCabc_GA-qgC3JYKQdhh00T2l5FzFS0DF12wU/edit?usp=sharing) that provides updated commands and (possibly in the future) more things to be changed in the docs This will also make getting the documentation up to date easier. Of course feel free to add comments and suggestions. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89429) Time Spent: 6h 10m (was: 6h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 6h 10m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89428=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89428 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 10/Apr/18 14:42 Start Date: 10/Apr/18 14:42 Worklog Time Spent: 10m Work Description: lgajowy commented on a change in pull request #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#discussion_r180447086 ## File path: sdks/java/io/jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java ## @@ -47,18 +47,16 @@ * This test requires a running instance of Postgres. Pass in connection information using * PipelineOptions: * - * mvn -e -Pio-it verify -pl sdks/java/io/jdbc -DintegrationTestPipelineOptions='[ + * ./gradlew integrationTest -p sdks/java/io/jdbc -DintegrationTestPipelineOptions='[ * "--postgresServerName=1.2.3.4", * "--postgresUsername=postgres", * "--postgresDatabaseName=myfancydb", * "--postgresPassword=mypass", * "--postgresSsl=false", * "--numberOfRecords=1000" ]' + * --tests org.apache.beam.sdk.io.jdbc.JdbcIOIT * * - * If you want to run this with a runner besides directrunner, there are profiles for dataflow - * and spark in the jdbc pom. You'll want to activate those in addition to the normal test runner Review comment: Indeed, it does not - we were not able to run spark even though there was a profile for it. I added a more descriptive comment in following commit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 89428) Time Spent: 6h (was: 5h 50m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 6h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89425=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89425
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:42
Start Date: 10/Apr/18 14:42
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446905
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
Review comment:
It mimcs the maven behaviour that was there + is implemented in one generic
place (code is not duplicated all over the build configurations as it was in
maven). Because we don't have profiles in gradle, a system property was used to
decide which runner to include - this is done by user by specifying
`-DintegrationTestRunner=`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89425)
Time Spent: 5.5h (was: 5h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89426=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89426
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:42
Start Date: 10/Apr/18 14:42
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446926
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
+ testCompile project(path: ":runners:google-cloud-dataflow-java",
configuration: 'shadowTest')
+} else {
+ testCompile project(path: ":runners:direct-java", configuration:
'shadowTest')
+}
+ }
+}
+ }
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+ext.createPerformanceTestHarness = {
+ println "createPerformanceTestHarness with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89426)
Time Spent: 5h 40m (was: 5.5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89424=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89424
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:42
Start Date: 10/Apr/18 14:42
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446870
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
Review comment:
This line effectively means that we should include the runner dependencies
only when the `integrationTest` task is run. So when user runs `./gradlew
integrationTest...` those dependencies are added while building the jar.
Now I see that this is not enough due the following reason: there is no way
to "prebuild" tests - we cannot build with a separate build command and then
just run the tests. Such prebuilding is needed by perfkit so it's a necessity I
forgot to take into account here. "Prebuilding" is invoked when we specify the
"beam_prebuilt" flag to false, see more in [Perfkit's
code](https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/blob/a3e6c0382809cb68cd294f34cbd5b3b558e53f56/perfkitbenchmarker/beam_benchmark_helper.py#L147).
I think we should remove this condition and include the runner dependency
only if the command is run with appropriate "integrationTestRunner" property .
I changed this in the next commit.
So now we can prebuild in Perfkit using this command:
`./gradlew clean build -xtest` -DintegrationTestRunner=dataflow
and run the tests using `integrationTest` task, for example like this:
and execute the tests using this one:
`./gradlew clean integrationTest -p sdks/java/io/file-based-io-tests/
-DintegrationTestPipelineOptions='[]' --tests
org.apache.beam.sdk.io.text.TextIOIT -DintegrationTestRunner=dataflow`
WDYT?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89424)
Time Spent: 5h 20m (was: 5h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL:
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89419=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89419
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:41
Start Date: 10/Apr/18 14:41
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446475
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
Review comment:
I think we can fix this on the perfkit side but I rather tend to think it
should be done in separate PR. I wanted to change the least possible amount of
code in perfkit to integrate with gradle and then improve. Do you agree with
me?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89419)
Time Spent: 4.5h (was: 4h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89421=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89421
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:41
Start Date: 10/Apr/18 14:41
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446503
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
Review comment:
I will fix this doc to be more clear
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89421)
Time Spent: 4h 50m (was: 4h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89420=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89420
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:41
Start Date: 10/Apr/18 14:41
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446493
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
Review comment:
(answered above)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89420)
Time Spent: 4h 40m (was: 4.5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89422=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89422
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:41
Start Date: 10/Apr/18 14:41
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446503
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
Review comment:
I fixed this doc to be more clear
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89422)
Time Spent: 5h (was: 4h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89423=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89423
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:41
Start Date: 10/Apr/18 14:41
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446578
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
Review comment:
Provided a link that clarifies how to use the implicit 'it' parameter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89423)
Time Spent: 5h 10m (was: 5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89417=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89417
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:40
Start Date: 10/Apr/18 14:40
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446421
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89417)
Time Spent: 4h 10m (was: 4h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89415=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89415
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:40
Start Date: 10/Apr/18 14:40
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446379
##
File path: sdks/java/io/cassandra/build.gradle
##
@@ -18,6 +18,7 @@
apply from: project(":").file("build_rules.gradle")
applyJavaNature(artifactId: "beam-sdks-java-io-cassandra")
+enableJavaPerformanceTesting()
Review comment:
This adds `integrationTest` task to cassandra io's build.gradle file. It
works the same way as `applyJavaNature()` but adds integration testing related
things. The way it is done now requires developers explicitly "append" the
tasks to every module that needs ITs by adding `enableJavaPerformanceTesting()`
in build.gradle file.
An alternative would be to include this task (along with
`packageIntegrationTests` ) to `applyJavaNature()` closure. This would add the
tasks (`integrationTests`, `packageIntegrationTest` to every module that uses
`applyJavaNature()`, so it's more implicit.
What do you think? Which option do you like better?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89415)
Time Spent: 3h 50m (was: 3h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89414=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89414
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:40
Start Date: 10/Apr/18 14:40
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446342
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
Review comment:
Currently we run the performance test only on dataflow runner. No other
runner is supported, eg. due to issues like this:
https://issues.apache.org/jira/browse/BEAM-3359
https://issues.apache.org/jira/browse/BEAM-3370
https://issues.apache.org/jira/browse/BEAM-3371
We didn't try to run ITs on runners like apex or gearpump so we don't know
whether ITs run there.
This is why I decided not to add them. I think we should add new runner
dependencies here only when they are ready to run ITs. Wdyt?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89414)
Time Spent: 3h 40m (was: 3.5h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89418=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89418
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:40
Start Date: 10/Apr/18 14:40
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446437
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
Review comment:
No, the timeout here is default and can be changed in case someone wants to
set a different one. In most cases 1200 sec will be enough.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89418)
Time Spent: 4h 20m (was: 4h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89413=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89413
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:40
Start Date: 10/Apr/18 14:40
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446301
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
Review comment:
No we shouldn't - this would mark benchmark results being marked as official
which even in Perfkit is defaulted to false. The task is for running them from
local machines so those are definitely not official test results. It should be
false then - my bad.
I added a comment describing this in more detail in next commit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89413)
Time Spent: 3.5h (was: 3h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89416=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89416
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 10/Apr/18 14:40
Start Date: 10/Apr/18 14:40
Worklog Time Spent: 10m
Work Description: lgajowy commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r180446402
##
File path: build.gradle
##
@@ -22,6 +22,7 @@
// tasks.
apply from: project(":").file("build_rules.gradle")
+createPerformanceTestHarness()
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 89416)
Time Spent: 4h (was: 3h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 4h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=1=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-1 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 09/Apr/18 10:46 Start Date: 09/Apr/18 10:46 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-379711162 @aaltay @chamikaramj thank you for all your comments and suggestions! fyi, please expect my answers on tuesday (I'm a little bit unavailable today). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 1) Time Spent: 3h 20m (was: 3h 10m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 3h 20m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88675=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88675
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r178998423
##
File path: build.gradle
##
@@ -175,3 +175,74 @@ task goPreCommit() {
dependsOn ":rat"
dependsOn ":sdks:go:test"
}
+
+// This task runs PerfKitBenchmarker, which does benchmarking of the IO ITs.
+// The arguments passed to it allows it to invoke gradle again with the
desired benchmark.
+//
+// To invoke this, run:
+//
+// ./gradlew performanceTest \
+// -DpkbLocation=""
+// -DintegrationTestPipelineOptions='["--numberOfRecords=1000", ""]' \
+// -DintegrationTest= \
+// -DitModule= \
+// -DintegrationTestRunner=direct/dataflow/
Review comment:
Can you give the exact value that should be used for DataflowRunner here
(without ) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88675)
Time Spent: 2h 10m (was: 2h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88673=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88673
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905570
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
Review comment:
Why do we have to set runner-specific dependencies here? Can't this be set
by the user command that invokes the Gradle task to run the test ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88673)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88683=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88683
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905413
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
Review comment:
Do all ITs have to have to same timeout
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88683)
Time Spent: 3h 10m (was: 3h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88678=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88678
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905431
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
Review comment:
Is this something we can fix at PerfKit side (and not specify empty strings
here) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88678)
Time Spent: 2.5h (was: 2h 20m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88672=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88672
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905373
##
File path: build.gradle
##
@@ -22,6 +22,7 @@
// tasks.
apply from: project(":").file("build_rules.gradle")
+createPerformanceTestHarness()
Review comment:
Let's add a comment here explaining why we need this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88672)
Time Spent: 2h (was: 1h 50m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88680=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88680
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905587
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
+ testCompile project(path: ":runners:google-cloud-dataflow-java",
configuration: 'shadowTest')
+} else {
+ testCompile project(path: ":runners:direct-java", configuration:
'shadowTest')
+}
+ }
+}
+ }
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+ext.createPerformanceTestHarness = {
+ println "createPerformanceTestHarness with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
Review comment:
Ditto. Please explain what 'it' parameter is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88680)
Time Spent: 2h 50m (was: 2h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88676=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88676
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905477
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
Review comment:
implicit 'it' parameter.
BTW who sets this parameter ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88676)
Time Spent: 2h 20m (was: 2h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88682=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88682 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 07/Apr/18 02:15 Start Date: 07/Apr/18 02:15 Worklog Time Spent: 10m Work Description: chamikaramj commented on a change in pull request #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#discussion_r179905678 ## File path: sdks/java/io/jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java ## @@ -47,18 +47,16 @@ * This test requires a running instance of Postgres. Pass in connection information using * PipelineOptions: * - * mvn -e -Pio-it verify -pl sdks/java/io/jdbc -DintegrationTestPipelineOptions='[ + * ./gradlew integrationTest -p sdks/java/io/jdbc -DintegrationTestPipelineOptions='[ * "--postgresServerName=1.2.3.4", * "--postgresUsername=postgres", * "--postgresDatabaseName=myfancydb", * "--postgresPassword=mypass", * "--postgresSsl=false", * "--numberOfRecords=1000" ]' + * --tests org.apache.beam.sdk.io.jdbc.JdbcIOIT * * - * If you want to run this with a runner besides directrunner, there are profiles for dataflow - * and spark in the jdbc pom. You'll want to activate those in addition to the normal test runner Review comment: Does this restriction not apply anymore ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 88682) Time Spent: 3h (was: 2h 50m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88677=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88677
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905401
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
Review comment:
Could you add single lines of comments explaining what these properties are
? For example, I'm not sure what 'official' is used for.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88677)
Time Spent: 2h 20m (was: 2h 10m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88681=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88681
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905624
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
+ testCompile project(path: ":runners:google-cloud-dataflow-java",
configuration: 'shadowTest')
+} else {
+ testCompile project(path: ":runners:direct-java", configuration:
'shadowTest')
+}
+ }
+}
+ }
+
+ // Task for running integration tests
+ task integrationTest(type: Test) {
+include "**/*IT.class"
+systemProperties.beamTestPipelineOptions =
configuration.integrationTestPipelineOptions
+ }
+}
+
+ext.createPerformanceTestHarness = {
+ println "createPerformanceTestHarness with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // This task runs PerfKitBenchmarker, which does benchmarking of the IO ITs.
+ // The arguments passed to it allows it to invoke gradle again with the
desired benchmark.
+ //
+ // To invoke this, run:
+ //
+ // ./gradlew performanceTest \
+ // -DpkbLocation=""
+ // -DintegrationTestPipelineOptions='["--numberOfRecords=1000", ""]' \
+ // -DintegrationTest= \
+ // -DitModule= \
+ // -DintegrationTestRunner=direct/dataflow/
+ //
+ // There are more options with default values that can be tweaked if needed
(see below).
+ task performanceTest(type: Exec) {
+
+// Perfkit needs to work in the Beam's root directory,
Review comment:
You mean, test has to be invoked from the root directory ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id:
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88684=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88684
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905518
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
Review comment:
So, tasks names starting with 'integrationTest' is a requirement ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88684)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88674
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:15
Start Date: 07/Apr/18 02:15
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5003: [BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905443
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
Review comment:
Ditto.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88674)
Time Spent: 2h 10m (was: 2h)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88669=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88669
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:10
Start Date: 07/Apr/18 02:10
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905569
##
File path: sdks/java/io/cassandra/build.gradle
##
@@ -18,6 +18,7 @@
apply from: project(":").file("build_rules.gradle")
applyJavaNature(artifactId: "beam-sdks-java-io-cassandra")
+enableJavaPerformanceTesting()
Review comment:
What does this line do? Does it enable for all build?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88669)
Time Spent: 1h 50m (was: 1h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88671=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88671
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:10
Start Date: 07/Apr/18 02:10
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905554
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
+ String benchmarks = System.getProperty('benchmarks',
'beam_integration_benchmark')
+
+ String beamPrebuilt = System.getProperty('beamPrebuilt', 'true')
+ String beamSdk = System.getProperty('beamSdk', 'java')
+
+ String timeout = System.getProperty('itTimeout', '1200')
+
+ String kubeconfig = System.getProperty('kubeconfig',
System.getProperty('user.home') + '/.kube/config')
+ String kubectl = System.getProperty('kubectl', 'kubectl')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no scripts are expected.
+ String kubernetesScripts = System.getProperty('kubernetesScripts', '')
+
+ String integrationTestPipelineOptions =
System.getProperty('integrationTestPipelineOptions')
+
+ // PerfKit will have trouble reading 'null' path. It expects empty string if
no config file is expected.
+ String optionsConfigFile = System.getProperty('beamITOptions', '')
+
+ String integrationTest = System.getProperty('integrationTest')
+ String itModule = System.getProperty('itModule')
+
+ String runner = System.getProperty('integrationTestRunner', 'direct')
+
+ String extraProperties = System.getProperty('beamExtraProperties', '')
+}
+
+// Configures a project with a set of tasks needed for running performance
tests
+ext.enableJavaPerformanceTesting = {
+ println "enableJavaPerformanceTesting with ${it ? "$it" : "default
configuration"} for project ${project.name}"
+
+ // Use the implicit it parameter of the closure to handle zero argument or
one argument map calls.
+ JavaPerformanceTestConfiguration configuration = it ? it as
JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration()
+
+ // Add runners needed to run integration tests on
+ task packageIntegrationTests(type: Jar) {
+if (gradle.startParameter.taskNames.contains('integrationTest')) {
+
+ def runner = configuration.runner
+ dependencies {
+if (runner.contains('dataflow')) {
Review comment:
Do we need to add hard coded branch for each possible runner here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88671)
Time Spent: 1h 50m (was: 1h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[
https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88670=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88670
]
ASF GitHub Bot logged work on BEAM-3942:
Author: ASF GitHub Bot
Created on: 07/Apr/18 02:10
Start Date: 07/Apr/18 02:10
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #5003:
[BEAM-3942] Update performance testing framework to use Gradle
URL: https://github.com/apache/beam/pull/5003#discussion_r179905523
##
File path: build_rules.gradle
##
@@ -593,6 +593,118 @@ ext.applyJavaNature = {
}
}
+// Reads and contains all necessary performance test parameters
+class JavaPerformanceTestConfiguration {
+ String pkbLocation = System.getProperty('pkbLocation')
+
+ String logLevel = System.getProperty('logLevel', 'INFO')
+ String gradleBinary = System.getProperty('gradleBinary', './gradlew')
+ String isOfficial = System.getProperty('official', 'true')
Review comment:
Do we want to default to `true` here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 88670)
Time Spent: 1h 50m (was: 1h 40m)
> Update performance testing framework to use Gradle.
> ---
>
> Key: BEAM-3942
> URL: https://issues.apache.org/jira/browse/BEAM-3942
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Chamikara Jayalath
>Assignee: Łukasz Gajowy
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This requires performing updates to PerfKitBenchmarker and Beam so that we
> can execute performance tests using Gradle.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=88410=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-88410 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 06/Apr/18 12:07 Start Date: 06/Apr/18 12:07 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-379233885 R: @udim This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 88410) Time Spent: 1h 40m (was: 1.5h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87552=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87552 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 04/Apr/18 14:09 Start Date: 04/Apr/18 14:09 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378612975 Thanks to @lukecwik suggestion, this PR got changed a little bit. The task remained the same, but their placement is different. Now those are located in the root's build_rules. Same as before, those are applied only to those subprojects which need it. All tasks reuse the JavaPerformanceTestConfiguration class to fetch and use system properties. One last important thing to notice is that `performanceTest` task is designed to be run from the root project directory. I found it easier, because: 1. Placing the task in each subproject would require adding -p option in order to run exactly one test. Running from root will invoke all performanceTest tasks which is not what we want. 2. Running with -p option makes it harder to use relative paths. Because Perfkit is now invoked in the subproject (eg. sdks/java/io/jdbc), we cannot pass it paths starting from root project (eg. .test-infra). This also applies to ./gradlew. @lukecwik could you also take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87552) Time Spent: 1.5h (was: 1h 20m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87331=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87331 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 23:23 Start Date: 03/Apr/18 23:23 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378430503 I had some ideas on how to improve this PR: https://github.com/lgajowy/beam/pull/3 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87331) Time Spent: 1h 20m (was: 1h 10m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87257=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87257 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 20:22 Start Date: 03/Apr/18 20:22 Worklog Time Spent: 10m Work Description: angoenka commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378384572 R: @chamikaramj This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87257) Time Spent: 1h 10m (was: 1h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87207=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87207 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 19:02 Start Date: 03/Apr/18 19:02 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378361011 Also cc: @chamikaramj This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87207) Time Spent: 1h (was: 50m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87092=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87092 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 13:30 Start Date: 03/Apr/18 13:30 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378250518 Sorry, the seed job i run does not make sens for now - jenkins (perfkit) still uses maven (yet), so there's no sense to run the ITs on this pr This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87092) Time Spent: 50m (was: 40m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87091=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87091 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 13:30 Start Date: 03/Apr/18 13:30 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378250518 Sorry, the seed job i run does not make sens for now - jenkins (perfkit) still uses maven (yet). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87091) Time Spent: 40m (was: 0.5h) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87089=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87089 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 13:29 Start Date: 03/Apr/18 13:29 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378249653 Run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87089) Time Spent: 0.5h (was: 20m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87085=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87085 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 13:27 Start Date: 03/Apr/18 13:27 Worklog Time Spent: 10m Work Description: lgajowy opened a new pull request #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003 This PR introduces three gradle tasks that are needed for IOITs. 1. integrationTest - which is for running integration tests manually 2. packageIntegrationTests - which provides necessary IOIT's dependencies (runners to be chosen using a command line arg) 3. performanceTest - which runs perfkit to setup all necessary testing infrastructure. At the time of writing this mesage, PerfKitBenchmarker is not using gradle yet. I'll migrate it as soon as this PR gets merged. More on that in the issues comments: https://issues.apache.org/jira/browse/BEAM-3942 @jbonofre could you take a look? CC: @lukecwik Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it). Trivial changes like typos do not require a JIRA issue. Your pull request should address just this issue, without pulling in other changes. - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue. - [ ] Write a pull request description that is detailed enough to understand: - [ ] What the pull request does - [ ] Why it does it - [ ] How it does it - [ ] Why this approach - [ ] Each commit in the pull request should have a meaningful subject line and body. - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87085) Time Spent: 10m Remaining Estimate: 0h > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3942) Update performance testing framework to use Gradle.
[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=87086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-87086 ] ASF GitHub Bot logged work on BEAM-3942: Author: ASF GitHub Bot Created on: 03/Apr/18 13:27 Start Date: 03/Apr/18 13:27 Worklog Time Spent: 10m Work Description: lgajowy commented on issue #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#issuecomment-378249653 Run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 87086) Time Spent: 20m (was: 10m) > Update performance testing framework to use Gradle. > --- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Chamikara Jayalath >Assignee: Łukasz Gajowy >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
