shunping commented on code in PR #40142:
URL: https://github.com/apache/beam/pull/40142#discussion_r4037470428
##########
it/google-cloud-platform/build.gradle:
##########
@@ -79,10 +85,31 @@ dependencies {
}
tasks.register(
- "GCSPerformanceTest", IoPerformanceTestUtilities.IoPerformanceTest,
project, 'google-cloud-platform', 'FileBasedIOLT',
+ "GCSPerformanceTest", IoPerformanceTestUtilities.IoPerformanceTest,
project, 'google-cloud-platform', 'TextIOLT',
['configuration':'large','project':'apache-beam-testing',
'artifactBucket':'io-performance-temp']
+ System.properties
)
+// Note: the task always passes 'configuration' down as a system property, so
the "local" default in
+// ParquetIOLT itself is never reached through gradle. The default here is
therefore the cheap local
+// preset: an accidental `./gradlew
:it:google-cloud-platform:ParquetPerformanceTest` must not launch
+// a full scale Dataflow job. Pass an explicit preset (or a json override) for
real runs, e.g.
+// -Dconfiguration=large or
-Dconfiguration='{"preset":"f100_s16","totalBytes":"10GB"}'.
+tasks.register(
+ "ParquetPerformanceTest",
IoPerformanceTestUtilities.IoPerformanceTest, project, 'google-cloud-platform',
'ParquetIOLT',
+ ['configuration':'local','project':'apache-beam-testing',
'artifactBucket':'io-performance-temp']
Review Comment:
At this stage, we only trigger the minimum configuration in our github
postcommit workflow. There will be a separate PR to add a new workflow to sweep
all the configurations weekly.
##########
it/google-cloud-platform/build.gradle:
##########
@@ -79,10 +85,31 @@ dependencies {
}
tasks.register(
- "GCSPerformanceTest", IoPerformanceTestUtilities.IoPerformanceTest,
project, 'google-cloud-platform', 'FileBasedIOLT',
+ "GCSPerformanceTest", IoPerformanceTestUtilities.IoPerformanceTest,
project, 'google-cloud-platform', 'TextIOLT',
['configuration':'large','project':'apache-beam-testing',
'artifactBucket':'io-performance-temp']
+ System.properties
)
+// Note: the task always passes 'configuration' down as a system property, so
the "local" default in
+// ParquetIOLT itself is never reached through gradle. The default here is
therefore the cheap local
+// preset: an accidental `./gradlew
:it:google-cloud-platform:ParquetPerformanceTest` must not launch
+// a full scale Dataflow job. Pass an explicit preset (or a json override) for
real runs, e.g.
+// -Dconfiguration=large or
-Dconfiguration='{"preset":"f100_s16","totalBytes":"10GB"}'.
+tasks.register(
+ "ParquetPerformanceTest",
IoPerformanceTestUtilities.IoPerformanceTest, project, 'google-cloud-platform',
'ParquetIOLT',
+ ['configuration':'local','project':'apache-beam-testing',
'artifactBucket':'io-performance-temp']
Review Comment:
At this stage, we only trigger the minimum configuration ("local") in our
github postcommit workflow. There will be a separate PR to add a new workflow
to sweep all the configurations weekly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
