[
https://issues.apache.org/jira/browse/BEAM-8281?focusedWorklogId=327776&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-327776
]
ASF GitHub Bot logged work on BEAM-8281:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Oct/19 12:02
Start Date: 14/Oct/19 12:02
Worklog Time Spent: 10m
Work Description: lgajowy commented on pull request #9638: [BEAM-8281]
Resize IOITs datasets
URL: https://github.com/apache/beam/pull/9638#discussion_r334443378
##########
File path:
sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/common/FileBasedIOTestPipelineOptions.java
##########
@@ -48,4 +42,10 @@
boolean getReportGcsPerformanceMetrics();
void setReportGcsPerformanceMetrics(boolean performanceMetrics);
+
+ @Description("Test configuration name")
+ @Validation.Required
+ String getTestConfigName();
+
+ void setTestConfigName(String testConfig);
Review comment:
I think this can't be required. In general, IOITs should allow to:
- be configured according to a specific "scenario" (this is what you're
doing in this PR and this is good)
- be configured however user likes, according to some arbitrary scenario
that isn't hardcoded
Once we're forcing the config to be passed to the test, we loose the 2
bullet from above.
On the other hand, I know that wasn't the case different test sizes because
the hash was hardcoded. Maybe we should allow users passing their custom
hashcode using pipeline options and specify fixed scenarios elsewhere (in code
that invokes the tests, rather than implements them).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 327776)
Time Spent: 50m (was: 40m)
> Resize file-based IOITs
> ------------------------
>
> Key: BEAM-8281
> URL: https://issues.apache.org/jira/browse/BEAM-8281
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Michal Walenia
> Assignee: Michal Walenia
> Priority: Minor
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Resize the IOITs to use known amounts of data saved on GCS so that it's
> possible to report dataset size and measure throughput as part of the test
> metrics
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
