Abacn commented on code in PR #40321:
URL: https://github.com/apache/beam/pull/40321#discussion_r4126343671
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/GcsStager.java:
##########
@@ -49,6 +56,16 @@ public static GcsStager fromOptions(PipelineOptions options)
{
*/
@Override
public List<DataflowPackage> stageFiles(List<StagedFile> filesToStage) {
+ String stagingLocation = options.getStagingLocation();
+ if (stagingLocation != null &&
TestDataflowRunner.class.equals(options.getRunner())) {
+ List<Object> cacheKey = Arrays.asList(stagingLocation, filesToStage);
Review Comment:
since the key is a pair of (String, List<StagedFile>), consider use it as
STAGED_FILES_CACHE's key type. One can use guava's
com.google.common.collect.Maps.immutableEntry or
org.apache.commons.lang3.tuple.Pair
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
