kennknowles commented on code in PR #30283:
URL: https://github.com/apache/beam/pull/30283#discussion_r1488065606
##########
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnHarness.java:
##########
@@ -144,6 +171,8 @@ public static void main(Function<String, String>
environmentVarGetter) throws Ex
}
System.out.format("Pipeline options %s%n", pipelineOptionsJson);
+ pipelineOptionsJson = removeNestedKey(pipelineOptionsJson,
"impersonateServiceAccount");
Review Comment:
OK I now realize why this one is doing a weird thing to the string of the
JSON, which is not a good pattern.
You _could_ have just set the pipeline option to `null` after parsing it.
But of course then you would have to take a dependency on GCP pipeline options
when the SDK harness should not have a dependency on GCP.
So it is a hack. What should really happen is the Dataflow service strips
this option before sending the options to the worker. Maybe what you actually
want is for the Dataflow pipeline to have two sets of options in the job
submission: (1) the options used to submit the job (2) the options used to run
the job.
Anyhow, please keep a bug open for this and make an extensive comment here.
Because even though you didn't add a dependency on the GCP library, you did
actually add a dependency on a GCP concept, plus a hack that someone who comes
later will wonder "what is going on here?"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
