kennknowles opened a new issue, #18765:
URL: https://github.com/apache/beam/issues/18765
When a dataflow is started from a template, the value of a ValueProvider vp
is evaluated either (1) when the template is created or (2) the first time that
vp.get() is called in each instantiation of that value provider.
There needs to be a mechanism of specifying that a ValueProvider is
evaluated once at the start of the running of the pipeline, and that the value
is the same among all instances. I cannot find any way to do so.
The two obvious examples I can come up with are:
```
ValueProvider<Date> startTime;
ValueProvider<String> shortLivedAccessToken;
```
The obvious rebuttal is that the user could pass \--startTimeMs or
\--shortTimeAccessToken as a parameter to the dataflow.
* For the access token, the user may not have the permissions to get this
token, and repeatedly requesting a new token is expensive and may hit system
request limits.
* For the "start time", the dataflow might be used to perform periodic
maintenance in which old entries are deleted. A bad argument (accidental or
malicious) putting startTime in the future could cause the system to think that
*everything* is old. There is no simple mechanism to check the passed
parameter for reasonableness.
I can get either of these as a side input, but not as a ValueProvider.
Imported from Jira
[BEAM-4295](https://issues.apache.org/jira/browse/BEAM-4295). Original Jira may
contain additional context.
Reported by: fyellin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
