There is no way to achieve this using ValueProvider. Its value is either fixed at construction time (StaticValueProvider), or completely dynamic (evaluated every time you call .get()). You'll need to implement this using a side input. E.g. take a look at implementation of BigQueryIO, how it generates unique job id tokens - https://github.com/apache/beam/blob/bf94e36f67a8bc5d24c795e40697ad2504c8594c/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L756
On Thu, May 3, 2018 at 5:42 PM Frank Yellin <[email protected]> wrote: > [Sorry, I accidentally hit send before I had finished typing . .] > > Is there any way to achieve what I'm looking for? Or is this just beyond > the scope of ValueProvider and templates? > > > > On Thu, May 3, 2018 at 5:36 PM, Frank Yellin <[email protected]> wrote: > >> I'm attempting to create a dataflow template, and within the template >> have a variable >> ValueProvider<DateTime> now >> such that now is the time the dataflow is started, note the time that the >> template was created. >> >> My first attempt was >> ValueProvider<DateTime> now = >> StaticValueProvider.of(DateTime.now(DateTimeZone.UTC)); >> >> My second attempt was >> >> public interface MyOptions extends PipelineOptions, >> DataflowPipelineOptions { >> @Description("Now") >> @Default.InstanceFactory(GetNow.class) >> ValueProvider<DateTime> getNow(); >> void setNow(ValueProvider<DateTime> value); >> } >> >> static class GetNow implements DefaultValueFactory<DateTime> { >> @Override >> public DateTime create(PipelineOptions options) { >> return DateTime.now(DateTimeZone.UTC); >> } >> } >> >> ValueProvider<DateTime> now = options.getNow() >> >> My final attempt was: >> >> ValueProvider<SerializableFunction<Void, DateTime>> nowFn = >> StaticValueProvider.of(x -> DateTime.now(DateTimeZone.UTC)); >> >> ValueProvider<DateTime> now = NestedValueProvider.of(nowFn, x -> >> x.apply(null)); >> >> >> >> In every case, it was clear that "now" was being set to template-creation >> time rather than actual runtime. >> >> I note that the documentation talks about a RuntimeValueProvider, but >> there is no user-visible constructor for this. >> >> >> >> >
