It sounds like it may simply be an oversight that @Default does not work with ValueProvider. Copying dev@ to see if there's some pitfall associated with adding this.
Kenn On Thu, Dec 20, 2018 at 5:17 PM Jeff Klukas <[email protected]> wrote: > I am also now realizing that runtime params show up in --help with type of > ValueProvider, so I can likely meet my documentation needs via a README for > the project that explains how runtime parameters behave and that > ValueProvider type is the flag that lets you know a parameter can be > overridden at runtime. > > > > On Thu, Dec 20, 2018 at 4:47 PM Jeff Klukas <[email protected]> wrote: > >> I did some more testing this afternoon, and it looks like I'm incorrect >> on some details here. >> >> Parameters implemented as ValueProviders can be provided at compile time >> and they do end up in the template. A value specified at runtime overrides. >> If not specified at runtime, the compile-time value is in effect. >> >> It still looks, though, like @Default has no effect for ValueProviders. >> If not value is specified at compile time or runtime, the ValueProvider is >> not available, even if it has a @Default configured. >> >> Does anybody have tips for documenting which params are overrideable at >> runtime? Or experience handling default values for ValueProviders? >> >> On Thu, Dec 20, 2018 at 3:49 PM Jeff Klukas <[email protected]> wrote: >> >>> I am deploying Beam pipelines with the DataflowRunner and would like to >>> move more of the pipeline options to use the ValueProvider interface so >>> they can be specified at runtime rather than at template compile time, but >>> running into various issues. >>> >>> First, it's been unclear to the operations engineers deploying pipelines >>> which options are runtime and which are compile-time. The engineers are >>> typically using the gcloud command-line interface to deploy rather than the >>> console, so I don't see much benefit from implementing a metadata json file. >>> >>> AFAICT, Dataflow happily accepts values for runtime parameters when >>> compiling the template, but those values are completely ignored and need to >>> be defined again at runtime, furthering the confusion. Should I just go >>> ahead and add "(runtime parameter)" to each of the @Description annotations >>> to at least document the distinction in --help output? >>> >>> Finally, it's not clear whether the @Default annotation supports runtime >>> parameters. The dataflow docs show an example where @Default is used on a >>> ValueProvider [0], but this doesn't appear to actually have any effect. If >>> I don't pass in a value for a runtime parameter when executing a template, >>> the pipeline throws a "Value only available at runtime" exception on calls >>> to .get(), rather than returning the default value. Have others encountered >>> this? Is there a pattern for providing defaults for runtime parameters? >>> >>> [0] >>> https://cloud.google.com/dataflow/docs/guides/templates/creating-templates#using-valueprovider-in-your-pipeline-options >>> >>
