Hi, I’m using Apache Beam via Google Cloud Dataflow, and I’m working on making a template for my job. My team is working to set up QA and prod environments within separate Google projects, so I’d like to be able to pass the project ID as a parameter at runtime. I’ve been playing around with the ValueProvider<String> type but this causes Java to complain at compile time “java.lang.RuntimeException: Not called from a runtime context.”
I did some googling around and found a few links, and I’ve been working from the official docs as well. https://stackoverflow.com/questions/43992120/valueprovider-issue https://stackoverflow.com/questions/42843495/google-dataflow-create-templates-with-runtime-parameters/42843682 https://cloud.google.com/dataflow/docs/templates/creating-templates#using-nestedvalueprovider The solution seems to be to use the NestedValueProvider type, but I’m still getting a runtime exception if I call .get() on it. I’ve got the following in my pipeline options (according to the Google Cloud Dataflow docs, I’m still supposed to have (Runtime)ValueProviders in my options file): ValueProvider<String> getDataflowProjectId(); void setDataflowProjectId(ValueProvider<String> value); And I’m composing it into a NestedValueProvider in the following manner: NestedValueProvider<String, String> pid = NestedValueProvider.of( pipelineOptions.getDataflowProjectId(), (SerializableFunction<String, String>) s -> s ); If I try following Google’s example and try to assign the output of NestedValueProvider.of directly to a String, I get the following compilation error: incompatible types: no instance(s) of type variable(s) T,X exist so that org.apache.beam.sdk.options.ValueProvider.NestedValueProvider<T,X> conforms to java.lang.String Java’s not really my language but I’ve been getting along pretty well so far. This however has me completely stumped. What am I doing wrong? Thanks, —Nathan Deren
