Behroz Sikander created BEAM-11060:
--------------------------------------
Summary: Bigquery readTableRowsWithSchema fails with
IllegalStateException
Key: BEAM-11060
URL: https://issues.apache.org/jira/browse/BEAM-11060
Project: Beam
Issue Type: Bug
Components: beam-community
Affects Versions: 2.24.0
Environment: local, dataflow
Reporter: Behroz Sikander
Assignee: Aizhamal Nurmamat kyzy
If you are templating the pipeline and use the function
BigQueryIO.readTableRowsWithSchema(), it fails with the following exception
even though no .get() was called on the ValueProvider
An exception occured while executing the Java class. Value only available at
runtime, but accessed from a non-runtime context: RuntimeValueProvider
The following stack trace gives more clue and the problem seems to be that
BigQueryQuerySourceDef is trying to call a .get on ValueProvider which is
wrong. Here is the exact code line in master
https://github.com/apache/beam/blame/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQuerySourceDef.java#L103
{code:java}
// code placeholder
java.lang.IllegalStateException: Value only available at runtime, but accessed
from a non-runtime context: RuntimeValueProvider{propertyName=tablename,
default=null}
at org.apache.beam.sdk.options.ValueProvider$RuntimeValueProvider.get
(ValueProvider.java:250)
at org.apache.beam.sdk.options.ValueProvider$NestedValueProvider.get
(ValueProvider.java:141)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryQuerySourceDef.getBeamSchema
(BigQueryQuerySourceDef.java:163)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO$TypedRead.expand
(BigQueryIO.java:1188)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO$TypedRead.expand
(BigQueryIO.java:701)
at org.apache.beam.sdk.Pipeline.applyInternal (Pipeline.java:542)
at org.apache.beam.sdk.Pipeline.applyTransform (Pipeline.java:493)
at org.apache.beam.sdk.values.PBegin.apply (PBegin.java:56)
at org.apache.beam.sdk.Pipeline.apply (Pipeline.java:186)
at com.abc.pipeline.main (BomboraIntentCrmIdPipeline.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)
at java.lang.Thread.run (Thread.java:836)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
