[
https://issues.apache.org/jira/browse/BEAM-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16627530#comment-16627530
]
Kevin Si commented on BEAM-5420:
--------------------------------
This is the error we get if we try to access runtime parameters when getting
display data:
2018-09-25 08:15:55 INFO DataflowRunner:706 - Executing pipeline on the
Dataflow Service, which will have billing implications related to Google
Compute Engine usage and other Google Cloud Services.
2018-09-25 08:15:55 INFO PackageUtil:315 - Uploading 1 files from
PipelineOptions.filesToStage to staging location to prepare for execution.
2018-09-25 08:15:55 INFO PackageUtil:242 - Uploading
/usr/local/google/_blaze_kevinsi/f06b53e3e81897f185aef98ca1141c77/execroot/google3/blaze-out/k8-fastbuild/bin/cloud/dataflow/testing/integration/teleport/teleport_bigtable_export_import_avro_file_uif_generate_template.runfiles/google3/third_party/java_src/cloud/teleport/teleport-all-bundled.jar
to
gs://deft-guitar-sandbox-e2e-test/df2-teleport-bigtable-to-avro-file-generate-template-prod/09250815492298/staging/teleport-all-bundled-Lato7X_J6JWX3aVWaViZHg.jar
2018-09-25 08:16:12 INFO PackageUtil:389 - Staging files complete: 0 files
cached, 1 files newly uploaded
Exception in thread "main"
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder$PopulateDisplayDataException:
Error while populating display data for component
'org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$BigtableSource': Value only
available at runtime, but accessed from a non-runtime context:
RuntimeValueProvider\{propertyName=bigtableTableId, default=null}
at
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include(DisplayData.java:792)
at
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include(DisplayData.java:741)
at org.apache.beam.sdk.io.Read$Bounded.populateDisplayData(Read.java:119)
at
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include(DisplayData.java:783)
at
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.access$100(DisplayData.java:711)
at org.apache.beam.sdk.transforms.display.DisplayData.from(DisplayData.java:78)
at
org.apache.beam.runners.core.construction.PTransformTranslation.translateAppliedPTransform(PTransformTranslation.java:381)
at
org.apache.beam.runners.core.construction.PTransformTranslation$KnownTransformPayloadTranslator.translate(PTransformTranslation.java:323)
at
org.apache.beam.runners.core.construction.PTransformTranslation.toProto(PTransformTranslation.java:141)
at
org.apache.beam.runners.core.construction.SdkComponents.registerPTransform(SdkComponents.java:97)
at
org.apache.beam.runners.core.construction.PipelineTranslation$1.visitPrimitiveTransform(PipelineTranslation.java:72)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:657)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)
at
org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)
at
org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)
at
org.apache.beam.runners.core.construction.PipelineTranslation.toProto(PipelineTranslation.java:44)
at
org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:164)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:732)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:177)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
at
com.google.cloud.teleport.bigtable.BigtableToAvro.main(BigtableToAvro.java:95)
Caused by: java.lang.IllegalStateException: Value only available at runtime,
but accessed from a non-runtime context:
RuntimeValueProvider\{propertyName=bigtableTableId, default=null}
at
org.apache.beam.sdk.options.ValueProvider$RuntimeValueProvider.get(ValueProvider.java:228)
at
org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$BigtableSource.populateDisplayData(BigtableIO.java:1098)
at
org.apache.beam.sdk.transforms.display.DisplayData$InternalBuilder.include(DisplayData.java:783)
... 23 more
> BigtableIO tries to get runtime parameters when collecting display data at
> pipeline construction time
> -----------------------------------------------------------------------------------------------------
>
> Key: BEAM-5420
> URL: https://issues.apache.org/jira/browse/BEAM-5420
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Kevin Si
> Assignee: Chamikara Jayalath
> Priority: Minor
>
> For example:
> [https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableConfig.java#L165]
> At Dataflow pipeline construction time calling getProjectId() gives an error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
