[
https://issues.apache.org/jira/browse/BEAM-5364?focusedWorklogId=145005&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-145005
]
ASF GitHub Bot logged work on BEAM-5364:
----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Sep/18 18:33
Start Date: 17/Sep/18 18:33
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#6379: [BEAM-5364] Check if validation is disabled when validating
BigtableSource
URL: https://github.com/apache/beam/pull/6379#discussion_r218179213
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableConfig.java
##########
@@ -159,13 +159,33 @@ void validate() {
"Could not obtain Bigtable instance id");
}
+ /**
+ * @param <T> parameter The runtime parameter.
+ * @return {@code null} if the input parameter is {@code null}, return the
String value of runtime
+ * parameter if the parameter is accessible, returns "Unavailable during
pipeline
+ * construction" if the runtime parameter is not accessible for
debugging purpose.
+ */
+ @Nullable
+ static <T> String getDisplayDataForParameter(@Nullable ValueProvider<T>
parameter) {
+ if (parameter == null) return null;
+
+ if (parameter.isAccessible()) {
+ return String.valueOf(parameter.get());
+ }
+ return "Unavailable during pipeline construction";
+ }
+
void populateDisplayData(DisplayData.Builder builder) {
builder
.addIfNotNull(
- DisplayData.item("projectId", getProjectId()).withLabel("Bigtable
Project Id"))
+ DisplayData.item("projectId",
getDisplayDataForParameter(getProjectId()))
+ .withLabel("Bigtable Project Id"))
+ .addIfNotNull(
+ DisplayData.item("instanceId",
getDisplayDataForParameter(getInstanceId()))
Review comment:
I'm not sure why we had to update display-data related logic for this PR
(fix validation issue). Should these updates be a separate PR ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 145005)
Time Spent: 0.5h (was: 20m)
> BigtableIO source tries to validate table ID even though validation is
> turned off
> ----------------------------------------------------------------------------------
>
> Key: BEAM-5364
> URL: https://issues.apache.org/jira/browse/BEAM-5364
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Kevin Si
> Assignee: Chamikara Jayalath
> Priority: Minor
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> [https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java#L1084|https://www.google.com/url?q=https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableIO.java%23L1084&sa=D&usg=AFQjCNEfHprTOvnwAwFSrXwUuLvc__JBWg]
> The validation can be turned off with following:
> BigtableIO.read()
> .withoutValidation() // skip validation when constructing the
> pipelline.
> A Dataflow template cannot be constructed due to this validation failure.
>
> Error log when trying to construct a template:
> Exception in thread "main" java.lang.IllegalArgumentException: tableId was
> not supplied
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
> at
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$BigtableSource.validate(BigtableIO.java:1084)
> at org.apache.beam.sdk.io.Read$Bounded.expand(Read.java:95)
> at org.apache.beam.sdk.io.Read$Bounded.expand(Read.java:85)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:44)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:167)
> at
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$Read.expand(BigtableIO.java:423)
> at
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$Read.expand(BigtableIO.java:179)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
> at
> com.google.cloud.teleport.bigtable.BigtableToAvro.main(BigtableToAvro.java:89)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
