Ahmed Abualsaud created BEAM-14119:
--------------------------------------
Summary: BigQuery DIRECT_READ does not validate pipeline's project
ID and instead tries to read from a null project
Key: BEAM-14119
URL: https://issues.apache.org/jira/browse/BEAM-14119
Project: Beam
Issue Type: Improvement
Components: io-java-gcp
Reporter: Ahmed Abualsaud
When a pipeline is created without a GCP project ID and tries to read from
BigQuery using Storage Read API, it runs into the following unhelpful error:
{code:python}
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
com.google.api.gax.rpc.PermissionDeniedException:
io.grpc.StatusRuntimeException: PERMISSION_DENIED: BigQuery Storage API has not
been used in project 770406736630 before or it is disabled. Enable it by
visiting
https://console.developers.google.com/apis/api/bigquerystorage.googleapis.com/overview?project=770406736630
then retry. If you enabled this API recently, wait a few minutes for the
action to propagate to our systems and retry.{code}
It looks like no validation for project ID is happening, and Beam tries to read
without a project ID. Project 770406736630 mentioned in the error is a {{null}}
project and throws off the user because it isn't their project.
Doing the same but using the EXPORT read method results in this more helpful
error.
{code:python}
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
java.lang.NullPointerException: Required parameter projectId must be
specified.{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
