[
https://issues.apache.org/jira/browse/BEAM-13962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494162#comment-17494162
]
Kenneth Knowles commented on BEAM-13962:
----------------------------------------
I've got https://github.com/apache/beam/pull/16721 open that will eliminate all
NPEs in BigQueryIO so it should take care of this one. I'll double check this
line of code too.
> Dataflow Pub/Sub to BigQuery NullPointerExceptionat
> ---------------------------------------------------
>
> Key: BEAM-13962
> URL: https://issues.apache.org/jira/browse/BEAM-13962
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.34.0
> Environment: GCP, Dataflow
> Reporter: Elon Chen
> Assignee: Kenneth Knowles
> Priority: P3
> Labels: google-cloud-dataflow, pubsub
>
> Here is the error message when I run streaming Pub/Sub to BigQuery in Dataflow
>
> {code:java}
> java.lang.RuntimeException: java.lang.NullPointerExceptionat
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll
> ( org/apache.beam.sdk.io.gcp.bigquery/BigQueryServicesImpl.java:1001 )at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll
> ( org/apache.beam.sdk.io.gcp.bigquery/BigQueryServicesImpl.java:1054 )at
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows (
> org/apache.beam.sdk.io.gcp.bigquery/BatchedStreamingWrite.java:421 )at
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$900 (
> org/apache.beam.sdk.io.gcp.bigquery/BatchedStreamingWrite.java:72 )at
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle
> ( org/apache.beam.sdk.io.gcp.bigquery/BatchedStreamingWrite.java:267 )Caused
> by: java.lang.NullPointerExceptionat
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getErrorInfo
> ( BigQueryServicesImpl.java:1075 )at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1
> ( BigQueryServicesImpl.java:922 )at
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call
> ( BigQueryServicesImpl.java:1594 )at java.util.concurrent.FutureTask.run (
> FutureTask.java:264 )at java.util.concurrent.ThreadPoolExecutor.runWorker (
> ThreadPoolExecutor.java:1128 )at
> java.util.concurrent.ThreadPoolExecutor$Worker.run (
> ThreadPoolExecutor.java:628 )at java.lang.Thread.run ( Thread.java:834 ){code}
> And here is another one.
> {code:java}
> Error message from worker: java.lang.RuntimeException:
> com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad
> Request
> POST
> https://bigquery.googleapis.com/bigquery/v2/projects/{project-id}/datasets/{dataset-id}/tables/{table-id}/insertAll?prettyPrint=false
> {
> "code" : 400,
> "errors" : [ {
> "domain" : "global",
> "message" : "No rows present in the request.",
> "reason" : "invalid"
> } ],
> "message" : "No rows present in the request.",
> "status" : "INVALID_ARGUMENT"
> }
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1001)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1054)
>
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows(BatchedStreamingWrite.java:421)
>
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$900(BatchedStreamingWrite.java:72)
>
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle(BatchedStreamingWrite.java:267)
> Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException:
> 400 Bad Request
> POST
> https://bigquery.googleapis.com/bigquery/v2/projects/{project-id}/datasets/{dataset-id}/tables/{table-id}/insertAll?prettyPrint=false
> {
> "code" : 400,
> "errors" : [ {
> "domain" : "global",
> "message" : "No rows present in the request.",
> "reason" : "invalid"
> } ],
> "message" : "No rows present in the request.",
> "status" : "INVALID_ARGUMENT"
> }
>
> com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
>
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
>
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
> com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
>
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:910)
>
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1594)
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> java.base/java.lang.Thread.run(Thread.java:834) {code}
> I'm wondering those errors will be affected my dataflow job? What kind of
> situation will happen? I just search my hold day, but I can't find out any
> similar problem. Thank you all.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
