damccorm opened a new issue, #21219:
URL: https://github.com/apache/beam/issues/21219
Many BigQuery Storage API errors get logged at ERROR level even though they
are expected and handled by the Beam's transform. They should be suppressed, or
output at DEBUG/INFO level to differentiate from abnormal behavior.
Example:
```
{
"jsonPayload":
{ "message": "Got error io.grpc.StatusRuntimeException: ALREADY_EXISTS: The
offset
is within stream, expected offset 52125, received 51264 Entity:
projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM
closing
projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM",
"step": "Save Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write
Records", "worker": "data-processing-streaming-10250846-kps9-harness-hnj3",
"stage": "P6", "job": "2021-10-25_08_46_20-4863073108555028756", "thread":
"258", "work": "3800026c09938fac-5cf2f68daacdd",
"logger":
"org.apache.beam.sdk.io.gcp.bigquery.StorageApiWritesShardedRecords" }
,
"resource":
{
"type": "dataflow_step",
"labels":
{ "project_id": "event-processing-demo", "step_id": "Save
Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write Records",
"job_id": "2021-10-25_08_46_20-4863073108555028756",
"region": "us-central1", "job_name":
"data-processing-streaming-storage-write-api-200-2" }
},
"timestamp":
"2021-10-25T15:57:43.617Z",
"severity": "ERROR",
"labels":
{ "compute.googleapis.com/resource_type":
"instance", "dataflow.googleapis.com/job_name":
"data-processing-streaming-storage-write-api-200-2",
"dataflow.googleapis.com/log_type": "supportability",
"compute.googleapis.com/resource_id": "6365565973148954518",
"dataflow.googleapis.com/job_id": "2021-10-25_08_46_20-4863073108555028756",
"dataflow.googleapis.com/region":
"us-central1", "compute.googleapis.com/resource_name":
"data-processing-streaming-10250846-kps9-harness-hnj3"
}
,
"logName":
"projects/event-processing-demo/logs/dataflow.googleapis.com%2Fworker",
"receiveTimestamp":
"2021-10-25T15:57:47.557344849Z"
}
```
Additional exceptions that fit this category:
* Got error `io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is
closed due to com.google.api.gax.rpc.UnavailableException:
io.grpc.StatusRuntimeException: UNAVAILABLE: Connection closed after GOAWAY.
HTTP/2 error code: NO_ERROR, debug data: server_shutting_down closing`
* Got error `com.google.api.gax.rpc.AbortedException:
io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is
restarted`. This is expected and client is advised to reconnect.
* Got error `io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is
closed due to com.google.api.gax.rpc.AbortedException:
io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is
restarted`. This is expected and client is advised to reconnect.
Imported from Jira
[BEAM-13162](https://issues.apache.org/jira/browse/BEAM-13162). Original Jira
may contain additional context.
Reported by: slilichenko.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
