hekk-kaori-maeda opened a new issue, #36831:
URL: https://github.com/apache/beam/issues/36831
### What would you like to happen?
**What is the problem?**
When using BigQuery Storage Write API (`STORAGE_WRITE_API` method) and write
operations fail, the `BigQueryStorageApiInsertError` object does not contain
information about which
table the error occurred on. This makes it difficult for users to identify
and troubleshoot errors, especially when writing to multiple tables.
**Current behavior:**
- `BigQueryStorageApiInsertError` only contains the row data and error
message
- Users cannot determine which table caused the error without additional
logging
**Expected behavior:**
- `BigQueryStorageApiInsertError` should include table identification
(project, dataset, table)
- API should be consistent with `BigQueryInsertError` (used by
`STREAMING_INSERTS` method), which provides table information via
`TableReference`
**Proposed solution:**
Add the following to `BigQueryStorageApiInsertError`:
- `tableUrn` field (format:
`projects/{project}/datasets/{dataset}/tables/{table}`)
- Convenience methods: `getProjectId()`, `getDatasetId()`, `getTableId()`
**Use case:**
This is particularly useful for:
- Error monitoring and alerting systems
- Debugging write failures in multi-table pipelines
- Logging and auditing
**Related:**
- Consistent with existing `BigQueryInsertError` API
- Uses standard `TableDestination.getTableUrn()` format
**Additional context:**
I'm willing to contribute a PR for this enhancement.
### Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [x] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
