[
https://issues.apache.org/jira/browse/BEAM-4257?focusedWorklogId=126041&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-126041
]
ASF GitHub Bot logged work on BEAM-4257:
----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Jul/18 14:28
Start Date: 23/Jul/18 14:28
Worklog Time Spent: 10m
Work Description: lgajowy edited a comment on issue #5341: [BEAM-4257]
Increases BigQuery streaming error information
URL: https://github.com/apache/beam/pull/5341#issuecomment-407076438
Just FYI: Some of the interfaces became package-private (BigQueryService,
FakeBigQueryService, FakeDatasetService etc). This caused Nexmark suites to
fail. Luckily, it seems that those classes were used only for testing purposes
(but outside BigQueryIO module).
Logs for reference:
https://builds.apache.org/view/A-D/view/Beam/job/beam_PostCommit_Java_Nexmark_Direct/112/console
I had trouble to think of a better solution than simply deleting the test,
because now it seems impossible to test the code in a similar way (please
correct me if I'm wrong): https://github.com/apache/beam/pull/6018. Please
merge it only if you strongly belive those interfaces/classes should remain
package-private.
Maybe it's a good idea to leave at least `BigQueryServices` public so that
it could be used by `withTestServices()` in other places (like nexmark)? WDYT?
CC: @calonso @reuvenlax
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 126041)
Time Spent: 5h 10m (was: 5h)
> Add error reason and table destination to BigQueryIO streaming failed inserts
> -----------------------------------------------------------------------------
>
> Key: BEAM-4257
> URL: https://issues.apache.org/jira/browse/BEAM-4257
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Reporter: Carlos Alonso
> Assignee: Carlos Alonso
> Priority: Minor
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> When using `BigQueryIO.Write` and getting `WriteResult.getFailedInserts()` we
> get a `PCollection<TableRow>` which is fine, but in order to properly work on
> the errors downstream having extended information such as the `InsertError`
> fields and the `TableReference` it was routed to would be really valuable.
>
> My suggestion is to create a new object that contains all that information
> and return a `PCollection` of those instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
