ee07dazn commented on pull request #7677: URL: https://github.com/apache/beam/pull/7677#issuecomment-955713230
@pabloem : Hi, looking for a way to automatically send back records to DLQ table, i came across [ticket](https://issues.apache.org/jira/browse/BEAM-4265), which references this PR... i believe what i get out from this is FAILED_ROWS as a list of tuples (destination, rows), which i need to process however i want to write to DLQ table as a separate transform - please correct me if i have understood anything wrong so far? Assuming thats correct, I believe we need the error message and/or stacktrace as part of that tuple such that its easier to look into and debug ...I do know that this currently goes into worker logs and i can look at the logs but it would be much easier to send the error message as part of the tuple....i believe the Java version of Pub/Sub--> BQ does this and has a set output BQ table schema that has payload(str), payload(bytes), errorMessage, stacktrace, timestamp...which all helps in debugging...let me know what you think ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
