johnjcasey commented on code in PR #30081:
URL: https://github.com/apache/beam/pull/30081#discussion_r1491369073
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageStreamBundleSource.java:
##########
@@ -355,6 +351,11 @@ private boolean readNextRecord() throws IOException {
// progress made in the current Stream gives us the overall StreamBundle
progress.
fractionOfStreamBundleConsumed =
(currentStreamBundleIndex + fractionOfCurrentStreamConsumed) /
source.streamBundle.size();
+
+ SchemaAndRecord schemaAndRecord = new
SchemaAndRecord(reader.readSingleRecord(), tableSchema);
+
+ current = parseFn.apply(schemaAndRecord);
+
Review Comment:
the .apply can throw an exception. This didn't matter previously, because
everything would be retried, but now we want to sometimes continue even through
an exception. This means we need to advance the state of the reader fraction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
