ffernandez92 commented on PR #29368:
URL: https://github.com/apache/beam/pull/29368#issuecomment-1805620144
I tried reading and writing in proto format with DF and it works fine.
Besides that I ran into an issue when trying to write into BQ but that's
unrelated to proto. These lines produce a null pointer
```
Integer numStreams = configuration.getNumStreams();
// Triggering frequency is only applicable for exactly-once
if (!configuration.getUseAtLeastOnceSemantics()) {
write =
write.withTriggeringFrequency(
(triggeringFrequency == null || triggeringFrequency <= 0)
? DEFAULT_TRIGGERING_FREQUENCY
: Duration.standardSeconds(triggeringFrequency));
}
// set num streams if specified, otherwise default to autoSharding
if (numStreams > 0) {
write = write.withNumStorageWriteApiStreams(numStreams);
} else if (autoSharding == null || autoSharding) {
write = write.withAutoSharding();
}
```
numStreams and configuration.getUseAtLeastOnceSemantics don't have default
values and are not mapped to standard_io.yml
@robertwb should I open an issue to fix those and make them fallback to a
default value or is someone else working on that?
<img width="884" alt="Screenshot 2023-11-10 at 13 02 29"
src="https://github.com/apache/beam/assets/20140348/ea83313a-dfd5-49b1-a8fe-cff138ca610a";>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
