Hello, I am running into a strange situation trying to use windows and FileIO properly.
I have a KafkaIO source, followed by a DoFn that assigns timestamps based on a field in the input record using outputWithTimestamp. After that, I apply FixedWindows of 1 hour duration on these elements. I need to write these windows to disk in parts, with a constraint that each part be of a certain size (except the last part). So, I made the FixedWindow trigger repeatedly, once per element, and implemented a stateful DoFn that collects these elements until the size limit is reached and outputs an Iterable. However, now I see that these elements are in separate panes. FileIO's behavior is that "writing happens by default per window and pane" (per javadoc), and this is what I am observing too. I get a bunch of files, instead of one for the Iterable. Is there any way I can make FileIO write that Iterable to a single file? Thank you - Pradyumna
