ahmedabu98 commented on PR #32939: URL: https://github.com/apache/beam/pull/32939#issuecomment-2437718999
We have a relatively thin [RecordWriter](https://github.com/apache/beam/blob/master/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/RecordWriter.java) wrapper that uses Parquet and Avro writers. Instances of this class are blind to the data file's partition key and spec. RecordWriterManager takes care of routing records to the correct destination and partition. If it helps, we can certainly move the new partition logic in this PR to RecordWriterManager. I can understand the argument that utils isn't the place for such logic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
