dungdm93 opened a new pull request #4132:
URL: https://github.com/apache/iceberg/pull/4132
Currently, `TaskWriter` implementations (with its inner classes) are quite
complex, it take several responsibilities:
* handle different kinds of records (e.g insert, update, delete)
* handle partition if needed
* rolling to new file when current file size is to large
Consequently, each engine usually has more than one class to handle
different cases.
Thanks to newly `FileWriter` and `PartitioningWriter` interfaces introduced
in #2945, now we can streamlined `TaskWriter` implementations as following:
```txt
Task Writer # handle record types (e.g insert, update, delete)
|
V
PartitioningWriter # handle partitioning, partition=null mean un-partitioned
|
V
RollingFileWriter # rolling to new file when current file size is to large
|
V
FileWriter # writing data of single type (i.e. data/delete) in one
spec/partition
|
V
FileAppender
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
