jackye1995 commented on PR #7105: URL: https://github.com/apache/iceberg/pull/7105#issuecomment-1482226297
> We may end up supporting both copy-on-write (single output file) and merge-on-read (multiple output files) as it boils down to whether the user wants faster read time or writing time. This makes me feel if we should really use a storage for this. Technically it sound like we should use a key-value store for `(partition, spec-id) -> (statistics values)` instead of a file-based storage. Maybe we should build an interface for it and just make storage a specific implementation, instead of saying it has to be backed by a file or multiple files, because it is has to be inefficient in read or write in that way. Just a thought. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
