mohitgargk commented on issue #3941: URL: https://github.com/apache/iceberg/issues/3941#issuecomment-1023912443
@flyrain Agree on that. With CoW, without any row-identifier, it is not possible to establish link with the new state and old state of the records without ambiguities. For the CDF record structure, we grokked a bit on the design and came up with [this](https://docs.google.com/document/d/14ZRQc1anvXN52HGoNpSbKy7Ua6_7IIkr6mlXe_gHrJA/edit?usp=sharing). Above proposal is based on our use case where we are interested in change of a certain column but also want the values of other columns. If the whole state is part of CDF record, the consumer can avoid query/lookup/join with the table to get rest of the state. cc - @SreeramGarlapati @rawataaryan9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
