rdblue commented on pull request #796: URL: https://github.com/apache/iceberg/pull/796#issuecomment-664638390
@jerryshao, I gave it another review pass. There are a few issues to fix in the source, and I think it would be great to clean up the tests to use separate cases, context in assertions, and to correct the order of values passed to assertions. We should also consider separating out the MicroBatch code to its own commit. I think that's ready to be committed and we could work on it in parallel. Specifically, the Avro readers now support a `_pos` metadata column that would allow using actual offsets even if the files are filtered. We'd need to add support for it, but we can do that separately and get the review for the structured streaming part done here. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
