azwanzuharimi commented on PR #3420: URL: https://github.com/apache/iceberg-python/pull/3420#issuecomment-5886556406
Thank you for this PR. I tested it as part of #4027. The bounded scan filter works, but with 20,000 keys the scan returns about 100,000 rows and the insert path then builds an exact Or over them and crashes (exit 138 on macOS, pyarrow 25.0.1). #4027 uses the same idea, a superset filter for the scan, and adds an exact anti join in Arrow for the insert path and a re-append of removed rows for the overwrite path. The result table is in that PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
