Fokko opened a new pull request, #5665: URL: https://github.com/apache/iceberg/pull/5665
I would love to get your input on this @aokolnychyi. The current PR isn't passing because it changes the output. I noticed https://github.com/apache/iceberg/issues/5641 last week, and it looks to be a bug. We read all the existing partition specs for some paths, and for some only the current one. This leads to incorrect results (an empty query, which is quite concerning) as shown in the issue. I fired up the debugger and found the issue here:  We can see the two values and just a single field in the struct. So the stats go to null, and when we read the data, Iceberg will skip the data. I also added this as a test case. The bug only shows in V2. And I think (but I'm probably missing some historical context here), that we only want to show the current partition spec (hence the change), but this would change the behavior. What do you think? Since you're the author of the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
