ammarchalifah commented on PR #16424: URL: https://github.com/apache/iceberg/pull/16424#issuecomment-4558270656
> I am running the tests you added to the PR, so far I have not reproduced the issue on any Spark build. > > Took your PR, removed the changes to Internal Row. Ran for example > > ``` > ./gradlew :iceberg-spark:iceberg-spark-4.1_2.13:test --tests "org.apache.iceberg.spark.sql.TestStoragePartitionedJoins.testJoinsWithBucketingOnStringColumn" --tests "org.apache.iceberg.spark.sql.TestStoragePartitionedJoins.testJoinsWithIdentityAndBucketOnStringColumn" -DsparkVersions=4.1 > ``` > > Passed > > Spark version testJoinsWithBucketingOnStringColumn testJoinsWithIdentityAndBucketOnStringColumn Total > 3.4 2/2 pass 2/2 pass 4/4 > 3.5 2/2 pass 2/2 pass 4/4 > 4.0 2/2 pass 2/2 pass 4/4 > 4.1 2/2 pass 2/2 pass 4/4 Hmm, this is confusing. In my production case, my actual table has more partition keys, and I'm doing a join on a subset of key with `spark.sql.sources.v2.bucketing.allowJoinKeysSubsetOfPartitionKeys.enabled`. Let me add additional test case that mirror this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
