MonkeyCanCode opened a new pull request, #4535: URL: https://github.com/apache/polaris/pull/4535
<!-- ๐ Describe what changes you're proposing, especially breaking or user-facing changes. ๐ See https://github.com/apache/polaris/blob/main/CONTRIBUTING.md for more. --> This PR added spark 4 support for polaris spark client (this was previously worked on by @adam-christian-software via https://github.com/apache/polaris/pull/3188), this also issues https://github.com/apache/polaris/issues/3021. Beside the spark 4 support, this PR also fixes couple nits while I was working on this change such as: 1. wrong reference path to README.md 2. typo in various files 3. Minor wording changes on `plugins/spark/README.md` to make it more generic to support both spark v3.5 and v4.0 For reference, while I was working on this one, I noticed we can't just go to latest Iceberg support spark version (4.1) due to lack of dependency for `org.apache.hudi:hudi-spark4.1-bundle` (the latest released one is `org.apache.hudi:hudi-spark4.0-bundle`). Thus, I used spark 4.0.2. While official delta doc mentioned 4.2.0 is supported for 4.0.x spark, but I ended up running into issues. For now, I am using delta `4.0.1` which is functional. I will do a follow up PR to bump those and debug more. ## Checklist - [x] ๐ก๏ธ Don't disclose security issues! (contact [email protected]) - [x] ๐ Clearly explained why the changes are needed, or linked related issues: Fixes # - [x] ๐งช Added/updated tests with good coverage, or manually tested (and explained how) - [x] ๐ก Added comments for complex logic - [x] ๐งพ Updated `CHANGELOG.md` (if needed) - [x] ๐ Updated documentation in `site/content/in-dev/unreleased` (if needed) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
