MonkeyCanCode commented on PR #4535: URL: https://github.com/apache/polaris/pull/4535#issuecomment-4539050648
> Thanks for pushing this forward, @MonkeyCanCode ! > > My main concern is reusing as much code as possible between Spark 3.5 and 4.0 modules. > > Also, it might be easier to move "fixes" into a separate PRs and merge them right away. > > Isolating "regtest" changes can also simplify review of the main Spark 4.0 code... but it's not critical from my POV. Hello @dimas-b , Thanks for the quick review. I am not sure if this is preferred to mixed regetests code between spark 3.5 and 4.0 as Iceberg handle them on diff dirs completely without shared code. There are couple of things which for sure can be shared by moving them to a base dir/package then refs them, but I am not sure if we will have breaking changes or not or making maintain two version too messy (e.g. only only changes in spark itself, there can be incompatible changes on the table format as well such as delta lake 3.x and 4.x are not compatible across spark version.) Thanks, Yong Zheng -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
