MonkeyCanCode commented on PR #4535: URL: https://github.com/apache/polaris/pull/4535#issuecomment-4551464567
@dimas-b So I did found out why 4.2.0 version was not working for delta-io (due to they default to 4.1 spark and we needed to update package name to add `_4.0_` to use the spark 4.0 version). However, after fixed this issue, I ran into more issue with 4.2.0 delta-io. It turns out they restructed `DeltaCatalog` which the `isUnityCatalog` got moved from `DeltaCatalog` to a new base trait called `AbstractDeltaCatalog`. This means, the existed reflection we have no longer work as it only looks for concrete class. Here is are some references I found: Delta 3.3 (in-used version by spark v.3.5): https://github.com/delta-io/delta/blob/branch-3.3/spark/src/main/scala/org/apache/spark/sql/delta/catalog/DeltaCatalog.scala#L75 Delta 4.0 (version which are using in this PR): https://github.com/delta-io/delta/blob/branch-4.0/spark/src/main/scala/org/apache/spark/sql/delta/catalog/DeltaCatalog.scala#L78 Delta 4.2 (latest): https://github.com/delta-io/delta/blob/branch-4.2/spark/src/main/scala/org/apache/spark/sql/delta/catalog/AbstractDeltaCatalog.scala#L86 (no longer existed: https://github.com/delta-io/delta/blob/branch-4.2/spark/src/main/scala/org/apache/spark/sql/delta/catalog/DeltaCatalog.scala) i would let more people review current PR before raise fixes to bump them to the latest in the next PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
