malinjawi commented on PR #12823: URL: https://github.com/apache/gluten/pull/12823#issuecomment-5437083815
Hi @infvg, we found an issue after this change when the Iceberg runtime is on the classpath. Tracking issue: #12912. `IcebergSparkSessionExtensions` is now enabled for every Gluten session. Iceberg’s row-lineage rules then process non-Iceberg DataSource V2 tables, causing `UPDATE` and `MERGE` operations to fail with: ```text scala.MatchError: org.apache.spark.sql.connector.catalog.InMemoryRowLevelOperationTable ``` The extension can also unexpectedly wrap other configured SQL parsers. Community CI passed because the Iceberg runtime was not on the `gluten-ut` test classpath, so the Iceberg component remained inactive during those tests. The safest fix seems to be keeping the new component extension mechanism, but not registering the Iceberg extension globally. Iceberg applications and tests can configure it explicitly, and we can add the runtime to `gluten-ut` so mixed-format sessions are covered. Does this approach make sense to you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
