eric-maynard commented on code in PR #1503: URL: https://github.com/apache/polaris/pull/1503#discussion_r2071257662
########## plugins/spark/README.md: ########## @@ -72,24 +77,26 @@ bin/spark-shell \ --conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,io.delta.sql.DeltaSparkSessionExtension \ --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog \ --conf spark.sql.catalog.polaris.warehouse=<catalog-name> \ ---conf spark.sql.catalog.polaris.header.X-Iceberg-Access-Delegation=true \ +--conf spark.sql.catalog.polaris.header.X-Iceberg-Access-Delegation=vended-credentials \ --conf spark.sql.catalog.polaris=org.apache.polaris.spark.SparkCatalog \ --conf spark.sql.catalog.polaris.uri=http://localhost:8181/api/catalog \ --conf spark.sql.catalog.polaris.credential="root:secret" \ --conf spark.sql.catalog.polaris.scope='PRINCIPAL_ROLE:ALL' \ --conf spark.sql.catalog.polaris.token-refresh-enabled=true \ ---conf spark.sql.catalog.polaris.type=rest \ --conf spark.sql.sources.useV1SourceList='' ``` # Limitations The Polaris Spark client supports catalog management for both Iceberg and Delta tables, it routes all Iceberg table requests to the Iceberg REST endpoints, and routes all Delta table requests to the Generic Table REST endpoints. -Following describes the current limitations of the Polaris Spark client: +The Spark Client requires at least delta 3.2.1 to work with Delta tables, which requires at least Apache Spark 3.5.3. +Following describes the current functionality limitations of the Polaris Spark client: 1) Create table as select (CTAS) is not supported for Delta tables. As a result, the `saveAsTable` method of `Dataframe` is also not supported, since it relies on the CTAS support. 2) Create a Delta table without explicit location is not supported. 3) Rename a Delta table is not supported. 4) ALTER TABLE ... SET LOCATION/SET FILEFORMAT/ADD PARTITION is not supported for DELTA table. -5) For other non-iceberg tables like csv, there is no specific guarantee provided today. +5) For other non-Iceberg tables like csv, there is no specific guarantee provided today. +6) TABLE_WRITE_DATA privilege is not supported for Delta Table. +7) Credential Vending is not supported for Delta Table. Review Comment: It's more complicated than that; it implies exactly what it says. With Delta, the catalog doesn't take over the responsibility of the writes. But that doesn't mean that the client can't write to the Delta table. However, it can't use vended credentials to do so. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
