lgbaeza commented on issue #5369:
URL: https://github.com/apache/iceberg/issues/5369#issuecomment-1217304300
> Updated the config to this (based on iceberg table configuration):
>
> `spark = SparkSession.builder.config("spark.driver.memory",
"25g").config("spark.sql.catalog.spark_catalog",
"org.apache.iceberg.spark.SparkSessionCatalog").config("spark.sql.catalog.spark_catalog.type",
"hive").appName(app_name).getOrCreate()`
>
> I'm getting this new error - `An error occurred while calling o87.sql.
Cannot find catalog plugin class for catalog 'spark_catalog':
org.apache.iceberg.spark.SparkSessionCatalog`
That error seems to be due to not having the dependencies for iceberg. Have
you configured your glue job with the iceberg connector from marketplace? Here
you can find how
https://aws.amazon.com/es/blogs/big-data/implement-a-cdc-based-upsert-in-a-data-lake-using-apache-iceberg-and-aws-glue/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
