RussellSpitzer commented on issue #2604: URL: https://github.com/apache/iceberg/issues/2604#issuecomment-843280647
The issue here is that SparkSession.getOrCreate will use any session that already exists if one does. In this case the extensions are not applied. You cannot modify the SparkConf after it has been made which is why you get the "can't modify" spark conf error. So in your case you have basically 2 options I think ... 1. find the original session creation in your code and add the property there, if this is a notebook usually you can just set the properties in the kernel definition. If this is your own code you need to find the first init yourself. 2. Force the creation of a new session (newSession) and make sure you are using that. I'm not sure this will pick up the session.conf from the old session but it couldn't hurt -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
