AngersZhuuuu commented on issue #25218: [SPARK-21067][SQL][FOLLOW-UP] fix sts FileSystem closed error when to call Hive.moveFile URL: https://github.com/apache/spark/pull/25218#issuecomment-513593803 > Yikes, this looks complicated. Is there any other simpler fix if this is meant to be a quick follow up or was this a separate existing issue? Since about HDFS Encryption, code in each version of hive is different, in hive v1.1/v1.2/v1.3/v2.0/v2.0 hdfsEncryptionShim will be initialized on one FS, then above error will happen, in v2.2/v2.3/v3.0, hdfsEncryptionShim's API has changed a lot. It's hard to change hive code, and many user may use their own hive version jars.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
