Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5407#discussion_r28050423
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala ---
@@ -248,18 +253,23 @@ object CheckpointReader extends Logging {
checkpointFiles.foreach(file => {
logInfo("Attempting to load checkpoint from file " + file)
try {
- val fis = fs.open(file)
- // ObjectInputStream uses the last defined user-defined class
loader in the stack
- // to find classes, which maybe the wrong class loader. Hence, a
inherited version
- // of ObjectInputStream is used to explicitly use the current
thread's default class
- // loader to find and load classes. This is a well know Java issue
and has popped up
- // in other places (e.g.,
http://jira.codehaus.org/browse/GROOVY-1627)
- val zis = compressionCodec.compressedInputStream(fis)
- val ois = new ObjectInputStreamWithLoader(zis,
- Thread.currentThread().getContextClassLoader)
- val cp = ois.readObject.asInstanceOf[Checkpoint]
- ois.close()
- fs.close()
+ var ois: ObjectInputStreamWithLoader = null
+ var cp: Checkpoint = null
+ Utils.tryWithSafeFinally {
+ val fis = fs.open(file)
+ // ObjectInputStream uses the last defined user-defined class
loader in the stack
+ // to find classes, which maybe the wrong class loader. Hence, a
inherited version
+ // of ObjectInputStream is used to explicitly use the current
thread's default class
+ // loader to find and load classes. This is a well know Java
issue and has popped up
+ // in other places (e.g.,
http://jira.codehaus.org/browse/GROOVY-1627)
+ val zis = compressionCodec.compressedInputStream(fis)
+ ois = new ObjectInputStreamWithLoader(zis,
+ Thread.currentThread().getContextClassLoader)
+ cp = ois.readObject.asInstanceOf[Checkpoint]
+ } {
+ ois.close()
+ fs.close()
--- End diff --
I know this code was there before, but, am I crazy -- are we not supposed
to close HDFS `FileSystem` objects? this may be closing a cached `FileSystem`.
It also looks like it meant to close `fis`. I suggest just closing `ois`.
No idea about the MiMa warning.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
