Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19277#discussion_r139894546
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -351,11 +351,11 @@ private[spark] object EventLoggingListener extends
Logging {
// Since we sanitize the app ID to not include periods, it is safe to
split on it
val logName = log.getName.stripSuffix(IN_PROGRESS)
val codecName: Option[String] = logName.split("\\.").tail.lastOption
- val codec = codecName.map { c =>
- codecMap.getOrElseUpdate(c, CompressionCodec.createCodec(new
SparkConf, c))
- }
-
+
try {
+ val codec = codecName.map { c =>
+ codecMap.getOrElseUpdate(c, CompressionCodec.createCodec(new
SparkConf, c))
+ }
codec.map(_.compressedInputStream(in)).getOrElse(in)
} catch {
case e: Exception =>
--- End diff --
I also note this doesn't handle Throwable, nor does a similar block earlier
in the file. In case of say OutOfMemoryError it wouldn't be closed
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
