Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/22557#discussion_r221056816
--- Diff:
core/src/main/scala/org/apache/spark/security/CryptoStreamUtils.scala ---
@@ -157,6 +165,111 @@ private[spark] object CryptoStreamUtils extends
Logging {
}
+ /**
+ * SPARK-25535. The commons-cryto library will throw InternalError if
something goes
+ * wrong, and leave bad state behind in the Java wrappers, so it's not
safe to use them
+ * afterwards. This wrapper detects that situation and avoids further
calls into the
+ * commons-crypto code, while still allowing the underlying streams to
be closed.
+ *
+ * This should be removed once CRYPTO-141 is fixed (and Spark upgrades
its commons-crypto
+ * dependency).
+ */
+ trait BaseErrorHandler extends Closeable {
+
+ private var closed = false
+
+ protected def cipherStream: Closeable
+ protected def wrapped: Closeable
--- End diff --
its a little confusing that there are two closeables. I'd add a comment
here that `wrapped` is what the `cipherStream` is already wrapping, and we want
to make sure it gets closed if the `cipherStream` has an internal error.
(when I see a variable named `wrapped`, I assumed it was what *this* was
wrapping, not what the cipherStream was wrapping)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
