sadikovi commented on PR #44129:
URL: https://github.com/apache/spark/pull/44129#issuecomment-1849246681
@dongjoon-hyun Could you check if `SPARK-46216: FileSystemPersistenceEngine
with compression` passes on Java 8? I have encountered an error when running it
on Java 8.
```
*** 1 TEST FAILED ***
PersistenceEngineSuite:
- SPARK-46216: FileSystemPersistenceEngine with compression *** FAILED ***
(660 milliseconds)
java.io.IOException: StreamClosed
at
com.github.luben.zstd.ZstdOutputStreamNoFinalizer.flush(ZstdOutputStreamNoFinalizer.java:377)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:141)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at
org.apache.spark.deploy.master.FileSystemPersistenceEngine.$anonfun$serializeIntoFile$3(FileSystemPersistenceEngine.scala:76)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:73)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:103)
at
org.apache.spark.deploy.master.FileSystemPersistenceEngine.serializeIntoFile(FileSystemPersistenceEngine.scala:72)
at
org.apache.spark.deploy.master.FileSystemPersistenceEngine.persist(FileSystemPersistenceEngine.scala:47)
at
org.apache.spark.deploy.master.PersistenceEngineSuite.testPersistenceEngine(PersistenceEngineSuite.scala:103)
```
Seems Zstd already closes the stream so calling fileOut.close() causes an
exception to be thrown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
