w l created SPARK-38664:
---------------------------
Summary: Compact EventLog fail when Event Log path has illegal
characters
Key: SPARK-38664
URL: https://issues.apache.org/jira/browse/SPARK-38664
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.2.1, 2.4.8
Reporter: w l
When a path has illegal characters, it is escaped. The file will not be found
and compactor will not work.
The path: "/tmp/app a"
Exception:
java.io.FileNotFoundException: /tmp/app%25a/events_7_app.compact.inprogress (No
such file or directory)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.(FileOutputStream.java:213)
at java.io.FileOutputStream.(FileOutputStream.java:101)
at
org.apache.spark.deploy.history.EventLogFileWriter.initLogFile(EventLogFileWriters.scala:95)
at
org.apache.spark.deploy.history.SingleEventLogFileWriter.start(EventLogFileWriters.scala:223)
at
org.apache.spark.deploy.history.EventLogFileCompactor.rewrite(EventLogFileCompactor.scala:152)
at
org.apache.spark.deploy.history.EventLogFileCompactor.compact(EventLogFileCompactor.scala:94)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
