This is an automated email from the ASF dual-hosted git repository.
srowen pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 775a829 [SPARK-38305][CORE] Explicitly check if source exists in
unpack() before calling FileUtil methods
775a829 is described below
commit 775a829c9de3717a8f298146dde0d57dd7c0ab11
Author: Sean Owen <[email protected]>
AuthorDate: Fri Feb 25 08:34:04 2022 -0600
[SPARK-38305][CORE] Explicitly check if source exists in unpack() before
calling FileUtil methods
### What changes were proposed in this pull request?
Explicitly check existence of source file in Utils.unpack before calling
Hadoop FileUtil methods
### Why are the changes needed?
A discussion from the Hadoop community raised a potential issue in calling
these methods when a file doesn't exist.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests
Closes #35632 from srowen/SPARK-38305.
Authored-by: Sean Owen <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
(cherry picked from commit 64e1f28f1626247cc1361dcb395288227454ca8f)
Signed-off-by: Sean Owen <[email protected]>
---
core/src/main/scala/org/apache/spark/util/Utils.scala | 3 +++
1 file changed, 3 insertions(+)
diff --git a/core/src/main/scala/org/apache/spark/util/Utils.scala
b/core/src/main/scala/org/apache/spark/util/Utils.scala
index 31514c8..776c8c4 100644
--- a/core/src/main/scala/org/apache/spark/util/Utils.scala
+++ b/core/src/main/scala/org/apache/spark/util/Utils.scala
@@ -585,6 +585,9 @@ private[spark] object Utils extends Logging {
* basically copied from `org.apache.hadoop.yarn.util.FSDownload.unpack`.
*/
def unpack(source: File, dest: File): Unit = {
+ if (!source.exists()) {
+ throw new FileNotFoundException(source.getAbsolutePath)
+ }
val lowerSrc = StringUtils.toLowerCase(source.getName)
if (lowerSrc.endsWith(".jar")) {
RunJar.unJar(source, dest, RunJar.MATCH_ANY)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
