dongjoon-hyun opened a new pull request, #57150: URL: https://github.com/apache/spark/pull/57150
### What changes were proposed in this pull request? This PR aims to remove the Java-based un-tarring workaround (`unTarUsingJava`) from `Utils.unpack` and use Hadoop's `FileUtil.unTar` for `.tar` files again, like `.tar.gz` and `.tgz` files. ### Why are the changes needed? SPARK-38631 introduced this workaround because Hadoop's `FileUtil.unTar` did not escape the file name before passing it to a shell command, which allowed arbitrary shell command injection. This was fixed by HADOOP-18136 via `FileUtil.makeSecureShellPath` and shipped in Apache Hadoop 3.4.0+. Since Apache Spark requires Hadoop 3.4.0 or later (SPARK-58053) and the master branch uses Hadoop 3.5.0, the workaround is no longer needed. Removing it also has the following benefits. - `.tar` files keep the original file permissions and symlinks because the un-tarring is delegated to the native `tar` utility, which resolves the original TODO of SPARK-38632. - The `setAccessible(true)` reflection hack on a Hadoop private method is removed. ### Does this PR introduce _any_ user-facing change? Yes, unpacking `.tar` archives now preserves file permissions and symlinks like `.tar.gz` and `.tgz` archives. This is the original behavior. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
