Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20706#discussion_r171927685
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -1018,70 +1016,18 @@ private[spark] object Utils extends Logging {
" " + (System.currentTimeMillis - startTimeMs) + " ms"
}
- private def listFilesSafely(file: File): Seq[File] = {
- if (file.exists()) {
- val files = file.listFiles()
- if (files == null) {
- throw new IOException("Failed to list files for dir: " + file)
- }
- files
- } else {
- List()
- }
- }
-
- /**
- * Lists files recursively.
- */
- def recursiveList(f: File): Array[File] = {
--- End diff --
I disagree that we should keep things in production code "just in case".
This is also an example of a function that if used without care is
dangerous. `Files.walk` is a much better way of achieving this.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
