Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/3293#discussion_r20417560
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -759,18 +759,9 @@ private[spark] object Utils extends Logging {
if (file != null) {
try {
if (file.isDirectory && !isSymlink(file)) {
- var savedIOException: IOException = null
- for (child <- listFilesSafely(file)) {
- try {
- deleteRecursively(child)
- } catch {
- // In case of multiple exceptions, only last one will be
thrown
- case ioe: IOException => savedIOException = ioe
- }
- }
- if (savedIOException != null) {
- throw savedIOException
- }
+ listFilesSafely(file).map(child =>
Try(deleteRecursively(child))).
--- End diff --
Iterators are lazy, however. If listFilesSafely were changed to return an
iterator instead of list, this would subtly change behavior here. I am not for
this change, as it makes the behavior less obvious. The prior form clearly
indicated the intended and actual behavior; here it is not clear which one is
intended, even if it is clear to the particular reader which one is actually
occurring.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
