HyukjinKwon commented on a change in pull request #31464:
URL: https://github.com/apache/spark/pull/31464#discussion_r569906864
##########
File path: core/src/main/scala/org/apache/spark/util/Utils.scala
##########
@@ -2989,6 +2989,9 @@ private[spark] object Utils extends Logging {
metadata.append(paths(index).toString)
index += 1
}
+ if (paths.length > index) {
+ metadata.append(s", ... ${paths.length - index} more")
Review comment:
The original PR reduced the size of memory usage, which is fine. I think
the current change is okay as well to make sure raw metadata shows that the
paths are abbreviate and truncated.
We will just have to make sure it looks pretty in both printed output and
raw metadata.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
