Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19458
Yes, I agree in any case it should not throw an exception. But in this PR
you filtered out temp shuffle/local blocks, do you think this block is valid or
not, are they blocks?
So I'd like not filtering out those blocks, instead adding two parsing
rules for those blocks. And for any other illegal files (cannot be parsed)
catch and log the exception.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
