Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/16386
Only regarding the comment,
https://github.com/apache/spark/pull/16386#issuecomment-269386229, I have a
similar (rather combined) idea that we provide another option such as corrupt
file name optionally (meaning maybe the column appears only when user
explicitly set for backwards compatibility), don't add a column by
`columnNameOfCorruptRecord` with a proper documentation in `wholeFile` mode and
issue a warning message if `columnNameOfCorruptRecord` is set by user in
`wholeFile` mode. This is a bit complicated idea that might make users confused
though. I am not sure if it is the best idea.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
