sadikovi commented on PR #38277:
URL: https://github.com/apache/spark/pull/38277#issuecomment-1280241811
@dongjoon-hyun Would you be able to review this PR?
I have read the comments on the original PR and it seems the strategy was to
just document the behaviour change. I would like to point out that
SymlinkTextInputFormat is one of the cases where this change is not safe as it
silently causes incorrect results instead of throwing an error.
I also considered an alternative fix of substituting
`SymlinkTextInputFormat` with a shim input format in `HiveTableScanExec` that
correctly sets those fields. This would be transparent to the users, they could
still specify the original one.
Maybe it would be better to implement this instead of disabling the flag, so
let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
