Joorgem commented on PR #57608:
URL: https://github.com/apache/spark/pull/57608#issuecomment-5180571193
Took your wording verbatim in 4fac147 — it covers all four points more
cleanly than patching mine would have, and dropping the token-count clause is
what removes the contradiction rather than just softening it.
On point 1, I checked the bullet rather than take it on faith, and you're
right — `UnivocityParser.convert` says so in its own comment:
```scala
var badRecordException: Option[Throwable] = if (tokens.length !=
parsedSchema.length) {
// If the number of tokens doesn't match the schema, we should treat it as
a malformed record.
// However, we still have chance to parse some of the tokens. It continues
to parses the
// tokens normally and sets null when `ArrayIndexOutOfBoundsException`
occurs for missing
// tokens.
```
So it is both things at once: flagged malformed *and* parsed as far as it
goes. The null-padding half of that sentence was right and the "is not a
corrupted record" half was wrong, which is presumably how it survived — the
visible behaviour matches the true half.
Corrected in #57658 as you suggested (d61ba6b), since that PR already edits
this bullet and this one would otherwise land a self-contradicting table cell.
Flagging it there explicitly rather than slipping it in.
Point 4 is worth recording as more than a nit: that em dash was the only
non-ASCII character this PR introduced, and the file is otherwise pure ASCII.
Your text puts it back to ASCII with no substitute needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
