andygrove commented on pull request #35573:
URL: https://github.com/apache/spark/pull/35573#issuecomment-1046116794
> I see, so the most consistent behavior would be to not allow `"NaN"`,
whether allowing non-numeric inputs to be parsed as numbers or not?
Yes.
> So a raw literal like `NaN` is not handled by this section of code in
Spark, but by Jackson?
That is correct.
> Well in any event, I suppose it's a little better to be permissive in this
narrow case, to achieve a little more consistency, rather than 'break' existing
functionality.
Yes, I agree.
> `allowNonNumericNumbers` is on by default, so adding the additional
constraint here wouldn't affect most paths. If someone had explicitly
disallowed it, now Spark would parse things like `"NaN"` as NaN. It could be
worth a note in core-migration-guide.md for good measure.
Ok, I will add a note to the migration guide in this PR later today.
Thanks for the review so far.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
