Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/6222#issuecomment-105083162
I've pushed a change which will cause `ClassCastExceptions` to be thrown as
early as possible in the Scala Row -> Catalyst Row conversion process. I've
realized that this is going to fail a different set of tests due some
null-handling corner-cases that I've missed, so I'm going to create a new
`CatalystTypeConvertersSuite` and will try to write some test cases to serve as
a spec for the converter behavior.
Technically, it looks like our Scaladoc API docs give us the leeway to
throw exceptions if the input data doesn't match the expected schema and
there's no mention of implicit casting / widening, so I think it's
_technically_ okay to change this undefined behavior, although it could be
frustrating for users.
```scala
* :: DeveloperApi ::
* Creates a [[DataFrame]] from an [[RDD]] containing [[Row]]s using the
given schema.
* It is important to make sure that the structure of every [[Row]] of
the provided RDD matches
* the provided schema. Otherwise, there will be runtime exception.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
