GitHub user dgingrich opened a pull request:
https://github.com/apache/spark/pull/17227
[SPARK-19507][PySpark][SQL] Show field name in _verify_type error
## What changes were proposed in this pull request?
Add better error messages to _verify_type to track down which columns are
not compliant with the schema.
## How was this patch tested?
Unit tests (incomplete), doctest, hand inspection in REPL.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dgingrich/spark topic-spark-19507-verify-types
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17227.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17227
----
commit ad5e5e5e5ed8396efca4c61eb0219fcd5a5e2caf
Author: David Gingrich <da...@textio.com>
Date: 2017-02-28T08:05:00Z
Remove "# noqa" comment from docstring
commit 5f72a547a948b5c5a787aace52df04bc8503888b
Author: David Gingrich <da...@textio.com>
Date: 2017-02-28T08:09:59Z
WIP: Add name parameter and better debugging to _verify_types
* Add name paramter to _verify_types
* Include name parameter in debug messages
* Build name message for nested structs, arrays, and maps
* Add detailed tests to flesh out spec for _verify_types (WIP)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
