GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22895
[SPARK-25886][SQL] Improve error message of `FailureSafeParser` and
`from_avro` in FAILFAST mode
## What changes were proposed in this pull request?
Currently in `FailureSafeParser` and `from_avro`, the exception is created
with such code
```
throw new SparkException("Malformed records are detected in record parsing.
" +
s"Parse Mode: ${FailFastMode.name}.", e.cause)
```
1. The cause part should be `e` instead of `e.cause`
2. If `e` contains non-null message, it should be shown in
`from_json`/`from_csv`/`from_avro`, e.g.
```
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('1'
(code 49)): was expecting a colon to separate field name and value
at [Source: (InputStreamReader); line: 1, column: 7]
```
3.Kindly show hint for trying PERMISSIVE in error message.
## How was this patch tested?
Unit test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gengliangwang/spark improve_error_msg
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22895.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22895
----
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
