yihua opened a new pull request, #10705:
URL: https://github.com/apache/hudi/pull/10705
### Change Logs
This PR improves the exception thrown for conversion error of Avro record
for error table. Before this PR, `NullPointerException` is thrown; after this
PR, `HoodieException` with the message "Failed to convert illegal record to
json" is thrown.
Before:
```
Job aborted due to stage failure: Task 0 in stage 47.0 failed 4 times, most
recent failure: Lost task 0.3 in stage 47.0 (TID 3129) (172.18.239.121 executor
1): java.lang.NullPointerException: null of string in field *** of ***
at
org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:184)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:178)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.hudi.avro.HoodieAvroUtils.avroToJsonHelper(HoodieAvroUtils.java:216)
at
org.apache.hudi.avro.HoodieAvroUtils.avroToJsonString(HoodieAvroUtils.java:199)
at
org.apache.hudi.utilities.streamer.HoodieStreamerUtils.lambda$null$60a9e756$1(HoodieStreamerUtils.java:111)
```
After:
```
8726 [dag-scheduler-event-loop] INFO
org.apache.spark.scheduler.DAGScheduler [] - ResultStage 1 (collect at
TestHoodieStreamerUtils.java:80) failed in 0.075 s due to Job aborted due to
stage failure: Task 3 in stage 1.0 failed 1 times, most recent failure: Lost
task 3.0 in stage 1.0 (TID 7) (192.168.1.29 executor driver):
org.apache.hudi.exception.HoodieException: Failed to convert illegal record to
json
at
org.apache.hudi.utilities.streamer.HoodieStreamerUtils.generateErrorRecord(HoodieStreamerUtils.java:166)
at
org.apache.hudi.utilities.streamer.HoodieStreamerUtils.lambda$null$0(HoodieStreamerUtils.java:141)
```
New test is added to test the handling.
### Impact
Makes error easier to understand.
### Risk level
none
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
