Suren Nihalani created AVRO-2764:
------------------------------------
Summary: Request to improve error message on type mismatch
Key: AVRO-2764
URL: https://issues.apache.org/jira/browse/AVRO-2764
Project: Apache Avro
Issue Type: Wish
Affects Versions: 1.7.7
Reporter: Suren Nihalani
Let's say I am looking at a stacktrace like this (see at the bottom). It's hard
for me to know what data field is corrupted. I'd like exception message to tell
me what's wrong so I know what I need to do
{{Caused by: org.apache.avro.AvroTypeException: Found TUPLE_1, expecting union}}
{{at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)}}
{{at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)}}
{{at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)}}
{{at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155)}}
{{at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)}}
{{at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)}}
{{at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)}}
{{at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)}}
{{at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)}}
{{at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)}}
{{at
org.apache.spark.sql.avro.AvroFileFormat$$anonfun$buildReader$1$$anon$1.next(AvroFileFormat.scala:302)}}
{{at
org.apache.spark.sql.avro.AvroFileFormat$$anonfun$buildReader$1$$anon$1.next(AvroFileFormat.scala:282)}}
{{at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)}}
{{at
org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.next(FileScanRDD.scala:101)}}
{{at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.agg_doAggregateWithKeys$(Unknown
Source)}}
{{at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown
Source)}}
{{at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)}}
{{at
org.apache.spark.sql.execution.WholeStageCodegenExec$$anonfun$12$$anon$1.hasNext(WholeStageCodegenExec.scala:634)}}
{{at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:408)}}
{{at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:125)}}
{{at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)}}
{{at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)}}
{{at org.apache.spark.scheduler.Task.run(Task.scala:109)}}
{{at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:384)}}
{{at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{at java.lang.Thread.run(Thread.java:748)}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
