OLD:
{
"namespace": "com.hello.world",
"type": "record",
"name": "Toto",
"fields": [
{
"name": "a",
"type": [
"string",
"null"
]
},
{
"name": "b",
"type": "string"
}
]
}
NEW:
{
"namespace": "com.hello.world",
"type": "record",
"name": "Toto",
"fields": [
{
"name": "a",
"type": [
"string",
"null"
]
},
{
"name": "b",
"type": "string"
},
{
"name": "c",
"type": "string",
"default": "na"
}
]
}
Sorry bad copy paste. The Avro Schema should be fine because I'm using
sbt-avro to generate the class from it.
On Wed, Apr 8, 2015 at 6:57 PM, Lukas Steiblys <[email protected]> wrote:
> The schema is not valid JSON. Maybe you forgot the “[“ after “fields:”?
>
> Lukas
> *From:* Nicolas Phung <[email protected]>
> *Sent:* Wednesday, April 8, 2015 9:45 AM
> *To:* [email protected]
> *Subject:* Issue with reading old data with a new Avro Schema
>
> Hello,
>
> I'm trying to read old avro binary data with a new schema (I add a new
> field).
>
> This is the Avro Schema (OLD) I was using to write Avro binary data
> before:
> {
> "namespace": "com.hello.world",
> "type": "record",
> "name": "Toto",
> "fields":
> {
> "name": "a",
> "type": [
> "string",
> "null"
> ]
> },
> {
> "name": "b",
> "type": "string"
> }
> ]
> }
>
> This is the Avro Schema (NEW) I'm using to read the Avro binary data :
>
> {
> "namespace": "com.hello.world",
> "type": "record",
> "name": "Toto",
> "fields":
> {
> "name": "a",
> "type": [
> "string",
> "null"
> ]
> },
> {
> "name": "b",
> "type": "string"
> },
> {
> "name": "c",
> "type": "string",
> "default": "na"
> }
> ]
> }
>
> However, I can't read the old data with the new Schema. I've got the
> following errors :
>
> 15/04/08 17:32:22 ERROR executor.Executor: Exception in task 0.0 in stage
> 3.0 (TID 3)
> java.io.EOFException
> at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
> at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
> at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259)
> at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:272)
> at
> org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:113)
> at
> org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:353)
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:157)
> at
> org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
> at
> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
> at com.miguno.kafka.avro.AvroDecoder.fromBytes(AvroDecoder.scala:31)
>
> From my understanding, I should be able to read the old data with the new
> schema that contains a new field with a default value. But it doesn't seem
> to work. Am I doing something wrong ?
>
> I have posted a report https://issues.apache.org/jira/browse/AVRO-1661
>
> Regards,
> Nicolas PHUNG
>
