RE: Need help with default in avro

[Tejal Khot]

Thanks Scott for your response, but I am still confused.
Can you please clarify? I am not finding any examples on this.
If a field is specified as default and is not present as part of actual data, 
doesn't the reader use the default value for missing field?

So does that mean one should always have a reader's schema and writer's schema?

Please help.

Thanks in advance.

________________________________
From: Scott Carey [mailto:sc...@richrelevance.com]
Sent: Thursday, November 04, 2010 2:13 PM
To: user@avro.apache.org; Tejal Khot
Subject: Re: Need help with default in avro

I suspect that you need to inform the GenericDatumReader what the "actual" 
versus "expected" schemas are.

The stack trace below has ValidatingDecoder in it, but not ResolvingDecoder.  
ResolvingDecoder is what populates defaults.

GenericDatumReader has two methods for the schema:

setSchema() and setExpected()

expected is the "reader's" schema.  The actual schema of the data is set via 
setSchema() and is sometimes referred to as the "writer's" schema.

When GenericDatumReader has its actual and expected schemas set differently, it 
activates a ResolvingDecoder to handle making the written schema data appear 
like the reader's expected schema.

-Scott



On Nov 4, 2010, at 10:23 AM, Tejal Khot wrote:




Hi,

I am new to avro and I have been trying to resolve the AvroTypeException when 
using "default" with the avro schema:
In the schema below, the service_wait field is a Boolean and I want to specify 
a default value. However, I keep getting the exception below. What is the 
correct way to specify defaults?

org.apache.avro.AvroTypeException: Expected field name service_wait got 
select_branch
            at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:396)
            at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)

Sample Schema:

{
  "type": "record",
  "name": "TestRequest",
  "fields" : [
    {"name": "ver", "type": "string"},
    {"name": "trace_id", "type": "int"},
    {"name": "error_at", "type": "string"},
    {"name": "exception_at", "type": "string"},
    {"name": "bytes", "type": "bytes"},
    {"name": "need_trace_buffer", "type": "boolean"},
    {"name": "service_wait", "type": "boolean", "default" :"true"},
    {"name": "select_branch", "type": "string"},
    {
      "name": "serviceData",
      "type": {
        "type": "array",
        "items":  {
          "type": "record",
          "name": "ServiceData",
          "fields" : [
            {"name": "nm", "type": "string"},
            {"name": "latency", "type": "int"},
            {"name": "req", "type": "bytes"}
          ]
        }
      }
    }


  ]
}

Sample data file is as below:
{
    "ver" : "1.0",
    "trace_id" : 3000301,
    "error_at" : "",
    "exception_at" : "",
    "bytes" : "1234567" ,
    "need_trace_buffer" : false,
    "service_wait" : true,
    "select_branch" : "",
    "serviceData" : [
        {
            "nm" : "HttpService3",
            "latency" : 5,
            "req" : "/"
        }
    ]
}



The exception:

org.apache.avro.AvroTypeException: Expected field name service_wait got 
select_branch
            at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:396)
            at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
            at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:67)
            at org.apache.avro.io.JsonDecoder.readBoolean(JsonDecoder.java:96)
            at 
org.apache.avro.io.ValidatingDecoder.readBoolean(ValidatingDecoder.java:63)
            at 
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:92)
            at 
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:108)
            at 
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:80)
            at 
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:71)
            at 
com.yahoo.dataintensive.streaming.streaming2transporter.AvroSchemaTest1.main(AvroSchemaTest1.java:32)


Thanks,
Tejal