avdl schema compatibility

[Camp, Jonathan]

Hi All, I am trying to use avro's protocol definition to provide the basis for 
an RPC system. In this case, I'll be handling the handshaking and transport 
layer rather the avro's IPC package. Are the schemas generated from the 
protocol files compatible with the standard DataFileWriter/Reader schemas (I am 
using python3)? If I try to load a file generated from an avdl message (using 
DataFileWriter(bytes, DatumWriter(), protocol.message_map['hash'].request)) 
then when I try to load it, I receive error messages stating that the 
writer_schema is essentially invalid (see stacktrace below).

Here is a code snippet that seems to show an invalid schema being generated. I 
think its because the value of the 'type' field (for objfsptr) is a python/json 
dictionary rather that the 'record' type i think its supposed to be.

import avro.protocol
p = avro.protocol.Parse(open("service.avpr").read())
print(p.message_map['hash'].request.to_json())

[{'name': 'objfsptr',
  'type': {'fields': [{'name': 'uri', 'type': 'string'}],
           'name': 'ObjFSPtr',
           'namespace': 'FooServiceType',
           'type': 'record'}},
 {'default': 10, 'name': 'x', 'type': 'int'}]


Stacktrace from a deserialization attempt:

Traceback (most recent call last):
  File "./schema_test.py", line 74, in <module>
    requests = deserializer.deserialize('hash', srequest)
  File "/data/bluecoat/services.py", line 123, in deserialize
    reader = DataFileReader(io.BytesIO(raw_bytes), DatumReader())
  File "/data/avro/datafile.py", line 363, in __init__
    schema.Parse(self.GetMeta(SCHEMA_KEY).decode('utf-8')))
  File "/data/avro/schema.py", line 1283, in Parse
    return SchemaFromJSONData(json_data, names)
  File "/data/avro/schema.py", line 1254, in SchemaFromJSONData
    return parser(json_data, names=names)
  File "/data/avro/schema.py", line 1142, in _SchemaFromJSONArray
    return UnionSchema(map(MakeSchema, json_array))
  File "/data/avro/schema.py", line 866, in __init__
    self._schemas = tuple(schemas)
  File "/data/avro/schema.py", line 1141, in MakeSchema
    return SchemaFromJSONData(json_data=desc, names=names)
  File "/data/avro/schema.py", line 1254, in SchemaFromJSONData
    return parser(json_data, names=names)
  File "/data/avro/schema.py", line 1154, in _SchemaFromJSONObject
    if type in PRIMITIVE_TYPES:
TypeError: unhashable type: 'dict'




service.avdl

@namespace("FooService")
protocol MyProtocol {

    @namespace("FooServiceType")
    record ObjFSPtr {
        string uri;
    }

    string hash(FooServiceType.ObjFSPtr objfsptr, int x=10);
}


generated service.avpr:

{
  "protocol" : "MyProtocol",
  "namespace" : "FooService",
  "types" : [ {
    "type" : "record",
    "name" : "ObjFSPtr",
    "namespace" : "FooServiceType",
    "fields" : [ {
      "name" : "uri",
      "type" : "string"
    } ]
  } ],
  "messages" : {
    "hash" : {
      "request" : [ {
        "name" : "objfsptr",
        "type" : "FooServiceType.ObjFSPtr"
      }, {
        "name" : "x",
        "type" : "int",
        "default" : 10
      } ],
      "response" : "string"
    }
  }
}


Thanks!

Jonathan Camp