Hi Doug, thanks for responding.
If you look at the parsing code here:
http://svn.apache.org/viewvc/avro/trunk/lang/py3/avro/schema.py?view=markup#l1154
you can see it checking to see if what the "value" of the type field is. In
this code, it _must_ be a "primitive type" (int, string, etc.) or a "named
type" (record, enum, etc.). But in this case, it's neither. The value of the
type field is a dictionary which _contains_ the record definition.
It throws an exception because you can't use a dictionary like that.
But you say that this objfsptr def is valid?
[{'name': 'objfsptr',
'type': {'fields': [{'name': 'uri', 'type': 'string'}],
'name': 'ObjFSPtr',
'namespace': 'FooServiceType',
'type': 'record'}}]
Is this also valid? Identical?
[{'name': 'objfsptr',
'fields': [{'name': 'uri', 'type': 'string'}],
'name': 'ObjFSPtr',
'namespace': 'FooServiceType',
'type': 'record'}]
Jonathan
________________________________________
From: Doug Cutting [[email protected]]
Sent: Monday, October 27, 2014 17:31
To: [email protected]
Subject: Re: avdl schema compatibility
The type of the objfsptr parameter looks correct to me. Rather than a
reference to the name of the record it has the record's definition.
What looks incorrect about this to you?
That said, I have no idea what's causing that error.
Doug
On Sun, Oct 26, 2014 at 12:20 PM, Camp, Jonathan
<[email protected]> wrote:
> Hi All, I am trying to use avro's protocol definition to provide the basis
> for an RPC system. In this case, I'll be handling the handshaking and
> transport layer rather the avro's IPC package. Are the schemas generated from
> the protocol files compatible with the standard DataFileWriter/Reader schemas
> (I am using python3)? If I try to load a file generated from an avdl message
> (using DataFileWriter(bytes, DatumWriter(),
> protocol.message_map['hash'].request)) then when I try to load it, I receive
> error messages stating that the writer_schema is essentially invalid (see
> stacktrace below).
>
> Here is a code snippet that seems to show an invalid schema being generated.
> I think its because the value of the 'type' field (for objfsptr) is a
> python/json dictionary rather that the 'record' type i think its supposed to
> be.
>
> import avro.protocol
> p = avro.protocol.Parse(open("service.avpr").read())
> print(p.message_map['hash'].request.to_json())
>
> [{'name': 'objfsptr',
> 'type': {'fields': [{'name': 'uri', 'type': 'string'}],
> 'name': 'ObjFSPtr',
> 'namespace': 'FooServiceType',
> 'type': 'record'}},
> {'default': 10, 'name': 'x', 'type': 'int'}]
>
>
> Stacktrace from a deserialization attempt:
>
> Traceback (most recent call last):
> File "./schema_test.py", line 74, in <module>
> requests = deserializer.deserialize('hash', srequest)
> File "/data/bluecoat/services.py", line 123, in deserialize
> reader = DataFileReader(io.BytesIO(raw_bytes), DatumReader())
> File "/data/avro/datafile.py", line 363, in __init__
> schema.Parse(self.GetMeta(SCHEMA_KEY).decode('utf-8')))
> File "/data/avro/schema.py", line 1283, in Parse
> return SchemaFromJSONData(json_data, names)
> File "/data/avro/schema.py", line 1254, in SchemaFromJSONData
> return parser(json_data, names=names)
> File "/data/avro/schema.py", line 1142, in _SchemaFromJSONArray
> return UnionSchema(map(MakeSchema, json_array))
> File "/data/avro/schema.py", line 866, in __init__
> self._schemas = tuple(schemas)
> File "/data/avro/schema.py", line 1141, in MakeSchema
> return SchemaFromJSONData(json_data=desc, names=names)
> File "/data/avro/schema.py", line 1254, in SchemaFromJSONData
> return parser(json_data, names=names)
> File "/data/avro/schema.py", line 1154, in _SchemaFromJSONObject
> if type in PRIMITIVE_TYPES:
> TypeError: unhashable type: 'dict'
>
>
>
>
> service.avdl
>
> @namespace("FooService")
> protocol MyProtocol {
>
> @namespace("FooServiceType")
> record ObjFSPtr {
> string uri;
> }
>
> string hash(FooServiceType.ObjFSPtr objfsptr, int x=10);
> }
>
>
> generated service.avpr:
>
> {
> "protocol" : "MyProtocol",
> "namespace" : "FooService",
> "types" : [ {
> "type" : "record",
> "name" : "ObjFSPtr",
> "namespace" : "FooServiceType",
> "fields" : [ {
> "name" : "uri",
> "type" : "string"
> } ]
> } ],
> "messages" : {
> "hash" : {
> "request" : [ {
> "name" : "objfsptr",
> "type" : "FooServiceType.ObjFSPtr"
> }, {
> "name" : "x",
> "type" : "int",
> "default" : 10
> } ],
> "response" : "string"
> }
> }
> }
>
>
> Thanks!
>
> Jonathan Camp
