[
https://issues.apache.org/jira/browse/AVRO-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861698#action_12861698
]
Jeff Hodges commented on AVRO-529:
----------------------------------
Okay, so a more complete example I got from Rael is:
{quote}
{
"namespace": "somenamespace"
"protocol": "Composition",
"types": [],
"messages": {
"get_things": {
"request": [
{ "name": "id", "type": "array", "items": "long" }
],
"response": { "type": "array", "items": "long" }
}
}
}
{quote}
The problem line for Wanli and Rael is the field in the request:
{quote}
"request": [
{ "name": "id", "type": "array", "items": "long" }
],
{quote}
This, unfortunately, is not allowed in avro! You see, this type has been
"flattened". It should look like
{quote}
"request": [
{ "name": "id", "type": {"type": "array", "items": "long" }}
],
{quote}
This is unintuitive but allowing it parse like above leads to other problems.
Say we mean to defined a Users type for the first time in the request field
like so:
{quote}
{"name": "mode", "type":{"name":"FileMode", "type": "enum", "symbols": ["w",
"r"] }
{quote}
but "flatten" the definition like the "bad" protocol given:
{quote}
{"name": "mode", "type": "enum", "symbols": ["w", "r"]}
{quote}
This second field is weird! We have the term "name" being overloaded! In the
first one, we have a field named "mode" that takes a type named "FileMode"
(remember that enum's require names!). In the second one we have a field named
"mode" and then some stuff that could be creating an enum named "mode" but
really is just incomplete!
This behavior is slightly confusing when dealing with array and map types, but
is consistent with the necessity to "nest" the named types enum, fixed, record,
error and, of course, user-defined types.
NOW! We can think about two different things. One is to improve this error
message. Basically, if there's more than "type", "name", "doc" and "default" in
a record's field, we should through a very specific error saying "nest this
type". This is relatively straightforward.
The second thing is to think about changing the spec to allow array and map
types to be "flattened" in this manner as they have no "name" conflicts. It
does, however, cause weird problems where people will expect primitive behavior
when they get complex behavior.
This is really just confusing because the "simple" versions of "primitive"
types (i.e. "string" instead of {"type":"string"}) are really confusing for
newbies! If we disallowed them, people would immediately glom on to this
nesting problem!
So, the first thing is probably doable. The second one requires more thought.
> Cannot use array type in avro request
> -------------------------------------
>
> Key: AVRO-529
> URL: https://issues.apache.org/jira/browse/AVRO-529
> Project: Avro
> Issue Type: Bug
> Components: ruby
> Affects Versions: 1.4.0
> Environment: Mac OS X
> Reporter: Wanli Yang
>
> While it is OK to use array in avro response, we get the following error when
> using it in the type definition. e.g.:
> {
> "namespace": "my_namespace",
> "protocol": "MyProtocol",
> "types": [
> {
> "name": "Thing", "type": "record", "fields": [
> { "name": "id", "type": "long" },
> { "name": "other_ids", "type": "array", "items": "long" }
> ...
> /script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:78:in `real_parse':
> "array" is not a schema we know about. (Avro::SchemaParseError)
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:135:in
> `subparse'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:369:in
> `initialize'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:180:in
> `new'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:180:in
> `make_field_objects'
> from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in
> `each_with_index'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:174:in
> `each'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:174:in
> `each_with_index'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:174:in
> `make_field_objects'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:200:in
> `initialize'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:53:in
> `new'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/schema.rb:53:in
> `real_parse'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/protocol.rb:73:in
> `parse_types'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/protocol.rb:70:in
> `collect'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/protocol.rb:70:in
> `parse_types'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/protocol.rb:54:in
> `initialize'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/protocol.rb:31:in
> `new'
> from ./script/../vendor/gems/avro-1.4.0.pre1/lib/avro/protocol.rb:31:in
> `parse'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
