[
https://issues.apache.org/jira/browse/AVRO-680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14318976#comment-14318976
]
Ryan Blue commented on AVRO-680:
--------------------------------
bq. For user-defined classes like `Map<Foo, Bar>`, we cannot generate
`FooBarPair` because the Foo/Bar names may be present in multiple packages.
True, but this is only a problem if you have a conflict that is used in the
same position in the resulting Avro schema. So you need both {{one.Foo}} and
{{two.Foo}} classes, present in the same Avro schema, both used as keys (or
both used as values), and paired with the same type. That seems unlikely enough
that I'm comfortable with it. Otherwise, we could restrict it to just primitive
types.
bq. Can you give me the code which did not give array schema for a non-string
map?
{code:java}
Map<Integer, String> m = new HashMap<Integer, String>();
m.put(34, "s");
ReflectData.get().getSchema(m);
{code}
bq. we changed [normal array write logic] to the current implementation based
on some discussion that happened above
I think you're referring to this comment:
bq. Apparently, it isn't exactly possible to call
ReflectDatumWriter#writeNonStringMap() from inside
ReflectDatumWriter#writeArray(). This is so because if that happens, then it
means the array-part of non-string-map is already processed and we are at the
individual record level. And at this level, we cannot distinguish the map-entry
from a normal array-element. Due to this, mapEntry.getKey() and
mapEntry.getValue() could not be called, leading us back to relying on the
names "key" and "value".
Does this still apply? In {{ReflectDatumWriter#writeNonStringMap}}, you call
{{writeField(mapEntry, keyField, outEnc, state)}} that doesn't call
{{mapEntry.getKey()}}. It looks like you use the accessor and rely on the
field's name being "key" anyway. It would also make sense to have the accessor
fall back to {{Map.Entry#getKey()}} if the underlying field isn't available. I
think you might be able to change back to the regular array logic now.
> Allow for non-string keys
> -------------------------
>
> Key: AVRO-680
> URL: https://issues.apache.org/jira/browse/AVRO-680
> Project: Avro
> Issue Type: Improvement
> Affects Versions: 1.7.6, 1.7.7
> Reporter: Jeremy Hanna
> Attachments: AVRO-680.patch, AVRO-680.patch, PERF_8000_cycles.zip,
> isMap_Call_Hierarchy.png, non_string_map_keys.zip, non_string_map_keys2.zip,
> non_string_map_keys3.zip, non_string_map_keys4.patch,
> non_string_map_keys5.patch, non_string_map_keys6.patch,
> non_string_map_keys7.patch, non_string_map_perf.txt,
> non_string_map_perf2.txt, original_perf.txt
>
>
> Based on an email thread back in April, Doug Cutting proposed a possible
> solution for having non-string keys:
> Stu Hood wrote:
> > I can understand the reasoning behind AVRO-9, but now I need to look for an
> > alternative to a 'map' that will allow me to store an association of bytes
> > keys to values.
> A map of Foo has the same binary format as an array of records, each
> with a string field and a Foo field. So an application can use an array
> schema similar to this to represent map-like structures with, e.g.,
> non-string keys.
> Perhaps we could establish standard properties that indicate that a
> given array of records should be represented in a map-like way if
> possible? E.g.,:
> {"type": "array", "isMap": true, "items": {"type":"record", ...}}
> Doug
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
