On 11/16/09 11:24 AM, "Doug Cutting" said:
> Scott Banachowski wrote:
>> Regarding union resolution, the spec says to take the first match when
>> comparing them. What if the first match is "promotable"? I would expect
>> preference for the first "exact" match, if both an exact and promotable
>> match existed.
>
> Good question. I agree, and that's what the Java implementation does,
> in GenericDatumReader#resolveExpected(). It first looks for a branch
> which "exactly" matches the type written, where "exactly" means, for
> named types, has the same name, and for other types has the same base
> type, e.g., array, map, int. If exact match fails, then it looks for a
> match via promotion.
>
> Should we fix the spec to better document this?
Yes, I think the spec can say something along the lines of, first try to
find an exact match, and if not present, use the first match via promotion.
>
>> Second clarification is about usage of unset vs. signaling error: in the
>> rules if we cannot match enums, we declare it unset, but if we cannot match
>> unions, we signal error. It seems for one type we take a best effort
>> approach and in the other we give up. I'm just trying to understand what is
>> different about the two cases.
>
> I think it's reasonable to differ in how we handle unmatched field names
> and mismatched types. If the type for a field changes in a way that
> cannot be promoted, that's an incompatible schema. But if a field is
> renamed, then we ignore the value under the old name and might leave the
> new name unset. Union mismatches are type differences, not field name
> differences, so errors seem appropriate. Does that make sense?
It makes sense. But let me ask this question in the context of a specific
example:
Let's say my writer is "[int double]" and my reader is "int".
If the writer sets the value as an int, obviously my reader can proceed
without error. But if the writer sets the value as double, my reader
should flag an error.
What I am really wondering is what flagging an error means in this context.
What if this is a field of a record--in this case, does error mean that the
entire record is bad? Or do I simply ignore this field and process the rest
of the record? And if I do the latter, how is this different than "unset"?
thanks,
Scott
>
> There's an open issue about whether leaving fields unset is reasonable.
>
> https://issues.apache.org/jira/browse/AVRO-75
>
> What do folks think is best here? Should we require all values that
> don't have a default values to be present, i.e., record fields with a
> default value are optional, all others are mandatory?
>
>> And assuming we're doing this matching at
>> runtime, for example, reading an rpc message, does signaling error mean we
>> drop the message, or still pass parts of the message that could be resolved?
>
> If we adopt the "unset" concept, then implementations could pass the
> parts of the message that can be resolved. Note that we should never
> simply drop the message, but rather return an error response, but that's
> probably what you meant.
>
> Doug