[
https://issues.apache.org/jira/browse/AVRO-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614912#comment-16614912
]
ASF GitHub Bot commented on AVRO-2226:
--------------------------------------
akellehe commented on a change in pull request #331: [AVRO-2226] Fixes
UnionSchema specificity
URL: https://github.com/apache/avro/pull/331#discussion_r217716914
##########
File path: lang/py3/avro/io.py
##########
@@ -135,9 +135,17 @@ def Validate(expected_schema, datum):
return any(Validate(union_branch, datum)
for union_branch in expected_schema.schemas)
elif schema_type in ['record', 'error', 'request']:
- return (isinstance(datum, dict)
- and all(Validate(field.type, datum.get(field.name))
- for field in expected_schema.fields))
+ if not isinstance(datum, dict):
+ return False
+ expected_schema_field_names = set()
+ for field in expected_schema.fields:
+ expected_schema_field_names.add(field.name)
+ if not Validate(field.type, datum.get(field.name)):
+ return False
+ for datum_field in datum.keys():
Review comment:
This block ensures all fields in the `datum` exist in the `expected_schema`.
If they do not then the datum does not validate against the schema.
A stricter (but incompatible) change would be to create a `Null` type and
require that all datum set all null fields to `Null` rather than relying on
python's `None` type. The approach here is probably better for compatibility.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> UnionSchema deduction is too permissive
> ---------------------------------------
>
> Key: AVRO-2226
> URL: https://issues.apache.org/jira/browse/AVRO-2226
> Project: Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.8.2
> Reporter: Andrew Kelleher
> Priority: Major
> Attachments: AVRO-2226.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> When given a schema of the form
> {code:java}
> {
> "type" : "record",
> "name" : "A",
> "namespace" : "com.example",
> "fields" : [
> {
> "name" : "foo",
> "type" : ["string", "null"]
> }
> ]
> }
> {
> "type" : "record",
> "name" : "B",
> "namespace" : "com.example",
> "fields" : [
> {
> "name" : "bar",
> "type" : ["string", "null"]
> }
> ]
> }
> {
> "type" : "record",
> "name" : "AOrB",
> "namespace" : "com.example",
> "fields" : [
> {
> "name" : "entity",
> "type" : [
> "com.example.A",
> "com.example.B"
> ]
> }
> ]
> }
> {code}
> And a datum of the form
> {code}
> {'entity': {'foo': 'this is an instance of schema A'}}{code}
> Converting to a message, and then from a message chooses the incorrect
> `entity` schema:
> {code}
> {'entity': {'bar': None}}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
