[jira] [Commented] (AVRO-2072) ResolvingGrammarGenerator doesn't implement schema resolution correctly for unions

2017-09-07 Thread Nandor Kollar (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156991#comment-16156991
 ] 

Nandor Kollar commented on AVRO-2072:
-

Thanks [~epkanol], I uploaded AVRO-2072_2.patch with your additional test 
cases, modified the failing ones in {{TestReadingWritingDataInEvolvedSchemas}} 
which shouldn't fail after the fix in {{ResolvingGrammarGenerator}}, and 
deleted the ignored case from {{TestSchemaCompatibility}}, since we should 
support those promotions: changing the specification is not the best idea 
according to [~cutting] and [~busbey], so we should allow this promotion, thus 
this is a bug in the implementation (see discussion on Avro user's mailing 
list). Anders, could you please help with a review?

> ResolvingGrammarGenerator doesn't implement schema resolution correctly for 
> unions
> --
>
> Key: AVRO-2072
> URL: https://issues.apache.org/jira/browse/AVRO-2072
> Project: Avro
>  Issue Type: Bug
>Reporter: Nandor Kollar
>Assignee: Nandor Kollar
> Attachments: 0001-AVRO-1931-Additional-test-cases.patch, 
> AVRO-2072_2.patch, AVRO-2072.patch
>
>
> According to 
> [specification|https://avro.apache.org/docs/current/spec.html#Schema+Resolution],
>  int and long is promotable to float, but when using SchemaValidator, a union 
> with a single int or long branch is not readable by an union with a float 
> branch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-2072) ResolvingGrammarGenerator doesn't implement schema resolution correctly for unions

2017-09-07 Thread Nandor Kollar (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AVRO-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nandor Kollar updated AVRO-2072:

Attachment: AVRO-2072_2.patch

> ResolvingGrammarGenerator doesn't implement schema resolution correctly for 
> unions
> --
>
> Key: AVRO-2072
> URL: https://issues.apache.org/jira/browse/AVRO-2072
> Project: Avro
>  Issue Type: Bug
>Reporter: Nandor Kollar
>Assignee: Nandor Kollar
> Attachments: 0001-AVRO-1931-Additional-test-cases.patch, 
> AVRO-2072_2.patch, AVRO-2072.patch
>
>
> According to 
> [specification|https://avro.apache.org/docs/current/spec.html#Schema+Resolution],
>  int and long is promotable to float, but when using SchemaValidator, a union 
> with a single int or long branch is not readable by an union with a float 
> branch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-2003) Report specific location of schema incompatibilities

2017-09-07 Thread Elliot West (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156962#comment-16156962
 ] 

Elliot West commented on AVRO-2003:
---

[~nkollar] certainly, I'll look at this shortly.

> Report specific location of schema incompatibilities
> 
>
> Key: AVRO-2003
> URL: https://issues.apache.org/jira/browse/AVRO-2003
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.8.1
> Environment: Any java env
>Reporter: Elliot West
>Assignee: Elliot West
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: AVRO-2003.patch
>
>
> h2. Overview
> Building on the work to improve schema incompatibility reporting in 
> AVRO-1933, it would be useful if the {{SchemaCompatibility}} classes could 
> also report the location in the schema where any incompatibility was 
> encountered.
> It is recommended that the location reported is both easily readable by 
> humans and machines. In the first case this would assist schema developers to 
> pin-point issues in there schema documents, and in the latter case it 
> provides a useful mechanism to schema tooling, such as IDEs and editors, to 
> easily select the pertinent nodes in the Schema document tree.
> h2. Implementation specifics
> To meet this requirements it is suggested that the location is encoded using 
> the [JSON Pointer specification|https://tools.ietf.org/html/rfc6901]. This is 
> both easily parsed by users, but is also supported by a number of libraries 
> for a range of common programming languages and platforms.
> h2. Examples
> Given the following example schema, consider some incompatibility scenarios. 
> For each case an expected JSON Pointer description of the incompatibility 
> location is described:
> {code}
> {
>   "type": "record",
>   "name": "myRecord",
>   "fields" : [
> {"name": "pField", "type": "long"},
> {"name": "uField", "type":
>   ["null", "int", "string"]
> },
> {"name": "eField", "type": 
>   { "type": "enum", "name": "Suit", "symbols" : ["SPADES", "HEARTS", 
> "DIAMONDS", "CLUBS"] }
> },
> {"name": "aField", "type":
>   {"type": "array", "items": "string"}
> },
> {"name": "mField", "type": 
>   {"type": "map", "values": "long"}
> },
> {"name": "fField", "type": 
>   {"type": "fixed", "size": 16, "name": "md5"}
> }
>   ]
> }
> {code}
> Possible incompatibility scenarions and the location that would be reported 
> back to the user/tool: 
> * Root type incompatibility; report location: {{/}}
> * Record name mismatch; report location: {{/name}}
> * {{pField}} type incompatibility; report location: {{/fields/0/type}}
> * {{uField}} field type incompatibility; report location: {{/fields/1/type}}
> * {{uField}} missing union branch {{string}}; report location: 
> {{/fields/1/type/2}}
> * {{eField}} field type incompatibility; report location: {{/fields/2/type}}
> * {{eField}} missing enum symbol; report location: {{/fields/2/type/symbols}}
> * {{eField}} enum name mismatch; report location: {{/fields/2/type/name}}
> * {{aField}} field type incompatibility; report location: {{/fields/3/type}}
> * {{aField}} array element type incompatibility; report location: 
> {{/fields/3/type/items}}
> * {{mField}} field type incompatibility; report location: {{/fields/4/type}}
> * {{mField}} map value type incompatibility; report location: 
> {{/fields/4/type/values}}
> * {{fField}} field type incompatibility; report location: {{/fields/5/type}}
> * {{fField}} fixed name mismatch; report location: {{/fields/5/type/name}}
> * {{fField}} fixed size type incompatibility; report location: 
> {{/fields/5/type/size}}
> * {{fField}} missing default value; report location: {{/fields/5}}
> h2. Notes
> * This ticket depends on AVRO-1933 and associated patches.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-2003) Report specific location of schema incompatibilities

2017-09-07 Thread Nandor Kollar (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156938#comment-16156938
 ] 

Nandor Kollar commented on AVRO-2003:
-

[~teabot] now AVRO-1933 is committed to master, due to this your PR has several 
merge conflicts. Could you please update it with merging your changes to latest 
Apache master?

> Report specific location of schema incompatibilities
> 
>
> Key: AVRO-2003
> URL: https://issues.apache.org/jira/browse/AVRO-2003
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.8.1
> Environment: Any java env
>Reporter: Elliot West
>Assignee: Elliot West
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: AVRO-2003.patch
>
>
> h2. Overview
> Building on the work to improve schema incompatibility reporting in 
> AVRO-1933, it would be useful if the {{SchemaCompatibility}} classes could 
> also report the location in the schema where any incompatibility was 
> encountered.
> It is recommended that the location reported is both easily readable by 
> humans and machines. In the first case this would assist schema developers to 
> pin-point issues in there schema documents, and in the latter case it 
> provides a useful mechanism to schema tooling, such as IDEs and editors, to 
> easily select the pertinent nodes in the Schema document tree.
> h2. Implementation specifics
> To meet this requirements it is suggested that the location is encoded using 
> the [JSON Pointer specification|https://tools.ietf.org/html/rfc6901]. This is 
> both easily parsed by users, but is also supported by a number of libraries 
> for a range of common programming languages and platforms.
> h2. Examples
> Given the following example schema, consider some incompatibility scenarios. 
> For each case an expected JSON Pointer description of the incompatibility 
> location is described:
> {code}
> {
>   "type": "record",
>   "name": "myRecord",
>   "fields" : [
> {"name": "pField", "type": "long"},
> {"name": "uField", "type":
>   ["null", "int", "string"]
> },
> {"name": "eField", "type": 
>   { "type": "enum", "name": "Suit", "symbols" : ["SPADES", "HEARTS", 
> "DIAMONDS", "CLUBS"] }
> },
> {"name": "aField", "type":
>   {"type": "array", "items": "string"}
> },
> {"name": "mField", "type": 
>   {"type": "map", "values": "long"}
> },
> {"name": "fField", "type": 
>   {"type": "fixed", "size": 16, "name": "md5"}
> }
>   ]
> }
> {code}
> Possible incompatibility scenarions and the location that would be reported 
> back to the user/tool: 
> * Root type incompatibility; report location: {{/}}
> * Record name mismatch; report location: {{/name}}
> * {{pField}} type incompatibility; report location: {{/fields/0/type}}
> * {{uField}} field type incompatibility; report location: {{/fields/1/type}}
> * {{uField}} missing union branch {{string}}; report location: 
> {{/fields/1/type/2}}
> * {{eField}} field type incompatibility; report location: {{/fields/2/type}}
> * {{eField}} missing enum symbol; report location: {{/fields/2/type/symbols}}
> * {{eField}} enum name mismatch; report location: {{/fields/2/type/name}}
> * {{aField}} field type incompatibility; report location: {{/fields/3/type}}
> * {{aField}} array element type incompatibility; report location: 
> {{/fields/3/type/items}}
> * {{mField}} field type incompatibility; report location: {{/fields/4/type}}
> * {{mField}} map value type incompatibility; report location: 
> {{/fields/4/type/values}}
> * {{fField}} field type incompatibility; report location: {{/fields/5/type}}
> * {{fField}} fixed name mismatch; report location: {{/fields/5/type/name}}
> * {{fField}} fixed size type incompatibility; report location: 
> {{/fields/5/type/size}}
> * {{fField}} missing default value; report location: {{/fields/5}}
> h2. Notes
> * This ticket depends on AVRO-1933 and associated patches.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1933) SchemaCompatibility class could be more user-friendly about incompatibilities

2017-09-07 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156873#comment-16156873
 ] 

ASF GitHub Bot commented on AVRO-1933:
--

Github user asfgit closed the pull request at:

https://github.com/apache/avro/pull/200


> SchemaCompatibility class could be more user-friendly about incompatibilities
> -
>
> Key: AVRO-1933
> URL: https://issues.apache.org/jira/browse/AVRO-1933
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Any Java env
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: AVRO-1933-compatible-with-AVRO-1931.patch, 
> AVRO-1933.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Today, the class SchemaCompatibility reports incompatibilities with quite 
> little detail. The whole reader and the whole writer schema is listed, and no 
> particular detail about what was incompatible.
> The attached patch fixes this, introducing a new enum 
> (SchemaIncompatibilityType), and more specific sub-schemas that were 
> incompatible.
> The old, overall picture, is still there - the new compatibility state is 
> encapsulated in the SchemaCompatibilityDetails class.
> Lots of test cases have been added, and there has been refactoring done in 
> the TestSchemaCompatibility and other test classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1933) SchemaCompatibility class could be more user-friendly about incompatibilities

2017-09-07 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156871#comment-16156871
 ] 

ASF subversion and git services commented on AVRO-1933:
---

Commit db8ed216eaf13c8f3862eb31152185f0504c4467 in avro's branch 
refs/heads/master from [~epkanol]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=db8ed21 ]

AVRO-1933: Add more specific error details to SchemaCompatibility class

Closes #200

Signed-off-by: Sriharsha Chintalapani 
Signed-off-by: Anna Szonyi 
Signed-off-by: Nandor Kollar 
Signed-off-by: Gabor Szadovszky 


> SchemaCompatibility class could be more user-friendly about incompatibilities
> -
>
> Key: AVRO-1933
> URL: https://issues.apache.org/jira/browse/AVRO-1933
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Any Java env
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: AVRO-1933-compatible-with-AVRO-1931.patch, 
> AVRO-1933.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Today, the class SchemaCompatibility reports incompatibilities with quite 
> little detail. The whole reader and the whole writer schema is listed, and no 
> particular detail about what was incompatible.
> The attached patch fixes this, introducing a new enum 
> (SchemaIncompatibilityType), and more specific sub-schemas that were 
> incompatible.
> The old, overall picture, is still there - the new compatibility state is 
> encapsulated in the SchemaCompatibilityDetails class.
> Lots of test cases have been added, and there has been refactoring done in 
> the TestSchemaCompatibility and other test classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)