[jira] [Commented] (AVRO-1574) Suboptimal arraylist creation

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133016#comment-16133016
 ] 

Sean Busbey commented on AVRO-1574:
---

+1 pushing shortly.

> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened AVRO-1929:
---

> SchemaCompatibility fails to recognize string<->bytes promotion
> ---
>
> Key: AVRO-1929
> URL: https://issues.apache.org/jira/browse/AVRO-1929
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.0, 1.8.1, 1.9.0
> Environment: Any Java env
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
> Fix For: 1.9.0, 1.8.2
>
> Attachments: AVRO-1929.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The class SchemaCompatibility fails to realize the change made as of 
> AVRO-1553, where string-to/from-bytes promotions were allowed (according to 
> docs, this was integrated into 1.7.7)
> The submitted fix corrects this (also updating the relevant unit test)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1574) Suboptimal arraylist creation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133066#comment-16133066
 ] 

ASF subversion and git services commented on AVRO-1574:
---

Commit 2f6eb2f4ec2071bc03e52a3ba8ac1d6aeb954c88 in avro's branch 
refs/heads/branch-1.8 from [~sekikn]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=2f6eb2f ]

AVRO-1574: Create list for schema symbols with specified size.

Signed-off-by: Nandor Kollar 
Signed-off-by: Sean Busbey 
(cherry picked from commit 09c997e206615c4b23ab9ae2ab5f6a3d793e9afb)


> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133062#comment-16133062
 ] 

ASF subversion and git services commented on AVRO-1929:
---

Commit 492ab98be6b718df6439d782c8b208d11cdf000a in avro's branch 
refs/heads/branch-1.7 from [~cutting]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=492ab98 ]

AVRO-1929: Java: SchemaCompatibility fails to recognize string<->bytes 
promotion.  Contributed by Anders Sundelin.

(cherry picked from commit 1b91c06e92c4b5dd8e3a3e936b36e54fb9c9e023)


> SchemaCompatibility fails to recognize string<->bytes promotion
> ---
>
> Key: AVRO-1929
> URL: https://issues.apache.org/jira/browse/AVRO-1929
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.0, 1.8.1, 1.9.0
> Environment: Any Java env
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
> Fix For: 1.7.8, 1.9.0, 1.8.2
>
> Attachments: AVRO-1929.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The class SchemaCompatibility fails to realize the change made as of 
> AVRO-1553, where string-to/from-bytes promotions were allowed (according to 
> docs, this was integrated into 1.7.7)
> The submitted fix corrects this (also updating the relevant unit test)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1574) Suboptimal arraylist creation

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1574:
--
   Resolution: Fixed
Fix Version/s: 1.8.3
   1.9.0
   1.7.8
   Status: Resolved  (was: Patch Available)

pushed to all branches. Thanks Kengo!

> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved AVRO-1929.
---
   Resolution: Fixed
Fix Version/s: 1.7.8
 Release Note: The java SchemaCompatibility class now recognizes the 
possible promotion between string and bytes fields.

pulled this back to branch-1.7 as well.

> SchemaCompatibility fails to recognize string<->bytes promotion
> ---
>
> Key: AVRO-1929
> URL: https://issues.apache.org/jira/browse/AVRO-1929
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.0, 1.8.1, 1.9.0
> Environment: Any Java env
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
> Fix For: 1.7.8, 1.9.0, 1.8.2
>
> Attachments: AVRO-1929.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The class SchemaCompatibility fails to realize the change made as of 
> AVRO-1553, where string-to/from-bytes promotions were allowed (according to 
> docs, this was integrated into 1.7.7)
> The submitted fix corrects this (also updating the relevant unit test)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133068#comment-16133068
 ] 

ASF subversion and git services commented on AVRO-1931:
---

Commit 15651fc95e058d1b3cc165a70c367d1dc2bad8b7 in avro's branch 
refs/heads/master from [~epkanol]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=15651fc ]

AVRO-1931: Reader is now compatible if able to read all branches of union

Closes #199

Signed-off-by: Sriharsha Chintalapani 
Signed-off-by: Nandor Kollar 
Signed-off-by: Sean Busbey 


> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1574) Suboptimal arraylist creation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133064#comment-16133064
 ] 

ASF subversion and git services commented on AVRO-1574:
---

Commit 58ebac451d176280febe5b2e95f78e6a065be4ec in avro's branch 
refs/heads/branch-1.7 from [~sekikn]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=58ebac4 ]

AVRO-1574: Create list for schema symbols with specified size.

Signed-off-by: Nandor Kollar 
Signed-off-by: Sean Busbey 
(cherry picked from commit 09c997e206615c4b23ab9ae2ab5f6a3d793e9afb)
(cherry picked from commit 2f6eb2f4ec2071bc03e52a3ba8ac1d6aeb954c88)


> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133074#comment-16133074
 ] 

ASF GitHub Bot commented on AVRO-1931:
--

Github user asfgit closed the pull request at:

https://github.com/apache/avro/pull/199


> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133063#comment-16133063
 ] 

ASF subversion and git services commented on AVRO-1931:
---

Commit 522b59b04f3cf1300908bb91b30b89af1089621e in avro's branch 
refs/heads/branch-1.7 from [~epkanol]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=522b59b ]

AVRO-1931: Reader is now compatible if able to read all branches of union

Closes #199

Signed-off-by: Sriharsha Chintalapani 
Signed-off-by: Nandor Kollar 
Signed-off-by: Sean Busbey 
(cherry picked from commit 15651fc95e058d1b3cc165a70c367d1dc2bad8b7)
(cherry picked from commit 83cdd2bd70a4f5f16a52177fc46aa6dec412548c)


> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133065#comment-16133065
 ] 

ASF subversion and git services commented on AVRO-1931:
---

Commit 83cdd2bd70a4f5f16a52177fc46aa6dec412548c in avro's branch 
refs/heads/branch-1.8 from [~epkanol]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=83cdd2b ]

AVRO-1931: Reader is now compatible if able to read all branches of union

Closes #199

Signed-off-by: Sriharsha Chintalapani 
Signed-off-by: Nandor Kollar 
Signed-off-by: Sean Busbey 
(cherry picked from commit 15651fc95e058d1b3cc165a70c367d1dc2bad8b7)


> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1574) Suboptimal arraylist creation

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133070#comment-16133070
 ] 

ASF subversion and git services commented on AVRO-1574:
---

Commit 09c997e206615c4b23ab9ae2ab5f6a3d793e9afb in avro's branch 
refs/heads/master from [~sekikn]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=09c997e ]

AVRO-1574: Create list for schema symbols with specified size.

Signed-off-by: Nandor Kollar 
Signed-off-by: Sean Busbey 


> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] avro pull request #199: AVRO-1931: Reader is now compatible if able to read ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/avro/pull/199


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1931:
--
   Resolution: Fixed
Fix Version/s: 1.8.3
   1.9.0
   1.7.8
 Release Note: Reader Schemas that have a non-union field which corresponds 
to a union field in a Writer Schema are now recognized as compatible if the 
non-union field is compatible with all entries in the writer's union field.
   Status: Resolved  (was: Patch Available)

pushd to branches. Thanks for the reviews everyone!

> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Fix For: 1.7.8, 1.9.0, 1.8.3
>
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1213) Dependency on Jetty Servlet API in IPC

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1213:
--
Priority: Blocker  (was: Minor)

> Dependency on Jetty Servlet API in IPC
> --
>
> Key: AVRO-1213
> URL: https://issues.apache.org/jira/browse/AVRO-1213
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.2
>Reporter: Sharmarke Aden
>Priority: Blocker
> Fix For: 1.9.0
>
>
> The compile scoped dependency on jetty servlet-api in the IPC pom file can be 
> problematic if using Avro in a webapp environment. Would it be possible to 
> make this dependency either optional or provided? Or maybe Avro modularize 
> into sub-modules in such a way that desired features can be assembled 
> piecemeal?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1213) Dependency on Jetty Servlet API in IPC

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1213:
--
Fix Version/s: 1.9.0

> Dependency on Jetty Servlet API in IPC
> --
>
> Key: AVRO-1213
> URL: https://issues.apache.org/jira/browse/AVRO-1213
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.2
>Reporter: Sharmarke Aden
>Priority: Blocker
> Fix For: 1.9.0
>
>
> The compile scoped dependency on jetty servlet-api in the IPC pom file can be 
> problematic if using Avro in a webapp environment. Would it be possible to 
> make this dependency either optional or provided? Or maybe Avro modularize 
> into sub-modules in such a way that desired features can be assembled 
> piecemeal?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1680) Problems with code snippet in Decoder#readMapStart javadocs

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1680:
--
Labels: beginner  (was: )

> Problems with code snippet in Decoder#readMapStart javadocs
> ---
>
> Key: AVRO-1680
> URL: https://issues.apache.org/jira/browse/AVRO-1680
> Project: Avro
>  Issue Type: Improvement
>  Components: doc, java
>Affects Versions: 1.7.7
>Reporter: Eugene Kirpichov
>Priority: Trivial
>  Labels: beginner
>
> https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/Decoder.java
>  javadocs ( 
> http://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/io/Decoder.html#readMapStart()
>  ) say:
> {code:java}
> Map m = new HashMap();
> Record reuse = new Record();
> for(long i = in.readMapStart(); i != 0; i = in.readMapNext()) {
>   for (long j = 0; j < i; j++) {
> String key = in.readString();
> reuse.intField = in.readInt();
> reuse.boolField = in.readBoolean();
> m.put(key, reuse);
>   }
>}
> {code}
> This can be improved in two ways:
> 1) Javadoc ate the generic arguments. This can be fixed by wrapping into 
> \{@code}.
> 2) The mutable record object is being reused; as a result, the map will have 
> the same shared object mapped to every key. I don't think this is likely to 
> be the user's intention, so a new Record should be created on every iteration.
> Actually in 3 ways.
> 3) a much better name for "i" would be "numRecords", because otherwise it 
> seems like "j" and "i" have similar roles (indices over some containers), 
> which they don't - "i" is not an index into any container, only j is. Then 
> "j" can be renamed to "i".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1680) Problems with code snippet in Decoder#readMapStart javadocs

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1680:
--
Component/s: doc

> Problems with code snippet in Decoder#readMapStart javadocs
> ---
>
> Key: AVRO-1680
> URL: https://issues.apache.org/jira/browse/AVRO-1680
> Project: Avro
>  Issue Type: Improvement
>  Components: doc, java
>Affects Versions: 1.7.7
>Reporter: Eugene Kirpichov
>Priority: Trivial
>  Labels: beginner
>
> https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/io/Decoder.java
>  javadocs ( 
> http://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/io/Decoder.html#readMapStart()
>  ) say:
> {code:java}
> Map m = new HashMap();
> Record reuse = new Record();
> for(long i = in.readMapStart(); i != 0; i = in.readMapNext()) {
>   for (long j = 0; j < i; j++) {
> String key = in.readString();
> reuse.intField = in.readInt();
> reuse.boolField = in.readBoolean();
> m.put(key, reuse);
>   }
>}
> {code}
> This can be improved in two ways:
> 1) Javadoc ate the generic arguments. This can be fixed by wrapping into 
> \{@code}.
> 2) The mutable record object is being reused; as a result, the map will have 
> the same shared object mapped to every key. I don't think this is likely to 
> be the user's intention, so a new Record should be created on every iteration.
> Actually in 3 ways.
> 3) a much better name for "i" would be "numRecords", because otherwise it 
> seems like "j" and "i" have similar roles (indices over some containers), 
> which they don't - "i" is not an index into any container, only j is. Then 
> "j" can be renamed to "i".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (AVRO-1574) Suboptimal arraylist creation

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reassigned AVRO-1574:
-

Assignee: Kengo Seki

> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1640) Does SpecificData.schemaCache actually do anything?

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16132166#comment-16132166
 ] 

Sean Busbey commented on AVRO-1640:
---

this would be much better as a DISCUSS thread on dev@, could someone please 
bring it up there?

> Does SpecificData.schemaCache actually do anything?
> ---
>
> Key: AVRO-1640
> URL: https://issues.apache.org/jira/browse/AVRO-1640
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Silas Davis
>Priority: Minor
>  Labels: should_be_dev_discussion
>
> I've been looking at the source code for SpecificData. On line 182 a 
> schemaCache is defined as a WeakHashMap.
> Having looked into how WeakHashMap works - it stores a WeakReference for the 
> keys of the map but not the values, I can't see how the schemaCache actually 
> provides a cache. I think it just stores all Schema objects it has ever seen, 
> and never expires any. This is because it stores keys of type 
> java.lang.reflect.Type key, typlically a Class, which typically never fall 
> out of reference. The only way the Class (a static singleton object) will be 
> garbage collected is if the ClassLoader that holds a reference to it is 
> garbage collected. This mostly doesn't happen, so the schemaCache is really 
> just behaving as a HashMap.
> It seems like WeakHashMap is commonly mistakenly used as a cache:
> - 
> http://stackoverflow.com/questions/1802809/javas-weakhashmap-and-caching-why-is-it-referencing-the-keys-not-the-values
> - 
> http://www.codeinstructions.com/2008/09/weakhashmap-is-not-cache-understanding.html
> Perhaps I am missing something, like Avro's use of ClassLoaders meaning that 
> Class objects do get collected, apologies if so.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1640) Does SpecificData.schemaCache actually do anything?

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1640:
--
Labels: should_be_dev_discussion  (was: )

> Does SpecificData.schemaCache actually do anything?
> ---
>
> Key: AVRO-1640
> URL: https://issues.apache.org/jira/browse/AVRO-1640
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Silas Davis
>Priority: Minor
>  Labels: should_be_dev_discussion
>
> I've been looking at the source code for SpecificData. On line 182 a 
> schemaCache is defined as a WeakHashMap.
> Having looked into how WeakHashMap works - it stores a WeakReference for the 
> keys of the map but not the values, I can't see how the schemaCache actually 
> provides a cache. I think it just stores all Schema objects it has ever seen, 
> and never expires any. This is because it stores keys of type 
> java.lang.reflect.Type key, typlically a Class, which typically never fall 
> out of reference. The only way the Class (a static singleton object) will be 
> garbage collected is if the ClassLoader that holds a reference to it is 
> garbage collected. This mostly doesn't happen, so the schemaCache is really 
> just behaving as a HashMap.
> It seems like WeakHashMap is commonly mistakenly used as a cache:
> - 
> http://stackoverflow.com/questions/1802809/javas-weakhashmap-and-caching-why-is-it-referencing-the-keys-not-the-values
> - 
> http://www.codeinstructions.com/2008/09/weakhashmap-is-not-cache-understanding.html
> Perhaps I am missing something, like Avro's use of ClassLoaders meaning that 
> Class objects do get collected, apologies if so.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1580) Use newer version of surefire, 2.17 instead of 2.12

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16132165#comment-16132165
 ] 

Sean Busbey commented on AVRO-1580:
---

as a part of this, let's upgrade the apache parent pom (atleast in 1.9+) since 
we're relying on version 10, which is very very out of date.

> Use newer version of surefire, 2.17 instead of 2.12
> ---
>
> Key: AVRO-1580
> URL: https://issues.apache.org/jira/browse/AVRO-1580
> Project: Avro
>  Issue Type: Improvement
>  Components: build, java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Priority: Minor
>  Labels: beginner, build
>
> version 2.12 does not work well with Netbeans 8.0 and makes development 
> cumbersome.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1310) Avro Maven project can't be built from scratch

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1310:
--
Labels: should_be_dev_discussion  (was: )

> Avro Maven project can't be built from scratch
> --
>
> Key: AVRO-1310
> URL: https://issues.apache.org/jira/browse/AVRO-1310
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.4
> Environment: Maven on Eclipse
>Reporter: Nir Zamir
>Priority: Minor
>  Labels: should_be_dev_discussion
>
> When getting the Java 'trunk' from SVN and trying to use Maven Install ('mvn 
> install') there are errors.
> Most of the errors are in tests so I tried skipping the tests but it still 
> fails.
> See more details in my post on Avro Users: 
> http://apache-avro.679487.n3.nabble.com/help-with-Avro-compilation-td4026946.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1213) Revisit dependencies on Jetty, servlet-api, and Netty

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1213:
--
Summary: Revisit dependencies on Jetty, servlet-api, and Netty  (was: 
Dependency on Jetty Servlet API in IPC)

> Revisit dependencies on Jetty, servlet-api, and Netty
> -
>
> Key: AVRO-1213
> URL: https://issues.apache.org/jira/browse/AVRO-1213
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.2
>Reporter: Sharmarke Aden
>Priority: Blocker
> Fix For: 1.9.0
>
>
> The compile scoped dependency on jetty servlet-api in the IPC pom file can be 
> problematic if using Avro in a webapp environment. Would it be possible to 
> make this dependency either optional or provided? Or maybe Avro modularize 
> into sub-modules in such a way that desired features can be assembled 
> piecemeal?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[Sean Busbey (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16132986#comment-16132986
 ] 

Sean Busbey commented on AVRO-1931:
---

+1 pushing shortly. just need an author string from one of the reviewers.

> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1574) Suboptimal arraylist creation

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1574:
--
Labels: beginner  (was: )

> Suboptimal arraylist creation
> -
>
> Key: AVRO-1574
> URL: https://issues.apache.org/jira/browse/AVRO-1574
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Assignee: Kengo Seki
>Priority: Trivial
>  Labels: beginner
> Attachments: AVRO-1574.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> in Schema.java:
> 1231 LockableArrayList symbols = new 
> LockableArrayList();
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
> should be changed to:
> 1231 LockableArrayList symbols = new 
> LockableArrayList(symbolsNode.size());
>  
> 1232 for (JsonNode n : symbolsNode)
>  
> 1233   symbols.add(n.getTextValue());
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1865) GenericData.Array class missing no arg constructor for Kryo serialization

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1865:
--
Status: Patch Available  (was: Open)

> GenericData.Array class missing no arg constructor for Kryo serialization
> -
>
> Key: AVRO-1865
> URL: https://issues.apache.org/jira/browse/AVRO-1865
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Alexander Kasper
>Assignee: Alexander Kasper
>  Labels: newbie, patch
> Attachments: GenericArray-no-args-ctor-kryo.patch
>
>
> When trying to serialize Java classes generated by Avro that contain a field 
> as follows:
> {{array myclasses = []}}
> serialization fails with the following error message
> {{com.esotericsoftware.kryo.KryoException: Class cannot be created (missing 
> no-arg constructor)}}
> Minimum example to recreate this:
> Avro IDL definition:
> {code:java}
> @namespace("com.adello")
> protocol KryoTest {
>   record TestRecord {
>   array values = [];
>   }
> }
> {code}
> This gets compiled using {{java -jar avro-tools-1.8.1.jar idl2schemata 
> kryo.avdl .}}
> Resulting Avro schema:
> {code:javascript}
> {
>   "type" : "record",
>   "name" : "TestRecord",
>   "namespace" : "com.adello",
>   "fields" : [ {
> "name" : "values",
> "type" : {
>   "type" : "array",
>   "items" : "int"
> },
> "default" : [ ]
>   } ]
> }
> {code}
> Generate Java class using {{java -jar avro-tools-1.8.1.jar compile schema 
> TestRecord.avsc}}
> Kryo serialization test code:
> {code:java}
> Kryo kryo = new Kryo();
> try {
> Output output = new Output(new FileOutputStream("test.bin"));
> TestRecord o = TestRecord.newBuilder().build();
> System.out.println("Before serialization: ");
> System.out.println(o.toString());
> kryo.writeObject(output, o);
> output.close();
> Input input = new Input(new FileInputStream("test.bin"));
> TestRecord i = kryo.readObject(input, TestRecord.class);
> input.close();
> System.out.println("After deserialization: ");
> System.out.println(i.toString());
> System.out.println(Objects.deepEquals(o, i));
> } catch (FileNotFoundException e) {
> e.printStackTrace();
> }
> {code}
> This is a common enough issue with Kryo that they have a dedicated section on 
> their website on it: 
> http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
> The attached patch adds a no args constructor for the inner class Array of 
> GenericData with zero entries and a dummy schema. My personal tests for 
> serialization with Kryo were successful when using it. Since I do not have 
> complete insight into Avro I'd like to know if this could be a breaking 
> change and how to test it if so.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (AVRO-1865) GenericData.Array class missing no arg constructor for Kryo serialization

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reassigned AVRO-1865:
-

Assignee: Alexander Kasper

> GenericData.Array class missing no arg constructor for Kryo serialization
> -
>
> Key: AVRO-1865
> URL: https://issues.apache.org/jira/browse/AVRO-1865
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Alexander Kasper
>Assignee: Alexander Kasper
>  Labels: beginner, patch
> Attachments: GenericArray-no-args-ctor-kryo.patch
>
>
> When trying to serialize Java classes generated by Avro that contain a field 
> as follows:
> {{array myclasses = []}}
> serialization fails with the following error message
> {{com.esotericsoftware.kryo.KryoException: Class cannot be created (missing 
> no-arg constructor)}}
> Minimum example to recreate this:
> Avro IDL definition:
> {code:java}
> @namespace("com.adello")
> protocol KryoTest {
>   record TestRecord {
>   array values = [];
>   }
> }
> {code}
> This gets compiled using {{java -jar avro-tools-1.8.1.jar idl2schemata 
> kryo.avdl .}}
> Resulting Avro schema:
> {code:javascript}
> {
>   "type" : "record",
>   "name" : "TestRecord",
>   "namespace" : "com.adello",
>   "fields" : [ {
> "name" : "values",
> "type" : {
>   "type" : "array",
>   "items" : "int"
> },
> "default" : [ ]
>   } ]
> }
> {code}
> Generate Java class using {{java -jar avro-tools-1.8.1.jar compile schema 
> TestRecord.avsc}}
> Kryo serialization test code:
> {code:java}
> Kryo kryo = new Kryo();
> try {
> Output output = new Output(new FileOutputStream("test.bin"));
> TestRecord o = TestRecord.newBuilder().build();
> System.out.println("Before serialization: ");
> System.out.println(o.toString());
> kryo.writeObject(output, o);
> output.close();
> Input input = new Input(new FileInputStream("test.bin"));
> TestRecord i = kryo.readObject(input, TestRecord.class);
> input.close();
> System.out.println("After deserialization: ");
> System.out.println(i.toString());
> System.out.println(Objects.deepEquals(o, i));
> } catch (FileNotFoundException e) {
> e.printStackTrace();
> }
> {code}
> This is a common enough issue with Kryo that they have a dedicated section on 
> their website on it: 
> http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
> The attached patch adds a no args constructor for the inner class Array of 
> GenericData with zero entries and a dummy schema. My personal tests for 
> serialization with Kryo were successful when using it. Since I do not have 
> complete insight into Avro I'd like to know if this could be a breaking 
> change and how to test it if so.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1976) Add Input/OutputFormat to read/write encoded objects

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1976:
--
Labels: beginner  (was: newbie)

> Add Input/OutputFormat to read/write encoded objects
> 
>
> Key: AVRO-1976
> URL: https://issues.apache.org/jira/browse/AVRO-1976
> Project: Avro
>  Issue Type: Improvement
>  Components: java
> Environment: hadoop
>Reporter: Marius Posta
>Assignee: Marius Posta
>Priority: Minor
>  Labels: beginner
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> In certain cases, performance of some Avro map-reduce jobs can be 
> considerably improved by de-coupling Avro encoding from actual Avro container 
> file IO.
> In my case, a complex schema (100+ record fields) and large HDFS blocks 
> resulted in Spark jobs where a lot of workers were idling while a couple of 
> them were busy decoding their input splits.Furthermore, the objects then 
> needed to be re-encoded in order to be shuffled about, which crippled 
> performance further.
> I propose the addition of an InputFormat which reads a container file input 
> split as key-value pairs in which the key is the file header and the value is 
> the decompressed file data block. Also, an OutputFormat which follows the 
> same logic for writing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-376) Example or two of data files for future compatibility testing

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-376:
-
Labels: beginner  (was: )

> Example or two of data files for future compatibility testing
> -
>
> Key: AVRO-376
> URL: https://issues.apache.org/jira/browse/AVRO-376
> Project: Avro
>  Issue Type: Test
>  Components: compatibility, java
>Reporter: Philip Zeyliger
>Priority: Minor
>  Labels: beginner
>
> As AVRO-160 (file format) and AVRO-135 (compression) have settled down, I'd 
> like to check in examples of data files with specific data in them, and write 
> tests against those checked-in files.  The current tests for data files rely 
> on code to generate the files, and then read them back, which is perfect for 
> testing that an AVRO version is self-consistent, but doesn't help if we 
> accidentally break compatibility.  (This test would fail, for example, if 
> someone changed the metadata key for the schema, say.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-376) Example or two of data files for future compatibility testing

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-376:
-
Component/s: compatibility

> Example or two of data files for future compatibility testing
> -
>
> Key: AVRO-376
> URL: https://issues.apache.org/jira/browse/AVRO-376
> Project: Avro
>  Issue Type: Test
>  Components: compatibility, java
>Reporter: Philip Zeyliger
>Priority: Minor
>  Labels: beginner
>
> As AVRO-160 (file format) and AVRO-135 (compression) have settled down, I'd 
> like to check in examples of data files with specific data in them, and write 
> tests against those checked-in files.  The current tests for data files rely 
> on code to generate the files, and then read them back, which is perfect for 
> testing that an AVRO version is self-consistent, but doesn't help if we 
> accidentally break compatibility.  (This test would fail, for example, if 
> someone changed the metadata key for the schema, say.)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1365) NPE thrown when comparing objects using GenericData.compare

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1365:
--
Labels: beginner  (was: )

> NPE thrown when comparing objects using GenericData.compare
> ---
>
> Key: AVRO-1365
> URL: https://issues.apache.org/jira/browse/AVRO-1365
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.5
>Reporter: Douglas Kaminsky
>Priority: Minor
>  Labels: beginner
>
> When comparing two objects using GenericData.compare (directly or 
> indirectly), null values in fields of record type objects are not 
> sufficiently protected against, resulting in NPE
> e.g.
> {code}
> case ENUM:
>   return s.getEnumOrdinal(o1.toString()) - s.getEnumOrdinal(o2.toString());
> {code}
> This is prevalent throughout the {{compare}} method. This impacts 
> {{compareTo}}, and {{equals}} implementations as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (AVRO-1310) Avro Maven project can't be built from scratch

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved AVRO-1310.
---
Resolution: Won't Fix

Please use the dev@ mailing list to troubleshoot compilation problems.

> Avro Maven project can't be built from scratch
> --
>
> Key: AVRO-1310
> URL: https://issues.apache.org/jira/browse/AVRO-1310
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.4
> Environment: Maven on Eclipse
>Reporter: Nir Zamir
>Priority: Minor
>
> When getting the Java 'trunk' from SVN and trying to use Maven Install ('mvn 
> install') there are errors.
> Most of the errors are in tests so I tried skipping the tests but it still 
> fails.
> See more details in my post on Avro Users: 
> http://apache-avro.679487.n3.nabble.com/help-with-Avro-compilation-td4026946.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1480) AvroTypeException should include fieldName as distinct field

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1480:
--
Labels: beginner  (was: )

> AvroTypeException should include fieldName as distinct field
> 
>
> Key: AVRO-1480
> URL: https://issues.apache.org/jira/browse/AVRO-1480
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.6
>Reporter: Will Sargent
>Priority: Minor
>  Labels: beginner
>
> Currently AvroTypeException is defined as 
> public class AvroTypeException extends AvroRuntimeException {
>   public AvroTypeException(String message) { super(message); }
>   public AvroTypeException(String message, Throwable cause) {
> super(message, cause);
>   }
> }
> And when used in JSONDecoder, field name failures are defined like this:
> throw new AvroTypeException("Expected field name not found: " + fa.fname);
> This means that, in the event of an AvroTypeException, the only way to find 
> which field has the error is to scan through the error message.
> If the AvroTypeException had a constructor (message, throwable, field) or a 
> subclass which explicitly provided fields, then it would make error 
> processing much easier.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (AVRO-1931) SchemaCompatibility fails to recognize reader compatible with all branches of a union

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reassigned AVRO-1931:
-

Assignee: Anders Sundelin

> SchemaCompatibility fails to recognize reader compatible with all branches of 
> a union
> -
>
> Key: AVRO-1931
> URL: https://issues.apache.org/jira/browse/AVRO-1931
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Java
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
>  Labels: patch
> Attachments: AVRO-1931-2.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> It is stated in the Avro spec
> "if writer's is a union, but reader's is not:
> If the reader's schema matches the selected writer's schema, it is 
> recursively resolved against it. If they do not match, an error is signalled."
> In case a the chosen reader is compatible with all branches of the union in 
> the writer, then the class SchemaCompatibility should reflect this. Currently 
> it does not.
> The submitted patch corrects this (also added tests showing this behaviour in 
> Avro)
> The new tests, in the class TestReadingWritingDataInEvolvedSchemas, could be 
> redundant, but they were very useful when exploring how Avro actually works 
> during de-/serialization
> I will try to continue working a little bit on the SchemaCompatibility class, 
> adding more user-friendly error messages (suitable for deeper structures than 
> todays error message). Feel free to contact me if you have any ideas or 
> pointers to existing work.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (AVRO-1976) Add Input/OutputFormat to read/write encoded objects

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reassigned AVRO-1976:
-

Assignee: Marius Posta

> Add Input/OutputFormat to read/write encoded objects
> 
>
> Key: AVRO-1976
> URL: https://issues.apache.org/jira/browse/AVRO-1976
> Project: Avro
>  Issue Type: Improvement
>  Components: java
> Environment: hadoop
>Reporter: Marius Posta
>Assignee: Marius Posta
>Priority: Minor
>  Labels: newbie
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> In certain cases, performance of some Avro map-reduce jobs can be 
> considerably improved by de-coupling Avro encoding from actual Avro container 
> file IO.
> In my case, a complex schema (100+ record fields) and large HDFS blocks 
> resulted in Spark jobs where a lot of workers were idling while a couple of 
> them were busy decoding their input splits.Furthermore, the objects then 
> needed to be re-encoded in order to be shuffled about, which crippled 
> performance further.
> I propose the addition of an InputFormat which reads a container file input 
> split as key-value pairs in which the key is the file header and the value is 
> the decompressed file data block. Also, an OutputFormat which follows the 
> same logic for writing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1976) Add Input/OutputFormat to read/write encoded objects

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1976:
--
Status: Patch Available  (was: Open)

> Add Input/OutputFormat to read/write encoded objects
> 
>
> Key: AVRO-1976
> URL: https://issues.apache.org/jira/browse/AVRO-1976
> Project: Avro
>  Issue Type: Improvement
>  Components: java
> Environment: hadoop
>Reporter: Marius Posta
>Assignee: Marius Posta
>Priority: Minor
>  Labels: newbie
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> In certain cases, performance of some Avro map-reduce jobs can be 
> considerably improved by de-coupling Avro encoding from actual Avro container 
> file IO.
> In my case, a complex schema (100+ record fields) and large HDFS blocks 
> resulted in Spark jobs where a lot of workers were idling while a couple of 
> them were busy decoding their input splits.Furthermore, the objects then 
> needed to be re-encoded in order to be shuffled about, which crippled 
> performance further.
> I propose the addition of an InputFormat which reads a container file input 
> split as key-value pairs in which the key is the file header and the value is 
> the decompressed file data block. Also, an OutputFormat which follows the 
> same logic for writing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1312) Make use of the sun.misc.Unsafe class in the IO streams implementation if a JDK supports it

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1312:
--
Priority: Major  (was: Minor)

> Make use of the sun.misc.Unsafe class in the IO streams implementation if a 
> JDK supports it
> ---
>
> Key: AVRO-1312
> URL: https://issues.apache.org/jira/browse/AVRO-1312
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Reporter: Leo Romanoff
>
> This is a follow-up of the AVRO-1282 issue.
> AVRO-1282 has used Unsafe to significantly improve performance of 
> Reflection-based serialization. But Unsafe can be also used to improve 
> performance of IO streams, which would be beneficial not only for 
> Reflection-based, but for all kinds of serializers. Experience with Kryo 
> shows that it can boost performance even higher that the speedups provided by 
> AVRO-1282. 
> Pros:
> - Overall performance boost
> - Biggest speedups of this optimizations are expected for the arrays of 
> primitive types, as they can be very efficiently written using bulk 
> operations instead of writing their elements one by one.
> - It is possible to write directly into the off-heap memory buffers at the 
> native speed, without using intermediate byte arrays. This can be interesting 
> for Big Data apps, which often keep a lot of data off-heap
> Cons:
> - Unsafe can efficiently write only primitive types in their native byte 
> order and using their fixed size. This is not quite compatible with Avro's 
> format. 
> (While one can still use Unsafe to more efficiently write single elements 
> even using their variable length encoding, the biggest benefits of bulk array 
> serialization would be lost.)
> - Introducing this feature may require a definition of a new format for Avro. 
> This format would be very fast, but not very space efficient as it would use 
> fixed-size representation.
> BTW, initial tests where a sketch of the proposed optimization is applied 
> only to Floats and Doubles has shown immediate boost of 35%. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1312) Make use of the sun.misc.Unsafe class in the IO streams implementation if a JDK supports it

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1312:
--
Issue Type: Improvement  (was: Bug)

> Make use of the sun.misc.Unsafe class in the IO streams implementation if a 
> JDK supports it
> ---
>
> Key: AVRO-1312
> URL: https://issues.apache.org/jira/browse/AVRO-1312
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Reporter: Leo Romanoff
>Priority: Minor
>
> This is a follow-up of the AVRO-1282 issue.
> AVRO-1282 has used Unsafe to significantly improve performance of 
> Reflection-based serialization. But Unsafe can be also used to improve 
> performance of IO streams, which would be beneficial not only for 
> Reflection-based, but for all kinds of serializers. Experience with Kryo 
> shows that it can boost performance even higher that the speedups provided by 
> AVRO-1282. 
> Pros:
> - Overall performance boost
> - Biggest speedups of this optimizations are expected for the arrays of 
> primitive types, as they can be very efficiently written using bulk 
> operations instead of writing their elements one by one.
> - It is possible to write directly into the off-heap memory buffers at the 
> native speed, without using intermediate byte arrays. This can be interesting 
> for Big Data apps, which often keep a lot of data off-heap
> Cons:
> - Unsafe can efficiently write only primitive types in their native byte 
> order and using their fixed size. This is not quite compatible with Avro's 
> format. 
> (While one can still use Unsafe to more efficiently write single elements 
> even using their variable length encoding, the biggest benefits of bulk array 
> serialization would be lost.)
> - Introducing this feature may require a definition of a new format for Avro. 
> This format would be very fast, but not very space efficient as it would use 
> fixed-size representation.
> BTW, initial tests where a sketch of the proposed optimization is applied 
> only to Floats and Doubles has shown immediate boost of 35%. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1580) Use newer version of surefire, 2.17 instead of 2.12

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1580:
--
Labels: beginner build  (was: build)

> Use newer version of surefire, 2.17 instead of 2.12
> ---
>
> Key: AVRO-1580
> URL: https://issues.apache.org/jira/browse/AVRO-1580
> Project: Avro
>  Issue Type: Improvement
>  Components: build, java
>Affects Versions: 1.7.7
>Reporter: Zoltan Farkas
>Priority: Minor
>  Labels: beginner, build
>
> version 2.12 does not work well with Netbeans 8.0 and makes development 
> cumbersome.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Assigned] (AVRO-1933) SchemaCompatibility class could be more user-friendly about incompatibilities

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reassigned AVRO-1933:
-

Assignee: Anders Sundelin

> SchemaCompatibility class could be more user-friendly about incompatibilities
> -
>
> Key: AVRO-1933
> URL: https://issues.apache.org/jira/browse/AVRO-1933
> Project: Avro
>  Issue Type: Improvement
>  Components: java
>Affects Versions: 1.7.7, 1.8.1
> Environment: Any Java env
>Reporter: Anders Sundelin
>Assignee: Anders Sundelin
>Priority: Minor
> Fix For: 1.9.0
>
> Attachments: AVRO-1933-compatible-with-AVRO-1931.patch, 
> AVRO-1933.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Today, the class SchemaCompatibility reports incompatibilities with quite 
> little detail. The whole reader and the whole writer schema is listed, and no 
> particular detail about what was incompatible.
> The attached patch fixes this, introducing a new enum 
> (SchemaIncompatibilityType), and more specific sub-schemas that were 
> incompatible.
> The old, overall picture, is still there - the new compatibility state is 
> encapsulated in the SchemaCompatibilityDetails class.
> Lots of test cases have been added, and there has been refactoring done in 
> the TestSchemaCompatibility and other test classes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1927) If a default value is set, Avro allows null values in non-nullable fields.

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1927:
--
Labels:   (was: newbie)

> If a default value is set, Avro allows null values in non-nullable fields.
> --
>
> Key: AVRO-1927
> URL: https://issues.apache.org/jira/browse/AVRO-1927
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Andreas Maier
>
> With an avro schema like
> {code}
> {
>   "name": "myfield",
>   "type": "string",
>   "default": ""
> }
> {code}
> the following code should throw an exception
> {code}
> MyObject myObject = MyObject.newBuilder().setMyfield(null).build();
> {code}
> But instead the value of myfield is set to null, which causes an exception 
> later when serializing myObject, because null is not a valid value for 
> myfield. 
> I believe in this case setMyfield(null) should throw an exception, 
> independent of the value of default. 
> See also
> https://stackoverflow.com/questions/38509279/generated-avro-builder-set-null-doesnt-overwrite-with-default



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1979) Errors when using avroccp.lib in Visual studio 2015

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1979:
--
Labels:   (was: newbie)

> Errors when using avroccp.lib in Visual studio 2015
> ---
>
> Key: AVRO-1979
> URL: https://issues.apache.org/jira/browse/AVRO-1979
> Project: Avro
>  Issue Type: Bug
>  Components: c++
>Affects Versions: 1.8.1
> Environment: Windows 7, 64Bit, Visual studio 2015 build for 32 bit 
> debug and release library  
>Reporter: Dimitrios Klimis
>Priority: Blocker
>
> In my current c++ project I need to use avro c++ serializer. Firstly I built 
> boost 1.6.2 and based on it I build via Visual Studio 2015 the Avro-cpp.sln 
> file as it is mentioned in avro relevant site. From the sIn file I build 
> avrocpp.lib and avrocpp.dll and I added the first to my linker.
>  The code I've created uses the following include files
>  #include "avro/DataFile.hh"   
>  #include "avro/Compiler.hh"   
>  #include "avro/ValidSchema.hh"   
>  #include "avro/Schema.hh"  
>  #include "avro/Generic.hh"
>  #include "avro/Specific.hh"
> However when I'm trying to build the project I get the following errors
> error LNK2019: unresolved external symbol "public: void __thiscall 
> avro::DataFileWriterBase::syncIfNeeded(void)  
> error LNK2019: unresolved external symbol "public: __thiscall 
> avro::DataFileWriterBase::DataFileWriterBase(char const *,class 
> avro::ValidSchema const &,unsigned int,enum avro::Codec)"  
> error LNK2019: unresolved external symbol "public: __thiscall 
> avro::DataFileWriterBase::~DataFileWriterBase(void)" 
> (??1DataFileWriterBase@avro@@QAE@XZ) referenced in function "public: void * 
> __thiscall avro::DataFileWriterBase::`scalar deleting destructor'(unsigned 
> int)"  
> error LNK2019: unresolved external symbol "public: void __thiscall 
> avro::DataFileWriterBase::close(void)"  
> error LNK2019: unresolved external symbol "public: void __thiscall 
> avro::DataFileWriterBase::close(void)"  
> error LNK2019: unresolved external symbol "void __cdecl 
> avro::compileJsonSchema(class std::basic_istream std::char_traits > &,class avro::ValidSchema &)"  
> error LNK2019: "public: void __thiscall avro::DataFileWriterBase::flush(void)"
> I assume that the linker does not find the definition of the the specific 
> classes DataFileWriterBase and compileJsonSchema.  Please can you give any 
> hint on what is wrong with the above issue?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (AVRO-1865) GenericData.Array class missing no arg constructor for Kryo serialization

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated AVRO-1865:
--
Labels: beginner patch  (was: newbie patch)

> GenericData.Array class missing no arg constructor for Kryo serialization
> -
>
> Key: AVRO-1865
> URL: https://issues.apache.org/jira/browse/AVRO-1865
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Alexander Kasper
>Assignee: Alexander Kasper
>  Labels: beginner, patch
> Attachments: GenericArray-no-args-ctor-kryo.patch
>
>
> When trying to serialize Java classes generated by Avro that contain a field 
> as follows:
> {{array myclasses = []}}
> serialization fails with the following error message
> {{com.esotericsoftware.kryo.KryoException: Class cannot be created (missing 
> no-arg constructor)}}
> Minimum example to recreate this:
> Avro IDL definition:
> {code:java}
> @namespace("com.adello")
> protocol KryoTest {
>   record TestRecord {
>   array values = [];
>   }
> }
> {code}
> This gets compiled using {{java -jar avro-tools-1.8.1.jar idl2schemata 
> kryo.avdl .}}
> Resulting Avro schema:
> {code:javascript}
> {
>   "type" : "record",
>   "name" : "TestRecord",
>   "namespace" : "com.adello",
>   "fields" : [ {
> "name" : "values",
> "type" : {
>   "type" : "array",
>   "items" : "int"
> },
> "default" : [ ]
>   } ]
> }
> {code}
> Generate Java class using {{java -jar avro-tools-1.8.1.jar compile schema 
> TestRecord.avsc}}
> Kryo serialization test code:
> {code:java}
> Kryo kryo = new Kryo();
> try {
> Output output = new Output(new FileOutputStream("test.bin"));
> TestRecord o = TestRecord.newBuilder().build();
> System.out.println("Before serialization: ");
> System.out.println(o.toString());
> kryo.writeObject(output, o);
> output.close();
> Input input = new Input(new FileInputStream("test.bin"));
> TestRecord i = kryo.readObject(input, TestRecord.class);
> input.close();
> System.out.println("After deserialization: ");
> System.out.println(i.toString());
> System.out.println(Objects.deepEquals(o, i));
> } catch (FileNotFoundException e) {
> e.printStackTrace();
> }
> {code}
> This is a common enough issue with Kryo that they have a dedicated section on 
> their website on it: 
> http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
> The attached patch adds a no args constructor for the inner class Array of 
> GenericData with zero entries and a dummy schema. My personal tests for 
> serialization with Kryo were successful when using it. Since I do not have 
> complete insight into Avro I'd like to know if this could be a breaking 
> change and how to test it if so.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)