[jira] [Updated] (AVRO-1788) python api differs unnecessarily between python2 and python3

[Suraj Acharya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suraj Acharya updated AVRO-1788:

Status: Patch Available  (was: Open)

> python api differs unnecessarily between python2 and python3
> 
>
> Key: AVRO-1788
> URL: https://issues.apache.org/jira/browse/AVRO-1788
> Project: Avro
>  Issue Type: Wish
>  Components: python
>Reporter: Alexander Hasha
>Priority: Minor
>
> It is difficult to use the python avro module in projects that maintain 
> simultaneous compatibility between python 2 and python 3, because there are 
> small differences in the API between the two versions in the module that seem 
> unnecessary.
> For example, to parse a schema string in python 2:
> {code}
> from avro.schema import parse
> {code}
> and in python 3:
> {code}
> from avro.schema import Parse
> {code}
> In Python2, DatumReader's constructor has keyword argument "writers_schema", 
> and in Python 3 it becomes "writer_schema" (no s).
> Would it be possible to align method names and arguments across the two 
> modules?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (AVRO-1788) python api differs unnecessarily between python2 and python3

[Suraj Acharya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suraj Acharya reassigned AVRO-1788:
---

Assignee: Suraj Acharya

> python api differs unnecessarily between python2 and python3
> 
>
> Key: AVRO-1788
> URL: https://issues.apache.org/jira/browse/AVRO-1788
> Project: Avro
>  Issue Type: Wish
>  Components: python
>Reporter: Alexander Hasha
>Assignee: Suraj Acharya
>Priority: Minor
>
> It is difficult to use the python avro module in projects that maintain 
> simultaneous compatibility between python 2 and python 3, because there are 
> small differences in the API between the two versions in the module that seem 
> unnecessary.
> For example, to parse a schema string in python 2:
> {code}
> from avro.schema import parse
> {code}
> and in python 3:
> {code}
> from avro.schema import Parse
> {code}
> In Python2, DatumReader's constructor has keyword argument "writers_schema", 
> and in Python 3 it becomes "writer_schema" (no s).
> Would it be possible to align method names and arguments across the two 
> modules?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AVRO-1788) python api differs unnecessarily between python2 and python3

[Suraj Acharya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suraj Acharya updated AVRO-1788:

Assignee: (was: Suraj Acharya)

> python api differs unnecessarily between python2 and python3
> 
>
> Key: AVRO-1788
> URL: https://issues.apache.org/jira/browse/AVRO-1788
> Project: Avro
>  Issue Type: Wish
>  Components: python
>Reporter: Alexander Hasha
>Priority: Minor
>
> It is difficult to use the python avro module in projects that maintain 
> simultaneous compatibility between python 2 and python 3, because there are 
> small differences in the API between the two versions in the module that seem 
> unnecessary.
> For example, to parse a schema string in python 2:
> {code}
> from avro.schema import parse
> {code}
> and in python 3:
> {code}
> from avro.schema import Parse
> {code}
> In Python2, DatumReader's constructor has keyword argument "writers_schema", 
> and in Python 3 it becomes "writer_schema" (no s).
> Would it be possible to align method names and arguments across the two 
> modules?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AVRO-1788) python api differs unnecessarily between python2 and python3

[Suraj Acharya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/AVRO-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Suraj Acharya updated AVRO-1788:

Component/s: python

> python api differs unnecessarily between python2 and python3
> 
>
> Key: AVRO-1788
> URL: https://issues.apache.org/jira/browse/AVRO-1788
> Project: Avro
>  Issue Type: Wish
>  Components: python
>Reporter: Alexander Hasha
>Priority: Minor
>
> It is difficult to use the python avro module in projects that maintain 
> simultaneous compatibility between python 2 and python 3, because there are 
> small differences in the API between the two versions in the module that seem 
> unnecessary.
> For example, to parse a schema string in python 2:
> {code}
> from avro.schema import parse
> {code}
> and in python 3:
> {code}
> from avro.schema import Parse
> {code}
> In Python2, DatumReader's constructor has keyword argument "writers_schema", 
> and in Python 3 it becomes "writer_schema" (no s).
> Would it be possible to align method names and arguments across the two 
> modules?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

[Niels Basjes (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529733#comment-15529733
 ] 

Niels Basjes commented on AVRO-1923:


Yes, that is better (see updated pull request). 

One thing I left out is removing it in a finally block.
It is an operation that I think is not needed (at the end of the toString the 
entire Map is discarded anyway).
Why do you do it?
Is it needed?


> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1849) C++ printJson fails on record with no fields

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529473#comment-15529473
 ] 

ASF GitHub Bot commented on AVRO-1849:
--

Github user Simon24601 closed the pull request at:

https://github.com/apache/avro/pull/97


> C++ printJson fails on record with no fields
> 
>
> Key: AVRO-1849
> URL: https://issues.apache.org/jira/browse/AVRO-1849
> Project: Avro
>  Issue Type: Bug
>  Components: c++
>Affects Versions: 1.7.7, 1.8.0
> Environment: Not relevant
>Reporter: Simon Woodford
>Assignee: Simon Woodford
>Priority: Minor
> Fix For: 1.9.0
>
>
> The function NodeRecord::printJson in lang/c++/impl/NodeImpl.cc does not 
> correctly handle a record with no fields. It injects an extra closing curly 
> brace, and the result is invalid JSON. 
> Starting with a schema
> {
>   "type": "record", 
>   "name": "Example",
>   "fields" : [
>   ]
> }
> and parsing it to create a ValidSchema, then calling toJson on the 
> ValidSchema generates
> {
>   "type": "record", 
>   "name": "Example",
>   "fields" : [
> }
>   ]
> }
> A record with no fields is unusual but we have encountered use cases for it, 
> avro does not invalidate a record with no fields, and I've confirmed that 
> Java, C and C# handle this case correctly. (I have not checked the other 
> supported languages.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] avro pull request #97: AVRO-1849 Fix C++ schema to JSON converter

[Simon24601]

Github user Simon24601 closed the pull request at:

https://github.com/apache/avro/pull/97


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (AVRO-1853) Compiler::toBin(const std::string& s) crashes on empty string

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529351#comment-15529351
 ] 

ASF GitHub Bot commented on AVRO-1853:
--

Github user zicl closed the pull request at:

https://github.com/apache/avro/pull/127


> Compiler::toBin(const std::string& s) crashes on empty string
> -
>
> Key: AVRO-1853
> URL: https://issues.apache.org/jira/browse/AVRO-1853
> Project: Avro
>  Issue Type: Bug
>  Components: c++
>Affects Versions: 1.8.1
> Environment: Win64 debug
>Reporter: Hans Thulin
>Assignee: Zoltan Ivanfi
>Priority: Critical
> Fix For: 1.9.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Compiler::toBin(const std::string& s) crashes if s is empty. A simple sanity 
> check should fix it:
> static vector toBin(const std::string& s)
> {
> vector result;
> if(!s.empty())
> {   
> result.resize(s.size());
> std::copy(s.c_str(), s.c_str() + s.size(), [0]);
> }
> return result;
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] avro pull request #127: AVRO-1853: Compiler::toBin crashes on empty string

[zicl]

Github user zicl closed the pull request at:

https://github.com/apache/avro/pull/127


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

[Gabor Szadovszky (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529342#comment-15529342
 ] 

Gabor Szadovszky commented on AVRO-1605:


Thanks a lot, [~tomwhite]. 

The friend package approach seems to be the better choice. As we are using the 
Jackson API in many different packages I'll create the accessor in 
{{org.apache.avro.util.internal}}. ({{*.internal}} packages are already skipped 
in the javadoc generation.) 

Schema immutability would not be easy as its super class {{JsonProperties}} has 
public methods for adding properties which are not deprecated yet. 
({{addProp(String, String)}} and {{addProp(String, Object))}}). 

> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] avro pull request #134: AOT support

[dolow]

GitHub user dolow opened a pull request:

https://github.com/apache/avro/pull/134

AOT support

For supporting compilers which do not allow JIT, such as iOS compilation, I 
removed lines where Reflection.Emit was used and refactored related methods.

The part Reflection.Emit was used was making a constructor of given type.
However it could be replaced with Activator.CreateInstance (it is OK for 
AOT) by using ObjectCreator.GetType method with the same arguments as 
GetConstructor methods.

I hope this is acceptable for Avro.
Thank you.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dolow/avro feature/aot_support

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/134.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #134


commit aad24a7b1bbeb39105b8b3ee9578dbc792d8cfea
Author: dolow 
Date:   2016-09-28T08:45:43Z

replaced Reflection.Emit to simple instantiation for supporting AOT.

commit 0cf2081569f3244ae535710d46ea41e15e57bbcc
Author: dolow 
Date:   2016-09-28T11:10:17Z

removed unused GetConstructor methods




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

[Tom White (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529260#comment-15529260
 ] 

Tom White commented on AVRO-1605:
-

I see your point. The motivation behind this change is to upgrade to Jackson 2 
(AVRO-1126), so another way to approach this would be to see how the deprecated 
methods could be removed in making that change.

I think the main problem here is that there are public methods that should 
really only be used by the internals, like JsonProperties#addProp(). In the 
absence of any Java-level mechanism for dealing with that, our options are i) 
relying on documentation (or an annotation) to say that a method is internal 
only, or ii) using a friend package schema and excluding the friend package 
from javadoc (see 
http://wiki.apidesign.org/wiki/APIDesignPatterns:FriendPackages).

Ideally, we'd also make Schema immutable at the same time, but that doesn't 
make this problem any easier.

> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

[Gabor Szadovszky (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529070#comment-15529070
 ] 

Gabor Szadovszky commented on AVRO-1605:


I completely agree that the best way would be removing all the public methods 
related to Jackson classes. 
I've started with this {{GeneralJsonNode}} idea because  otherwise I have to 
refactor/redesign every code part where we are building up/using Jackson 
structures outside of the package {{org.apache.avro}}. For example {{Idl}} 
builds up {{JsonNode}} structures and uses {{JsonProperties#addProp(String, 
JsonNode)}}.
If we want to really remove all the leaking public methods we will have to 
design a kind of json structure builder solution independently from any 3rd 
party json libs (or wrapping the whole). Or is it just me who cannot see the 
"mechanical change"? :)


> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

[Tom White (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528994#comment-15528994
 ] 

Tom White commented on AVRO-1605:
-

I'm not sure it gains much to make incompatible changes to deprecated methods 
over removing them (e.g. changing JsonProperties#getJsonProp to return 
GeneralJsonNode - defined in Avro - instead of Jackson's JsonNode).

Perhaps the first task is actually to change the internals so they no longer 
use any of the methods deprecated in AVRO-1585, then remove them as a second 
step once we know that the replacements are sufficient. There are lots of 
places in the codebase to change, but it should be a fairly mechanical change, 
and one limited to internal classes, so I don't see it as a problem.

> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

[Gabor Szadovszky (JIRA)]

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528821#comment-15528821
 ] 

Gabor Szadovszky commented on AVRO-1605:


There are many public methods leaking Jackson classes (mainly JsonNode) which 
are called internally from other packages. Therefore, it is really not easy to 
redesign the whole nor to tighten the visibility.

I've already started working on an idea. Could you please check my incomplete 
changes if I'm in a good direction?
https://github.com/apache/avro/compare/master...gszadovszky:AVRO-1605?expand=1

> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)