Re: backporting a fix to avro 1.7

2016-10-04 Thread radai 
Thanks,

I've opened a PR to backport to 1.8 (https://github.com/apache/avro/pull/140
- has everything that landed originally except additions to CHANGES.txt)

For 1.7, I had to 1st bring the branch to compile (PR here -
https://github.com/apache/avro/pull/139) and will backport in a subsequent
PR.


On Tue, Oct 4, 2016 at 6:28 AM, Sean Busbey  wrote:

> (For others, the jira under discussion is AVRO-1901
> https://issues.apache.org/jira/browse/AVRO-1901)
>
> On Tue, Oct 4, 2016 at 8:28 AM, Sean Busbey  wrote:
> > yes, it looks like the jira landed in 1.9, though there isn't a fix
> > version set. I don't see anything in the patch that would prevent a
> > backport to 1.7.
> >
> > On Mon, Oct 3, 2016 at 2:28 PM, radai 
> wrote:
> >> Hi,
> >>
> >> I've recently encountered a bug in avro code generation, for which I
> >> submitted a fix (
> >> https://github.com/apache/avro/commit/be33922c03bd229b885fb9d664b55c
> e308f6ffaf
> >> ).
> >>
> >> The issue was discovered while trying to migrate a sizable codebase from
> >> avro 1.4 to 1.7
> >>
> >> Would it be possible to back-port this fix to 1.7? bumping 1.7 to latest
> >> would be a lot easier than trying to upgrade 1.4 and now 1.7 code to 1.9
> >> (which is where this will will end up in, when its released?)
> >>
> >> I can just open a PR porting the fix to the 1.7 branch (maybe 1.8
> branch as
> >> well, for consistency), if thats the procedure.
> >>
> >> Thanks in advance,
> >>
> >>
> >> Radai.
> >
> >
> >
> > --
> > busbey
>
>
>
> --
> busbey
>

[jira] [Commented] (AVRO-1901) creating a record called "Exception" leads to generated code not compiling

2016-10-04 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546946#comment-15546946
 ] 

ASF GitHub Bot commented on AVRO-1901:
--

GitHub user radai-rosenblatt opened a pull request:

https://github.com/apache/avro/pull/140

AVRO-1901: Record named Exception generated bad code

NOTE - I did not update CHANGES.txt

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/radai-rosenblatt/avro backport-1901-to-1.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/140.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #140


commit 4ae6deb82ce7341f0285559e38111a836f46d430
Author: radai-rosenblatt 
Date:   2016-09-06T16:16:42Z

AVRO-1901: Record named Exception generated bad code




> creating a record called "Exception" leads to generated code not compiling
> --
>
> Key: AVRO-1901
> URL: https://issues.apache.org/jira/browse/AVRO-1901
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.1, 1.9.0
>Reporter: radai rosenblatt
>Assignee: Niels Basjes
>
> consider the following *.avsc file:
> {code:title=Exception.avsc}
> {
>   "type": "record",
>   "name": "Exception",
>   "namespace": "com.acme",
>   "fields" : [
> {"name": "value", "type": "int"}
>   ],
>   "doc": "Will not submit to your fascist naming conventions"
> }
> {code}
> the generated java code will contain:
> {code:title=Exception.java}
> ...
> @override
> public Exception build() {
>try {
>   ...
>   return record;
>} catch (Exception e) { //boom
>   throw new org.apache.avro.AvroRuntimeException(e);
>}
> }
> {code}
> which fails to compile because Exception refers to the generated class (which 
> does not extend Throwable)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] avro pull request #140: AVRO-1901: Record named Exception generated bad code

2016-10-04 Thread radai-rosenblatt 
GitHub user radai-rosenblatt opened a pull request:

https://github.com/apache/avro/pull/140

AVRO-1901: Record named Exception generated bad code

NOTE - I did not update CHANGES.txt

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/radai-rosenblatt/avro backport-1901-to-1.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/140.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #140


commit 4ae6deb82ce7341f0285559e38111a836f46d430
Author: radai-rosenblatt 
Date:   2016-09-06T16:16:42Z

AVRO-1901: Record named Exception generated bad code




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] avro pull request #139: Make 1.7 branch build

2016-10-04 Thread radai-rosenblatt 
GitHub user radai-rosenblatt opened a pull request:

https://github.com/apache/avro/pull/139

Make 1.7 branch build

Seems like AVRO-1681 was only partially applied to the 1.7 branch. 
Specifically, the expected test output was not updated to reflect changes in 
javadoc.
This commit fixes the tests and makes the build pass

Signed-off-by: radai-rosenblatt 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/radai-rosenblatt/avro make-1.7-build

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/avro/pull/139.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #139


commit ae3e189927aaf42d36dbb6248915b97a9dad766b
Author: radai-rosenblatt 
Date:   2016-10-04T22:36:18Z

Make 1.7 branch actually build

Seems like AVRO-1681 was only partially applied to the 1.7 branch. 
Specifically, the expected test output was not updated to reflect changes in 
javadoc.
This commit fixes the tests and makes the build pass

Signed-off-by: radai-rosenblatt 




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

2016-10-04 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545527#comment-15545527
 ] 

ASF GitHub Bot commented on AVRO-1923:
--

Github user nielsbasjes closed the pull request at:

https://github.com/apache/avro/pull/128


> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] avro pull request #128: AVRO-1923: Stop infinite recursion in GenericData.to...

2016-10-04 Thread nielsbasjes 
Github user nielsbasjes closed the pull request at:

https://github.com/apache/avro/pull/128


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

2016-10-04 Thread Niels Basjes (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated AVRO-1923:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

2016-10-04 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545480#comment-15545480
 ] 

ASF subversion and git services commented on AVRO-1923:
---

Commit e276aa5e40cf51e36b581750c135c9c3875cac84 in avro's branch 
refs/heads/master from [~nielsbasjes]
[ https://git-wip-us.apache.org/repos/asf?p=avro.git;h=e276aa5 ]

AVRO-1923: Stop infinite recursion in GenericData.toString


> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1923) Recursive record causes StackOverflow in GenericData.toString

2016-10-04 Thread Niels Basjes (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545481#comment-15545481
 ] 

Niels Basjes commented on AVRO-1923:


Committed.

> Recursive record causes StackOverflow in GenericData.toString
> -
>
> Key: AVRO-1923
> URL: https://issues.apache.org/jira/browse/AVRO-1923
> Project: Avro
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: AVRO-1923-Unittest.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion

2016-10-04 Thread Anders Sundelin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anders Sundelin updated AVRO-1929:
--
Status: Patch Available  (was: Open)

This time, added the patch as a attached file
HTH
/Anders

> SchemaCompatibility fails to recognize string<->bytes promotion
> ---
>
> Key: AVRO-1929
> URL: https://issues.apache.org/jira/browse/AVRO-1929
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1, 1.8.0, 1.7.7, 1.9.0
> Environment: Any Java env
>Reporter: Anders Sundelin
>Priority: Minor
> Attachments: AVRO-1929.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The class SchemaCompatibility fails to realize the change made as of 
> AVRO-1553, where string-to/from-bytes promotions were allowed (according to 
> docs, this was integrated into 1.7.7)
> The submitted fix corrects this (also updating the relevant unit test)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion

2016-10-04 Thread Anders Sundelin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/AVRO-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anders Sundelin updated AVRO-1929:
--
Attachment: AVRO-1929.patch

Patch for AVRO-1929 issue

> SchemaCompatibility fails to recognize string<->bytes promotion
> ---
>
> Key: AVRO-1929
> URL: https://issues.apache.org/jira/browse/AVRO-1929
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.7.7, 1.8.0, 1.8.1, 1.9.0
> Environment: Any Java env
>Reporter: Anders Sundelin
>Priority: Minor
> Attachments: AVRO-1929.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> The class SchemaCompatibility fails to realize the change made as of 
> AVRO-1553, where string-to/from-bytes promotions were allowed (according to 
> docs, this was integrated into 1.7.7)
> The submitted fix corrects this (also updating the relevant unit test)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (AVRO-1929) SchemaCompatibility fails to recognize string<->bytes promotion

2016-10-04 Thread Anders Sundelin (JIRA) 
Anders Sundelin created AVRO-1929:
-

 Summary: SchemaCompatibility fails to recognize string<->bytes 
promotion
 Key: AVRO-1929
 URL: https://issues.apache.org/jira/browse/AVRO-1929
 Project: Avro
  Issue Type: Bug
  Components: java
Affects Versions: 1.8.1, 1.8.0, 1.7.7, 1.9.0
 Environment: Any Java env
Reporter: Anders Sundelin
Priority: Minor


The class SchemaCompatibility fails to realize the change made as of 
AVRO-1553, where string-to/from-bytes promotions were allowed (according to 
docs, this was integrated into 1.7.7)
The submitted fix corrects this (also updating the relevant unit test)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: backporting a fix to avro 1.7

2016-10-04 Thread Sean Busbey 
(For others, the jira under discussion is AVRO-1901
https://issues.apache.org/jira/browse/AVRO-1901)

On Tue, Oct 4, 2016 at 8:28 AM, Sean Busbey  wrote:
> yes, it looks like the jira landed in 1.9, though there isn't a fix
> version set. I don't see anything in the patch that would prevent a
> backport to 1.7.
>
> On Mon, Oct 3, 2016 at 2:28 PM, radai  wrote:
>> Hi,
>>
>> I've recently encountered a bug in avro code generation, for which I
>> submitted a fix (
>> https://github.com/apache/avro/commit/be33922c03bd229b885fb9d664b55ce308f6ffaf
>> ).
>>
>> The issue was discovered while trying to migrate a sizable codebase from
>> avro 1.4 to 1.7
>>
>> Would it be possible to back-port this fix to 1.7? bumping 1.7 to latest
>> would be a lot easier than trying to upgrade 1.4 and now 1.7 code to 1.9
>> (which is where this will will end up in, when its released?)
>>
>> I can just open a PR porting the fix to the 1.7 branch (maybe 1.8 branch as
>> well, for consistency), if thats the procedure.
>>
>> Thanks in advance,
>>
>>
>> Radai.
>
>
>
> --
> busbey



-- 
busbey

Re: backporting a fix to avro 1.7

2016-10-04 Thread Sean Busbey 
yes, it looks like the jira landed in 1.9, though there isn't a fix
version set. I don't see anything in the patch that would prevent a
backport to 1.7.

On Mon, Oct 3, 2016 at 2:28 PM, radai  wrote:
> Hi,
>
> I've recently encountered a bug in avro code generation, for which I
> submitted a fix (
> https://github.com/apache/avro/commit/be33922c03bd229b885fb9d664b55ce308f6ffaf
> ).
>
> The issue was discovered while trying to migrate a sizable codebase from
> avro 1.4 to 1.7
>
> Would it be possible to back-port this fix to 1.7? bumping 1.7 to latest
> would be a lot easier than trying to upgrade 1.4 and now 1.7 code to 1.9
> (which is where this will will end up in, when its released?)
>
> I can just open a PR porting the fix to the 1.7 branch (maybe 1.8 branch as
> well, for consistency), if thats the procedure.
>
> Thanks in advance,
>
>
> Radai.



-- 
busbey

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

2016-10-04 Thread Tom White (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544956#comment-15544956
 ] 

Tom White commented on AVRO-1605:
-

I'm not sure how long the accessors would need to be there. In principle, they 
could be removed if the internal code started using the Object overrides for 
JSON values (e.g. {{Field(String, Schema, String, Object)}} rather than 
{{Field(String, Schema, String, JsonNode)}}); however I don't know how feasible 
(or performant) that would be.

It's more likely that we'd continue to use them even if we switched to a new 
JSON API internally, since this would allow us to create new Field objects 
(say), using a hidden {{Field(String, Schema, String, NewJsonRepresentation)}} 
constructor. How would we remove the accessor for Field without exposing new 
JSON APIs?



> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1861) Avro Schema parser treats Avro float type as Java Double for default values

2016-10-04 Thread Gabor Szadovszky (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544838#comment-15544838
 ] 

Gabor Szadovszky commented on AVRO-1861:


One fix for both.

> Avro Schema parser treats Avro float type as Java Double for default values
> ---
>
> Key: AVRO-1861
> URL: https://issues.apache.org/jira/browse/AVRO-1861
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Andy Mok
>Assignee: Gabor Szadovszky
>
> The following code snippet in the [Schema 
> class|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Schema.java]
>  shows that we explicitly treat Avro {{FLOAT}} and {{DOUBLE}} as a Java 
> {{Double}}.
> {code:java}
> JsonNode defaultValue = field.get("default");
> if (defaultValue != null
> && (Type.FLOAT.equals(fieldSchema.getType())
> || Type.DOUBLE.equals(fieldSchema.getType()))
> && defaultValue.isTextual())
>   defaultValue =
> new DoubleNode(Double.valueOf(defaultValue.getTextValue()));
> {code}
> Jackson has support for 
> [FloatNode|https://fasterxml.github.io/jackson-databind/javadoc/2.3.0/com/fasterxml/jackson/databind/node/FloatNode.html]
>  so why don't we use that?
> This is a problem when someone calls 
> [Schema.Field#defaultVal|https://avro.apache.org/docs/1.8.1/api/java/org/apache/avro/Schema.Field.html#defaultVal()]
>  for an Avro field with Avro type {{FLOAT}} and they try to typecast the 
> object to a Java {{float}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1605) Remove Jackson classes from public API

2016-10-04 Thread Gabor Szadovszky (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544756#comment-15544756
 ] 

Gabor Szadovszky commented on AVRO-1605:


My understanding about this issue was to not "leak" the jackson classes outside 
of the public API. I didn't think we would like to get rid of the whole jackson 
lib. Or what would be your concept for it?
There are some places like the class Idl where we are also relying on the 
jackson classes and using the related methods in the package org.apache.avro. 
It does not seem to be an easy job to rewrite these code parts (at least for 
me).

> Remove Jackson classes from public API
> --
>
> Key: AVRO-1605
> URL: https://issues.apache.org/jira/browse/AVRO-1605
> Project: Avro
>  Issue Type: Sub-task
>  Components: java
>Affects Versions: 1.7.8
>Reporter: Tom White
>Assignee: Gabor Szadovszky
> Fix For: 1.9.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (AVRO-1861) Avro Schema parser treats Avro float type as Java Double for default values

2016-10-04 Thread Gabor Szadovszky (JIRA) 

[ 
https://issues.apache.org/jira/browse/AVRO-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544668#comment-15544668
 ] 

Gabor Szadovszky commented on AVRO-1861:


You are absolutely right. I just wanted to resolve this one the same way as 
AVRO-1860 but it would not make sense. Removing the requires reference...

> Avro Schema parser treats Avro float type as Java Double for default values
> ---
>
> Key: AVRO-1861
> URL: https://issues.apache.org/jira/browse/AVRO-1861
> Project: Avro
>  Issue Type: Bug
>  Components: java
>Affects Versions: 1.8.1
>Reporter: Andy Mok
>Assignee: Gabor Szadovszky
>
> The following code snippet in the [Schema 
> class|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Schema.java]
>  shows that we explicitly treat Avro {{FLOAT}} and {{DOUBLE}} as a Java 
> {{Double}}.
> {code:java}
> JsonNode defaultValue = field.get("default");
> if (defaultValue != null
> && (Type.FLOAT.equals(fieldSchema.getType())
> || Type.DOUBLE.equals(fieldSchema.getType()))
> && defaultValue.isTextual())
>   defaultValue =
> new DoubleNode(Double.valueOf(defaultValue.getTextValue()));
> {code}
> Jackson has support for 
> [FloatNode|https://fasterxml.github.io/jackson-databind/javadoc/2.3.0/com/fasterxml/jackson/databind/node/FloatNode.html]
>  so why don't we use that?
> This is a problem when someone calls 
> [Schema.Field#defaultVal|https://avro.apache.org/docs/1.8.1/api/java/org/apache/avro/Schema.Field.html#defaultVal()]
>  for an Avro field with Avro type {{FLOAT}} and they try to typecast the 
> object to a Java {{float}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)