[jira] [Created] (AVRO-2402) Generics, provide access to parent object
Werner Daehn created AVRO-2402: -- Summary: Generics, provide access to parent object Key: AVRO-2402 URL: https://issues.apache.org/jira/browse/AVRO-2402 Project: Apache Avro Issue Type: Improvement Components: java Affects Versions: 1.10.0 Reporter: Werner Daehn Imagine you are within the GenericRecord Employee->Addresses. You have all fields of the Addresses Schema of course but what if you need to read something from the parent, the employee record? You would need to remember the entire parent tree, which is fine if you are just one level deep but can be cumbersome when the structure is 10 levels deep. I'd like the GenericDatumReader to return objects that have the methods * getParent() * getRoot() as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2401) Initializing decimal fields with fixed fields can cause padding issues in Java
[ https://issues.apache.org/jira/browse/AVRO-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gray updated AVRO-2401: --- Description: I'm having a hard time figuring out if this a a bug or expected behavior. If we initialize a decimal field with a fixed logical type that has fewer bytes then the fixed size has, then the bytes get padded on the right with 0s. This changes their resulting decimal value. I think they should be padded from the left. The spec is vague here so I'm not sure if this is a bug or just unexpected outcome. I've attached my junit test. The result of the test is: AssertionError: expected:<660.51> but was:<169090.56> was: I'm having a hard time figuring out if this a a bug or expected behavior. If we initialize a decimal field with a fixed logical type that has fewer bytes then the fixed size has, then the bytes get padded on the right with 0s. This changes their resulting decimal value. I think they should be padded from the left. The spec is vague here so I'm not sure if this is a bug or just unexpected outcome. I'll attach my junit test. The result of the test is: AssertionError: expected:<660.51> but was:<169090.56> > Initializing decimal fields with fixed fields can cause padding issues in Java > -- > > Key: AVRO-2401 > URL: https://issues.apache.org/jira/browse/AVRO-2401 > Project: Apache Avro > Issue Type: Bug > Components: java >Reporter: Gray >Priority: Minor > Attachments: DecimalConverterTest.java > > > I'm having a hard time figuring out if this a a bug or expected behavior. If > we initialize a decimal field with a fixed logical type that has fewer bytes > then the fixed size has, then the bytes get padded on the right with 0s. > This changes their resulting decimal value. I think they should be padded > from the left. The spec is vague here so I'm not sure if this is a bug or > just unexpected outcome. > I've attached my junit test. The result of the test is: AssertionError: > expected:<660.51> but was:<169090.56> -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [VOTE] Release Apache Avro 1.9.0 RC4
Thanks for fixing that Fokko. Maven Repository always takes 'ages' to catch up but since they are in Maven Central [1] I think we are good to announce. [1] https://search.maven.org/artifact/org.apache.avro/avro/1.9.0/bundle On Tue, May 21, 2019 at 2:47 PM Driesprong, Fokko wrote: > > My mistake. I've removed the 1.8.2 version from the mirror. In the > meantime, I'm still waiting for it to appear on Maven Repository: > https://mvnrepository.com/artifact/org.apache.avro/avro > > It would expect to have it been indexed already. Anyone any idea? > > Cheers, Fokko Driesprong > > Op di 21 mei 2019 om 10:06 schreef Ismaël Mejía : > > > Seems the jars are now everywhere. Are we going to do an Announcement > > (specially to announcement mailing list)? > > Also the download link points still to both 1.8.2 and 1.9.0 is this > > wished or we just forgot to remove 1.8.2 ? > > https://www.apache.org/dyn/closer.cgi/avro/ > > > > On Sun, May 19, 2019 at 6:59 PM Driesprong, Fokko > > wrote: > > > > > > Can you elaborate Raman? How did you confirm that it isn't the latest > > > version? > > > > > > Thanks for the heads-up Rumeshkrishnan, I'll check why 1.9.0 isn't > > showing > > > up in Maven Central. > > > > > > Cheers, Fokko > > > > > > Op vr 17 mei 2019 om 22:28 schreef Raman Gupta : > > > > > > > That's just mvnrepository not showing the latest information. See: > > > > https://search.maven.org/artifact/org.apache.avro/avro/1.9.0/bundle > > > > > > > > On Fri, May 17, 2019 at 4:16 PM RumeshKrishnan Mohan > > > > wrote: > > > > > > > > > > Hi everyone, > > > > > > > > > > I am unable to see 1.9.0 in the artifact in maven, > > > > > https://mvnrepository.com/artifact/org.apache.avro/avro > > > > > > > > > > By - Rumeshkrishnan > > > > > > > > > > > > > > > On Fri, May 17, 2019 at 10:09 PM rocketra...@gmail.com < > > > > > rocketra...@gmail.com> wrote: > > > > > > > > > > > Fokko, did you have a chance to look into this? I just built a > > project > > > > > > with 1.9.0, and am running into several issues that should have > > been > > > > fixed > > > > > > during the RCs: > > > > > > https://issues.apache.org/jira/browse/AVRO-2360, > > > > > > https://issues.apache.org/jira/browse/AVRO-2386, and > > > > > > https://issues.apache.org/jira/browse/AVRO-2383 which should all > > have > > > > > > been fixed. > > > > > > > > > > > > Looking at the avro-compiler.jar, all the class files are dated > > Apr 26 > > > > > > 9:32 EDT, which is before some of those issues were even reported. > > > > > > > > > > > > Regards, > > > > > > Raman > > > > > > > > > > > > On 2019/05/15 18:46:03, "Driesprong, Fokko" > > > > wrote: > > > > > > > Hi Jacob, > > > > > > > > > > > > > > This looks off, I'll dive into it. I think it got released for > > some > > > > > > reason. > > > > > > > While releasing it today, I also noticed that the artifact wasn't > > > > staged > > > > > > in > > > > > > > Nexus anymore. However, RC4 is the same as the release. > > > > > > > > > > > > > > Cheers, Fokko > > > > > > > > > > > > > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar > > > > > > > : > > > > > > > > > > > > > > > Great news! We have been looking forward to the 1.9 release. > > > > > > > > > > > > > > > > I see 1.9.0 published on Maven Central. It appears to have been > > > > > > published > > > > > > > > 5/8 > > > > > > > > (the same day as RC4 was announced but ~1week before vote > > passed). > > > > > > > > > > > > > > > > Is this the release build (presumably the same as RC4) or an > > > > earlier > > > > > > RC? > > > > > > > > > > > > > > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/ > > > > > > > > > > > > > > > > Would it be helpful to have maven include the git sha > > somewhere in > > > > > > > > MANIFEST.MF? > > > > > > > > > > > > > > > > jacob > > > > > > > > > > > > > > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Thank you all, > > > > > > > > > > > > > > > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed. > > > > > > > > > > > > > > > > > > Binding: > > > > > > > > > +1 Daniel Kulp > > > > > > > > > +1 Sean Busbey > > > > > > > > > +1 Fokko Driesprong > > > > > > > > > > > > > > > > > > Non-binding: > > > > > > > > > +1 Lunjie Jin > > > > > > > > > +1 Michael A. Smith > > > > > > > > > +1 Ismaël Mejía > > > > > > > > > +1 Brian Lachniet > > > > > > > > > > > > > > > > > > I would like to thank the community from participating in the > > > > > > release. > > > > > > > > Both > > > > > > > > > in providing patches for Apache Avro and giving the RC's a > > try. > > > > > > > > > > > > > > > > > > Yesterday I did some additional testing against Apache Spark > > > > > > (similar to > > > > > > > > > Nandor :-) and Apache Flink and did not find any blocking > > issues. > > > > > > There > > > > > > > > are > > > > > > > > > some incompatibilities, but most of them are; revoking > > Jackson > > > > from > > > > > >
Re: [VOTE] Release Apache Avro 1.9.0 RC4
My mistake. I've removed the 1.8.2 version from the mirror. In the meantime, I'm still waiting for it to appear on Maven Repository: https://mvnrepository.com/artifact/org.apache.avro/avro It would expect to have it been indexed already. Anyone any idea? Cheers, Fokko Driesprong Op di 21 mei 2019 om 10:06 schreef Ismaël Mejía : > Seems the jars are now everywhere. Are we going to do an Announcement > (specially to announcement mailing list)? > Also the download link points still to both 1.8.2 and 1.9.0 is this > wished or we just forgot to remove 1.8.2 ? > https://www.apache.org/dyn/closer.cgi/avro/ > > On Sun, May 19, 2019 at 6:59 PM Driesprong, Fokko > wrote: > > > > Can you elaborate Raman? How did you confirm that it isn't the latest > > version? > > > > Thanks for the heads-up Rumeshkrishnan, I'll check why 1.9.0 isn't > showing > > up in Maven Central. > > > > Cheers, Fokko > > > > Op vr 17 mei 2019 om 22:28 schreef Raman Gupta : > > > > > That's just mvnrepository not showing the latest information. See: > > > https://search.maven.org/artifact/org.apache.avro/avro/1.9.0/bundle > > > > > > On Fri, May 17, 2019 at 4:16 PM RumeshKrishnan Mohan > > > wrote: > > > > > > > > Hi everyone, > > > > > > > > I am unable to see 1.9.0 in the artifact in maven, > > > > https://mvnrepository.com/artifact/org.apache.avro/avro > > > > > > > > By - Rumeshkrishnan > > > > > > > > > > > > On Fri, May 17, 2019 at 10:09 PM rocketra...@gmail.com < > > > > rocketra...@gmail.com> wrote: > > > > > > > > > Fokko, did you have a chance to look into this? I just built a > project > > > > > with 1.9.0, and am running into several issues that should have > been > > > fixed > > > > > during the RCs: > > > > > https://issues.apache.org/jira/browse/AVRO-2360, > > > > > https://issues.apache.org/jira/browse/AVRO-2386, and > > > > > https://issues.apache.org/jira/browse/AVRO-2383 which should all > have > > > > > been fixed. > > > > > > > > > > Looking at the avro-compiler.jar, all the class files are dated > Apr 26 > > > > > 9:32 EDT, which is before some of those issues were even reported. > > > > > > > > > > Regards, > > > > > Raman > > > > > > > > > > On 2019/05/15 18:46:03, "Driesprong, Fokko" > > > wrote: > > > > > > Hi Jacob, > > > > > > > > > > > > This looks off, I'll dive into it. I think it got released for > some > > > > > reason. > > > > > > While releasing it today, I also noticed that the artifact wasn't > > > staged > > > > > in > > > > > > Nexus anymore. However, RC4 is the same as the release. > > > > > > > > > > > > Cheers, Fokko > > > > > > > > > > > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar > > > > > > : > > > > > > > > > > > > > Great news! We have been looking forward to the 1.9 release. > > > > > > > > > > > > > > I see 1.9.0 published on Maven Central. It appears to have been > > > > > published > > > > > > > 5/8 > > > > > > > (the same day as RC4 was announced but ~1week before vote > passed). > > > > > > > > > > > > > > Is this the release build (presumably the same as RC4) or an > > > earlier > > > > > RC? > > > > > > > > > > > > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/ > > > > > > > > > > > > > > Would it be helpful to have maven include the git sha > somewhere in > > > > > > > MANIFEST.MF? > > > > > > > > > > > > > > jacob > > > > > > > > > > > > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Thank you all, > > > > > > > > > > > > > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed. > > > > > > > > > > > > > > > > Binding: > > > > > > > > +1 Daniel Kulp > > > > > > > > +1 Sean Busbey > > > > > > > > +1 Fokko Driesprong > > > > > > > > > > > > > > > > Non-binding: > > > > > > > > +1 Lunjie Jin > > > > > > > > +1 Michael A. Smith > > > > > > > > +1 Ismaël Mejía > > > > > > > > +1 Brian Lachniet > > > > > > > > > > > > > > > > I would like to thank the community from participating in the > > > > > release. > > > > > > > Both > > > > > > > > in providing patches for Apache Avro and giving the RC's a > try. > > > > > > > > > > > > > > > > Yesterday I did some additional testing against Apache Spark > > > > > (similar to > > > > > > > > Nandor :-) and Apache Flink and did not find any blocking > issues. > > > > > There > > > > > > > are > > > > > > > > some incompatibilities, but most of them are; revoking > Jackson > > > from > > > > > the > > > > > > > > public API and deprecating Joda. > > > > > > > > > > > > > > > > I'll publish the artifacts to the different repositories > soon. On > > > > > this, I > > > > > > > > consider myself a polyglot programmer, but my knowledge is > > > limited on > > > > > > > .Net, > > > > > > > > Perl, and Javascript, so I might ask some help from the > community > > > > > here as > > > > > > > > well. > > > > > > > > > > > > > > > > Thanks all, and keep up the good work. > > > > > > > > > > > > > > > > Cheers, Fokko Driesprong > > > > > > > > > > > > >
Re: [VOTE] Release Apache Avro 1.9.0 RC4
Seems the jars are now everywhere. Are we going to do an Announcement (specially to announcement mailing list)? Also the download link points still to both 1.8.2 and 1.9.0 is this wished or we just forgot to remove 1.8.2 ? https://www.apache.org/dyn/closer.cgi/avro/ On Sun, May 19, 2019 at 6:59 PM Driesprong, Fokko wrote: > > Can you elaborate Raman? How did you confirm that it isn't the latest > version? > > Thanks for the heads-up Rumeshkrishnan, I'll check why 1.9.0 isn't showing > up in Maven Central. > > Cheers, Fokko > > Op vr 17 mei 2019 om 22:28 schreef Raman Gupta : > > > That's just mvnrepository not showing the latest information. See: > > https://search.maven.org/artifact/org.apache.avro/avro/1.9.0/bundle > > > > On Fri, May 17, 2019 at 4:16 PM RumeshKrishnan Mohan > > wrote: > > > > > > Hi everyone, > > > > > > I am unable to see 1.9.0 in the artifact in maven, > > > https://mvnrepository.com/artifact/org.apache.avro/avro > > > > > > By - Rumeshkrishnan > > > > > > > > > On Fri, May 17, 2019 at 10:09 PM rocketra...@gmail.com < > > > rocketra...@gmail.com> wrote: > > > > > > > Fokko, did you have a chance to look into this? I just built a project > > > > with 1.9.0, and am running into several issues that should have been > > fixed > > > > during the RCs: > > > > https://issues.apache.org/jira/browse/AVRO-2360, > > > > https://issues.apache.org/jira/browse/AVRO-2386, and > > > > https://issues.apache.org/jira/browse/AVRO-2383 which should all have > > > > been fixed. > > > > > > > > Looking at the avro-compiler.jar, all the class files are dated Apr 26 > > > > 9:32 EDT, which is before some of those issues were even reported. > > > > > > > > Regards, > > > > Raman > > > > > > > > On 2019/05/15 18:46:03, "Driesprong, Fokko" > > wrote: > > > > > Hi Jacob, > > > > > > > > > > This looks off, I'll dive into it. I think it got released for some > > > > reason. > > > > > While releasing it today, I also noticed that the artifact wasn't > > staged > > > > in > > > > > Nexus anymore. However, RC4 is the same as the release. > > > > > > > > > > Cheers, Fokko > > > > > > > > > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar > > > > > : > > > > > > > > > > > Great news! We have been looking forward to the 1.9 release. > > > > > > > > > > > > I see 1.9.0 published on Maven Central. It appears to have been > > > > published > > > > > > 5/8 > > > > > > (the same day as RC4 was announced but ~1week before vote passed). > > > > > > > > > > > > Is this the release build (presumably the same as RC4) or an > > earlier > > > > RC? > > > > > > > > > > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/ > > > > > > > > > > > > Would it be helpful to have maven include the git sha somewhere in > > > > > > MANIFEST.MF? > > > > > > > > > > > > jacob > > > > > > > > > > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Thank you all, > > > > > > > > > > > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed. > > > > > > > > > > > > > > Binding: > > > > > > > +1 Daniel Kulp > > > > > > > +1 Sean Busbey > > > > > > > +1 Fokko Driesprong > > > > > > > > > > > > > > Non-binding: > > > > > > > +1 Lunjie Jin > > > > > > > +1 Michael A. Smith > > > > > > > +1 Ismaël Mejía > > > > > > > +1 Brian Lachniet > > > > > > > > > > > > > > I would like to thank the community from participating in the > > > > release. > > > > > > Both > > > > > > > in providing patches for Apache Avro and giving the RC's a try. > > > > > > > > > > > > > > Yesterday I did some additional testing against Apache Spark > > > > (similar to > > > > > > > Nandor :-) and Apache Flink and did not find any blocking issues. > > > > There > > > > > > are > > > > > > > some incompatibilities, but most of them are; revoking Jackson > > from > > > > the > > > > > > > public API and deprecating Joda. > > > > > > > > > > > > > > I'll publish the artifacts to the different repositories soon. On > > > > this, I > > > > > > > consider myself a polyglot programmer, but my knowledge is > > limited on > > > > > > .Net, > > > > > > > Perl, and Javascript, so I might ask some help from the community > > > > here as > > > > > > > well. > > > > > > > > > > > > > > Thanks all, and keep up the good work. > > > > > > > > > > > > > > Cheers, Fokko Driesprong > > > > > > > > > > > > > > Op di 14 mei 2019 om 14:48 schreef Nandor Kollar > > > > > > > : > > > > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > > > > > * verified signatures and checksums > > > > > > > > * ran unit tests, passed > > > > > > > > * did a quick sanity test against Spark master: upgraded Spark > > Avro > > > > > > > > dependency to this release candidate, ran spark-avro tests, all > > > > passed > > > > > > > (for > > > > > > > > the record, two minor Maven changes are required due to > > removal of > > > > > > > Hadoop 1 > > > > > > > > support and making org.tukaani.xz provided dependency - looks > > like
