Re: Getting Avro 1.9 into downstream Apache projects

[Doug Cutting]

I've added you as an owner of both.  You should also now be able to add
others in the future as needed.

Thanks for all your work, making this release happen!

Doug

On Thu, May 16, 2019 at 1:00 PM Driesprong, Fokko 
wrote:

> Thanks Doug,
>
> My username for both PyPi and Rubygems is Fokko. Also, feel free to push
> yourself :-)
>
> Cheers, Fokko
>
> Op do 16 mei 2019 om 19:50 schreef Doug Cutting :
>
> > I'm happy to push these myself, or better yet, permit others.
> >
> > Please send me PyPi and rubygems.org usernames to add.
> >
> > Thanks,
> >
> > Doug
> >
> > On Thu, May 16, 2019 at 10:13 AM Daniel Kulp  wrote:
> >
> > >
> > >
> > > >
> > > > Also I'm blocked on verifying the python/ruby code due to the pip/gem
> > > > packages not yet having been pushed. Is there an ETA for those?
> > >
> > > The issue here is that none of the “currently active” Avro committers
> > have
> > > access to the repositories for those.   We’ve asked to have our
> accounts
> > > added to the Avro “groups” on those repositories, but that hasn’t
> > happened
> > > yet.   We need for some of the Avro old timers that do have access to
> > > either do the push or grant us access so we can do the push.   We’re
> > > working on it, just trying to get ahold of them.
> > >
> > > Dan
> > >
> > >
> > >
> > >
> > > >
> > > > Patrick
> > > >
> > > > On Thu, May 16, 2019 at 7:43 AM Ismaël Mejía 
> > wrote:
> > > >
> > > >> This is a fork of the discussion started in the release vote thread
> > [1].
> > > >>
> > > >> Just to keep the discussion going and follow up on issues and
> projects
> > > >> that deserve attention.
> > > >> So far open issues are:
> > > >>
> > > >> BEAM-7328 Update Avro to version 1.9.0 in Java SDK
> > > >> HIVE-21737 Upgrade Avro to version 1.9.0
> > > >> FLINK-12532 Upgrade Avro to version 1.9.0
> > > >> PARQUET-1576 Upgrade to Avro 1.9.0
> > > >> SPARK-27733 Upgrade to Avro 1.9.x
> > > >>
> > > >> Don't hesitate to create new issues if required and bring info in
> this
> > > >> thread that could be useful for others working on this.
> > > >>
> > > >> [1]
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/ab598c36fd823a3fc563f7c621f957f0c2ef09513d4e8428391ddb69@%3Cdev.avro.apache.org%3E
> > > >>
> > >
> > > --
> > > Daniel Kulp
> > > dk...@apache.org  - http://dankulp.com/blog <
> > > http://dankulp.com/blog>
> > > Talend Community Coder - http://talend.com 
> > >
> >
>

Re: Getting Avro 1.9 into downstream Apache projects

[Driesprong, Fokko]

Thanks Doug,

My username for both PyPi and Rubygems is Fokko. Also, feel free to push
yourself :-)

Cheers, Fokko

Op do 16 mei 2019 om 19:50 schreef Doug Cutting :

> I'm happy to push these myself, or better yet, permit others.
>
> Please send me PyPi and rubygems.org usernames to add.
>
> Thanks,
>
> Doug
>
> On Thu, May 16, 2019 at 10:13 AM Daniel Kulp  wrote:
>
> >
> >
> > >
> > > Also I'm blocked on verifying the python/ruby code due to the pip/gem
> > > packages not yet having been pushed. Is there an ETA for those?
> >
> > The issue here is that none of the “currently active” Avro committers
> have
> > access to the repositories for those.   We’ve asked to have our accounts
> > added to the Avro “groups” on those repositories, but that hasn’t
> happened
> > yet.   We need for some of the Avro old timers that do have access to
> > either do the push or grant us access so we can do the push.   We’re
> > working on it, just trying to get ahold of them.
> >
> > Dan
> >
> >
> >
> >
> > >
> > > Patrick
> > >
> > > On Thu, May 16, 2019 at 7:43 AM Ismaël Mejía 
> wrote:
> > >
> > >> This is a fork of the discussion started in the release vote thread
> [1].
> > >>
> > >> Just to keep the discussion going and follow up on issues and projects
> > >> that deserve attention.
> > >> So far open issues are:
> > >>
> > >> BEAM-7328 Update Avro to version 1.9.0 in Java SDK
> > >> HIVE-21737 Upgrade Avro to version 1.9.0
> > >> FLINK-12532 Upgrade Avro to version 1.9.0
> > >> PARQUET-1576 Upgrade to Avro 1.9.0
> > >> SPARK-27733 Upgrade to Avro 1.9.x
> > >>
> > >> Don't hesitate to create new issues if required and bring info in this
> > >> thread that could be useful for others working on this.
> > >>
> > >> [1]
> > >>
> >
> https://lists.apache.org/thread.html/ab598c36fd823a3fc563f7c621f957f0c2ef09513d4e8428391ddb69@%3Cdev.avro.apache.org%3E
> > >>
> >
> > --
> > Daniel Kulp
> > dk...@apache.org  - http://dankulp.com/blog <
> > http://dankulp.com/blog>
> > Talend Community Coder - http://talend.com 
> >
>

Re: Getting Avro 1.9 into downstream Apache projects

[Doug Cutting]

I'm happy to push these myself, or better yet, permit others.

Please send me PyPi and rubygems.org usernames to add.

Thanks,

Doug

On Thu, May 16, 2019 at 10:13 AM Daniel Kulp  wrote:

>
>
> >
> > Also I'm blocked on verifying the python/ruby code due to the pip/gem
> > packages not yet having been pushed. Is there an ETA for those?
>
> The issue here is that none of the “currently active” Avro committers have
> access to the repositories for those.   We’ve asked to have our accounts
> added to the Avro “groups” on those repositories, but that hasn’t happened
> yet.   We need for some of the Avro old timers that do have access to
> either do the push or grant us access so we can do the push.   We’re
> working on it, just trying to get ahold of them.
>
> Dan
>
>
>
>
> >
> > Patrick
> >
> > On Thu, May 16, 2019 at 7:43 AM Ismaël Mejía  wrote:
> >
> >> This is a fork of the discussion started in the release vote thread [1].
> >>
> >> Just to keep the discussion going and follow up on issues and projects
> >> that deserve attention.
> >> So far open issues are:
> >>
> >> BEAM-7328 Update Avro to version 1.9.0 in Java SDK
> >> HIVE-21737 Upgrade Avro to version 1.9.0
> >> FLINK-12532 Upgrade Avro to version 1.9.0
> >> PARQUET-1576 Upgrade to Avro 1.9.0
> >> SPARK-27733 Upgrade to Avro 1.9.x
> >>
> >> Don't hesitate to create new issues if required and bring info in this
> >> thread that could be useful for others working on this.
> >>
> >> [1]
> >>
> https://lists.apache.org/thread.html/ab598c36fd823a3fc563f7c621f957f0c2ef09513d4e8428391ddb69@%3Cdev.avro.apache.org%3E
> >>
>
> --
> Daniel Kulp
> dk...@apache.org  - http://dankulp.com/blog <
> http://dankulp.com/blog>
> Talend Community Coder - http://talend.com 
>

Re: Getting Avro 1.9 into downstream Apache projects

[Daniel Kulp]

> 
> Also I'm blocked on verifying the python/ruby code due to the pip/gem
> packages not yet having been pushed. Is there an ETA for those?

The issue here is that none of the “currently active” Avro committers have 
access to the repositories for those.   We’ve asked to have our accounts added 
to the Avro “groups” on those repositories, but that hasn’t happened yet.   We 
need for some of the Avro old timers that do have access to either do the push 
or grant us access so we can do the push.   We’re working on it, just trying to 
get ahold of them.

Dan




> 
> Patrick
> 
> On Thu, May 16, 2019 at 7:43 AM Ismaël Mejía  wrote:
> 
>> This is a fork of the discussion started in the release vote thread [1].
>> 
>> Just to keep the discussion going and follow up on issues and projects
>> that deserve attention.
>> So far open issues are:
>> 
>> BEAM-7328 Update Avro to version 1.9.0 in Java SDK
>> HIVE-21737 Upgrade Avro to version 1.9.0
>> FLINK-12532 Upgrade Avro to version 1.9.0
>> PARQUET-1576 Upgrade to Avro 1.9.0
>> SPARK-27733 Upgrade to Avro 1.9.x
>> 
>> Don't hesitate to create new issues if required and bring info in this
>> thread that could be useful for others working on this.
>> 
>> [1]
>> https://lists.apache.org/thread.html/ab598c36fd823a3fc563f7c621f957f0c2ef09513d4e8428391ddb69@%3Cdev.avro.apache.org%3E
>> 

-- 
Daniel Kulp
dk...@apache.org  - http://dankulp.com/blog 

Talend Community Coder - http://talend.com 

Re: Getting Avro 1.9 into downstream Apache projects

[Patrick Hunt]

Is there a migration guide? I tried upgrading my project
https://github.com/phunt/avro-rpc-quickstart
last night and ran into some issues wrt avro refactoring (ipc in my case)
that might be good to highlight in a migration guide.

Also I'm blocked on verifying the python/ruby code due to the pip/gem
packages not yet having been pushed. Is there an ETA for those?

Patrick

On Thu, May 16, 2019 at 7:43 AM Ismaël Mejía  wrote:

> This is a fork of the discussion started in the release vote thread [1].
>
> Just to keep the discussion going and follow up on issues and projects
> that deserve attention.
> So far open issues are:
>
> BEAM-7328 Update Avro to version 1.9.0 in Java SDK
> HIVE-21737 Upgrade Avro to version 1.9.0
> FLINK-12532 Upgrade Avro to version 1.9.0
> PARQUET-1576 Upgrade to Avro 1.9.0
> SPARK-27733 Upgrade to Avro 1.9.x
>
> Don't hesitate to create new issues if required and bring info in this
> thread that could be useful for others working on this.
>
> [1]
> https://lists.apache.org/thread.html/ab598c36fd823a3fc563f7c621f957f0c2ef09513d4e8428391ddb69@%3Cdev.avro.apache.org%3E
>

Getting Avro 1.9 into downstream Apache projects

[Ismaël Mejía]

This is a fork of the discussion started in the release vote thread [1].

Just to keep the discussion going and follow up on issues and projects
that deserve attention.
So far open issues are:

BEAM-7328 Update Avro to version 1.9.0 in Java SDK
HIVE-21737 Upgrade Avro to version 1.9.0
FLINK-12532 Upgrade Avro to version 1.9.0
PARQUET-1576 Upgrade to Avro 1.9.0
SPARK-27733 Upgrade to Avro 1.9.x

Don't hesitate to create new issues if required and bring info in this
thread that could be useful for others working on this.

[1] 
https://lists.apache.org/thread.html/ab598c36fd823a3fc563f7c621f957f0c2ef09513d4e8428391ddb69@%3Cdev.avro.apache.org%3E

Re: [VOTE] Release Apache Avro 1.9.0 RC4

[Ismaël Mejía]

Yes, sorry for polluting this thread, will fork it.

On Thu, May 16, 2019 at 1:45 PM Sean Busbey  wrote:
>
> The matter of upgrading downstream projects should get its own thread
> on dev@avro rather than occurring on this VOTE.
>
> On Thu, May 16, 2019 at 5:14 AM Ismaël Mejía  wrote:
> >
> > Argh this looks like problematic if Spark somehow wants to align the
> > deps, hope it won't be mandatory.
> >
> > I filled some more tickets, so the list of tickets so far:
> >
> > BEAM-7328 Update Avro to version 1.9.0 in Java SDK
> > HIVE-21737 Upgrade Avro to version 1.9.0
> > FLINK-12532 Upgrade Avro to version 1.9.0
> > PARQUET-1576 Upgrade to Avro 1.9.0
> > SPARK-27733 Upgrade to Avro 1.9.x
> >
> > On Thu, May 16, 2019 at 10:48 AM Driesprong, Fokko  
> > wrote:
> > >
> > > Thanks Nandor,
> > >
> > > Upgrading Hive 2.x (Hadoop 2) is going to be painful since this is still 
> > > on
> > > Java 1.7. It is still relying on Avro 1.7.7. This is an issue because this
> > > is also blocking upgrading Parquet, this implies that Hive 2.x does not
> > > support reading lz4 and zstd compression.
> > >
> > > From Hive 3.x the Java version is 1.8 and we could upgrade Avro. The
> > > current version is 1.8.2, so that should be doable.
> > >
> > > Cheers, Fokko
> > >
> > > Op wo 15 mei 2019 om 23:51 schreef Nandor Kollar
> > > :
> > >
> > > > If Spark is upgraded, then I'm afraid first Hive should be upgraded to
> > > > 1.9.0. When I did a quick release verification, I remember I saw several
> > > > failures due to Hive using deprecated and removed Avro methods (related 
> > > > to
> > > > Jackson removal from public API).
> > > >
> > > > On Wed, May 15, 2019 at 11:04 PM Ismaël Mejía  wrote:
> > > >
> > > > > Getting a new release of Avro feels amazing, thanks a lot Fokko for
> > > > > all your wokr to get this out with the release.
> > > > >
> > > > > An interesting next step is downstreaming. I saw Fokko created a
> > > > > ticket for Parquet, is anyone working on the Spark one? It will be
> > > > > really nice to get this into Spark 3. Anything I can do to help there
> > > > > or that we can sync work?
> > > > > I filled https://issues.apache.org/jira/browse/SPARK-27733 for the
> > > > moment.
> > > > >
> > > > > I am tackling the upgrade on Beam (the other project I work mostly on
> > > > too.
> > > > > https://issues.apache.org/jira/browse/BEAM-7328
> > > > >
> > > > > Any other project worth the sync/work?
> > > > >
> > > > >
> > > > > On Wed, May 15, 2019 at 8:54 PM Driesprong, Fokko 
> > > > > 
> > > > > wrote:
> > > > > >
> > > > > > Hi Jacob,
> > > > > >
> > > > > > This looks off, I'll dive into it. I think it got released for some
> > > > > reason.
> > > > > > While releasing it today, I also noticed that the artifact wasn't
> > > > staged
> > > > > in
> > > > > > Nexus anymore. However, RC4 is the same as the release.
> > > > > >
> > > > > > Cheers, Fokko
> > > > > >
> > > > > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar
> > > > > > :
> > > > > >
> > > > > > > Great news! We have been looking forward to the 1.9 release.
> > > > > > >
> > > > > > > I see 1.9.0 published on Maven Central. It appears to have been
> > > > > published
> > > > > > > 5/8
> > > > > > > (the same day as RC4 was announced but ~1week before vote passed).
> > > > > > >
> > > > > > > Is this the release build (presumably the same as RC4) or an 
> > > > > > > earlier
> > > > > RC?
> > > > > > >
> > > > > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/
> > > > > > >
> > > > > > > Would it be helpful to have maven include the git sha somewhere in
> > > > > > > MANIFEST.MF?
> > > > > > >
> > > > > > > jacob
> > > > > > >
> > > > > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko
> > > >  > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thank you all,
> > > > > > > >
> > > > > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed.
> > > > > > > >
> > > > > > > > Binding:
> > > > > > > > +1 Daniel Kulp
> > > > > > > > +1 Sean Busbey
> > > > > > > > +1 Fokko Driesprong
> > > > > > > >
> > > > > > > > Non-binding:
> > > > > > > > +1 Lunjie Jin
> > > > > > > > +1 Michael A. Smith
> > > > > > > > +1 Ismaël Mejía
> > > > > > > > +1 Brian Lachniet
> > > > > > > >
> > > > > > > > I would like to thank the community from participating in the
> > > > > release.
> > > > > > > Both
> > > > > > > > in providing patches for Apache Avro and giving the RC's a try.
> > > > > > > >
> > > > > > > > Yesterday I did some additional testing against Apache Spark
> > > > > (similar to
> > > > > > > > Nandor :-) and Apache Flink and did not find any blocking 
> > > > > > > > issues.
> > > > > There
> > > > > > > are
> > > > > > > > some incompatibilities, but most of them are; revoking Jackson 
> > > > > > > > from
> > > > > the
> > > > > > > > public API and deprecating Joda.
> > > > > > > >
> > > > > > > > I'll publish the artifacts to the different repositories soon. 
> > > > > > > > On
> > > > > this, I
> > > > > > > > 

Re: [VOTE] Release Apache Avro 1.9.0 RC4

[Sean Busbey]

The matter of upgrading downstream projects should get its own thread
on dev@avro rather than occurring on this VOTE.

On Thu, May 16, 2019 at 5:14 AM Ismaël Mejía  wrote:
>
> Argh this looks like problematic if Spark somehow wants to align the
> deps, hope it won't be mandatory.
>
> I filled some more tickets, so the list of tickets so far:
>
> BEAM-7328 Update Avro to version 1.9.0 in Java SDK
> HIVE-21737 Upgrade Avro to version 1.9.0
> FLINK-12532 Upgrade Avro to version 1.9.0
> PARQUET-1576 Upgrade to Avro 1.9.0
> SPARK-27733 Upgrade to Avro 1.9.x
>
> On Thu, May 16, 2019 at 10:48 AM Driesprong, Fokko  
> wrote:
> >
> > Thanks Nandor,
> >
> > Upgrading Hive 2.x (Hadoop 2) is going to be painful since this is still on
> > Java 1.7. It is still relying on Avro 1.7.7. This is an issue because this
> > is also blocking upgrading Parquet, this implies that Hive 2.x does not
> > support reading lz4 and zstd compression.
> >
> > From Hive 3.x the Java version is 1.8 and we could upgrade Avro. The
> > current version is 1.8.2, so that should be doable.
> >
> > Cheers, Fokko
> >
> > Op wo 15 mei 2019 om 23:51 schreef Nandor Kollar
> > :
> >
> > > If Spark is upgraded, then I'm afraid first Hive should be upgraded to
> > > 1.9.0. When I did a quick release verification, I remember I saw several
> > > failures due to Hive using deprecated and removed Avro methods (related to
> > > Jackson removal from public API).
> > >
> > > On Wed, May 15, 2019 at 11:04 PM Ismaël Mejía  wrote:
> > >
> > > > Getting a new release of Avro feels amazing, thanks a lot Fokko for
> > > > all your wokr to get this out with the release.
> > > >
> > > > An interesting next step is downstreaming. I saw Fokko created a
> > > > ticket for Parquet, is anyone working on the Spark one? It will be
> > > > really nice to get this into Spark 3. Anything I can do to help there
> > > > or that we can sync work?
> > > > I filled https://issues.apache.org/jira/browse/SPARK-27733 for the
> > > moment.
> > > >
> > > > I am tackling the upgrade on Beam (the other project I work mostly on
> > > too.
> > > > https://issues.apache.org/jira/browse/BEAM-7328
> > > >
> > > > Any other project worth the sync/work?
> > > >
> > > >
> > > > On Wed, May 15, 2019 at 8:54 PM Driesprong, Fokko 
> > > > wrote:
> > > > >
> > > > > Hi Jacob,
> > > > >
> > > > > This looks off, I'll dive into it. I think it got released for some
> > > > reason.
> > > > > While releasing it today, I also noticed that the artifact wasn't
> > > staged
> > > > in
> > > > > Nexus anymore. However, RC4 is the same as the release.
> > > > >
> > > > > Cheers, Fokko
> > > > >
> > > > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar
> > > > > :
> > > > >
> > > > > > Great news! We have been looking forward to the 1.9 release.
> > > > > >
> > > > > > I see 1.9.0 published on Maven Central. It appears to have been
> > > > published
> > > > > > 5/8
> > > > > > (the same day as RC4 was announced but ~1week before vote passed).
> > > > > >
> > > > > > Is this the release build (presumably the same as RC4) or an earlier
> > > > RC?
> > > > > >
> > > > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/
> > > > > >
> > > > > > Would it be helpful to have maven include the git sha somewhere in
> > > > > > MANIFEST.MF?
> > > > > >
> > > > > > jacob
> > > > > >
> > > > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko
> > >  > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thank you all,
> > > > > > >
> > > > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed.
> > > > > > >
> > > > > > > Binding:
> > > > > > > +1 Daniel Kulp
> > > > > > > +1 Sean Busbey
> > > > > > > +1 Fokko Driesprong
> > > > > > >
> > > > > > > Non-binding:
> > > > > > > +1 Lunjie Jin
> > > > > > > +1 Michael A. Smith
> > > > > > > +1 Ismaël Mejía
> > > > > > > +1 Brian Lachniet
> > > > > > >
> > > > > > > I would like to thank the community from participating in the
> > > > release.
> > > > > > Both
> > > > > > > in providing patches for Apache Avro and giving the RC's a try.
> > > > > > >
> > > > > > > Yesterday I did some additional testing against Apache Spark
> > > > (similar to
> > > > > > > Nandor :-) and Apache Flink and did not find any blocking issues.
> > > > There
> > > > > > are
> > > > > > > some incompatibilities, but most of them are; revoking Jackson 
> > > > > > > from
> > > > the
> > > > > > > public API and deprecating Joda.
> > > > > > >
> > > > > > > I'll publish the artifacts to the different repositories soon. On
> > > > this, I
> > > > > > > consider myself a polyglot programmer, but my knowledge is limited
> > > on
> > > > > > .Net,
> > > > > > > Perl, and Javascript, so I might ask some help from the community
> > > > here as
> > > > > > > well.
> > > > > > >
> > > > > > > Thanks all, and keep up the good work.
> > > > > > >
> > > > > > > Cheers, Fokko Driesprong
> > > > > > >
> > > > > > > Op di 14 mei 2019 om 14:48 schreef Nandor Kollar
> > > > > > > :
> > > > > > >
> > > > 

Re: [VOTE] Release Apache Avro 1.9.0 RC4

[Ismaël Mejía]

Argh this looks like problematic if Spark somehow wants to align the
deps, hope it won't be mandatory.

I filled some more tickets, so the list of tickets so far:

BEAM-7328 Update Avro to version 1.9.0 in Java SDK
HIVE-21737 Upgrade Avro to version 1.9.0
FLINK-12532 Upgrade Avro to version 1.9.0
PARQUET-1576 Upgrade to Avro 1.9.0
SPARK-27733 Upgrade to Avro 1.9.x

On Thu, May 16, 2019 at 10:48 AM Driesprong, Fokko  wrote:
>
> Thanks Nandor,
>
> Upgrading Hive 2.x (Hadoop 2) is going to be painful since this is still on
> Java 1.7. It is still relying on Avro 1.7.7. This is an issue because this
> is also blocking upgrading Parquet, this implies that Hive 2.x does not
> support reading lz4 and zstd compression.
>
> From Hive 3.x the Java version is 1.8 and we could upgrade Avro. The
> current version is 1.8.2, so that should be doable.
>
> Cheers, Fokko
>
> Op wo 15 mei 2019 om 23:51 schreef Nandor Kollar
> :
>
> > If Spark is upgraded, then I'm afraid first Hive should be upgraded to
> > 1.9.0. When I did a quick release verification, I remember I saw several
> > failures due to Hive using deprecated and removed Avro methods (related to
> > Jackson removal from public API).
> >
> > On Wed, May 15, 2019 at 11:04 PM Ismaël Mejía  wrote:
> >
> > > Getting a new release of Avro feels amazing, thanks a lot Fokko for
> > > all your wokr to get this out with the release.
> > >
> > > An interesting next step is downstreaming. I saw Fokko created a
> > > ticket for Parquet, is anyone working on the Spark one? It will be
> > > really nice to get this into Spark 3. Anything I can do to help there
> > > or that we can sync work?
> > > I filled https://issues.apache.org/jira/browse/SPARK-27733 for the
> > moment.
> > >
> > > I am tackling the upgrade on Beam (the other project I work mostly on
> > too.
> > > https://issues.apache.org/jira/browse/BEAM-7328
> > >
> > > Any other project worth the sync/work?
> > >
> > >
> > > On Wed, May 15, 2019 at 8:54 PM Driesprong, Fokko 
> > > wrote:
> > > >
> > > > Hi Jacob,
> > > >
> > > > This looks off, I'll dive into it. I think it got released for some
> > > reason.
> > > > While releasing it today, I also noticed that the artifact wasn't
> > staged
> > > in
> > > > Nexus anymore. However, RC4 is the same as the release.
> > > >
> > > > Cheers, Fokko
> > > >
> > > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar
> > > > :
> > > >
> > > > > Great news! We have been looking forward to the 1.9 release.
> > > > >
> > > > > I see 1.9.0 published on Maven Central. It appears to have been
> > > published
> > > > > 5/8
> > > > > (the same day as RC4 was announced but ~1week before vote passed).
> > > > >
> > > > > Is this the release build (presumably the same as RC4) or an earlier
> > > RC?
> > > > >
> > > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/
> > > > >
> > > > > Would it be helpful to have maven include the git sha somewhere in
> > > > > MANIFEST.MF?
> > > > >
> > > > > jacob
> > > > >
> > > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko
> >  > > >
> > > > > wrote:
> > > > >
> > > > > > Thank you all,
> > > > > >
> > > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed.
> > > > > >
> > > > > > Binding:
> > > > > > +1 Daniel Kulp
> > > > > > +1 Sean Busbey
> > > > > > +1 Fokko Driesprong
> > > > > >
> > > > > > Non-binding:
> > > > > > +1 Lunjie Jin
> > > > > > +1 Michael A. Smith
> > > > > > +1 Ismaël Mejía
> > > > > > +1 Brian Lachniet
> > > > > >
> > > > > > I would like to thank the community from participating in the
> > > release.
> > > > > Both
> > > > > > in providing patches for Apache Avro and giving the RC's a try.
> > > > > >
> > > > > > Yesterday I did some additional testing against Apache Spark
> > > (similar to
> > > > > > Nandor :-) and Apache Flink and did not find any blocking issues.
> > > There
> > > > > are
> > > > > > some incompatibilities, but most of them are; revoking Jackson from
> > > the
> > > > > > public API and deprecating Joda.
> > > > > >
> > > > > > I'll publish the artifacts to the different repositories soon. On
> > > this, I
> > > > > > consider myself a polyglot programmer, but my knowledge is limited
> > on
> > > > > .Net,
> > > > > > Perl, and Javascript, so I might ask some help from the community
> > > here as
> > > > > > well.
> > > > > >
> > > > > > Thanks all, and keep up the good work.
> > > > > >
> > > > > > Cheers, Fokko Driesprong
> > > > > >
> > > > > > Op di 14 mei 2019 om 14:48 schreef Nandor Kollar
> > > > > > :
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > * verified signatures and checksums
> > > > > > > * ran unit tests, passed
> > > > > > > * did a quick sanity test against Spark master: upgraded Spark
> > Avro
> > > > > > > dependency to this release candidate, ran spark-avro tests, all
> > > passed
> > > > > > (for
> > > > > > > the record, two minor Maven changes are required due to removal
> > of
> > > > > > Hadoop 1
> > > > > > > support and making 

Re: [VOTE] Release Apache Avro 1.9.0 RC4

[Driesprong, Fokko]

Thanks Nandor,

Upgrading Hive 2.x (Hadoop 2) is going to be painful since this is still on
Java 1.7. It is still relying on Avro 1.7.7. This is an issue because this
is also blocking upgrading Parquet, this implies that Hive 2.x does not
support reading lz4 and zstd compression.

>From Hive 3.x the Java version is 1.8 and we could upgrade Avro. The
current version is 1.8.2, so that should be doable.

Cheers, Fokko

Op wo 15 mei 2019 om 23:51 schreef Nandor Kollar
:

> If Spark is upgraded, then I'm afraid first Hive should be upgraded to
> 1.9.0. When I did a quick release verification, I remember I saw several
> failures due to Hive using deprecated and removed Avro methods (related to
> Jackson removal from public API).
>
> On Wed, May 15, 2019 at 11:04 PM Ismaël Mejía  wrote:
>
> > Getting a new release of Avro feels amazing, thanks a lot Fokko for
> > all your wokr to get this out with the release.
> >
> > An interesting next step is downstreaming. I saw Fokko created a
> > ticket for Parquet, is anyone working on the Spark one? It will be
> > really nice to get this into Spark 3. Anything I can do to help there
> > or that we can sync work?
> > I filled https://issues.apache.org/jira/browse/SPARK-27733 for the
> moment.
> >
> > I am tackling the upgrade on Beam (the other project I work mostly on
> too.
> > https://issues.apache.org/jira/browse/BEAM-7328
> >
> > Any other project worth the sync/work?
> >
> >
> > On Wed, May 15, 2019 at 8:54 PM Driesprong, Fokko 
> > wrote:
> > >
> > > Hi Jacob,
> > >
> > > This looks off, I'll dive into it. I think it got released for some
> > reason.
> > > While releasing it today, I also noticed that the artifact wasn't
> staged
> > in
> > > Nexus anymore. However, RC4 is the same as the release.
> > >
> > > Cheers, Fokko
> > >
> > > Op wo 15 mei 2019 om 20:27 schreef Jacob Tolar
> > > :
> > >
> > > > Great news! We have been looking forward to the 1.9 release.
> > > >
> > > > I see 1.9.0 published on Maven Central. It appears to have been
> > published
> > > > 5/8
> > > > (the same day as RC4 was announced but ~1week before vote passed).
> > > >
> > > > Is this the release build (presumably the same as RC4) or an earlier
> > RC?
> > > >
> > > > https://repo1.maven.org/maven2/org/apache/avro/avro/1.9.0/
> > > >
> > > > Would it be helpful to have maven include the git sha somewhere in
> > > > MANIFEST.MF?
> > > >
> > > > jacob
> > > >
> > > > On Tue, May 14, 2019 at 1:04 PM Driesprong, Fokko
>  > >
> > > > wrote:
> > > >
> > > > > Thank you all,
> > > > >
> > > > > I'm happy to announce, the vote for Avro 1.9.0 has passed.
> > > > >
> > > > > Binding:
> > > > > +1 Daniel Kulp
> > > > > +1 Sean Busbey
> > > > > +1 Fokko Driesprong
> > > > >
> > > > > Non-binding:
> > > > > +1 Lunjie Jin
> > > > > +1 Michael A. Smith
> > > > > +1 Ismaël Mejía
> > > > > +1 Brian Lachniet
> > > > >
> > > > > I would like to thank the community from participating in the
> > release.
> > > > Both
> > > > > in providing patches for Apache Avro and giving the RC's a try.
> > > > >
> > > > > Yesterday I did some additional testing against Apache Spark
> > (similar to
> > > > > Nandor :-) and Apache Flink and did not find any blocking issues.
> > There
> > > > are
> > > > > some incompatibilities, but most of them are; revoking Jackson from
> > the
> > > > > public API and deprecating Joda.
> > > > >
> > > > > I'll publish the artifacts to the different repositories soon. On
> > this, I
> > > > > consider myself a polyglot programmer, but my knowledge is limited
> on
> > > > .Net,
> > > > > Perl, and Javascript, so I might ask some help from the community
> > here as
> > > > > well.
> > > > >
> > > > > Thanks all, and keep up the good work.
> > > > >
> > > > > Cheers, Fokko Driesprong
> > > > >
> > > > > Op di 14 mei 2019 om 14:48 schreef Nandor Kollar
> > > > > :
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > * verified signatures and checksums
> > > > > > * ran unit tests, passed
> > > > > > * did a quick sanity test against Spark master: upgraded Spark
> Avro
> > > > > > dependency to this release candidate, ran spark-avro tests, all
> > passed
> > > > > (for
> > > > > > the record, two minor Maven changes are required due to removal
> of
> > > > > Hadoop 1
> > > > > > support and making org.tukaani.xz provided dependency - looks
> like
> > > > Spark
> > > > > > tests against each codec). spark-hive tests failed, because
> > deprecated
> > > > > > methods were removed in Avro (related to Jackson removal from
> > public
> > > > API,
> > > > > > and Hive uses them), but I think it is expected, upgrading to a
> > major
> > > > > > release could have breaking changes.
> > > > > >
> > > > > > Nandor
> > > > > >
> > > > > > On Sun, May 12, 2019 at 7:11 AM Sean Busbey 
> > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > -- Good:
> > > > > > > * signatures
> > > > > > > * checksums
> > > > > > > * tag/ref lines up with src artifact (except for one file
>