Hi all,

I've been doing a little bit of testing with Samza and Hadoop 3.3.4;
afaict, in light testing, Samza seems to work fine using the 2.10.x
hadoop-yarn library against a YARN cluster running 3.3.x. As Jamie pointed
out, YARN didn't incorporate Java 11 compatibility until v3.3.0 (
https://hadoop.apache.org/docs/r3.3.0/index.html). Are there any unit tests
in Samza that verify compatibility against a YARN cluster? If so, that
could be a place to validate YARN v2.10/v3.3 cross-compatibility.

Just throwing my 2 cents out there,
Malcolm McFarland
Cavulus

On Fri, Sep 2, 2022 at 6:27 PM James DeMichele
<james.demich...@redfin.com.invalid> wrote:

> Hey Yi,
>
> Thanks for getting back to me. I have not tried the older yarn cluster
> version yet in the Samza app running against 3.3.4 but I am wary it would
> work. Yarn itself is not compatible at 2.10.1 with Java 11 so you would
> have to update yarn even if the Java library here wasn't updated.
>
> Could we move this version I'm proposing to a 2.x version of Samza? So
> people that wanted to move forward with yarn upgrade and Samza and Java 11
> (like us) could do so? Then 1.x could only be java 8 compatible and 2.x
> could be java 11.
>
> Jamie
>
> On Fri, Sep 2, 2022, 6:44 PM Yi Pan <nickpa...@gmail.com> wrote:
>
> > Hey, James,
> >
> > Thanks for the ping. @prateek, can we have someone to review this change?
> >
> > One question: have you tested the change w/ the older YARN cluster
> version
> > (running 2.10.1)? If this change requires YARN cluster upgrade to 3.3.4
> as
> > well, that may be a breaking change to existing Samza users (i.e.
> LinkedIn
> > is still running a YARN cluster with version 2.10.1).
> >
> > Best and apologize for the delay.
> >
> > -Yi
> >
> > On Fri, Sep 2, 2022 at 8:56 AM James DeMichele
> > <james.demich...@redfin.com.invalid> wrote:
> >
> > > Hey y'all. I just am not sure how to get some traction on these Java 11
> > > PRs.
> > >
> > > https://github.com/apache/samza/pull/1628
> > > https://github.com/apache/samza-hello-samza/pull/87
> > >
> > > Would someone that is a maintainer for Samza just let us know that
> y'all
> > > are looking at them? I can stop pestering you :)
> > >
> > > I ran all tests in both PRs, all pass. I also confirmed that using my
> > Samza
> > > PR in the Hello World app all works with Java 11.
> > >
> > > Thanks!
> > >
> > > -Jamie
> > >
> >
>

Reply via email to