Good point, fair enough.
Regards
JB
On 06/29/2017 10:26 AM, Aljoscha Krettek wrote:
I think it’s a bug because if you start a Flink cluster on bare-metal it works,
just when it’s started in YARN it doesn’t. And I feel that the way you start
your cluster should not affect how you can submit jobs to it.
Best,
Aljoscha
On 29. Jun 2017, at 10:15, Jean-Baptiste Onofré <[email protected]> wrote:
Yes, it's the same with the spark runner using bin/spark-submit. From my
standpoint, it's not a bug, it's a feature request.
Regards
JB
On 06/29/2017 10:12 AM, Aljoscha Krettek wrote:
I also responded to a separate mail by Will. The problem is that currently we
cannot submit a job using the remote client to a Flink cluster that was started
on YARN. (It’s a bug or “feature” of how communication with a Flink cluster
from a client works.)
The workaround for that is to use the bin/flink command to submit a Beam
fat-jar on a Flink YARN cluster.
Best,
Aljoscha
On 29. Jun 2017, at 07:23, Jean-Baptiste Onofré <[email protected]> wrote:
Hi Will,
assuming you are using Beam 2.0.0, the Flink runner uses Flink 1.2.1 by
default. So, I would recommend this version or 1.2.x.
Regards
JB
On 06/28/2017 10:39 PM, Will Walters wrote:
Hello,
I've been attempting to run Beam through Flink on a Yarn cluster and have run
into trouble with getting a job to submit, partly because of incompatibility
between versions. Does anyone know what versions of Beam and Flink I should be
using to give myself the best chance of finding compatibility?
Thank you,
Will.
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
