I also responded to a separate mail by Will. The problem is that currently we cannot submit a job using the remote client to a Flink cluster that was started on YARN. (It’s a bug or “feature” of how communication with a Flink cluster from a client works.)
The workaround for that is to use the bin/flink command to submit a Beam fat-jar on a Flink YARN cluster. Best, Aljoscha > On 29. Jun 2017, at 07:23, Jean-Baptiste Onofré <[email protected]> wrote: > > Hi Will, > > assuming you are using Beam 2.0.0, the Flink runner uses Flink 1.2.1 by > default. So, I would recommend this version or 1.2.x. > > Regards > JB > > On 06/28/2017 10:39 PM, Will Walters wrote: >> Hello, >> I've been attempting to run Beam through Flink on a Yarn cluster and have >> run into trouble with getting a job to submit, partly because of >> incompatibility between versions. Does anyone know what versions of Beam and >> Flink I should be using to give myself the best chance of finding >> compatibility? >> Thank you, >> Will. > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com
