Hi Marco, as Klou said, -m yarn-cluster should try to deploy a Yarn per job cluster on your Yarn cluster. Could you maybe share a bit more details about what is going wrong? E.g. the cli logs could be helpful to pinpoint the problem.
I've tested that both `bin/flink run -m yarn-cluster examples/streaming/WindowJoin.jar` as well as `bin/flink run -t yarn-per-job examples/streamingWindowJoin.jar` start a Flink per job cluster. What was -yna supposed to do? -ynm should set the custom name of the Yarn application. @kkloudas <kklou...@apache.org> should we maybe improve the existing documentation to better reflect the usage of -t/--target? The CLI documentation [1] does not include a single example where we use the target option. Moreover, we could think about retiring -m yarn-cluster in favour of -t yarn-per-job. Moreover, should we somewhere document which `execution.target` are all supported? What do you think? [1] https://ci.apache.org/projects/flink/flink-docs-stable/ops/cli.html#job-submission-examples Cheers, Till On Tue, Nov 10, 2020 at 4:00 PM Kostas Kloudas <kklou...@gmail.com> wrote: > Hi Marco, > > I agree with you that the -m help message is misleading but I do not > think it has changed between releases. > You can specify the address of the jobmanager or, for example, you can > put "-m yarn-cluster" and depending on your environment setup Flink > will pick up a session cluster or will create a per-job cluster. > This was always the case. > > For the -t and -e the change is that -e was deprecated (although still > active) in favour of -t. But it still has the same meaning. > > Finally on how to run Flink on EMR, I am not an expert so I will pull > in Till who may have some input. > > Cheers, > Kostas > > On Mon, Nov 9, 2020 at 10:46 PM Marco Villalobos > <mvillalo...@kineteque.com> wrote: > > > > The flink CLI documentation says that the -m option is to specify the > job manager. > > > > but the examples are passing in an execution target. I am quite > confused by this. > > > > ./bin/flink run -m yarn-cluster \ > > ./examples/batch/WordCount.jar \ > > --input hdfs:///user/hamlet.txt --output > hdfs:///user/wordcount_out > > > > > > So what is it? > > > > I am trying to run Flink in EMR 6.1.0 but I have failed. > > > > It appears as though some of the command line parameters changed from > version 1.10 to 1.11. > > > > For example, -yna is now -ynm. > > > > -e is now -t. > > > > But I am still confused by the -m option in both documentation. > > > > Can somebody please explain? > > >
