Hi Pankaj,
Always using `-yid` to submit a flink job to an existing yarn session
cluster is a safe way. For example, `flink run -d -yid application_1234
examples/streaming/WordCount.jar`. Maybe the magic properties file will be
removed in the future.
Best,
Yang
Pankaj Chand <pankajchanda...@gmail.com> 于2019年12月13日周五 下午1:16写道：
> Vino and Kostas:
>
> Thank you for the info!
>
> I was using Flink 1.9.1 with Pre-bundled Hadoop 2.7.5.
>
> Cloudlab has quarantined my cluster experiment without notice 😕, so I'll
> let you know if and when they allow me to access the files in the future.
>
> regards,
>
> Pankaj
>
> On Thu, Dec 12, 2019 at 8:35 AM Kostas Kloudas <kklou...@gmail.com> wrote:
>
>> Hi Pankaj,
>>
>> When you start a session cluster with the bin/yarn-session.sh script,
>> Flink will create the cluster and then write a "Yarn Properties file"
>> named ".yarn-properties-YOUR_USER_NAME" in the directory:
>> either the one specified by the option "yarn.properties-file.location"
>> in the flink-conf.yaml or in your local
>> System.getProperty("java.io.tmpdir"). This file will contain the
>> applicationId of the cluster and
>> it will be picked up by any future calls to `flink run`. Could you
>> check if this file exists and if it is updated every time you create a
>> cluster?
>>
>> Thanks,
>> Kostas
>>
>> On Thu, Dec 12, 2019 at 2:22 PM vino yang <yanghua1...@gmail.com> wrote:
>> >
>> > Hi Pankaj,
>> >
>> > Can you tell us what's Flink version do you use? And can you share the
>> Flink client and job manager log with us?
>> >
>> > This information would help us to locate your problem.
>> >
>> > Best,
>> > Vino
>> >
>> > Pankaj Chand <pankajchanda...@gmail.com> 于2019年12月12日周四 下午7:08写道：
>> >>
>> >> Hello,
>> >>
>> >> When using Flink on YARN in session mode, each Flink job client would
>> automatically know the YARN cluster to connect to. It says this somewhere
>> in the documentation.
>> >>
>> >> So, I killed the Flink session cluster by simply killing the YARN
>> application using the "yarn kill" command. However, when starting a new
>> Flink session cluster and trying to submit Flink jobs to yarn-session,
>> Flink complains that the old cluster (it gave the port number and YARN
>> application ID) is not available.
>> >>
>> >> It seems like the details of the old cluster were still stored
>> somewhere in Flink. So, I had to completely replace the Flink folder with a
>> new one.
>> >>
>> >> Does anyone know the proper way to kill a Flink+YARN session cluster
>> to completely remove it so that jobs will get submitted to a new Flink
>> session cluster?
>> >>
>> >> Thanks,
>> >>
>> >> Pankaj
>>
>
