Thanks for figuring this out, Shuyi!
> On 9. Oct 2018, at 09:09, Shuyi Chen wrote:
>
> I think the bug is introduced in FLINK-7643 (Rework FileSystem loading to use
> factories). In YarnApplicationMasterRunner, after the JIRA, FileSystem was
> not properly initialized with the correct flink co
I think the bug is introduced in FLINK-7643 (Rework FileSystem loading to
use factories). In YarnApplicationMasterRunner, after the JIRA, FileSystem
was not properly initialized with the correct flink configuration before
calling runApplicationMaster(). W/o the initialization, a call of
FileSystem.
Hi Aljoscha,
Thanks for looking into this! Yes, we toggled Flink 1.4 back to Flink 1.3.2
and it works. So seems truly a feature disparity between 1.3.2 and 1.4.
Best,
Yan
On Thu, Oct 4, 2018 at 6:36 AM Aljoscha Krettek wrote:
> Another thing: when you retry this again with Flink 1.3.2 it work
Another thing: when you retry this again with Flink 1.3.2 it works? I'm trying
to rule out another problem in the setup.
> On 4. Oct 2018, at 15:17, Aljoscha Krettek wrote:
>
> Hi Yan,
>
> This seems to be a bug in the FileSystems and how they're initialized. I'm
> looking into this myself bu
Hi Yan,
This seems to be a bug in the FileSystems and how they're initialized. I'm
looking into this myself but I'm also looping in Stephan and Stefan how have
worked on this the most in the past. Maybe they have some valuable input.
Best,
Aljoscha
> On 4. Oct 2018, at 01:18, Yan Yan wrote:
Hi,
We recently bumped to Flink 1.4 from 1.3.2, and found out an issue on HDFS
configuration.
We are using *FlinkYarnSessionCli* to start the cluster and submit job.
In 1.3.2, we set below Flink properties when using checkpoints:
state.backend.fs.checkpointdir = hdfs://nameservice0/.../..
state.
