ramkrishna.s.vasudevan created FLINK-30128:
----------------------------------------------
Summary: Introduce Azure Data Lake Gen2 APIs in the Hadoop
Recoverable path
Key: FLINK-30128
URL: https://issues.apache.org/jira/browse/FLINK-30128
Project: Flink
Issue Type: Sub-task
Affects Versions: 1.13.1
Reporter: ramkrishna.s.vasudevan
Currently the HadoopRecoverableWriter assumes that the underlying FS is Hadoop
and so it checks for DistributedFileSystem. It also tries to do a truncate and
ensure the lease is recovered before the 'rename' operation is done.
In the Azure Data lake gen 2 world, the driver does not support truncate and
lease recovery API. We should be able to get the last committed size and if it
matches go for the rename. Will be back with more details here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
