Re: Understanding checkpoint/savepoint storage requirements

2024-04-02 Thread Robert Young
Thank you both for the information! Rob On Thu, Mar 28, 2024 at 7:08 PM Asimansu Bera wrote: > To add more details to it so that it will be clear why access to > persistent object stores for all JVM processes are required for a job graph > of Flink for consistent recovery. > *JoB Manager:* > >

Re: Understanding checkpoint/savepoint storage requirements

2024-03-28 Thread Asimansu Bera
To add more details to it so that it will be clear why access to persistent object stores for all JVM processes are required for a job graph of Flink for consistent recovery. *JoB Manager:* Flink's JobManager writes critical metadata during checkpoints for fault tolerance: - Job

Understanding checkpoint/savepoint storage requirements

2024-03-27 Thread Robert Young
Hi all, I have some questions about checkpoint and savepoint storage. >From what I understand a distributed, production-quality job with a lot of state should use durable shared storage for checkpoints and savepoints. All job managers and task managers should access the same volume. So typically