Hi, just thought of reaching out once again and seeking out your kind help to find out what is the best way to stop SPARK streaming gracefully. Do we still use the methods of creating a file as in SPARK 2.4.x which is several years old method or do we have a better approach in SPARK 3.1?
Regards, Gourav Sengupta ---------- Forwarded message --------- From: Gourav Sengupta <gourav.sengupta.develo...@gmail.com> Date: Wed, Apr 21, 2021 at 10:06 AM Subject: Graceful shutdown SPARK Structured Streaming To: <user@spark.apache.org> Dear friends, is there any documentation available for gracefully stopping SPARK Structured Streaming in 3.1.x? I am referring to articles which are 4 to 5 years old and was wondering whether there is a better way available today to gracefully shutdown a SPARK streaming job. Thanks a ton in advance for all your kind help. Regards, Gourav Sengupta
