Anish Shrigondekar created SPARK-42676: ------------------------------------------
Summary: Write temp checkpoints for streaming queries to local filesystem even if default FS is set differently Key: SPARK-42676 URL: https://issues.apache.org/jira/browse/SPARK-42676 Project: Spark Issue Type: Task Components: Structured Streaming Affects Versions: 3.4.0 Reporter: Anish Shrigondekar Write temp checkpoints for streaming queries to local filesystem even if default FS is set differently We have seen cases where the default FS could be a remote file system and since the path for streaming checkpoints is not specified explcitily, this could cause pileup under 2 cases: * query exits with exception and the flag to force checkpoint removal is not set * driver/cluster terminates without query being terminated gracefully -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org