Tathagata Das created SPARK-18416:
-------------------------------------
Summary: State Store leaks temporary files
Key: SPARK-18416
URL: https://issues.apache.org/jira/browse/SPARK-18416
Project: Spark
Issue Type: Sub-task
Components: Structured Streaming
Affects Versions: 2.0.1, 2.0.0, 2.0.2
Reporter: Tathagata Das
Assignee: Tathagata Das
StateStore.get() causes temporary files to be created immediately, even if the
store is not used to make updates for new version. The temp file is not closed
as store.commit() is not called in those cases, thus keeping the output stream
to temp file open forever.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
