Sunil Kumar created SPARK-18153:
-----------------------------------
Summary: CLONE - Ability to remove old metadata for structure
streaming MetadataLog
Key: SPARK-18153
URL: https://issues.apache.org/jira/browse/SPARK-18153
Project: Spark
Issue Type: Sub-task
Components: SQL, Streaming
Reporter: Sunil Kumar
Assignee: Saisai Shao
Fix For: 2.0.1, 2.1.0
Current MetadataLog lacks the ability to remove old checkpoint file, we'd
better add this functionality to the MetadataLog and honor it in the place
where MetadataLog is used, that will reduce unnecessary small files in the long
running scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
