[
https://issues.apache.org/jira/browse/SPARK-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760071#comment-16760071
]
Shixiong Zhu commented on SPARK-26824:
--------------------------------------
This will need a release note. After the fix, the paths to store metadata will
be changed. The user needs to move the metadata files which are in a wrong
place to the right location manually. Otherwise, their query will not pick up
the old metadata.
> Streaming queries may store checkpoint data in a wrong directory
> ----------------------------------------------------------------
>
> Key: SPARK-26824
> URL: https://issues.apache.org/jira/browse/SPARK-26824
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.4.0
> Reporter: Shixiong Zhu
> Assignee: Shixiong Zhu
> Priority: Major
> Labels: release-notes
>
> When a user specifies a checkpoint location containing special chars that
> need to be escaped in a path, the streaming query will store checkpoint in a
> wrong place. For example, if you use "/chk chk", the metadata will be stored
> in "/chk%20chk". File sink's "_spark_metadata" directory has the same issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
