vanzin commented on a change in pull request #22952: [SPARK-20568][SS] Provide
option to clean up completed files in streaming query
URL: https://github.com/apache/spark/pull/22952#discussion_r339709020
##########
File path: docs/structured-streaming-programming-guide.md
##########
@@ -546,6 +546,13 @@ Here are the details of all the sources in Spark.
"s3://a/dataset.txt"<br/>
"s3n://a/b/dataset.txt"<br/>
"s3a://a/b/c/dataset.txt"<br/>
+ <code>cleanSource</code>: option to clean up completed files after
processing.<br/>
+ Available options are "archive", "delete", "off". If the option is not
provided, the default value is "off".<br/>
+ When "archive" is provided, additional option
<code>sourceArchiveDir</code> must be provided as well. The value of
"sourceArchiveDir" must have 2 subdirectories (so depth of directory is greater
than 2). e.g. /archived/here This will ensure archived files are never included
as new source files.<br/>
Review comment:
`<code></code>` around directory path, period before `This`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
