This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.0 by this push:
     new c5683fc  [MINOR][SS][DOCS] fileNameOnly parameter description re-unite
c5683fc is described below

commit c5683fce6f43dd561677830d74af196bc6c97134
Author: Gabor Somogyi <gabor.g.somo...@gmail.com>
AuthorDate: Sat Jun 6 16:49:48 2020 +0900

    [MINOR][SS][DOCS] fileNameOnly parameter description re-unite
    
    ### What changes were proposed in this pull request?
    `fileNameOnly` parameter is split to 2 pieces in 
[this](https://github.com/apache/spark/commit/dbb8143501ab87865d6e202c17297b9a73a0b1c3)
 commit. This PR re-unites it.
    
    ### Why are the changes needed?
    Parameter description split in doc.
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    ```
    cd docs/
    SKIP_API=1 jekyll build
    ```
    Manual webpage check.
    
    Closes #28739 from gaborgsomogyi/datasettxtfix.
    
    Authored-by: Gabor Somogyi <gabor.g.somo...@gmail.com>
    Signed-off-by: HyukjinKwon <gurwls...@apache.org>
    (cherry picked from commit 04f66bfd4eb863253ac9c30594055b8d5997c321)
    Signed-off-by: HyukjinKwon <gurwls...@apache.org>
---
 docs/structured-streaming-programming-guide.md | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/docs/structured-streaming-programming-guide.md 
b/docs/structured-streaming-programming-guide.md
index 1776d23..69d744d 100644
--- a/docs/structured-streaming-programming-guide.md
+++ b/docs/structured-streaming-programming-guide.md
@@ -540,12 +540,13 @@ Here are the details of all the sources in Spark.
         <br/>
         <code>fileNameOnly</code>: whether to check new files based on only 
the filename instead of on the full path (default: false). With this set to 
`true`, the following files would be considered as the same file, because their 
filenames, "dataset.txt", are the same:
         <br/>
-        <code>maxFileAge</code>: Maximum age of a file that can be found in 
this directory, before it is ignored. For the first batch all files will be 
considered valid. If <code>latestFirst</code> is set to `true` and 
<code>maxFilesPerTrigger</code> is set, then this parameter will be ignored, 
because old files that are valid, and should be processed, may be ignored. The 
max age is specified with respect to the timestamp of the latest file, and not 
the timestamp of the current system.(d [...]
-        <br/>
         "file:///dataset.txt"<br/>
         "s3://a/dataset.txt"<br/>
         "s3n://a/b/dataset.txt"<br/>
-        "s3a://a/b/c/dataset.txt"<br/>
+        "s3a://a/b/c/dataset.txt"
+        <br/>
+        <code>maxFileAge</code>: Maximum age of a file that can be found in 
this directory, before it is ignored. For the first batch all files will be 
considered valid. If <code>latestFirst</code> is set to `true` and 
<code>maxFilesPerTrigger</code> is set, then this parameter will be ignored, 
because old files that are valid, and should be processed, may be ignored. The 
max age is specified with respect to the timestamp of the latest file, and not 
the timestamp of the current system.(d [...]
+        <br/>
         <code>cleanSource</code>: option to clean up completed files after 
processing.<br/>
         Available options are "archive", "delete", "off". If the option is not 
provided, the default value is "off".<br/>
         When "archive" is provided, additional option 
<code>sourceArchiveDir</code> must be provided as well. The value of 
"sourceArchiveDir" must not match with source pattern in depth (the number of 
directories from the root directory), where the depth is minimum of depth on 
both paths. This will ensure archived files are never included as new source 
files.<br/>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

Reply via email to