This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.1 by this push:
     new e9bead7  [MINOR][DOCS] Corrected spacing in structured streaming 
programming
e9bead7 is described below

commit e9bead79f8555faa8ba6a3b2ca9925a28022bee9
Author: mans2singh <mans2si...@yahoo.com>
AuthorDate: Tue Nov 2 11:01:57 2021 +0900

    [MINOR][DOCS] Corrected spacing in structured streaming programming
    
    ### What changes were proposed in this pull request?
    There is no space between `with` and  `<code>` as shown below:
    
    `... configured 
with<code>spark.sql.streaming.fileSource.cleaner.numThreads</code> ...`
    
    ### Why are the changes needed?
    Added space
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Only documentation was changed and no code was change.
    
    Closes #34458 from mans2singh/structured_streaming_programming_guide_space.
    
    Authored-by: mans2singh <mans2si...@yahoo.com>
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
    (cherry picked from commit 675071a38e47dc2c55cf4f71de7ad0bebc1b4f2b)
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
 docs/structured-streaming-programming-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/structured-streaming-programming-guide.md 
b/docs/structured-streaming-programming-guide.md
index d88cf91b..28d312e 100644
--- a/docs/structured-streaming-programming-guide.md
+++ b/docs/structured-streaming-programming-guide.md
@@ -553,7 +553,7 @@ Here are the details of all the sources in Spark.
         For example, suppose you provide '/hello?/spark/*' as source pattern, 
'/hello1/spark/archive/dir' cannot be used as the value of "sourceArchiveDir", 
as '/hello?/spark/*' and '/hello1/spark/archive' will be matched. 
'/hello1/spark' cannot be also used as the value of "sourceArchiveDir", as 
'/hello?/spark' and '/hello1/spark' will be matched. '/archived/here' would be 
OK as it doesn't match.<br/>
         Spark will move source files respecting their own path. For example, 
if the path of source file is <code>/a/b/dataset.txt</code> and the path of 
archive directory is <code>/archived/here</code>, file will be moved to 
<code>/archived/here/a/b/dataset.txt</code>.<br/>
         NOTE: Both archiving (via moving) or deleting completed files will 
introduce overhead (slow down, even if it's happening in separate thread) in 
each micro-batch, so you need to understand the cost for each operation in your 
file system before enabling this option. On the other hand, enabling this 
option will reduce the cost to list source files which can be an expensive 
operation.<br/>
-        Number of threads used in completed file cleaner can be configured 
with<code>spark.sql.streaming.fileSource.cleaner.numThreads</code> (default: 
1).<br/>
+        Number of threads used in completed file cleaner can be configured 
with <code>spark.sql.streaming.fileSource.cleaner.numThreads</code> (default: 
1).<br/>
         NOTE 2: The source path should not be used from multiple sources or 
queries when enabling this option. Similarly, you must ensure the source path 
doesn't match to any files in output directory of file stream sink.<br/>
         NOTE 3: Both delete and move actions are best effort. Failing to 
delete or move files will not fail the streaming query. Spark may not clean up 
some source files in some circumstances - e.g. the application doesn't shut 
down gracefully, too many files are queued to clean up.
         <br/><br/>

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org

Reply via email to