tejasiyer-dev commented on code in PR #40543:
URL: https://github.com/apache/beam/pull/40543#discussion_r4648311208


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java:
##########
@@ -315,34 +339,42 @@ public WriteFiles<UserT, DestinationT, OutputT> 
withNumShards(
    * #withRunnerDeterminedSharding()}):</b>
    *
    * <table>
-   *   <tr>
-   *     <th>Configuration</th>
-   *     <th>Behavior when {@code maxNumWritersPerBundle} is reached</th>
-   *     <th>Worker Memory Consumption</th>
-   *     <th>Shuffle Cost</th>
-   *     <th>Output File Size / Count</th>
-   *   </tr>
-   *   <tr>
-   *     <td><b>Default (Spill to Shuffle)</b><br>{@code 
maxNumWritersPerBundle > 0},<br>{@code evictWritersWhenFull = false}</td>
-   *     <td>Keeps first {@code N} writers open; spills remaining records to a 
{@link GroupByKey} shuffle stage</td>
-   *     <td>Bounded ({@code <= N} buffers per bundle)</td>
-   *     <td>High if many records spill across shuffle</td>
-   *     <td>Fewer, larger files</td>
-   *   </tr>
-   *   <tr>
-   *     <td><b>LRU Writer Eviction</b><br>{@code maxNumWritersPerBundle > 
0},<br>{@code evictWritersWhenFull = true}</td>
-   *     <td>Flushes and closes the least recently used (LRU) open writer to 
open a new writer inline</td>
-   *     <td>Bounded ({@code <= N} buffers per bundle)</td>
-   *     <td>None (no shuffle stage for unwritten records)</td>
-   *     <td>May produce more/smaller files (minimal if input is ordered by 
destination, high if random)</td>
-   *   </tr>
-   *   <tr>
-   *     <td><b>No Spilling</b><br>{@link #withNoSpilling()} ({@code 
maxNumWritersPerBundle = -1})</td>
-   *     <td>Opens a new writer for every destination in the bundle without 
limit</td>
-   *     <td>Unbounded (risk of OOM with many destinations)</td>
-   *     <td>None (no shuffle stage for unwritten records)</td>
-   *     <td>Fewer, larger files (1 file per destination per bundle)</td>
-   *   </tr>
+   * <tr>

Review Comment:
   Done



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java:
##########
@@ -137,21 +144,27 @@ public abstract class WriteFiles<UserT, DestinationT, 
OutputT>
   @Internal
   public static final Class<? extends WriteFiles> CONCRETE_CLASS = 
AutoValue_WriteFiles.class;
 
-  // The maximum number of file writers to keep open in a single bundle at a 
time, since file
-  // writers default to 64mb buffers. This comes into play when writing 
per-window or dynamic
-  // destination files. The first 20 files from a single WriteFiles transform 
will write files
-  // inline in the transform. Anything beyond that might be spilled to shuffle 
(default) or cause
+  // The maximum number of file writers to keep open in a single bundle at a 
time,
+  // since file
+  // writers default to 64mb buffers. This comes into play when writing 
per-window
+  // or dynamic
+  // destination files. The first 20 files from a single WriteFiles transform 
will
+  // write files

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to