This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new f8bbbdfccc96 [SPARK-58413][SQL][TEST] Rename `sequential fetch` label 
to `pipelined fetch (1 client)` in `NettyTransportBenchmark`
f8bbbdfccc96 is described below

commit f8bbbdfccc9645032ffc35bddfd84abe3e8d7b2d
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Jul 29 00:52:57 2026 -0700

    [SPARK-58413][SQL][TEST] Rename `sequential fetch` label to `pipelined 
fetch (1 client)` in `NettyTransportBenchmark`
    
    ### What changes were proposed in this pull request?
    
    This PR renames the benchmark case label `sequential fetch` to `pipelined 
fetch (1 client)` in the `File-Backed Shuffle Block Fetch` suite of 
`NettyTransportBenchmark`.
    
    ### Why are the changes needed?
    
    The label is misleading. The case fires all 100 `fetchChunk` requests at 
once on a single connection and then waits for all of them to complete via a 
semaphore (`fetchChunksSync`), so the client never waits for one chunk before 
requesting the next. This is a pipelined fetch over one connection, not a 
sequential request-response loop.
    
    The mislabel skews the interpretation of the results. For example:
    
    
https://github.com/apache/spark/blob/12785d56624a8b412e78559dd920b816d629704b/core/benchmarks/NettyTransportBenchmark-results.txt#L132-L133
    
    Read as "sequential vs parallel", the 1.8X looks like poor parallelization 
efficiency. The actual comparison is "pipelined over 1 socket vs pipelined over 
4 sockets", where the single pipelined connection already runs at ~4.2 GB/s. 
The new label makes this clear and contrasts naturally with `parallel fetch (4 
clients)`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Label-only change; manually reviewed.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Fable 5
    
    Closes #57616 from dongjoon-hyun/SPARK-58413.
    
    Authored-by: Dongjoon Hyun <[email protected]>
    Signed-off-by: Dongjoon Hyun <[email protected]>
    (cherry picked from commit 1fba3cc49139a5130febc2f408b378cc8b64d598)
    Signed-off-by: Dongjoon Hyun <[email protected]>
---
 .../test/scala/org/apache/spark/network/NettyTransportBenchmark.scala   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/core/src/test/scala/org/apache/spark/network/NettyTransportBenchmark.scala 
b/core/src/test/scala/org/apache/spark/network/NettyTransportBenchmark.scala
index 0a3831a675c3..cd87260f1cee 100644
--- a/core/src/test/scala/org/apache/spark/network/NettyTransportBenchmark.scala
+++ b/core/src/test/scala/org/apache/spark/network/NettyTransportBenchmark.scala
@@ -599,7 +599,7 @@ object NettyTransportBenchmark extends BenchmarkBase {
           output = output)
 
         Seq("NIO", "AUTO").foreach { mode =>
-          benchmark.addTimerCase(s"$mode, sequential fetch", numIters = 3) { 
timer =>
+          benchmark.addTimerCase(s"$mode, pipelined fetch (1 client)", 
numIters = 3) { timer =>
             val conf = createConf(mode)
             val streamManager = createFileStreamManager(conf, files)
             val rpcHandler = createStreamRpcHandler(streamManager)


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to