LuciferYang commented on PR #2749: URL: https://github.com/apache/uniffle/pull/2749#issuecomment-4441372482
Ran a local runtime smoke test — the Uniffle tab loads cleanly under Spark 4 with real shuffle traffic going through Uniffle. ### Setup - Uniffle coordinator + shuffle server built from this PR (commit `46a71436`) with `mvn -Pspark4,hadoop-dependencies-included install -DskipTests` - Local Spark 4.0.2 (`scala-2.13`, JDK 17) as driver - `spark-shell --jars rss-client-spark4-shaded-*.jar,rss-client-spark-ui-*.jar --conf spark.plugins=org.apache.spark.UnifflePlugin --conf spark.shuffle.manager=org.apache.spark.shuffle.RssShuffleManager --conf spark.rss.coordinator.quorum=127.0.0.1:19999 --conf spark.rss.storage.type=MEMORY_LOCALFILE --conf spark.shuffle.sort.io.plugin.class=org.apache.spark.shuffle.RssShuffleDataIo --conf spark.serializer=org.apache.spark.serializer.KryoSerializer` - Workload: `sc.parallelize(1 to 50000, 4).map(i => (i % 20, i.toLong)).reduceByKey(_ + _).collect()` ### Results `GET http://localhost:4040/uniffle` → HTTP 200, 15307 bytes. The tab appears in the navbar: ```html <li class="nav-item active"> <a class="nav-link" href="/uniffle/">Uniffle</a> </li> ``` All expected sections render with real data (stripped of HTML tags): ``` Total shuffle bytes: 343.7 KiB CompressionRatio: 683.6 KiB / 343.7 KiB = 1.0 Shuffle Duration (W+R): 0.3 s (0.3 s + 36 ms) / 0.8 s = 0.41 Client Observed Speed: 1.16 / 9.78 MB/s Uniffle Speed: 2.55 / 31.99 MB/s Reassign Status: partitionSplit=false, blockSentFailure=false, stageRetry=false ``` Every collapsible section is present and renders without error: Uniffle Build Information, Uniffle Properties, Shuffle Throughput Statistics, Hybrid Storage Read Statistics, Shuffle Server, Assignment, Shuffle Write Times, Shuffle Read Times, Shuffle Failures. ### History Server Not exercised in this run. The plugin class is registered via `META-INF/services/org.apache.spark.status.AppHistoryServerPlugin → org.apache.spark.UniffleHistoryServerPlugin`, which the Spark 4 History Server picks up through `ServiceLoader` exactly like Spark 3. Both paths attach the same `ShuffleTab` + `UniffleStatusStore` objects, so if the runtime UI renders, the History Server rendering should follow the same code path. Happy to follow up with an explicit HS run if you want one on the record. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
