dongjoon-hyun commented on PR #42326: URL: https://github.com/apache/spark/pull/42326#issuecomment-1664679561
I rebased and tested this test-only PR manually. ``` [info] MapOutputTrackerSuite: [info] - master start and stop (638 milliseconds) [info] - master register shuffle and fetch (169 milliseconds) [info] - master register and unregister shuffle (43 milliseconds) [info] - master register shuffle and unregister map output and fetch (30 milliseconds) [info] - remote fetch (486 milliseconds) [info] - remote fetch below max RPC message size (565 milliseconds) [info] - min broadcast size exceeds max RPC message size (21 milliseconds) [info] - getLocationsWithLargestOutputs with multiple outputs in same machine (107 milliseconds) [info] - remote fetch using broadcast (6 seconds, 409 milliseconds) [info] - equally divide map statistics tasks (15 milliseconds) [info] - zero-sized blocks should be excluded when getMapSizesByExecutorId (62 milliseconds) [info] - SPARK-32921: master register and unregister merge result (259 milliseconds) [info] - SPARK-32921: get map sizes with merged shuffle (163 milliseconds) [info] - SPARK-32921: get map statuses from merged shuffle (138 milliseconds) [info] - SPARK-32921: get map statuses for merged shuffle block chunks (116 milliseconds) [info] - SPARK-32921: getPreferredLocationsForShuffle with MergeStatus (939 milliseconds) [info] - SPARK-34939: remote fetch using broadcast if broadcasted value is destroyed (15 seconds, 957 milliseconds) [info] - SPARK-32921: test new protocol changes fetching both Map and Merge status in single RPC (18 seconds, 24 milliseconds) [info] - SPARK-32921: unregister merge result if it is present and contains the map Id (91 milliseconds) [info] - SPARK-32210: serialize mapStatuses to a nested Array and deserialize them (1 second, 50 milliseconds) [info] - SPARK-32210: serialize and deserialize over 2GB compressed mapStatuses !!! IGNORED !!! [info] - SPARK-36892: Batch fetch should be enabled in some scenarios with push based shuffle (151 milliseconds) [info] - SPARK-36892: Batch fetch should be disabled in some scenarios with push based shuffle (79 milliseconds) [info] - SPARK-37023: Avoid fetching merge status when useMergeResult is false (120 milliseconds) [info] - SPARK-40480: shuffle remove should cleanup merged files as well (335 milliseconds) [info] - SPARK-34826: Adaptive shuffle mergers (218 milliseconds) [info] - SPARK-39553: Multi-thread unregister shuffle shouldn't throw NPE (79 milliseconds) [info] - SPARK-42719: `MapOutputTracker#getMapLocation` should respect the config option (38 milliseconds) [info] - SPARK-44109: Remove duplicate preferred locations of each RDD partition (226 milliseconds) [info] - SPARK-44658: ShuffleStatus.getMapStatus should return None (13 milliseconds) [info] - SPARK-44661: getMapOutputLocation should not throw NPE (63 milliseconds) [info] Run completed in 49 seconds, 770 milliseconds. [info] Total number of tests run: 30 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 30, failed 0, canceled 0, ignored 1, pending 0 [info] All tests passed. [success] Total time: 99 s (01:39), completed Aug 3, 2023 2:35:12 PM ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
