This is an automated email from the ASF dual-hosted git repository.

viirya pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 5b3ee7503d5e [SPARK-59985][4.X][ML][PYTHON][FOLLOWUP] Drop the 
log_streaming_auth_secret assertion that branch-4.x does not have
5b3ee7503d5e is described below

commit 5b3ee7503d5e9e3b53f0526c033f9fdd1f723c14
Author: Liang-Chi Hsieh <[email protected]>
AuthorDate: Wed Oct 21 19:59:51 2026 -0700

    [SPARK-59985][4.X][ML][PYTHON][FOLLOWUP] Drop the log_streaming_auth_secret 
assertion that branch-4.x does not have
    
    ### What changes were proposed in this pull request?
    
    This is a branch-4.x follow-up of SPARK-59985 (#59242). It removes the 
`self.assertIsNone(dist.log_streaming_auth_secret)` assertion from 
`test_log_streaming_server_start_failure_warns` in 
`python/pyspark/ml/torch/tests/test_distributor.py`.
    
    `TorchDistributor.log_streaming_auth_secret` exists only on master. It was 
added by 77f1e05768f ("[PYTHON][MINOR] Improve TorchDistributor log server 
provisioning"), which is not on branch-4.x. When #59242 was cherry-picked onto 
branch-4.x as b628a945a24, the test was taken as written for master, including 
this assertion. The manual backports for 4.3/4.2/4.1/4.0 (#59381, #59382, 
#59383, #59384) already leave it out. This PR makes branch-4.x match them; the 
rest of the test is unchanged.
    
    ### Why are the changes needed?
    
    On branch-4.x the test fails with:
    
    ```
    AttributeError: 'TorchDistributor' object has no attribute 
'log_streaming_auth_secret'
    ```
    
    This breaks the PySpark CI for PRs against branch-4.x, for example #59676, 
which does not touch `pyspark.ml.torch`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. Test-only change.
    
    ### How was this patch tested?
    
    Ran 
`TorchDistributorBaselineUnitTests.test_log_streaming_server_start_failure_warns`
 locally. Before this change it fails with the `AttributeError` above; after 
it, it passes. Also checked that branch-4.x has no other reference to 
`log_streaming_auth_secret` under `python/`.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Claude Opus 5.5)
    
    Closes #59691 from viirya/SPARK-59985-4.x-followup.
    
    Authored-by: Liang-Chi Hsieh <[email protected]>
    Signed-off-by: Liang-Chi Hsieh <[email protected]>
---
 python/pyspark/ml/torch/tests/test_distributor.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/python/pyspark/ml/torch/tests/test_distributor.py 
b/python/pyspark/ml/torch/tests/test_distributor.py
index af994c81af40..1b9af7a7a3f1 100644
--- a/python/pyspark/ml/torch/tests/test_distributor.py
+++ b/python/pyspark/ml/torch/tests/test_distributor.py
@@ -294,7 +294,6 @@ class TorchDistributorBaselineUnitTestsMixin:
             dist._run_distributed_training(MagicMock(), MagicMock(), None, 
None)
 
         self.assertEqual(dist.log_streaming_server_port, -1)
-        self.assertIsNone(dist.log_streaming_auth_secret)
         self.assertEqual(len(logs.records), 1)
         message = logs.records[0].getMessage()
         self.assertIn("Start torch distributor log streaming server failed", 
message)


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to