jonathansudhakar1 commented on issue #25374:
URL: https://github.com/apache/beam/issues/25374#issuecomment-6881133387

   @robertwb, a polite follow-up, since you added the `use_subprocess` and 
`timeout` modes of `with_exception_handling` (#26526). The fix touches 
`transforms/core.py`, so I'd like a maintainer's view on the approach in [my 
comment 
above](https://github.com/apache/beam/issues/25374#issuecomment-6710411217) 
before I send a PR.
   
   It still reproduces on master (de0e417a696, Python 3.13). On the 
DirectRunner, an `AsIter` side input sends every element to the dead-letter 
output with `'_thread._local' object has no attribute 
'side_input_cache_tokens'`. That happens with `use_subprocess=True` and with 
`timeout=30`. The BundleBasedDirectRunner passes, because its side inputs are 
already lists.
   
   In short:
   1. **Subprocess mode:** pickle the call's arguments in the dispatching 
thread, which has the state context, instead of leaving it to the pool's queue 
feeder thread.
   2. **Timeout mode:** turn lazy `AsIter` side inputs into lists before 
handing the call to the worker thread.
   
   Is materializing the side input in the timeout path OK with you? Or would 
you rather carry the state context into the worker thread, or fix only the 
subprocess path?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to