JingsongLi commented on code in PR #11103:
URL: https://github.com/apache/paimon/pull/11103#discussion_r4546583957
##########
paimon-python/pypaimon/multimodal/video.py:
##########
@@ -460,6 +481,17 @@ def _map_decoders(self, requests, decode, parallelism,
executor=None):
return
items = iter(requests.items())
pending = deque()
+ torch = sys.modules.get("torch")
+ if torch is not None:
+ grad_enabled = torch.is_grad_enabled()
+ inference_enabled = torch.is_inference_mode_enabled()
+
+ def run_decode(decoder, value):
+ if torch is None:
+ return decode(decoder, value)
+ with torch.inference_mode(inference_enabled), \
+ torch.set_grad_enabled(grad_enabled):
+ return decode(decoder, value)
Review Comment:
**[P2] Preserve Torch device and autocast context in parallel callbacks**
This wrapper restores grad/inference modes, but the caller's default-device
and autocast contexts are also thread-local and are lost here. With the same
collator configured with `decode_parallelism=2`, a thread-safe callback such as
`lambda decoder, index, row: torch.tensor([index])`, called inside `with
torch.device("mps")`, returns an MPS tensor for a single-video batch but CPU
tensors for a two-video batch. I reproduced this on Torch 2.10 with MPS. The
serial path runs inline, so the output device now depends on the number of
distinct videos in the batch and can cause downstream device-mismatch errors.
Likewise, inside `torch.autocast("cpu", dtype=torch.bfloat16)`, a callback
performing `torch.mm(torch.ones(2, 2), torch.ones(2, 2))` returns bfloat16
serially and float32 in parallel.
Please capture and restore the relevant device/autocast contexts alongside
grad/inference modes, and add equivalence coverage for single-video and
multi-video batches using the same collator configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]