Hyukjin Kwon created SPARK-61067:
------------------------------------
Summary: Keep the tags of a session per thread in the Python
client as PySpark does
Key: SPARK-61067
URL: https://issues.apache.org/jira/browse/SPARK-61067
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
In PySpark Connect, the tags of {{spark.addTag}}, {{removeTag}}, {{getTags}}
and {{clearTags}} are those of the current thread: the operations that a thread
starts carry its tags only, so {{spark.interruptTag(tag)}} interrupts only
those of the threads that added the tag, and a new thread starts without tags.
The Rust-backed Python client keeps one set of tags for all the threads of a
session: a tag that one thread adds is on the operations of every thread, so
{{interruptTag}} interrupts them all, and {{getTags}} of a new thread returns
the tags of the others. Keep them per thread as PySpark does.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
