Tian Gao created SPARK-54619:
--------------------------------
Summary: Add sanity check for runner conf
Key: SPARK-54619
URL: https://issues.apache.org/jira/browse/SPARK-54619
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 4.2.0
Reporter: Tian Gao
Having some sanity checks on worker side is helpful to avoid deadlock caused by
invalid message. For example if the message is malformed, the worker could wait
for an unreasonable amount of data - if we are unlucky, there won't be any
errors.
Checking number of configurations is safe, effective and fast - we should do
that.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
