Shay Elbaz created SPARK-26438:
----------------------------------
Summary: Driver waits to spark.sql.broadcastTimeout before
throwing OutOfMemoryError - is this by design?
Key: SPARK-26438
URL: https://issues.apache.org/jira/browse/SPARK-26438
Project: Spark
Issue Type: Question
Components: Spark Core, SQL
Affects Versions: 2.3.0
Reporter: Shay Elbaz
When broadcasting too large DataFrame, the driver does not fail immediately
when the broadcast thread throws OutOfMemoryError. Instead it waits for
`spark.sql.broadcastTimeout` to meet. Is that by design or a bug?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
