ivoson commented on code in PR #59122:
URL: https://github.com/apache/spark/pull/59122#discussion_r4192340591
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -5433,6 +5433,33 @@ object SQLConf {
"be -1 (no limit) or greater than zero and less than or equal to
INT_MAX.")
.createWithDefault(-1)
+ val PYTHON_UDF_ROW_SIZE_GUARD_ENABLED =
+ buildConf("spark.sql.execution.python.udf.rowSizeGuard.enabled")
+ .internal()
+ .doc("When true, guard pickle-serialized (non-Arrow) Python UDF
evaluation against " +
+ "top-level string and binary argument payloads whose combined size
exceeds " +
+ "rowSizeGuard.maxRowHeapFraction of executor heap. The guard checks
the projected " +
+ "arguments before conversion and pickling. Nested inputs are not
estimated.")
+ .version("4.3.0")
Review Comment:
Done — updated both row-size guard config versions to 4.4.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
