vanzin commented on a change in pull request #23169: [SPARK-26103][SQL] Limit
the length of debug strings for query plans
URL: https://github.com/apache/spark/pull/23169#discussion_r264900387
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1625,6 +1625,17 @@ object SQLConf {
.intConf
.createWithDefault(25)
+ val MAX_PLAN_STRING_LENGTH = buildConf("spark.sql.maxPlanStringLength")
+ .doc("Maximum number of characters to output for a plan string. If the
plan is " +
+ "longer, further output will be truncated. The default setting always
generates a full " +
+ "plan. Set this to a lower value such as 8192 if plan strings are
taking up too much " +
+ "memory or are causing OutOfMemory errors in the driver or UI
processes.")
+ .intConf
Review comment:
Argh, should have commented before. But I think it's more user-friendly to
use `bytesConf` here so that users can set the limit to e.g. "32k".
Also if the limit is `0` the plan string will be a little odd. Not a big
deal though, but it could be improved (e.g. using a different message when it's
0).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
