hvanhovell commented on issue #23169: [SPARK-26103][SQL] Limit the length of debug strings for query plans URL: https://github.com/apache/spark/pull/23169#issuecomment-456004821 @DaveDeCaprio what kind of process is processing these plans? As soon as we are dealing with plans of this scale you will start to suffer from all kinds of slowness in the analyzer/optimizer/planner. I am wondering if this can be fixed in a better way by attacking the root problem (deep/wide plans) and if we need to add constructs/primitives to spark that will allow you handle more gracefully.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
