HyukjinKwon commented on issue #23285: [SPARK-26224][SQL] Warn and advice the user when creating many project on subsequent calls to withColumn URL: https://github.com/apache/spark/pull/23285#issuecomment-454264153 How about we simply leave a `note` in the doc that says for instance multiple calls of this causes deeply nested plan and workaround is, for instance, `select` multiple columns? I think that's going to reach the goal here. We can just document and advise. Changes for instance of configuration, benchmarking to find out the most appropriate number, or finding out a general fix the root cause looks quite over kill considering the goal is just to let users know about the limitation and workaround.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
