HyukjinKwon commented on issue #23285: [SPARK-26224][SQL] Warn and advice the user when creating many project on subsequent calls to withColumn URL: https://github.com/apache/spark/pull/23285#issuecomment-454641067 I agree that this is a bad pattern and rather common mistake(?) that users do time to time. Warning can be more effective. However, I was wondering if it's worth enough to make the current change considering that it's going to be more complicated and cause a bit of overhead to maintain this code. For instance, if we happened to improve the deeply nested plan problem, we should find another number fo set as default. Also, configurations for logging does look an overkill .. We could go for documentation first and consider the current fix later when users keep making this pattern.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
