Changgyoo Park created SPARK-51601:
--------------------------------------
Summary: Chained union/withColumn expressions slow down query
analysis
Key: SPARK-51601
URL: https://issues.apache.org/jira/browse/SPARK-51601
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 4.1.0
Reporter: Changgyoo Park
Example.
```scala
import org.apache.spark.sql.functions.col
var df = sql("select id from range(16)")
for (i <- 0 until 1024) {
df = df.withColumn(s"id$i", col("id") + 1)
}
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
