GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21824
[SPARK-24871][SQL] Refactor Concat and MapConcat to avoid creating concatenator object for each row. ## What changes were proposed in this pull request? Refactor `Concat` and `MapConcat` to: - avoid creating concatenator object for each row. - make `Concat` handle `containsNull` properly. - make `Concat` shortcut if `null` child is found. ## How was this patch tested? Added some tests and existing tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ueshin/apache-spark issues/SPARK-24871/refactor_concat_mapconcat Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21824.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21824 ---- commit c4d4b28756d93ecc5c1e9a50e705e2a9cb4b8460 Author: Takuya UESHIN <ueshin@...> Date: 2018-07-19T08:32:15Z Refactor `Concat` and `MapConcat` to avoid creating concatenator for each row. commit 644a30d32e390f39f25515bb251e120e69502f27 Author: Takuya UESHIN <ueshin@...> Date: 2018-07-19T10:38:20Z Make `Concat` shortcut if null child is found. ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org