Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20035#discussion_r158071129
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
---
@@ -214,11 +213,12 @@ case class Stack(children: Seq[Expression]) extends
Generator {
// Create the collection.
val wrapperClass = classOf[mutable.WrappedArray[_]].getName
- ctx.addMutableState(
- s"$wrapperClass<InternalRow>",
- ev.value,
- v => s"$v = $wrapperClass$$.MODULE$$.make($rowData);", useFreshName
= false)
- ev.copy(code = code, isNull = "false")
+ ev.copy(code =
+ s"""
+ |InternalRow[] $rowData = new InternalRow[$numRows];
--- End diff --
this creates a large array every time, and I don't think we have data copy
issues for generator expressions...
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
