Andy Sautins created SPARK-53873:
------------------------------------
Summary: ExplodeBase.eval Iterate directly on input
Key: SPARK-53873
URL: https://issues.apache.org/jira/browse/SPARK-53873
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 4.0.1
Reporter: Andy Sautins
It was noticed that `ExplodeBase.eval` returns an IterableOnce[InternalRow].
The current implementation creates a pre-allocated array, populates the array
appropriately, and returns the Array. This works as the is an implicit
conversion from Array to IterableOnce.
However Allocating and populating an array does not seem to provide benefits
over exposing an iterator over the input data type.
A proposed PR removes the creation and population of the array and instead
returns a IterableOnce object that iterates over the underlying input.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
