srowen commented on code in PR #37843:
URL: https://github.com/apache/spark/pull/37843#discussion_r972490951
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/expressions/Expression.java:
##########
@@ -44,7 +46,12 @@ public interface Expression {
* List of fields or columns that are referenced by this expression.
*/
default NamedReference[] references() {
- return Arrays.stream(children()).map(e -> e.references())
- .flatMap(Arrays::stream).distinct().toArray(NamedReference[]::new);
+ // SPARK-40398: Replace `Arrays.stream()...distinct()`
+ // to this for perf gain, the result order is not important.
+ Set<NamedReference> set = new HashSet<>();
+ for (Expression e : children()) {
+ Collections.addAll(set, e.references());
+ }
+ return set.toArray(new NamedReference[0]);
Review Comment:
I have one last tiny suggestion - either pass an array of size set.size(),
or make a static final empty array and pass it here, to avoid allocating an
empty array. It's tiny but hey we are micro-optimizing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
