jon-wei commented on a change in pull request #9800:
URL: https://github.com/apache/druid/pull/9800#discussion_r419212188
##########
File path:
processing/src/main/java/org/apache/druid/query/filter/InDimFilter.java
##########
@@ -84,9 +90,11 @@ public InDimFilter(
Preconditions.checkNotNull(dimension, "dimension can not be null");
Preconditions.checkArgument(values != null, "values can not be null");
- this.values = new TreeSet<>(Comparators.naturalNullsFirst());
- for (String value : values) {
- this.values.add(NullHandling.emptyToNullIfNeeded(value));
+ // The values set can be huge. Try to avoid copying the set if possible.
+ if (values instanceof Set &&
values.stream().noneMatch(NullHandling::needsEmptyToNull)) {
+ this.values = (Set<String>) values;
Review comment:
I don't think we have any usages right now where this would be an issue,
but maybe it's good to have a comment somewhere noting that the set passed in
to InDimFilter shouldn't be modified afterwards since it can be reused
##########
File path:
processing/src/main/java/org/apache/druid/query/filter/InDimFilter.java
##########
@@ -132,31 +141,43 @@ public FilterTuning getFilterTuning()
@Override
public byte[] getCacheKey()
{
- boolean hasNull = false;
- for (String value : values) {
- if (value == null) {
- hasNull = true;
- break;
- }
+ if (cacheKey == null) {
+ final boolean hasNull = values.stream().anyMatch(Objects::isNull);
Review comment:
If `values` is a set, I think you could determine `hasNull` beforehand
if you've already scanned values for empty/nulls, and avoid scanning again
##########
File path:
processing/src/main/java/org/apache/druid/query/filter/InDimFilter.java
##########
@@ -84,9 +90,11 @@ public InDimFilter(
Preconditions.checkNotNull(dimension, "dimension can not be null");
Preconditions.checkArgument(values != null, "values can not be null");
- this.values = new TreeSet<>(Comparators.naturalNullsFirst());
- for (String value : values) {
- this.values.add(NullHandling.emptyToNullIfNeeded(value));
+ // The values set can be huge. Try to avoid copying the set if possible.
+ if (values instanceof Set &&
values.stream().noneMatch(NullHandling::needsEmptyToNull)) {
Review comment:
If `NullHandling.replaceWithDefault()` returns false, I think you can
skip the scan here and just reuse the set
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
