clintropolis commented on a change in pull request #5712: HllSketch module URL: https://github.com/apache/incubator-druid/pull/5712#discussion_r217678885
########## File path: extensions-core/datasketches/src/main/java/io/druid/query/aggregation/datasketches/hll/HllSketchAggregatorFactory.java ########## @@ -0,0 +1,193 @@ +/* + * Licensed to Metamarkets Group Inc. (Metamarkets) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. Metamarkets licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package io.druid.query.aggregation.datasketches.hll; + +import java.util.Arrays; +import java.util.Comparator; +import java.util.List; +import java.util.Objects; + +import javax.annotation.Nullable; + +import com.fasterxml.jackson.annotation.JsonProperty; +import com.yahoo.sketches.hll.HllSketch; +import com.yahoo.sketches.hll.TgtHllType; +import com.yahoo.sketches.hll.Union; + +import io.druid.java.util.common.IAE; +import io.druid.query.aggregation.AggregatorFactory; +import io.druid.query.cache.CacheKeyBuilder; + +/** + * Base class for both build and merge factories + * @author Alexander Saydakov + */ +abstract class HllSketchAggregatorFactory extends AggregatorFactory Review comment: I had to add an override of `makeAggregateCombiner` to make this work with rollup during indexing. This is what I added which seemed to make it work and spit out values similar to the existing `hyperUnique` aggregator on the query side. ``` @Override public AggregateCombiner makeAggregateCombiner() { return new ObjectAggregateCombiner<HllSketch>() { private Union union = new Union(lgK); @Override public void reset(ColumnValueSelector selector) { union.reset(); fold(selector); } @Override public void fold(ColumnValueSelector selector) { HllSketch sketch = (HllSketch) selector.getObject(); if (sketch != null) { union.update(sketch); } } @Override public Class<HllSketch> classOfObject() { return HllSketch.class; } @Nullable @Override public HllSketch getObject() { return union.getResult(tgtHllType); } }; } ``` I'm not certain if this is where this makes sense to be defined, or if it should be in `HllSketchBuildAggregatorFactory` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
