alex-plekhanov commented on code in PR #10153:
URL: https://github.com/apache/ignite/pull/10153#discussion_r961652350
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/IndexScan.java:
##########
@@ -159,6 +162,44 @@ public IndexScan(
}
}
+ /**
+ * Gets first or last records from all segments.
Review Comment:
1. Theoretically, `IndexScan` should not provide to users any information
about segments.
2. Output of `firstOrLast` should be sorted, with hash aggregate it will
work as is, but with sort aggregate it will return wrong results.
3. It should return `Cursor`, not `List`. In the current implementation
first/last scan will be executed during execution nodes building (in
`LogicalRelImplementor`), but actually scan should only be run on execution
stage.
I propose to refactor a little bit:
1. Extend `IndexScan` with some kind of new class (`IndexFirstLastScan` for
example), implement `indexQueryContext` and `iterator` methods, and new tree
index wrapper class which will wrap `idx.findFirst`/`idx.findLast` methods.
2. Add new methods findFirst/findLast to the `InlineIndexImpl` to get sorted
values for all segments (see find method and `SegmentedIndexCursor` class),
perhaps it's worth to return only one value by these methods to maintain API
consistency (for example create some `SingleValueSegmentedIndexCursor extends
SegmentedIndexCursor`)
##########
modules/core/src/main/java/org/apache/ignite/internal/util/lang/GridFunc.java:
##########
@@ -1186,6 +1187,42 @@ public static <T> boolean isEmptyOrNulls(@Nullable T[]
c) {
return true;
}
+ /**
+ * Tests if the given object is null, is empty array, empty map or
contains only nulls.
+ *
+ * @param o Object to test.
+ * @return {@code True}, is given object is null, is empty array, empty
map or contains only nulls.
+ * {@code False} otherwise.
+ */
+ public static boolean isEmptyOrNulls(@Nullable Object o) {
Review Comment:
Redundant (if system view first/last scan will be removed).
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rule/IndexMinMaxRule.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.query.calcite.rule;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.sql.SqlAggFunction;
+import org.apache.calcite.sql.SqlKind;
+import
org.apache.ignite.internal.processors.query.calcite.rel.AbstractIndexScan;
+import org.apache.ignite.internal.processors.query.calcite.rel.IgniteAggregate;
+import
org.apache.ignite.internal.processors.query.calcite.rel.IgniteIndexBound;
+import org.apache.ignite.internal.processors.query.calcite.rel.IgniteIndexScan;
+import org.apache.ignite.internal.processors.query.calcite.schema.IgniteIndex;
+import org.apache.ignite.internal.processors.query.calcite.schema.IgniteTable;
+import
org.apache.ignite.internal.processors.query.calcite.trait.RewindabilityTrait;
+import org.apache.ignite.internal.util.typedef.F;
+import org.immutables.value.Value;
+
+/**
+ * Tries to optimize MIN() and MAX() so that taking only first or last index
record is engaged.
+ */
[email protected]
+public class IndexMinMaxRule extends RelRule<IndexMinMaxRule.Config> {
+ /** */
+ public static final IndexMinMaxRule INSTANCE = Config.DEFAULT.toRule();
+
+ /** Ctor. */
+ private IndexMinMaxRule(IndexMinMaxRule.Config cfg) {
+ super(cfg);
+ }
+
+ /** */
+ @Override public void onMatch(RelOptRuleCall call) {
+ IgniteAggregate aggr = call.rel(0);
+ IgniteIndexScan idxScan = call.rel(1);
+ IgniteTable table = idxScan.getTable().unwrap(IgniteTable.class);
+ IgniteIndex idx = table.getIndex(idxScan.indexName());
+
+ if (
+ table.isIndexRebuildInProgress() ||
+ idxScan.condition() != null ||
Review Comment:
Projections should also be null
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/schema/SystemViewIndexImpl.java:
##########
@@ -102,6 +105,25 @@ public SystemViewIndexImpl(SystemViewTableImpl tbl) {
return tbl.descriptor().systemView().size();
}
+ /** {@inheritDoc} */
+ @Override public <Row> List<Row> findFirstOrLast(boolean first,
ExecutionContext<Row> ectx,
+ ColocationGroup grp, @Nullable ImmutableBitSet requiredColumns) {
+ Iterator<Row> it = scan(ectx, grp, null, null, null, null,
requiredColumns).iterator();
Review Comment:
Since collation for system view indexes is always empty, there is impossible
to rich this code, I think we should just throw assertion error here instead of
implementation.
In any case, `isNullOrEmpty` should not be used, see
https://github.com/apache/ignite/pull/10117#discussion_r916578233
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/LogicalRelImplementor.java:
##########
@@ -428,6 +429,44 @@ public LogicalRelImplementor(
}
}
+ /** {@inheritDoc} */
+ @Override public Node<Row> visit(IgniteIndexBound rel) {
+ IgniteTable tbl = rel.getTable().unwrap(IgniteTable.class);
+ IgniteIndex idx = tbl.getIndex(rel.indexName());
+ IgniteTypeFactory typeFactory = ctx.getTypeFactory();
+ ColocationGroup grp = ctx.group(rel.sourceId());
+ ImmutableBitSet requiredColumns = rel.requiredColumns();
+ RelDataType rowType = tbl.getRowType(typeFactory, requiredColumns);
+
+ if (idx != null && !tbl.isIndexRebuildInProgress()) {
+ return new ScanNode<>(ctx, rowType,
idx.findFirstOrLast(rel.first(), ctx, ctx.group(rel.sourceId()),
+ requiredColumns));
+ }
+ else {
+ Iterable<Row> rowsIter = tbl.scan(
+ ctx,
+ grp,
+ null,
+ null,
+ rel.requiredColumns()
+ );
+
+ Node<Row> scanNode = new ScanNode<>(ctx, rowType, rowsIter);
+
+ RelCollation collation =
idx.collation().apply(LogicalScanConverterRule.createMapping(
+ null,
+ requiredColumns,
+ tbl.getRowType(typeFactory).getFieldCount()
+ ));
+
+ SortNode<Row> sortNode = new SortNode<>(ctx, rowType,
expressionFactory.comparator(collation));
Review Comment:
Sort with limit(1) should be used
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/rule/IndexMinMaxRule.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.processors.query.calcite.rule;
+
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.sql.SqlAggFunction;
+import org.apache.calcite.sql.SqlKind;
+import
org.apache.ignite.internal.processors.query.calcite.rel.AbstractIndexScan;
+import org.apache.ignite.internal.processors.query.calcite.rel.IgniteAggregate;
+import
org.apache.ignite.internal.processors.query.calcite.rel.IgniteIndexBound;
+import org.apache.ignite.internal.processors.query.calcite.rel.IgniteIndexScan;
+import org.apache.ignite.internal.processors.query.calcite.schema.IgniteIndex;
+import org.apache.ignite.internal.processors.query.calcite.schema.IgniteTable;
+import
org.apache.ignite.internal.processors.query.calcite.trait.RewindabilityTrait;
+import org.apache.ignite.internal.util.typedef.F;
+import org.immutables.value.Value;
+
+/**
+ * Tries to optimize MIN() and MAX() so that taking only first or last index
record is engaged.
+ */
[email protected]
+public class IndexMinMaxRule extends RelRule<IndexMinMaxRule.Config> {
+ /** */
+ public static final IndexMinMaxRule INSTANCE = Config.DEFAULT.toRule();
+
+ /** Ctor. */
+ private IndexMinMaxRule(IndexMinMaxRule.Config cfg) {
+ super(cfg);
+ }
+
+ /** */
+ @Override public void onMatch(RelOptRuleCall call) {
+ IgniteAggregate aggr = call.rel(0);
+ IgniteIndexScan idxScan = call.rel(1);
+ IgniteTable table = idxScan.getTable().unwrap(IgniteTable.class);
+ IgniteIndex idx = table.getIndex(idxScan.indexName());
+
+ if (
+ table.isIndexRebuildInProgress() ||
+ idxScan.condition() != null ||
+ aggr.getGroupCount() > 0 ||
+ aggr.getAggCallList().stream().filter(a ->
a.getAggregation().getKind() == SqlKind.MIN
+ || a.getAggregation().getKind() == SqlKind.MAX).count() !=
1 ||
+ idx.collation().getFieldCollations().isEmpty() ||
+ idx.collation().getFieldCollations().get(0).getFieldIndex() !=
idxScan.requiredColumns().asList().get(0)
+ )
+ return;
+
+ SqlAggFunction aggFun = aggr.getAggCallList().get(0).getAggregation();
+ boolean firstIdxValue = (aggFun.getKind() == SqlKind.MIN) !=
+
idx.collation().getFieldCollations().get(0).getDirection().isDescending();
+
+ IgniteIndexBound newAggrInput = new IgniteIndexBound(
+ idxScan.getTable(),
+ idxScan.getCluster(),
+ idxScan.getTraitSet().replace(RewindabilityTrait.REWINDABLE),
+ idxScan.indexName(),
+ firstIdxValue,
+ idx.collation()
+ );
+
+ call.transformTo(aggr.clone(aggr.getCluster(),
F.asList(newAggrInput)));
+ }
+
+ /** The rule config. */
+ @Value.Immutable
+ public interface Config extends RelRule.Config {
+ /** */
+ IndexMinMaxRule.Config DEFAULT = ImmutableIndexMinMaxRule.Config.of()
+ .withDescription("IndexMinMaxRule")
+ .withOperandSupplier(r -> r.operand(IgniteAggregate.class)
+ .oneInput(i ->
i.operand(AbstractIndexScan.class).anyInputs()));
Review Comment:
Why `AbstractIndexScan` but not `IgniteIndexScan` (physical node)? Using
only physical node will reduce search space in my opinion.
##########
modules/calcite/src/main/java/org/apache/ignite/internal/processors/query/calcite/exec/exp/agg/Accumulators.java:
##########
@@ -973,7 +973,7 @@ private DecimalMinMax(AggregateCall aggCall,
RowHandler<Row> hnd, boolean min) {
private final boolean min;
/** */
- private final Function<IgniteTypeFactory, RelDataType> typeSupplier;
+ private final transient Function<IgniteTypeFactory, RelDataType>
typeSupplier;
Review Comment:
Is it really required? Looks like accumulators is not serialized
##########
modules/core/src/main/java/org/apache/ignite/cache/QueryEntity.java:
##########
@@ -780,7 +780,7 @@ private static void processAnnotationsInClass(boolean key,
Class<?> cls, QueryEn
@Nullable QueryEntityClassProperty parent) {
if (U.isJdk(cls) || QueryUtils.isGeometryClass(cls)) {
if (parent == null && !key && QueryUtils.isSqlType(cls)) { // We
have to index primitive _val.
- String idxName = cls.getSimpleName() + "_" +
QueryUtils.VAL_FIELD_NAME + "_idx";
+ String idxName = QueryUtils.indexName(cls.getSimpleName(),
QueryUtils.VAL_FIELD_NAME);
Review Comment:
1. Local method is enough
2. Looks like redundant check at all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
