This is an automated email from the ASF dual-hosted git repository.
jackie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 679c9e6 Add QueryContext to replace BrokerRequest in the query engine
(#5483)
679c9e6 is described below
commit 679c9e6637eab310020b4bac20f48b44f616919c
Author: Xiaotian (Jackie) Jiang <[email protected]>
AuthorDate: Mon Jun 15 11:00:13 2020 -0700
Add QueryContext to replace BrokerRequest in the query engine (#5483)
Motivation:
For historic reason, BrokerRequest does not support expressions natively.
In order to support expressions (transform), the current solution is to
store the expressions as String and compile them when constructing the
operators.
The problem with this approach is that the same compilation is performed
multiple times on EACH segment. This could cause performance degradation
when each server is hosting lots of segments.
In this PR, we introduced a new Object - QueryContext to encapsulate all
the query related information to be used by the query engine. It is
already fully compiled and can be shared by all the segments.
The reason of introducing the new QueryContext over using the PinotQuery
is that with QueryContext:
- It is not used in the wiring layer so that execution layer can be
decoupled from the wiring layer, and changes for one layer won't affect
the other layer.
- It is very hard to change wiring Object (PinotQuery) because it involves
protocol change, so we should make it as generic as possible to support
future features. Instead, QueryContext can be upgraded along with the
new future support in query engine as needed, and we don't have to make
it very generic, which can help save the overhead of handling generic
Objects.
- In case we need to change the wiring Object (e.g. switch from Thrift to
Protobuf), we don't need to change the whole query engine.
- We can add some helper variables or methods in the context classes which
can be shared for all segments to reduce the repetitive work for each
segment.
This PR only introduces the new QueryContext, the related context classes
and a Util class to get QueryContext from BrokerRequest. The following PRs
will make incremental changes to move the query engine to use the new
QueryContext.
---
.../request/transform/TransformExpressionTree.java | 5 +-
.../org/apache/pinot/pql/parsers/Pql2Compiler.java | 13 +-
.../query/request/context/ExpressionContext.java | 118 ++++++
.../core/query/request/context/FilterContext.java | 116 ++++++
.../query/request/context/FunctionContext.java | 97 +++++
.../request/context/OrderByExpressionContext.java | 74 ++++
.../core/query/request/context/QueryContext.java | 257 ++++++++++++
.../request/context/predicate/EqPredicate.java | 72 ++++
.../request/context/predicate/InPredicate.java | 79 ++++
.../context/predicate/IsNotNullPredicate.java | 66 +++
.../request/context/predicate/IsNullPredicate.java | 66 +++
.../request/context/predicate/NotEqPredicate.java | 72 ++++
.../request/context/predicate/NotInPredicate.java | 79 ++++
.../query/request/context/predicate/Predicate.java | 47 +++
.../request/context/predicate/RangePredicate.java | 125 ++++++
.../context/predicate/RegexpLikePredicate.java | 72 ++++
.../context/predicate/TextMatchPredicate.java | 72 ++++
.../BrokerRequestToQueryContextConverter.java | 433 ++++++++++++++++++++
.../request/context/utils/QueryContextUtils.java | 76 ++++
.../BrokerRequestToQueryContextConverterTest.java | 447 +++++++++++++++++++++
20 files changed, 2379 insertions(+), 7 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/request/transform/TransformExpressionTree.java
b/pinot-common/src/main/java/org/apache/pinot/common/request/transform/TransformExpressionTree.java
index 93d9839..7de0be6 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/request/transform/TransformExpressionTree.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/request/transform/TransformExpressionTree.java
@@ -67,6 +67,9 @@ public class TransformExpressionTree {
return standardizeExpression(((FunctionCallAstNode)
astNode).getExpression());
} else if (astNode instanceof LiteralAstNode) {
// Literal
+ // NOTE: String is treated as column name for backward-compatibility
+ // TODO: This can cause problem for string literals (e.g. in
DistinctCountThetaSketch where we have to add special
+ // handling). Fix this legacy behavior when we migrate to SQL
format.
return ((LiteralAstNode) astNode).getValueAsString();
} else {
throw new IllegalStateException("Cannot get standard expression from " +
astNode.getClass().getSimpleName());
@@ -87,7 +90,7 @@ public class TransformExpressionTree {
_expressionType = ExpressionType.FUNCTION;
_value = ((FunctionCallAstNode) root).getName().toLowerCase();
_children = new ArrayList<>();
- if(root.hasChildren()) {
+ if (root.hasChildren()) {
for (AstNode child : root.getChildren()) {
_children.add(new TransformExpressionTree(child));
}
diff --git
a/pinot-common/src/main/java/org/apache/pinot/pql/parsers/Pql2Compiler.java
b/pinot-common/src/main/java/org/apache/pinot/pql/parsers/Pql2Compiler.java
index 59733dc..193c79c 100644
--- a/pinot-common/src/main/java/org/apache/pinot/pql/parsers/Pql2Compiler.java
+++ b/pinot-common/src/main/java/org/apache/pinot/pql/parsers/Pql2Compiler.java
@@ -70,6 +70,7 @@ public class Pql2Compiler implements AbstractCompiler {
Boolean.valueOf(System.getProperty("pinot.query.converter.validate",
"false"));
public static boolean FAIL_ON_CONVERSION_ERROR =
Boolean.valueOf(System.getProperty("pinot.query.converter.fail_on_error",
"false"));
+
private static class ErrorListener extends BaseErrorListener {
@Override
@@ -149,7 +150,7 @@ public class Pql2Compiler implements AbstractCompiler {
}
}
- public TransformExpressionTree compileToExpressionTree(String expression) {
+ public AstNode parseToAstNode(String expression) {
CharStream charStream = new ANTLRInputStream(expression);
PQL2Lexer lexer = new PQL2Lexer(charStream);
lexer.setTokenFactory(new CommonTokenFactory(true));
@@ -158,13 +159,13 @@ public class Pql2Compiler implements AbstractCompiler {
parser.setErrorHandler(new BailErrorStrategy());
// Parse
- ParseTree parseTree = parser.expression();
-
- ParseTreeWalker walker = new ParseTreeWalker();
Pql2AstListener listener = new Pql2AstListener(expression);
- walker.walk(listener, parseTree);
+ new ParseTreeWalker().walk(listener, parser.expression());
+ return listener.getRootNode();
+ }
- return new TransformExpressionTree(listener.getRootNode());
+ public TransformExpressionTree compileToExpressionTree(String expression) {
+ return new TransformExpressionTree(parseToAstNode(expression));
}
private void validateHavingClause(AstNode rootNode) {
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/ExpressionContext.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/ExpressionContext.java
new file mode 100644
index 0000000..75fc467
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/ExpressionContext.java
@@ -0,0 +1,118 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context;
+
+import java.util.Objects;
+import java.util.Set;
+
+
+/**
+ * The {@code ExpressionContext} class represents an expression in the query.
+ * <p>The expression can be a LITERAL (e.g. 1, "abc"), an IDENTIFIER (e.g.
memberId, timestamp), or a FUNCTION (e.g.
+ * SUM(price), ADD(foo, bar)).
+ * <p>Currently the query engine processes all literals as strings, so we
store literals in string format (1 is stored
+ * as "1").
+ */
+public class ExpressionContext {
+ public enum Type {
+ LITERAL, IDENTIFIER, FUNCTION
+ }
+
+ private final Type _type;
+ private final String _value;
+ private final FunctionContext _function;
+
+ public static ExpressionContext forLiteral(String literal) {
+ return new ExpressionContext(Type.LITERAL, literal, null);
+ }
+
+ public static ExpressionContext forIdentifier(String identifier) {
+ return new ExpressionContext(Type.IDENTIFIER, identifier, null);
+ }
+
+ public static ExpressionContext forFunction(FunctionContext function) {
+ return new ExpressionContext(Type.FUNCTION, null, function);
+ }
+
+ private ExpressionContext(Type type, String value, FunctionContext function)
{
+ _type = type;
+ _value = value;
+ _function = function;
+ }
+
+ public Type getType() {
+ return _type;
+ }
+
+ public String getLiteral() {
+ return _value;
+ }
+
+ public String getIdentifier() {
+ return _value;
+ }
+
+ public FunctionContext getFunction() {
+ return _function;
+ }
+
+ /**
+ * Adds the columns (IDENTIFIER expressions) in the expression to the given
set.
+ */
+ public void getColumns(Set<String> columns) {
+ if (_type == Type.IDENTIFIER) {
+ if (!_value.equals("*")) {
+ columns.add(_value);
+ }
+ } else if (_type == Type.FUNCTION) {
+ _function.getColumns(columns);
+ }
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof ExpressionContext)) {
+ return false;
+ }
+ ExpressionContext that = (ExpressionContext) o;
+ return _type == that._type && Objects.equals(_value, that._value) &&
Objects.equals(_function, that._function);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_type, _value, _function);
+ }
+
+ @Override
+ public String toString() {
+ switch (_type) {
+ case LITERAL:
+ return '\'' + _value + '\'';
+ case IDENTIFIER:
+ return _value;
+ case FUNCTION:
+ return _function.toString();
+ default:
+ throw new IllegalStateException();
+ }
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/FilterContext.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/FilterContext.java
new file mode 100644
index 0000000..22d734a
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/FilterContext.java
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Set;
+import org.apache.pinot.core.query.request.context.predicate.Predicate;
+
+
+/**
+ * The {@code FilterContext} class encapsulates the information of a filter in
the query. Both WHERE clause and HAVING
+ * clause are modeled as a filter.
+ */
+public class FilterContext {
+ public enum Type {
+ AND, OR, PREDICATE
+ }
+
+ private final Type _type;
+
+ // For AND and OR
+ private final List<FilterContext> _children;
+
+ // For Predicate
+ private final Predicate _predicate;
+
+ public FilterContext(Type type, List<FilterContext> children, Predicate
predicate) {
+ _type = type;
+ _children = children;
+ _predicate = predicate;
+ }
+
+ public Type getType() {
+ return _type;
+ }
+
+ public List<FilterContext> getChildren() {
+ return _children;
+ }
+
+ public Predicate getPredicate() {
+ return _predicate;
+ }
+
+ /**
+ * Adds the columns (IDENTIFIER expressions) in the filter to the given set.
+ */
+ public void getColumns(Set<String> columns) {
+ if (_children != null) {
+ for (FilterContext child : _children) {
+ child.getColumns(columns);
+ }
+ } else {
+ _predicate.getLhs().getColumns(columns);
+ }
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof FilterContext)) {
+ return false;
+ }
+ FilterContext that = (FilterContext) o;
+ return _type == that._type && Objects.equals(_children, that._children) &&
Objects
+ .equals(_predicate, that._predicate);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_type, _children, _predicate);
+ }
+
+ @Override
+ public String toString() {
+ switch (_type) {
+ case AND:
+ StringBuilder stringBuilder = new
StringBuilder().append('(').append(_children.get(0).toString());
+ int numChildren = _children.size();
+ for (int i = 1; i < numChildren; i++) {
+ stringBuilder.append(" AND ").append(_children.get(i).toString());
+ }
+ return stringBuilder.append(')').toString();
+ case OR:
+ stringBuilder = new
StringBuilder().append('(').append(_children.get(0).toString());
+ numChildren = _children.size();
+ for (int i = 1; i < numChildren; i++) {
+ stringBuilder.append(" OR ").append(_children.get(i).toString());
+ }
+ return stringBuilder.append(')').toString();
+ case PREDICATE:
+ return _predicate.toString();
+ default:
+ throw new IllegalStateException();
+ }
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/FunctionContext.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/FunctionContext.java
new file mode 100644
index 0000000..2837305
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/FunctionContext.java
@@ -0,0 +1,97 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Set;
+
+
+/**
+ * The {@code FunctionContext} class represents the function in the expression.
+ * <p>Pinot currently supports 2 types of functions: Aggregation (e.g. SUM,
MAX) and Transform (e.g. ADD, SUB).
+ */
+public class FunctionContext {
+ public enum Type {
+ AGGREGATION, TRANSFORM
+ }
+
+ private final Type _type;
+ private final String _functionName;
+ private final List<ExpressionContext> _arguments;
+
+ public FunctionContext(Type type, String functionName,
List<ExpressionContext> arguments) {
+ _type = type;
+ // NOTE: Standardize the function name to lower case
+ _functionName = functionName.toLowerCase();
+ _arguments = arguments;
+ }
+
+ public Type getType() {
+ return _type;
+ }
+
+ public String getFunctionName() {
+ return _functionName;
+ }
+
+ public List<ExpressionContext> getArguments() {
+ return _arguments;
+ }
+
+ /**
+ * Adds the columns (IDENTIFIER expressions) in the function to the given
set.
+ */
+ public void getColumns(Set<String> columns) {
+ for (ExpressionContext argument : _arguments) {
+ argument.getColumns(columns);
+ }
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof FunctionContext)) {
+ return false;
+ }
+ FunctionContext that = (FunctionContext) o;
+ return _type == that._type && Objects.equals(_functionName,
that._functionName) && Objects
+ .equals(_arguments, that._arguments);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_type, _functionName, _arguments);
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder stringBuilder = new StringBuilder(_functionName).append('(');
+ int numArguments = _arguments.size();
+ if (numArguments > 0) {
+ stringBuilder.append(_arguments.get(0).toString());
+ for (int i = 1; i < numArguments; i++) {
+ stringBuilder.append(',').append(_arguments.get(i).toString());
+ }
+ }
+ return stringBuilder.append(')').toString();
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/OrderByExpressionContext.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/OrderByExpressionContext.java
new file mode 100644
index 0000000..3afc604
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/OrderByExpressionContext.java
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context;
+
+import java.util.Objects;
+import java.util.Set;
+
+
+/**
+ * The {@code OrderByExpressionContext} class represents an expression in the
ORDER-BY clause. It encapsulates an
+ * expression and the expected ordering of the expression.
+ */
+public class OrderByExpressionContext {
+ private final ExpressionContext _expression;
+ private final boolean _isAsc;
+
+ public OrderByExpressionContext(ExpressionContext expression, boolean isAsc)
{
+ _expression = expression;
+ _isAsc = isAsc;
+ }
+
+ public ExpressionContext getExpression() {
+ return _expression;
+ }
+
+ public boolean isAsc() {
+ return _isAsc;
+ }
+
+ /**
+ * Adds the columns (IDENTIFIER expressions) in the order-by expression to
the given set.
+ */
+ public void getColumns(Set<String> columns) {
+ _expression.getColumns(columns);
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof OrderByExpressionContext)) {
+ return false;
+ }
+ OrderByExpressionContext that = (OrderByExpressionContext) o;
+ return _isAsc == that._isAsc && Objects.equals(_expression,
that._expression);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_expression, _isAsc);
+ }
+
+ @Override
+ public String toString() {
+ return _expression.toString() + (_isAsc ? " ASC" : " DESC");
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java
new file mode 100644
index 0000000..a6c50cb
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/QueryContext.java
@@ -0,0 +1,257 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context;
+
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+import org.apache.pinot.common.request.BrokerRequest;
+
+
+/**
+ * The {@code QueryContext} class encapsulates all the query related
information extracted from the wiring Object.
+ * <p>The query engine should work on the QueryContext instead of the wiring
Object for the following benefits:
+ * <ul>
+ * <li>
+ * Execution layer can be decoupled from the wiring layer, so that changes
for one layer won't affect the other
+ * layer.
+ * </li>
+ * <li>
+ * It is very hard to change wiring Object because it involves protocol
change, so we should make it as generic as
+ * possible to support future features. Instead, QueryContext is extracted
from the wiring Object within each
+ * Broker/Server, and changing it won't cause any protocol change, so we
can upgrade it along with the new feature
+ * support in query engine as needed. Also, because of this, we don't have
to make QueryContext very generic, which
+ * can help save the overhead of handling generic Objects (e.g. we can
pre-compute the Predicates instead of the
+ * using the generic Expressions).
+ * </li>
+ * <li>
+ * In case we need to change the wiring Object (e.g. switch from Thrift to
Protobuf), we don't need to change the
+ * whole query engine.
+ * </li>
+ * <li>
+ * We can also add some helper variables or methods in the context classes
which can be shared for all segments to
+ * reduce the repetitive work for each segment.
+ * </li>
+ * </ul>
+ */
+public class QueryContext {
+ private final List<ExpressionContext> _selectExpressions;
+ private final Map<ExpressionContext, String> _aliasMap;
+ private final FilterContext _filter;
+ private final List<ExpressionContext> _groupByExpressions;
+ private final List<OrderByExpressionContext> _orderByExpressions;
+ private final FilterContext _havingFilter;
+ private final int _limit;
+ private final int _offset;
+ private final Map<String, String> _queryOptions;
+ private final Map<String, String> _debugOptions;
+
+ // Keep the BrokerRequest to make incremental changes
+ // TODO: Remove it once the whole query engine is using the QueryContext
+ private final BrokerRequest _brokerRequest;
+
+ private QueryContext(List<ExpressionContext> selectExpressions,
Map<ExpressionContext, String> aliasMap,
+ @Nullable FilterContext filter, @Nullable List<ExpressionContext>
groupByExpressions,
+ @Nullable List<OrderByExpressionContext> orderByExpressions, @Nullable
FilterContext havingFilter, int limit,
+ int offset, @Nullable Map<String, String> queryOptions, @Nullable
Map<String, String> debugOptions,
+ BrokerRequest brokerRequest) {
+ _selectExpressions = selectExpressions;
+ _aliasMap = aliasMap;
+ _filter = filter;
+ _groupByExpressions = groupByExpressions;
+ _orderByExpressions = orderByExpressions;
+ _havingFilter = havingFilter;
+ _limit = limit;
+ _offset = offset;
+ _queryOptions = queryOptions;
+ _debugOptions = debugOptions;
+ _brokerRequest = brokerRequest;
+ }
+
+ /**
+ * Returns a list of expressions in the SELECT clause.
+ */
+ public List<ExpressionContext> getSelectExpressions() {
+ return _selectExpressions;
+ }
+
+ /**
+ * Returns the alias of the given expression, or {@code null} if it does not
have an alias.
+ */
+ @Nullable
+ public String getAlias(ExpressionContext expression) {
+ return _aliasMap.get(expression);
+ }
+
+ /**
+ * Returns the filter in the WHERE clause, or {@code null} if there is no
WHERE clause.
+ */
+ @Nullable
+ public FilterContext getFilter() {
+ return _filter;
+ }
+
+ /**
+ * Returns a list of expressions in the GROUP-BY clause, or {@code null} if
there is no GROUP-BY clause.
+ */
+ @Nullable
+ public List<ExpressionContext> getGroupByExpressions() {
+ return _groupByExpressions;
+ }
+
+ /**
+ * Returns a list of order-by expressions in the ORDER-BY clause, or {@code
null} if there is no ORDER-BY clause.
+ */
+ @Nullable
+ public List<OrderByExpressionContext> getOrderByExpressions() {
+ return _orderByExpressions;
+ }
+
+ /**
+ * Returns the filter in the HAVING clause, or {@code null} if there is no
HAVING clause.
+ */
+ @Nullable
+ public FilterContext getHavingFilter() {
+ return _havingFilter;
+ }
+
+ /**
+ * Returns the limit of the query.
+ */
+ public int getLimit() {
+ return _limit;
+ }
+
+ /**
+ * Returns the offset of the query.
+ */
+ public int getOffset() {
+ return _offset;
+ }
+
+ /**
+ * Returns the query options of the query, or {@code null} if not exist.
+ */
+ @Nullable
+ public Map<String, String> getQueryOptions() {
+ return _queryOptions;
+ }
+
+ /**
+ * Returns the debug options of the query, or {@code null} if not exist.
+ */
+ @Nullable
+ public Map<String, String> getDebugOptions() {
+ return _debugOptions;
+ }
+
+ /**
+ * Returns the BrokerRequest where the QueryContext is extracted from.
+ */
+ public BrokerRequest getBrokerRequest() {
+ return _brokerRequest;
+ }
+
+ /**
+ * NOTE: For debugging only.
+ */
+ @Override
+ public String toString() {
+ return "QueryContext{" + "_selectExpressions=" + _selectExpressions + ",
_aliasMap=" + _aliasMap + ", _filter="
+ + _filter + ", _groupByExpressions=" + _groupByExpressions + ",
_orderByExpressions=" + _orderByExpressions
+ + ", _havingFilter=" + _havingFilter + ", _limit=" + _limit + ",
_offset=" + _offset + ", _queryOptions="
+ + _queryOptions + ", _debugOptions=" + _debugOptions + ",
_brokerRequest=" + _brokerRequest + '}';
+ }
+
+ public static class Builder {
+ private List<ExpressionContext> _selectExpressions;
+ private Map<ExpressionContext, String> _aliasMap;
+ private FilterContext _filter;
+ private List<ExpressionContext> _groupByExpressions;
+ private List<OrderByExpressionContext> _orderByExpressions;
+ private FilterContext _havingFilter;
+ private int _limit;
+ private int _offset;
+ private Map<String, String> _queryOptions;
+ private Map<String, String> _debugOptions;
+ private BrokerRequest _brokerRequest;
+
+ public Builder setSelectExpressions(List<ExpressionContext>
selectExpressions) {
+ _selectExpressions = selectExpressions;
+ return this;
+ }
+
+ public Builder setAliasMap(Map<ExpressionContext, String> aliasMap) {
+ _aliasMap = aliasMap;
+ return this;
+ }
+
+ public Builder setFilter(@Nullable FilterContext filter) {
+ _filter = filter;
+ return this;
+ }
+
+ public Builder setGroupByExpressions(@Nullable List<ExpressionContext>
groupByExpressions) {
+ _groupByExpressions = groupByExpressions;
+ return this;
+ }
+
+ public Builder setOrderByExpressions(@Nullable
List<OrderByExpressionContext> orderByExpressions) {
+ _orderByExpressions = orderByExpressions;
+ return this;
+ }
+
+ public Builder setHavingFilter(@Nullable FilterContext havingFilter) {
+ _havingFilter = havingFilter;
+ return this;
+ }
+
+ public Builder setLimit(int limit) {
+ _limit = limit;
+ return this;
+ }
+
+ public Builder setOffset(int offset) {
+ _offset = offset;
+ return this;
+ }
+
+ public Builder setQueryOptions(@Nullable Map<String, String> queryOptions)
{
+ _queryOptions = queryOptions;
+ return this;
+ }
+
+ public Builder setDebugOptions(@Nullable Map<String, String> debugOptions)
{
+ _debugOptions = debugOptions;
+ return this;
+ }
+
+ public Builder setBrokerRequest(BrokerRequest brokerRequest) {
+ _brokerRequest = brokerRequest;
+ return this;
+ }
+
+ public QueryContext build() {
+ // TODO: Add validation logic here
+
+ return new QueryContext(_selectExpressions, _aliasMap, _filter,
_groupByExpressions, _orderByExpressions,
+ _havingFilter, _limit, _offset, _queryOptions, _debugOptions,
_brokerRequest);
+ }
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/EqPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/EqPredicate.java
new file mode 100644
index 0000000..032f4af
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/EqPredicate.java
@@ -0,0 +1,72 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for EQ.
+ */
+public class EqPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+ private final String _value;
+
+ public EqPredicate(ExpressionContext lhs, String value) {
+ _lhs = lhs;
+ _value = value;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.EQ;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public String getValue() {
+ return _value;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof EqPredicate)) {
+ return false;
+ }
+ EqPredicate that = (EqPredicate) o;
+ return Objects.equals(_lhs, that._lhs) && Objects.equals(_value,
that._value);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _value);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + " = '" + _value + '\'';
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/InPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/InPredicate.java
new file mode 100644
index 0000000..1809563
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/InPredicate.java
@@ -0,0 +1,79 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.List;
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for IN.
+ */
+public class InPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+ private final List<String> _values;
+
+ public InPredicate(ExpressionContext lhs, List<String> values) {
+ _lhs = lhs;
+ _values = values;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.IN;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public List<String> getValues() {
+ return _values;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof InPredicate)) {
+ return false;
+ }
+ InPredicate that = (InPredicate) o;
+ return Objects.equals(_lhs, that._lhs) && Objects.equals(_values,
that._values);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _values);
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder stringBuilder =
+ new StringBuilder(_lhs.toString()).append(" IN
('").append(_values.get(0)).append('\'');
+ int numValues = _values.size();
+ for (int i = 1; i < numValues; i++) {
+ stringBuilder.append(",'").append(_values.get(i)).append('\'');
+ }
+ return stringBuilder.append(')').toString();
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/IsNotNullPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/IsNotNullPredicate.java
new file mode 100644
index 0000000..36907af
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/IsNotNullPredicate.java
@@ -0,0 +1,66 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for IS_NOT_NULL.
+ */
+public class IsNotNullPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+
+ public IsNotNullPredicate(ExpressionContext lhs) {
+ _lhs = lhs;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.IS_NOT_NULL;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof IsNotNullPredicate)) {
+ return false;
+ }
+ IsNotNullPredicate that = (IsNotNullPredicate) o;
+ return Objects.equals(_lhs, that._lhs);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + "IS NOT NULL";
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/IsNullPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/IsNullPredicate.java
new file mode 100644
index 0000000..cc11724
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/IsNullPredicate.java
@@ -0,0 +1,66 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for IS_NULL.
+ */
+public class IsNullPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+
+ public IsNullPredicate(ExpressionContext lhs) {
+ _lhs = lhs;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.IS_NULL;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof IsNullPredicate)) {
+ return false;
+ }
+ IsNullPredicate that = (IsNullPredicate) o;
+ return Objects.equals(_lhs, that._lhs);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + "IS NULL";
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/NotEqPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/NotEqPredicate.java
new file mode 100644
index 0000000..40d13c5
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/NotEqPredicate.java
@@ -0,0 +1,72 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for NOT_EQ.
+ */
+public class NotEqPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+ private final String _value;
+
+ public NotEqPredicate(ExpressionContext lhs, String value) {
+ _lhs = lhs;
+ _value = value;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.NOT_EQ;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public String getValue() {
+ return _value;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof NotEqPredicate)) {
+ return false;
+ }
+ NotEqPredicate that = (NotEqPredicate) o;
+ return Objects.equals(_lhs, that._lhs) && Objects.equals(_value,
that._value);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _value);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + " != '" + _value + '\'';
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/NotInPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/NotInPredicate.java
new file mode 100644
index 0000000..d60309e
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/NotInPredicate.java
@@ -0,0 +1,79 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.List;
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for NOT_IN.
+ */
+public class NotInPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+ private final List<String> _values;
+
+ public NotInPredicate(ExpressionContext lhs, List<String> values) {
+ _lhs = lhs;
+ _values = values;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.NOT_IN;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public List<String> getValues() {
+ return _values;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof NotInPredicate)) {
+ return false;
+ }
+ NotInPredicate that = (NotInPredicate) o;
+ return Objects.equals(_lhs, that._lhs) && Objects.equals(_values,
that._values);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _values);
+ }
+
+ @Override
+ public String toString() {
+ StringBuilder stringBuilder =
+ new StringBuilder(_lhs.toString()).append(" NOT IN
('").append(_values.get(0)).append('\'');
+ int numValues = _values.size();
+ for (int i = 1; i < numValues; i++) {
+ stringBuilder.append(",'").append(_values.get(i)).append('\'');
+ }
+ return stringBuilder.append(')').toString();
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/Predicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/Predicate.java
new file mode 100644
index 0000000..203e950
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/Predicate.java
@@ -0,0 +1,47 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * The {@code Predicate} class represents the predicate in the filter.
+ * <p>Currently the query engine only accepts string literals as the
right-hand side of the predicate, so we store the
+ * right-hand side of the predicate as string or list of strings.
+ */
+public interface Predicate {
+ enum Type {
+ EQ, NOT_EQ, IN, NOT_IN, RANGE, REGEXP_LIKE, TEXT_MATCH, IS_NULL,
IS_NOT_NULL;
+
+ public boolean isExclusive() {
+ return this == NOT_EQ || this == NOT_IN || this == IS_NOT_NULL;
+ }
+ }
+
+ /**
+ * Returns the type of the predicate.
+ */
+ Type getType();
+
+ /**
+ * Returns the left-hand side expression of the predicate.
+ */
+ ExpressionContext getLhs();
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/RangePredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/RangePredicate.java
new file mode 100644
index 0000000..0396283
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/RangePredicate.java
@@ -0,0 +1,125 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for RANGE.
+ * <p>Pinot uses RANGE to represent '>', '>=', '<', '<=', BETWEEN so that
intersection of multiple ranges can be merged.
+ */
+public class RangePredicate implements Predicate {
+ public static final String DELIMITER = "\t\t";
+ public static final char LOWER_INCLUSIVE = '[';
+ public static final char LOWER_EXCLUSIVE = '(';
+ public static final char UPPER_INCLUSIVE = ']';
+ public static final char UPPER_EXCLUSIVE = ')';
+ public static final String UNBOUNDED = "*";
+
+ private final ExpressionContext _lhs;
+ private final boolean _lowerInclusive;
+ private final String _lowerBound;
+ private final boolean _upperInclusive;
+ private final String _upperBound;
+
+ /**
+ * The range is formatted as 5 parts:
+ * <ul>
+ * <li>Lower inclusive '[' or exclusive '('</li>
+ * <li>Lower bound ('*' for unbounded)</li>
+ * <li>Delimiter ("\t\t")</li>
+ * <li>Upper bound ('*' for unbounded)</li>
+ * <li>Upper inclusive ']' or exclusive ')'</li>
+ * </ul>
+ */
+ public RangePredicate(ExpressionContext lhs, String range) {
+ _lhs = lhs;
+ String[] split = StringUtils.split(range, DELIMITER);
+ String lower = split[0];
+ String upper = split[1];
+ _lowerInclusive = lower.charAt(0) == LOWER_INCLUSIVE;
+ _lowerBound = lower.substring(1);
+ int upperLength = upper.length();
+ _upperInclusive = upper.charAt(upperLength - 1) == UPPER_INCLUSIVE;
+ _upperBound = upper.substring(0, upperLength - 1);
+ }
+
+ public RangePredicate(ExpressionContext lhs, boolean lowerInclusive, String
lowerBound, boolean upperInclusive,
+ String upperBound) {
+ _lhs = lhs;
+ _lowerInclusive = lowerInclusive;
+ _lowerBound = lowerBound;
+ _upperInclusive = upperInclusive;
+ _upperBound = upperBound;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.RANGE;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public boolean isLowerInclusive() {
+ return _lowerInclusive;
+ }
+
+ public String getLowerBound() {
+ return _lowerBound;
+ }
+
+ public boolean isUpperInclusive() {
+ return _upperInclusive;
+ }
+
+ public String getUpperBound() {
+ return _upperBound;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof RangePredicate)) {
+ return false;
+ }
+ RangePredicate that = (RangePredicate) o;
+ return _lowerInclusive == that._lowerInclusive && _upperInclusive ==
that._upperInclusive && Objects
+ .equals(_lhs, that._lhs) && Objects.equals(_lowerBound,
that._lowerBound) && Objects
+ .equals(_upperBound, that._upperBound);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _lowerInclusive, _lowerBound, _upperInclusive,
_upperBound);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + " IN RANGE " + (_lowerInclusive ? LOWER_INCLUSIVE :
LOWER_EXCLUSIVE) + _lowerBound + ',' + _upperBound
+ + (_upperInclusive ? UPPER_INCLUSIVE : UPPER_EXCLUSIVE);
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/RegexpLikePredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/RegexpLikePredicate.java
new file mode 100644
index 0000000..c4eb655
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/RegexpLikePredicate.java
@@ -0,0 +1,72 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for REGEXP_LIKE.
+ */
+public class RegexpLikePredicate implements Predicate {
+ private final ExpressionContext _lhs;
+ private final String _value;
+
+ public RegexpLikePredicate(ExpressionContext lhs, String value) {
+ _lhs = lhs;
+ _value = value;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.REGEXP_LIKE;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public String getValue() {
+ return _value;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof RegexpLikePredicate)) {
+ return false;
+ }
+ RegexpLikePredicate that = (RegexpLikePredicate) o;
+ return Objects.equals(_lhs, that._lhs) && Objects.equals(_value,
that._value);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _value);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + " REGEXP_LIKE '" + _value + '\'';
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/TextMatchPredicate.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/TextMatchPredicate.java
new file mode 100644
index 0000000..23944a2
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/predicate/TextMatchPredicate.java
@@ -0,0 +1,72 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.predicate;
+
+import java.util.Objects;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+
+
+/**
+ * Predicate for TEXT_MATCH.
+ */
+public class TextMatchPredicate implements Predicate {
+ private final ExpressionContext _lhs;
+ private final String _value;
+
+ public TextMatchPredicate(ExpressionContext lhs, String value) {
+ _lhs = lhs;
+ _value = value;
+ }
+
+ @Override
+ public Type getType() {
+ return Type.TEXT_MATCH;
+ }
+
+ @Override
+ public ExpressionContext getLhs() {
+ return _lhs;
+ }
+
+ public String getValue() {
+ return _value;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof TextMatchPredicate)) {
+ return false;
+ }
+ TextMatchPredicate that = (TextMatchPredicate) o;
+ return Objects.equals(_lhs, that._lhs) && Objects.equals(_value,
that._value);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(_lhs, _value);
+ }
+
+ @Override
+ public String toString() {
+ return _lhs + " TEXT_MATCH '" + _value + '\'';
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/BrokerRequestToQueryContextConverter.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/BrokerRequestToQueryContextConverter.java
new file mode 100644
index 0000000..261981e
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/BrokerRequestToQueryContextConverter.java
@@ -0,0 +1,433 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.utils;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import org.apache.commons.collections.CollectionUtils;
+import org.apache.pinot.common.function.AggregationFunctionType;
+import org.apache.pinot.common.function.FunctionDefinitionRegistry;
+import org.apache.pinot.common.request.AggregationInfo;
+import org.apache.pinot.common.request.BrokerRequest;
+import org.apache.pinot.common.request.Expression;
+import org.apache.pinot.common.request.ExpressionType;
+import org.apache.pinot.common.request.FilterOperator;
+import org.apache.pinot.common.request.Function;
+import org.apache.pinot.common.request.GroupBy;
+import org.apache.pinot.common.request.PinotQuery;
+import org.apache.pinot.common.request.Selection;
+import org.apache.pinot.common.request.SelectionSort;
+import org.apache.pinot.common.utils.request.FilterQueryTree;
+import org.apache.pinot.common.utils.request.RequestUtils;
+import org.apache.pinot.core.query.exception.BadQueryRequestException;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+import org.apache.pinot.core.query.request.context.FilterContext;
+import org.apache.pinot.core.query.request.context.FunctionContext;
+import org.apache.pinot.core.query.request.context.OrderByExpressionContext;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.predicate.EqPredicate;
+import org.apache.pinot.core.query.request.context.predicate.InPredicate;
+import
org.apache.pinot.core.query.request.context.predicate.IsNotNullPredicate;
+import org.apache.pinot.core.query.request.context.predicate.IsNullPredicate;
+import org.apache.pinot.core.query.request.context.predicate.NotEqPredicate;
+import org.apache.pinot.core.query.request.context.predicate.NotInPredicate;
+import org.apache.pinot.core.query.request.context.predicate.RangePredicate;
+import
org.apache.pinot.core.query.request.context.predicate.RegexpLikePredicate;
+import
org.apache.pinot.core.query.request.context.predicate.TextMatchPredicate;
+import org.apache.pinot.pql.parsers.Pql2Compiler;
+import org.apache.pinot.pql.parsers.pql2.ast.AstNode;
+import org.apache.pinot.pql.parsers.pql2.ast.FilterKind;
+import org.apache.pinot.pql.parsers.pql2.ast.FunctionCallAstNode;
+import org.apache.pinot.pql.parsers.pql2.ast.IdentifierAstNode;
+import org.apache.pinot.pql.parsers.pql2.ast.LiteralAstNode;
+
+
+public class BrokerRequestToQueryContextConverter {
+ private BrokerRequestToQueryContextConverter() {
+ }
+
+ private static final Pql2Compiler PQL_COMPILER = new Pql2Compiler();
+
+ /**
+ * Converts the given BrokerRequest to a QueryContext.
+ * <p>Use PinotQuery if available to avoid the unnecessary parsing of the
expression.
+ * <p>TODO: We cannot use PinotQuery to generate the {@code filter} because
{@code BrokerRequestOptimizer} only
+ * optimizes the BrokerRequest but not the PinotQuery.
+ */
+ public static QueryContext convert(BrokerRequest brokerRequest) {
+ PinotQuery pinotQuery = brokerRequest.getPinotQuery();
+
+ List<ExpressionContext> selectExpressions;
+ Map<ExpressionContext, String> aliasMap;
+ List<ExpressionContext> groupByExpressions = null;
+ int limit;
+ int offset = 0;
+ if (pinotQuery != null) {
+ aliasMap = new HashMap<>();
+ List<Expression> selectList = pinotQuery.getSelectList();
+ int numExpressions = selectList.size();
+ List<ExpressionContext> aggregationExpressions = new
ArrayList<>(numExpressions);
+ List<ExpressionContext> nonAggregationExpressions = new
ArrayList<>(numExpressions);
+ for (Expression thriftExpression : selectList) {
+ ExpressionContext expression;
+ if (thriftExpression.getType() == ExpressionType.FUNCTION &&
thriftExpression.getFunctionCall().getOperator()
+ .equalsIgnoreCase("AS")) {
+ // Handle alias
+ List<Expression> operands =
thriftExpression.getFunctionCall().getOperands();
+ expression = getExpression(operands.get(0));
+ aliasMap.put(expression, operands.get(1).getIdentifier().getName());
+ } else {
+ expression = getExpression(thriftExpression);
+ }
+ if (expression.getType() == ExpressionContext.Type.FUNCTION
+ && expression.getFunction().getType() ==
FunctionContext.Type.AGGREGATION) {
+ aggregationExpressions.add(expression);
+ } else {
+ nonAggregationExpressions.add(expression);
+ }
+ }
+ if (aggregationExpressions.isEmpty()) {
+ // NOTE: Pinot ignores the GROUP-BY clause when there is no
aggregation expressions in the SELECT clause.
+ selectExpressions = nonAggregationExpressions;
+ } else {
+ // NOTE: Pinot ignores the non-aggregation expressions when there are
aggregation expressions in the SELECT
+ // clause. E.g. SELECT a, SUM(b) -> SELECT SUM(b).
+ selectExpressions = aggregationExpressions;
+
+ List<Expression> groupByList = pinotQuery.getGroupByList();
+ if (CollectionUtils.isNotEmpty(groupByList)) {
+ groupByExpressions = new ArrayList<>(groupByList.size());
+ for (Expression thriftExpression : groupByList) {
+ groupByExpressions.add(getExpression(thriftExpression));
+ }
+ }
+ }
+ limit = pinotQuery.getLimit();
+ offset = pinotQuery.getOffset();
+ } else {
+ // NOTE: Alias is not supported for PQL queries.
+ aliasMap = Collections.emptyMap();
+ Selection selections = brokerRequest.getSelections();
+ if (selections != null) {
+ // Selection query
+ List<String> selectionColumns = selections.getSelectionColumns();
+ selectExpressions = new ArrayList<>(selectionColumns.size());
+ for (String expression : selectionColumns) {
+ selectExpressions.add(getExpression(expression));
+ }
+ // NOTE: Pinot ignores the GROUP-BY clause for selection queries.
+ groupByExpressions = null;
+ limit = brokerRequest.getLimit();
+ offset = selections.getOffset();
+ } else {
+ // Aggregation query
+ List<AggregationInfo> aggregationsInfo =
brokerRequest.getAggregationsInfo();
+ selectExpressions = new ArrayList<>(aggregationsInfo.size());
+ for (AggregationInfo aggregationInfo : aggregationsInfo) {
+ String functionName = aggregationInfo.getAggregationType();
+ List<String> stringExpressions = aggregationInfo.getExpressions();
+ int numArguments = stringExpressions.size();
+ List<ExpressionContext> arguments = new ArrayList<>(numArguments);
+ if
(functionName.equalsIgnoreCase(AggregationFunctionType.DISTINCTCOUNTTHETASKETCH.getName()))
{
+ // NOTE: For DistinctCountThetaSketch, because of the legacy
behavior of PQL compiler treating string
+ // literal as identifier in aggregation, here we treat all
expressions except for the first one as
+ // string literal.
+ arguments.add(getExpression(stringExpressions.get(0)));
+ for (int i = 1; i < numArguments; i++) {
+
arguments.add(ExpressionContext.forLiteral(stringExpressions.get(i)));
+ }
+ } else {
+ for (String expression : stringExpressions) {
+ arguments.add(getExpression(expression));
+ }
+ }
+ FunctionContext function = new
FunctionContext(FunctionContext.Type.AGGREGATION, functionName, arguments);
+ selectExpressions.add(ExpressionContext.forFunction(function));
+ }
+
+ GroupBy groupBy = brokerRequest.getGroupBy();
+ if (groupBy != null) {
+ // Aggregation group-by query
+ List<String> stringExpressions = groupBy.getExpressions();
+ groupByExpressions = new ArrayList<>(stringExpressions.size());
+ for (String stringExpression : stringExpressions) {
+ groupByExpressions.add(getExpression(stringExpression));
+ }
+
+ // NOTE: Use TOP in GROUP-BY clause as LIMIT for
backward-compatibility.
+ limit = (int) groupBy.getTopN();
+ } else {
+ limit = brokerRequest.getLimit();
+ }
+ }
+ }
+
+ List<OrderByExpressionContext> orderByExpressions = null;
+ if (pinotQuery != null) {
+ List<Expression> orderByList = pinotQuery.getOrderByList();
+ if (CollectionUtils.isNotEmpty(orderByList)) {
+ orderByExpressions = new ArrayList<>(orderByList.size());
+ for (Expression orderBy : orderByList) {
+ // NOTE: Order-by is always a Function with the ordering of the
Expression
+ Function thriftFunction = orderBy.getFunctionCall();
+ boolean isAsc = thriftFunction.getOperator().equalsIgnoreCase("ASC");
+ ExpressionContext expression =
getExpression(thriftFunction.getOperands().get(0));
+ orderByExpressions.add(new OrderByExpressionContext(expression,
isAsc));
+ }
+ }
+ } else {
+ List<SelectionSort> orderBy = brokerRequest.getOrderBy();
+ if (CollectionUtils.isNotEmpty(orderBy)) {
+ orderByExpressions = new ArrayList<>(orderBy.size());
+ for (SelectionSort selectionSort : orderBy) {
+ orderByExpressions
+ .add(new
OrderByExpressionContext(getExpression(selectionSort.getColumn()),
selectionSort.isIsAsc()));
+ }
+ }
+ }
+
+ // NOTE: Always use BrokerRequest to generate filter because
BrokerRequestOptimizer only optimizes the BrokerRequest
+ // but not the PinotQuery.
+ FilterContext filter = null;
+ FilterQueryTree root = RequestUtils.generateFilterQueryTree(brokerRequest);
+ if (root != null) {
+ filter = getFilter(root);
+ }
+
+ // NOTE: Always use PinotQuery to generate HAVING filter because PQL does
not support HAVING clause.
+ FilterContext havingFilter = null;
+ if (pinotQuery != null) {
+ Expression havingExpression = pinotQuery.getHavingExpression();
+ if (havingExpression != null) {
+ havingFilter = getFilter(havingExpression);
+ }
+ }
+
+ return new
QueryContext.Builder().setSelectExpressions(selectExpressions).setAliasMap(aliasMap).setFilter(filter)
+
.setGroupByExpressions(groupByExpressions).setOrderByExpressions(orderByExpressions)
+ .setHavingFilter(havingFilter).setLimit(limit).setOffset(offset)
+
.setQueryOptions(brokerRequest.getQueryOptions()).setDebugOptions(brokerRequest.getDebugOptions())
+ .setBrokerRequest(brokerRequest).build();
+ }
+
+ private static ExpressionContext getExpression(Expression thriftExpression) {
+ switch (thriftExpression.getType()) {
+ case LITERAL:
+ return
ExpressionContext.forLiteral(thriftExpression.getLiteral().getFieldValue().toString());
+ case IDENTIFIER:
+ return
ExpressionContext.forIdentifier(thriftExpression.getIdentifier().getName());
+ case FUNCTION:
+ return
ExpressionContext.forFunction(getFunction(thriftExpression.getFunctionCall()));
+ default:
+ throw new IllegalStateException();
+ }
+ }
+
+ private static FunctionContext getFunction(Function thriftFunction) {
+ String functionName = thriftFunction.getOperator();
+ if
(functionName.equalsIgnoreCase(AggregationFunctionType.COUNT.getName())) {
+ // NOTE: COUNT always take one single argument "*"
+ return new FunctionContext(FunctionContext.Type.AGGREGATION,
AggregationFunctionType.COUNT.getName(),
+ Collections.singletonList(ExpressionContext.forIdentifier("*")));
+ }
+ FunctionContext.Type functionType =
+ FunctionDefinitionRegistry.isAggFunc(functionName) ?
FunctionContext.Type.AGGREGATION
+ : FunctionContext.Type.TRANSFORM;
+ List<Expression> operands = thriftFunction.getOperands();
+ List<ExpressionContext> arguments = new ArrayList<>(operands.size());
+ for (Expression operand : operands) {
+ arguments.add(getExpression(operand));
+ }
+ return new FunctionContext(functionType, functionName, arguments);
+ }
+
+ private static ExpressionContext getExpression(String stringExpression) {
+ if (stringExpression.equals("*")) {
+ // For 'SELECT *' and 'SELECT COUNT(*)'
+ return ExpressionContext.forIdentifier("*");
+ } else {
+ return getExpression(PQL_COMPILER.parseToAstNode(stringExpression));
+ }
+ }
+
+ private static ExpressionContext getExpression(AstNode astNode) {
+ if (astNode instanceof IdentifierAstNode) {
+ return ExpressionContext.forIdentifier(((IdentifierAstNode)
astNode).getName());
+ }
+ if (astNode instanceof FunctionCallAstNode) {
+ return ExpressionContext.forFunction(getFunction((FunctionCallAstNode)
astNode));
+ }
+ if (astNode instanceof LiteralAstNode) {
+ return ExpressionContext.forLiteral(((LiteralAstNode)
astNode).getValueAsString());
+ }
+ throw new IllegalStateException();
+ }
+
+ private static FunctionContext getFunction(FunctionCallAstNode astNode) {
+ String functionName = astNode.getName();
+ FunctionContext.Type functionType =
+ FunctionDefinitionRegistry.isAggFunc(functionName) ?
FunctionContext.Type.AGGREGATION
+ : FunctionContext.Type.TRANSFORM;
+ List<? extends AstNode> children = astNode.getChildren();
+ List<ExpressionContext> arguments = new ArrayList<>(children.size());
+ for (AstNode child : children) {
+ arguments.add(getExpression(child));
+ }
+ return new FunctionContext(functionType, functionName, arguments);
+ }
+
+ private static FilterContext getFilter(FilterQueryTree node) {
+ FilterOperator filterOperator = node.getOperator();
+ switch (filterOperator) {
+ case AND:
+ List<FilterQueryTree> childNodes = node.getChildren();
+ List<FilterContext> children = new ArrayList<>(childNodes.size());
+ for (FilterQueryTree childNode : childNodes) {
+ children.add(getFilter(childNode));
+ }
+ return new FilterContext(FilterContext.Type.AND, children, null);
+ case OR:
+ childNodes = node.getChildren();
+ children = new ArrayList<>(childNodes.size());
+ for (FilterQueryTree childNode : childNodes) {
+ children.add(getFilter(childNode));
+ }
+ return new FilterContext(FilterContext.Type.OR, children, null);
+ case EQUALITY:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new EqPredicate(getExpression(node.getColumn()),
node.getValue().get(0)));
+ case NOT:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new NotEqPredicate(getExpression(node.getColumn()),
node.getValue().get(0)));
+ case IN:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new InPredicate(getExpression(node.getColumn()), node.getValue()));
+ case NOT_IN:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new NotInPredicate(getExpression(node.getColumn()),
node.getValue()));
+ case RANGE:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(getExpression(node.getColumn()),
node.getValue().get(0)));
+ case REGEXP_LIKE:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RegexpLikePredicate(getExpression(node.getColumn()),
node.getValue().get(0)));
+ case TEXT_MATCH:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new TextMatchPredicate(getExpression(node.getColumn()),
node.getValue().get(0)));
+ case IS_NULL:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new IsNullPredicate(getExpression(node.getColumn())));
+ case IS_NOT_NULL:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new IsNotNullPredicate(getExpression(node.getColumn())));
+ default:
+ throw new IllegalStateException();
+ }
+ }
+
+ /**
+ * NOTE: Currently the query engine only accepts string literals as the
right-hand side of the predicate, so we always
+ * convert the right-hand side expressions into strings.
+ */
+ private static FilterContext getFilter(Expression thriftExpression) {
+ Function thriftFunction = thriftExpression.getFunctionCall();
+ FilterKind filterKind = FilterKind.valueOf(thriftFunction.getOperator());
+ List<Expression> operands = thriftFunction.getOperands();
+ int numOperands = operands.size();
+ switch (filterKind) {
+ case AND:
+ List<FilterContext> children = new ArrayList<>(numOperands);
+ for (Expression operand : operands) {
+ children.add(getFilter(operand));
+ }
+ return new FilterContext(FilterContext.Type.AND, children, null);
+ case OR:
+ children = new ArrayList<>(numOperands);
+ for (Expression operand : operands) {
+ children.add(getFilter(operand));
+ }
+ return new FilterContext(FilterContext.Type.OR, children, null);
+ case EQUALS:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new EqPredicate(getExpression(operands.get(0)),
getStringValue(operands.get(1))));
+ case NOT_EQUALS:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new NotEqPredicate(getExpression(operands.get(0)),
getStringValue(operands.get(1))));
+ case IN:
+ List<String> values = new ArrayList<>(numOperands - 1);
+ for (int i = 1; i < numOperands; i++) {
+ values.add(getStringValue(operands.get(i)));
+ }
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new InPredicate(getExpression(operands.get(0)), values));
+ case NOT_IN:
+ values = new ArrayList<>(numOperands - 1);
+ for (int i = 1; i < numOperands; i++) {
+ values.add(getStringValue(operands.get(i)));
+ }
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new NotInPredicate(getExpression(operands.get(0)), values));
+ case GREATER_THAN:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(getExpression(operands.get(0)), false,
getStringValue(operands.get(1)), false,
+ RangePredicate.UNBOUNDED));
+ case GREATER_THAN_OR_EQUAL:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(getExpression(operands.get(0)), true,
getStringValue(operands.get(1)), false,
+ RangePredicate.UNBOUNDED));
+ case LESS_THAN:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(getExpression(operands.get(0)), false,
RangePredicate.UNBOUNDED, false,
+ getStringValue(operands.get(1))));
+ case LESS_THAN_OR_EQUAL:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(getExpression(operands.get(0)), false,
RangePredicate.UNBOUNDED, true,
+ getStringValue(operands.get(1))));
+ case BETWEEN:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(getExpression(operands.get(0)), true,
getStringValue(operands.get(1)), true,
+ getStringValue(operands.get(2))));
+ case REGEXP_LIKE:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new RegexpLikePredicate(getExpression(operands.get(0)),
getStringValue(operands.get(1))));
+ case TEXT_MATCH:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new TextMatchPredicate(getExpression(operands.get(0)),
getStringValue(operands.get(1))));
+ case IS_NULL:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new IsNullPredicate(getExpression(operands.get(0))));
+ case IS_NOT_NULL:
+ return new FilterContext(FilterContext.Type.PREDICATE, null,
+ new IsNotNullPredicate(getExpression(operands.get(0))));
+ default:
+ throw new IllegalStateException();
+ }
+ }
+
+ private static String getStringValue(Expression thriftExpression) {
+ if (thriftExpression.getType() != ExpressionType.LITERAL) {
+ throw new BadQueryRequestException(
+ "Pinot does not support column or expression on the right-hand side
of the predicate");
+ }
+ return thriftExpression.getLiteral().getFieldValue().toString();
+ }
+}
diff --git
a/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java
new file mode 100644
index 0000000..2d0972d
--- /dev/null
+++
b/pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java
@@ -0,0 +1,76 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.utils;
+
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+import org.apache.pinot.core.query.request.context.FilterContext;
+import org.apache.pinot.core.query.request.context.FunctionContext;
+import org.apache.pinot.core.query.request.context.OrderByExpressionContext;
+import org.apache.pinot.core.query.request.context.QueryContext;
+
+
+public class QueryContextUtils {
+ private QueryContextUtils() {
+ }
+
+ /**
+ * Returns all the columns (IDENTIFIER expressions) in the given query.
+ */
+ public static Set<String> getAllColumns(QueryContext query) {
+ Set<String> columns = new HashSet<>();
+
+ for (ExpressionContext expression : query.getSelectExpressions()) {
+ expression.getColumns(columns);
+ }
+ FilterContext filter = query.getFilter();
+ if (filter != null) {
+ filter.getColumns(columns);
+ }
+ List<ExpressionContext> groupByExpressions = query.getGroupByExpressions();
+ if (groupByExpressions != null) {
+ for (ExpressionContext expression : groupByExpressions) {
+ expression.getColumns(columns);
+ }
+ }
+ List<OrderByExpressionContext> orderByExpressions =
query.getOrderByExpressions();
+ if (orderByExpressions != null) {
+ for (OrderByExpressionContext orderByExpression : orderByExpressions) {
+ orderByExpression.getColumns(columns);
+ }
+ }
+ FilterContext havingFilter = query.getHavingFilter();
+ if (havingFilter != null) {
+ havingFilter.getColumns(columns);
+ }
+
+ return columns;
+ }
+
+ /**
+ * Returns {@code true} if the given query is an aggregation query, {@code
false} otherwise.
+ */
+ public static boolean isAggregationQuery(QueryContext query) {
+ ExpressionContext firstSelectExpression =
query.getSelectExpressions().get(0);
+ return firstSelectExpression.getType() == ExpressionContext.Type.FUNCTION
+ && firstSelectExpression.getFunction().getType() ==
FunctionContext.Type.AGGREGATION;
+ }
+}
diff --git
a/pinot-core/src/test/java/org/apache/pinot/core/query/request/context/utils/BrokerRequestToQueryContextConverterTest.java
b/pinot-core/src/test/java/org/apache/pinot/core/query/request/context/utils/BrokerRequestToQueryContextConverterTest.java
new file mode 100644
index 0000000..2e1b88e
--- /dev/null
+++
b/pinot-core/src/test/java/org/apache/pinot/core/query/request/context/utils/BrokerRequestToQueryContextConverterTest.java
@@ -0,0 +1,447 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.request.context.utils;
+
+import java.io.BufferedReader;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.pinot.core.query.request.context.ExpressionContext;
+import org.apache.pinot.core.query.request.context.FilterContext;
+import org.apache.pinot.core.query.request.context.FunctionContext;
+import org.apache.pinot.core.query.request.context.OrderByExpressionContext;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.predicate.InPredicate;
+import org.apache.pinot.core.query.request.context.predicate.RangePredicate;
+import
org.apache.pinot.core.query.request.context.predicate.TextMatchPredicate;
+import org.apache.pinot.pql.parsers.Pql2Compiler;
+import org.apache.pinot.sql.parsers.CalciteSqlCompiler;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.*;
+
+
+public class BrokerRequestToQueryContextConverterTest {
+ private static final Pql2Compiler PQL_COMPILER = new Pql2Compiler();
+ private static final CalciteSqlCompiler SQL_COMPILER = new
CalciteSqlCompiler();
+
+ @Test
+ public void testHardcodedQueries() {
+ // Select *
+ {
+ String query = "SELECT * FROM testTable";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 1);
+ assertEquals(selectExpressions.get(0),
ExpressionContext.forIdentifier("*"));
+ assertEquals(selectExpressions.get(0).toString(), "*");
+ assertNull(queryContext.getFilter());
+ assertNull(queryContext.getGroupByExpressions());
+ assertNull(queryContext.getOrderByExpressions());
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 10);
+ assertEquals(queryContext.getOffset(), 0);
+ assertTrue(QueryContextUtils.getAllColumns(queryContext).isEmpty());
+ assertFalse(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Select COUNT(*)
+ {
+ String query = "SELECT COUNT(*) FROM testTable";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 1);
+ assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "count",
+
Collections.singletonList(ExpressionContext.forIdentifier("*")))));
+ assertEquals(selectExpressions.get(0).toString(), "count(*)");
+ assertNull(queryContext.getFilter());
+ assertNull(queryContext.getGroupByExpressions());
+ assertNull(queryContext.getOrderByExpressions());
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 10);
+ assertEquals(queryContext.getOffset(), 0);
+ assertTrue(QueryContextUtils.getAllColumns(queryContext).isEmpty());
+ assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Order-by
+ {
+ String query = "SELECT foo, bar FROM testTable ORDER BY bar ASC, foo
DESC LIMIT 50, 100";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 2);
+ assertEquals(selectExpressions.get(0),
ExpressionContext.forIdentifier("foo"));
+ assertEquals(selectExpressions.get(0).toString(), "foo");
+ assertEquals(selectExpressions.get(1),
ExpressionContext.forIdentifier("bar"));
+ assertEquals(selectExpressions.get(1).toString(), "bar");
+ assertNull(queryContext.getFilter());
+ List<OrderByExpressionContext> orderByExpressions =
queryContext.getOrderByExpressions();
+ assertNotNull(orderByExpressions);
+ assertEquals(orderByExpressions.size(), 2);
+ assertEquals(orderByExpressions.get(0),
+ new
OrderByExpressionContext(ExpressionContext.forIdentifier("bar"), true));
+ assertEquals(orderByExpressions.get(0).toString(), "bar ASC");
+ assertEquals(orderByExpressions.get(1).toString(), "foo DESC");
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 100);
+ assertEquals(queryContext.getOffset(), 50);
+ assertEquals(QueryContextUtils.getAllColumns(queryContext), new
HashSet<>(Arrays.asList("foo", "bar")));
+ assertFalse(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Distinct with order-by
+ {
+ String pqlQuery = "SELECT DISTINCT(foo, bar, foobar) FROM testTable
ORDER BY bar DESC, foo LIMIT 15";
+ String sqlQuery = "SELECT DISTINCT foo, bar, foobar FROM testTable ORDER
BY bar DESC, foo LIMIT 15";
+ QueryContext[] queryContexts = getQueryContexts(pqlQuery, sqlQuery);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 1);
+ assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "distinct",
Arrays
+ .asList(ExpressionContext.forIdentifier("foo"),
ExpressionContext.forIdentifier("bar"),
+ ExpressionContext.forIdentifier("foobar")))));
+ assertEquals(selectExpressions.get(0).toString(),
"distinct(foo,bar,foobar)");
+ assertNull(queryContext.getFilter());
+ assertNull(queryContext.getGroupByExpressions());
+ List<OrderByExpressionContext> orderByExpressions =
queryContext.getOrderByExpressions();
+ assertNotNull(orderByExpressions);
+ assertEquals(orderByExpressions.size(), 2);
+ assertEquals(orderByExpressions.get(0),
+ new
OrderByExpressionContext(ExpressionContext.forIdentifier("bar"), false));
+ assertEquals(orderByExpressions.get(0).toString(), "bar DESC");
+ assertEquals(orderByExpressions.get(1),
+ new
OrderByExpressionContext(ExpressionContext.forIdentifier("foo"), true));
+ assertEquals(orderByExpressions.get(1).toString(), "foo ASC");
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 15);
+ assertEquals(queryContext.getOffset(), 0);
+ assertEquals(QueryContextUtils.getAllColumns(queryContext),
+ new HashSet<>(Arrays.asList("foo", "bar", "foobar")));
+ assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Transform with order-by
+ {
+ String query =
+ "SELECT ADD(foo, ADD(bar, 123)), SUB('456', foobar) FROM testTable
ORDER BY SUB(456, foobar) LIMIT 30, 20";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 2);
+ assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.TRANSFORM, "add", Arrays
+ .asList(ExpressionContext.forIdentifier("foo"),
ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.TRANSFORM, "add",
Arrays
+ .asList(ExpressionContext.forIdentifier("bar"),
ExpressionContext.forLiteral("123"))))))));
+ assertEquals(selectExpressions.get(0).toString(),
"add(foo,add(bar,'123'))");
+ assertEquals(selectExpressions.get(1), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.TRANSFORM, "sub",
+ Arrays.asList(ExpressionContext.forLiteral("456"),
ExpressionContext.forIdentifier("foobar")))));
+ assertEquals(selectExpressions.get(1).toString(), "sub('456',foobar)");
+ assertNull(queryContext.getFilter());
+ assertNull(queryContext.getGroupByExpressions());
+ List<OrderByExpressionContext> orderByExpressions =
queryContext.getOrderByExpressions();
+ assertNotNull(orderByExpressions);
+ assertEquals(orderByExpressions.size(), 1);
+ assertEquals(orderByExpressions.get(0), new
OrderByExpressionContext(ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.TRANSFORM, "sub",
+ Arrays.asList(ExpressionContext.forLiteral("456"),
ExpressionContext.forIdentifier("foobar")))), true));
+ assertEquals(orderByExpressions.get(0).toString(), "sub('456',foobar)
ASC");
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 20);
+ assertEquals(queryContext.getOffset(), 30);
+ assertEquals(QueryContextUtils.getAllColumns(queryContext),
+ new HashSet<>(Arrays.asList("foo", "bar", "foobar")));
+ assertFalse(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Aggregation group-by with transform, order-by
+ {
+ String pqlQuery =
+ "SELECT SUM(ADD(foo, bar)) FROM testTable GROUP BY SUB(foo, bar),
bar ORDER BY SUM(ADD(foo, bar)), SUB(foo, bar) DESC LIMIT 20";
+ String sqlQuery =
+ "SELECT SUB(foo, bar), bar, SUM(ADD(foo, bar)) FROM testTable GROUP
BY SUB(foo, bar), bar ORDER BY SUM(ADD(foo, bar)), SUB(foo, bar) DESC LIMIT 20";
+ QueryContext[] queryContexts = getQueryContexts(pqlQuery, sqlQuery);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 1);
+ assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
Collections.singletonList(ExpressionContext
+ .forFunction(new
FunctionContext(FunctionContext.Type.TRANSFORM, "add",
+ Arrays.asList(ExpressionContext.forIdentifier("foo"),
ExpressionContext.forIdentifier("bar"))))))));
+ assertEquals(selectExpressions.get(0).toString(), "sum(add(foo,bar))");
+ assertNull(queryContext.getFilter());
+ List<ExpressionContext> groupByExpressions =
queryContext.getGroupByExpressions();
+ assertNotNull(groupByExpressions);
+ assertEquals(groupByExpressions.size(), 2);
+ assertEquals(groupByExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.TRANSFORM, "sub",
+ Arrays.asList(ExpressionContext.forIdentifier("foo"),
ExpressionContext.forIdentifier("bar")))));
+ assertEquals(groupByExpressions.get(0).toString(), "sub(foo,bar)");
+ assertEquals(groupByExpressions.get(1),
ExpressionContext.forIdentifier("bar"));
+ assertEquals(groupByExpressions.get(1).toString(), "bar");
+ List<OrderByExpressionContext> orderByExpressions =
queryContext.getOrderByExpressions();
+ assertNotNull(orderByExpressions);
+ assertEquals(orderByExpressions.size(), 2);
+ assertEquals(orderByExpressions.get(0), new
OrderByExpressionContext(ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
Collections.singletonList(ExpressionContext
+ .forFunction(new
FunctionContext(FunctionContext.Type.TRANSFORM, "add",
+ Arrays.asList(ExpressionContext.forIdentifier("foo"),
ExpressionContext.forIdentifier("bar"))))))),
+ true));
+ assertEquals(orderByExpressions.get(0).toString(), "sum(add(foo,bar))
ASC");
+ assertEquals(orderByExpressions.get(1), new
OrderByExpressionContext(ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.TRANSFORM, "sub",
+ Arrays.asList(ExpressionContext.forIdentifier("foo"),
ExpressionContext.forIdentifier("bar")))),
+ false));
+ assertEquals(orderByExpressions.get(1).toString(), "sub(foo,bar)
DESC");
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 20);
+ assertEquals(queryContext.getOffset(), 0);
+ assertEquals(QueryContextUtils.getAllColumns(queryContext), new
HashSet<>(Arrays.asList("foo", "bar")));
+ assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Filter with transform
+ {
+ String query =
+ "SELECT * FROM testTable WHERE foo > 15 AND (DIV(bar, foo) BETWEEN
10 AND 20 OR TEXT_MATCH(foobar, 'potato'))";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 1);
+ assertEquals(selectExpressions.get(0),
ExpressionContext.forIdentifier("*"));
+ assertEquals(selectExpressions.get(0).toString(), "*");
+ FilterContext filter = queryContext.getFilter();
+ assertNotNull(filter);
+ assertEquals(filter.getType(), FilterContext.Type.AND);
+ List<FilterContext> children = filter.getChildren();
+ assertEquals(children.size(), 2);
+ assertEquals(children.get(0), new
FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(ExpressionContext.forIdentifier("foo"), false,
"15", false, "*")));
+ FilterContext orFilter = children.get(1);
+ assertEquals(orFilter.getType(), FilterContext.Type.OR);
+ assertEquals(orFilter.getChildren().size(), 2);
+ assertEquals(orFilter.getChildren().get(0), new
FilterContext(FilterContext.Type.PREDICATE, null,
+ new RangePredicate(ExpressionContext.forFunction(new
FunctionContext(FunctionContext.Type.TRANSFORM, "div",
+ Arrays.asList(ExpressionContext.forIdentifier("bar"),
ExpressionContext.forIdentifier("foo")))), true,
+ "10", true, "20")));
+ assertEquals(orFilter.getChildren().get(1), new
FilterContext(FilterContext.Type.PREDICATE, null,
+ new TextMatchPredicate(ExpressionContext.forIdentifier("foobar"),
"potato")));
+ assertEquals(filter.toString(),
+ "(foo IN RANGE (15,*) AND (div(bar,foo) IN RANGE [10,20] OR foobar
TEXT_MATCH 'potato'))");
+ assertNull(queryContext.getGroupByExpressions());
+ assertNull(queryContext.getOrderByExpressions());
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 10);
+ assertEquals(queryContext.getOffset(), 0);
+ assertEquals(QueryContextUtils.getAllColumns(queryContext),
+ new HashSet<>(Arrays.asList("foo", "bar", "foobar")));
+ assertFalse(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ // Alias (only supported in SQL format)
+ // NOTE: All the references to the alias should already be converted to
the original expressions.
+ {
+ String sqlQuery =
+ "SELECT SUM(foo) AS a, bar AS b FROM testTable WHERE b IN (5, 10,
15) GROUP BY b ORDER BY a DESC";
+ QueryContext queryContext =
+
BrokerRequestToQueryContextConverter.convert(SQL_COMPILER.compileToBrokerRequest(sqlQuery));
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 1);
+ assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
+
Collections.singletonList(ExpressionContext.forIdentifier("foo")))));
+ assertEquals(selectExpressions.get(0).toString(), "sum(foo)");
+ assertEquals(queryContext.getAlias(ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
+
Collections.singletonList(ExpressionContext.forIdentifier("foo"))))), "a");
+
assertEquals(queryContext.getAlias(ExpressionContext.forIdentifier("bar")),
"b");
+ FilterContext filter = queryContext.getFilter();
+ assertNotNull(filter);
+ assertEquals(filter, new FilterContext(FilterContext.Type.PREDICATE,
null,
+ new InPredicate(ExpressionContext.forIdentifier("bar"),
Arrays.asList("5", "10", "15"))));
+ assertEquals(filter.toString(), "bar IN ('5','10','15')");
+ List<ExpressionContext> groupByExpressions =
queryContext.getGroupByExpressions();
+ assertNotNull(groupByExpressions);
+ assertEquals(groupByExpressions.size(), 1);
+ assertEquals(groupByExpressions.get(0),
ExpressionContext.forIdentifier("bar"));
+ assertEquals(groupByExpressions.get(0).toString(), "bar");
+ List<OrderByExpressionContext> orderByExpressions =
queryContext.getOrderByExpressions();
+ assertNotNull(orderByExpressions);
+ assertEquals(orderByExpressions.size(), 1);
+ assertEquals(orderByExpressions.get(0), new
OrderByExpressionContext(ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
+
Collections.singletonList(ExpressionContext.forIdentifier("foo")))), false));
+ assertEquals(orderByExpressions.get(0).toString(), "sum(foo) DESC");
+ assertNull(queryContext.getHavingFilter());
+ assertEquals(queryContext.getLimit(), 10);
+ assertEquals(queryContext.getOffset(), 0);
+ assertEquals(QueryContextUtils.getAllColumns(queryContext), new
HashSet<>(Arrays.asList("foo", "bar")));
+ assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+
+ // TODO: Uncomment the following part after CalciteSqlParser supports
Having clause
+ // Having (only supported in SQL format)
+// {
+// String sqlQuery = "SELECT SUM(foo), bar FROM testTable GROUP BY bar
HAVING SUM(foo) IN (5, 10, 15)";
+// QueryContext queryContext =
+//
BrokerRequestToQueryContextConverter.convertToQueryContext(SQL_COMPILER.compileToBrokerRequest(sqlQuery));
+// List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+// assertEquals(selectExpressions.size(), 1);
+// assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+// new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
+//
Collections.singletonList(ExpressionContext.forIdentifier("foo")))));
+// assertEquals(selectExpressions.get(0).toString(), "sum(foo)");
+// assertNull(queryContext.getFilter());
+// List<ExpressionContext> groupByExpressions =
queryContext.getGroupByExpressions();
+// assertNotNull(groupByExpressions);
+// assertEquals(groupByExpressions.size(), 1);
+// assertEquals(groupByExpressions.get(0),
ExpressionContext.forIdentifier("bar"));
+// assertEquals(groupByExpressions.get(0).toString(), "bar");
+// assertNull(queryContext.getOrderByExpressions());
+// FilterContext havingFilter = queryContext.getHavingFilter();
+// assertNotNull(havingFilter);
+// assertEquals(havingFilter, new
FilterContext(FilterContext.Type.PREDICATE, null, new
InPredicate(ExpressionContext
+// .forFunction(new FunctionContext(FunctionContext.Type.AGGREGATION,
"sum",
+//
Collections.singletonList(ExpressionContext.forIdentifier("foo")))),
Arrays.asList("5", "10", "15"))));
+// assertEquals(havingFilter.toString(), "sum(foo) IN ('5','10','15')");
+// assertEquals(queryContext.getLimit(), 10);
+// assertEquals(queryContext.getOffset(), 0);
+// assertEquals(QueryContextUtils.getAllColumns(queryContext), new
HashSet<>(Arrays.asList("foo", "bar")));
+// assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+// }
+
+ // DistinctCountThetaSketch (string literal and escape quote)
+ {
+ String query =
+ "SELECT DISTINCTCOUNTTHETASKETCH(foo, 'nominalEntries=1000',
'bar=''a''', 'bar=''b''', 'bar=''a'' AND bar=''b''') FROM testTable WHERE bar
IN ('a', 'b')";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ FunctionContext function =
queryContext.getSelectExpressions().get(0).getFunction();
+ assertEquals(function.getType(), FunctionContext.Type.AGGREGATION);
+ assertEquals(function.getFunctionName(), "distinctcountthetasketch");
+ List<ExpressionContext> arguments = function.getArguments();
+ assertEquals(arguments.get(0), ExpressionContext.forIdentifier("foo"));
+ assertEquals(arguments.get(1),
ExpressionContext.forLiteral("nominalEntries=1000"));
+ assertEquals(arguments.get(2),
ExpressionContext.forLiteral("bar='a'"));
+ assertEquals(arguments.get(3),
ExpressionContext.forLiteral("bar='b'"));
+ assertEquals(arguments.get(4), ExpressionContext.forLiteral("bar='a'
AND bar='b'"));
+ }
+ }
+
+ // Legacy PQL behaviors
+ // Aggregation group-by with only TOP
+ {
+ String query = "SELECT COUNT(*) FROM testTable GROUP BY foo TOP 50";
+ QueryContext queryContext =
+
BrokerRequestToQueryContextConverter.convert(PQL_COMPILER.compileToBrokerRequest(query));
+ assertEquals(queryContext.getLimit(), 50);
+ }
+ // Aggregation group-by with both LIMIT and TOP
+ {
+ String query = "SELECT COUNT(*) FROM testTable GROUP BY foo LIMIT 0 TOP
50";
+ QueryContext queryContext =
+
BrokerRequestToQueryContextConverter.convert(PQL_COMPILER.compileToBrokerRequest(query));
+ assertEquals(queryContext.getLimit(), 50);
+ }
+ // Mixed column, aggregation and transform in select expressions
+ {
+ String query = "SELECT foo, ADD(foo, bar), MAX(foo), SUM(bar) FROM
testTable";
+ QueryContext[] queryContexts = getQueryContexts(query, query);
+ for (QueryContext queryContext : queryContexts) {
+ List<ExpressionContext> selectExpressions =
queryContext.getSelectExpressions();
+ assertEquals(selectExpressions.size(), 2);
+ assertEquals(selectExpressions.get(0), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "max",
+
Collections.singletonList(ExpressionContext.forIdentifier("foo")))));
+ assertEquals(selectExpressions.get(0).toString(), "max(foo)");
+ assertEquals(selectExpressions.get(1), ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
+
Collections.singletonList(ExpressionContext.forIdentifier("bar")))));
+ assertEquals(selectExpressions.get(1).toString(), "sum(bar)");
+ assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+ // Use string literal as identifier for aggregation
+ {
+ String query = "SELECT SUM('foo') FROM testTable";
+ QueryContext queryContext =
+
BrokerRequestToQueryContextConverter.convert(PQL_COMPILER.compileToBrokerRequest(query));
+ assertEquals(queryContext.getSelectExpressions().get(0),
ExpressionContext.forFunction(
+ new FunctionContext(FunctionContext.Type.AGGREGATION, "sum",
+
Collections.singletonList(ExpressionContext.forIdentifier("foo")))));
+ assertEquals(queryContext.getSelectExpressions().get(0).toString(),
"sum(foo)");
+ assertTrue(QueryContextUtils.isAggregationQuery(queryContext));
+ }
+ }
+
+ private QueryContext[] getQueryContexts(String pqlQuery, String sqlQuery) {
+ return new QueryContext[]{BrokerRequestToQueryContextConverter.convert(
+ PQL_COMPILER.compileToBrokerRequest(pqlQuery)),
BrokerRequestToQueryContextConverter.convert(
+ SQL_COMPILER.compileToBrokerRequest(sqlQuery))};
+ }
+
+ @Test
+ public void testPqlAndSqlCompatible()
+ throws Exception {
+ ClassLoader classLoader = getClass().getClassLoader();
+ InputStream pqlInputStream =
classLoader.getResourceAsStream("pql_queries.list");
+ assertNotNull(pqlInputStream);
+ InputStream sqlInputStream =
classLoader.getResourceAsStream("sql_queries.list");
+ assertNotNull(sqlInputStream);
+ try (BufferedReader pqlReader = new BufferedReader(new
InputStreamReader(pqlInputStream));
+ BufferedReader sqlReader = new BufferedReader(new
InputStreamReader(sqlInputStream))) {
+ String pqlQuery;
+ while ((pqlQuery = pqlReader.readLine()) != null) {
+ String sqlQuery = sqlReader.readLine();
+ assertNotNull(sqlQuery);
+ QueryContext pqlQueryContext =
+
BrokerRequestToQueryContextConverter.convert(PQL_COMPILER.compileToBrokerRequest(pqlQuery));
+ QueryContext sqlQueryContext =
+
BrokerRequestToQueryContextConverter.convert(SQL_COMPILER.compileToBrokerRequest(sqlQuery));
+ // NOTE: Do not compare alias and HAVING clause because they are not
supported in PQL.
+ // NOTE: Do not compare filter (WHERE clause) because:
+ // 1. It is always generated from the BrokerRequest so there is
no compatibility issue.
+ // 2. PQL and SQL compiler have different behavior on AND/OR
handling, and require BrokerRequestOptimizer
+ // to fix the discrepancy. Check
PqlAndCalciteSqlCompatibilityTest for more details.
+ assertEquals(pqlQueryContext.getSelectExpressions(),
sqlQueryContext.getSelectExpressions());
+ assertEquals(pqlQueryContext.getGroupByExpressions(),
sqlQueryContext.getGroupByExpressions());
+ assertEquals(pqlQueryContext.getOrderByExpressions(),
sqlQueryContext.getOrderByExpressions());
+ assertEquals(pqlQueryContext.getLimit(), sqlQueryContext.getLimit());
+ assertEquals(pqlQueryContext.getOffset(), sqlQueryContext.getOffset());
+ }
+ assertNull(sqlReader.readLine());
+ }
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
