[GitHub] [incubator-pinot] harleyjj opened a new pull request #5466: [TE] frontend - harleyjj/components - remove dead components
harleyjj opened a new pull request #5466: URL: https://github.com/apache/incubator-pinot/pull/5466 ## Description 1) Removes dead components from frontend code 2) Fixes minor bugs with Report Anomaly Modal and Delete Anomaly Modal This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] tag release-0.4.0 created (now 0ee9083)
This is an automated email from the ASF dual-hosted git repository. haibow pushed a change to tag release-0.4.0 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 0ee9083 (commit) No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch release-0.4.0-rc2 created (now 0ee9083)
This is an automated email from the ASF dual-hosted git repository. haibow pushed a change to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 0ee9083 [maven-release-plugin] prepare for next development iteration This branch includes the following new commits: new f26f19b Single Quote String Literal for function invoker results (#5456) new 75f58bc Update release note new e425f5a [maven-release-plugin] prepare release release-0.4.0-rc2 new 0ee9083 [maven-release-plugin] prepare for next development iteration The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/04: Single Quote String Literal for function invoker results (#5456)
This is an automated email from the ASF dual-hosted git repository.
haibow pushed a commit to branch release-0.4.0-rc2
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit f26f19b674604dbd7afce9e81954198b245e4338
Author: Xiang Fu
AuthorDate: Wed May 27 21:55:14 2020 -0700
Single Quote String Literal for function invoker results (#5456)
---
.../org/apache/pinot/sql/parsers/CalciteSqlParser.java | 3 +++
.../apache/pinot/sql/parsers/CalciteSqlCompilerTest.java | 6 +++---
.../integration/tests/OfflineClusterIntegrationTest.java | 14 ++
3 files changed, 20 insertions(+), 3 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
index 45b3274..2aec8ec 100644
---
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
+++
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
@@ -676,6 +676,9 @@ public class CalciteSqlParser {
try {
FunctionInvoker invoker = new FunctionInvoker(functionInfo);
Object result = invoker.process(arguments);
+if (result instanceof String) {
+ result = String.format("'%s'", result);
+}
return RequestUtils.getLiteralExpression(result);
} catch (Exception e) {
throw new SqlCompilationException(new
IllegalArgumentException("Unsupported function - " + funcName, e));
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index ac0a5cf..aec7917 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -1534,7 +1534,7 @@ public class CalciteSqlCompilerTest {
Function greaterThan = pinotQuery.getFilterExpression().getFunctionCall();
String today =
greaterThan.getOperands().get(1).getLiteral().getStringValue();
String expectedTodayStr =
-
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z"));
+"'" +
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z")) + "'";
Assert.assertEquals(today, expectedTodayStr);
}
@@ -1557,7 +1557,7 @@ public class CalciteSqlCompilerTest {
Assert.assertTrue(expression.getLiteral() != null);
String today = expression.getLiteral().getStringValue();
String expectedTodayStr =
-
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z"));
+"'" +
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z")) + "'";
Assert.assertEquals(today, expectedTodayStr);
expression =
CalciteSqlParser.compileToExpression("toDateTime(playerName)");
Assert.assertTrue(expression.getFunctionCall() != null);
@@ -1575,7 +1575,7 @@ public class CalciteSqlCompilerTest {
Assert.assertTrue(expression.getFunctionCall() != null);
expression =
CalciteSqlParser.invokeCompileTimeFunctionExpression(expression);
Assert.assertTrue(expression.getLiteral() != null);
-Assert.assertEquals(expression.getLiteral().getFieldValue(), "emaNreyalp");
+Assert.assertEquals(expression.getLiteral().getFieldValue(),
"'emaNreyalp'");
expression = CalciteSqlParser.compileToExpression("count(*)");
Assert.assertTrue(expression.getFunctionCall() != null);
expression =
CalciteSqlParser.invokeCompileTimeFunctionExpression(expression);
diff --git
a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
index f433a3e..223602c 100644
---
a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
+++
b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
@@ -24,6 +24,9 @@ import com.fasterxml.jackson.databind.node.ObjectNode;
import com.google.common.collect.ImmutableList;
import java.io.File;
import java.io.IOException;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
@@ -309,6 +312,17 @@ public class OfflineClusterIntegrationTest extends
BaseClusterIntegrationTestSet
}
@Test
+ public void testTimeFunc()
+ throws Exception {
+String sqlQuery = "SELECT toDateTime(now(), '-MM-dd z') FROM mytable";
+JsonNode response = postSqlQuery(sqlQuery, _brokerBaseApiUrl);
+String todayStr =
response.get("resultTable").get("rows").get(0).get(0).asText();
+String
[incubator-pinot] 02/04: Update release note
This is an automated email from the ASF dual-hosted git repository. haibow pushed a commit to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 75f58bc279988d3818969bd9b8ce850b82202eaa Author: Haibo Wang AuthorDate: Thu May 28 21:15:15 2020 -0700 Update release note --- LICENSE-binary | 155 ++--- NOTICE-binary | 53 +++- 2 files changed, 157 insertions(+), 51 deletions(-) diff --git a/LICENSE-binary b/LICENSE-binary index 5c0e3e4..04faf94 100644 --- a/LICENSE-binary +++ b/LICENSE-binary @@ -211,28 +211,32 @@ com.clearspring.analytics:stream:2.7.0 com.fasterxml.jackson.core:jackson-annotations:2.9.8 com.fasterxml.jackson.core:jackson-core:2.9.8 com.fasterxml.jackson.core:jackson-databind:2.9.8 +com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.9.8 com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.8 +com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.0 com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:2.9.8 com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.9.8 com.fasterxml.jackson.module:jackson-module-jaxb-annotations:2.9.8 com.fasterxml.jackson.module:jackson-module-paranamer:2.6.5 com.fasterxml.jackson.module:jackson-module-scala_2.11:2.6.5 +com.fasterxml.woodstox:woodstox-core:5.0.3 com.github.jnr:jffi:1.2.9 com.github.jnr:jnr-constants:0.8.7 com.github.jnr:jnr-ffi:2.0.3 +com.google.api-client:google-api-client com.google.api.grpc:proto-google-common-protos:1.17.0 com.google.api.grpc:proto-google-iam-v1:0.13.0 com.google.api:api-common:1.8.1 com.google.api:gax:1.49.1 com.google.api:gax-httpjson:0.66.1 -com.google.api:google-api-client:1.30.4 -com.google.api:google-api-services-storage:v1-rev20190910-1.30.3.jar +com.google.apis:google-api-services-storage:v1-rev20190910-1.30.3 com.google.auth:google-auth-library-credentials:0.18.0 com.google.auth:google-auth-library-oauth2-http:0.18.0 com.google.auto.value:auto-value-annotations:1.6.6 com.google.cloud:google-cloud-core:1.91.3 com.google.cloud:google-cloud-core-http:1.91.3 -com.google.cloud:google-cloud-storage:1.101.0 +com.google.cloud:google-cloud-nio:0.120.0-alpha +com.google.cloud:google-cloud-storage:1.102.0 com.google.code.findbugs:jsr305:3.0.0 com.google.code.gson:gson:2.2.4 com.google.errorprone:error_prone_annotations:2.3.2 @@ -240,21 +244,20 @@ com.google.guava:guava:20.0 com.google.http-client:google-http-client-appengine:1.32.1 com.google.http-client:google-http-client-jackson2:1.32.1 com.google.http-client:google-http-client:1.32.1 -com.google.inject.extensions:guice-servlet:3.0 -com.google.inject:guice:3.0 com.google.j2objc:j2objc-annotations:1.3 com.google.oauth-client:google-oauth-client:1.30.3 com.google.protobuf:protobuf-java-util:3.10.0 com.google.protobuf:protobuf-java:2.5.0 com.google.protobuf:protobuf-java:3.10.0 com.jamesmurty.utils:java-xmlbuilder:0.4 +com.jamesmurty.utils:java-xmlbuilder:1.0 +com.jayway.jsonpath:json-path:2.4.0 com.lmax:disruptor:3.3.4 com.ning:async-http-client:1.9.21 com.ning:compress-lzf:1.0.3 com.tdunning:t-digest:3.2 -com.twitter:chill-java:0.8.0 -com.twitter:chill_2.11:0.8.0 -com.typesafe:config:1.3.2 +com.typesafe.netty:netty-reactive-streams-http:2.0.4 +com.typesafe.netty:netty-reactive-streams:2.0.4 com.typesafe.scala-logging:scala-logging_2.11:3.9.0 com.yammer.metrics:metrics-core:2.2.0 commons-beanutils:commons-beanutils:1.8.3 @@ -271,15 +274,35 @@ commons-lang:commons-lang:2.6 commons-logging:commons-logging:1.2 commons-net:commons-net:3.1 io.airlift:aircompressor:0.10 +io.confluent:common-config:5.3.1 +io.confluent:common-utils:5.3.1 +io.confluent:kafka-avro-serializer:5.3.1 +io.confluent:kafka-schema-registry-client:5.3.1 io.dropwizard.metrics:metrics-core:3.2.3 -io.dropwizard.metrics:metrics-graphite:3.1.2 -io.dropwizard.metrics:metrics-json:3.1.2 -io.dropwizard.metrics:metrics-jvm:3.1.2 +io.dropwizard.metrics:metrics-graphite:3.2.3 +io.dropwizard.metrics:metrics-json:3.2.3 +io.dropwizard.metrics:metrics-jvm:3.2.3 io.grpc:grpc-context:1.22.1 +io.netty:netty-all:4.1.42.Final +io.netty:netty-buffer:4.1.42.Final +io.netty:netty-codec-http2:4.1.42.Final +io.netty:netty-codec-http:4.1.42.Final +io.netty:netty-codec-socks:4.1.42.Final +io.netty:netty-codec:4.1.42.Final +io.netty:netty-common:4.1.42.Final +io.netty:netty-handler-proxy:4.1.42.Final +io.netty:netty-handler:4.1.42.Final +io.netty:netty-resolver:4.1.42.Final +io.netty:netty-tcnative-boringssl-static:2.0.26.Final +io.netty:netty-transport-native-epoll:linux-x86_64:4.1.42.Final +io.netty:netty-transport-native-unix-common:4.1.42.Final +io.netty:netty-transport:4.1.42.Final io.netty:netty:3.9.6.Final -io.netty:netty-all:4.1.28.Final io.opencensus:opencensus-api:0.24.0 io.opencensus:opencensus-contrib-http-util:0.24.0 +io.projectreactor.addons:reactor-pool:0.1.0.RELEASE +io.projectreactor.netty:reactor-netty:0.9.0.RELEASE
[incubator-pinot] 04/04: [maven-release-plugin] prepare for next development iteration
This is an automated email from the ASF dual-hosted git repository. haibow pushed a commit to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 0ee908370bbca888bac5356304e1217391d54f78 Author: Haibo Wang AuthorDate: Thu May 28 22:00:09 2020 -0700 [maven-release-plugin] prepare for next development iteration --- licenses-binary/LICENSE-gpl-2.0.txt| 125 + pinot-broker/pom.xml | 2 +- pinot-clients/pinot-java-client/pom.xml| 2 +- pinot-clients/pom.xml | 2 +- pinot-common/pom.xml | 2 +- pinot-controller/pom.xml | 2 +- pinot-core/pom.xml | 2 +- pinot-distribution/pom.xml | 2 +- pinot-integration-tests/pom.xml| 2 +- pinot-minion/pom.xml | 2 +- pinot-perf/pom.xml | 2 +- .../pinot-batch-ingestion-common/pom.xml | 2 +- .../pinot-batch-ingestion-hadoop/pom.xml | 2 +- .../pinot-batch-ingestion-spark/pom.xml| 2 +- .../pinot-batch-ingestion-standalone/pom.xml | 2 +- pinot-plugins/pinot-batch-ingestion/pom.xml| 2 +- .../v0_deprecated/pinot-hadoop/pom.xml | 2 +- .../v0_deprecated/pinot-ingestion-common/pom.xml | 2 +- .../v0_deprecated/pinot-spark/pom.xml | 2 +- .../pinot-batch-ingestion/v0_deprecated/pom.xml| 2 +- pinot-plugins/pinot-file-system/pinot-adls/pom.xml | 2 +- pinot-plugins/pinot-file-system/pinot-gcs/pom.xml | 2 +- pinot-plugins/pinot-file-system/pinot-hdfs/pom.xml | 2 +- pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 2 +- pinot-plugins/pinot-file-system/pom.xml| 2 +- .../pinot-input-format/pinot-avro-base/pom.xml | 2 +- .../pinot-input-format/pinot-avro/pom.xml | 2 +- .../pinot-confluent-avro/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-csv/pom.xml | 2 +- .../pinot-input-format/pinot-json/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-orc/pom.xml | 2 +- .../pinot-input-format/pinot-parquet/pom.xml | 2 +- .../pinot-input-format/pinot-thrift/pom.xml| 2 +- pinot-plugins/pinot-input-format/pom.xml | 2 +- .../pinot-stream-ingestion/pinot-kafka-0.9/pom.xml | 2 +- .../pinot-stream-ingestion/pinot-kafka-2.0/pom.xml | 2 +- .../pinot-kafka-base/pom.xml | 2 +- pinot-plugins/pinot-stream-ingestion/pom.xml | 2 +- pinot-plugins/pom.xml | 2 +- pinot-server/pom.xml | 2 +- pinot-spi/pom.xml | 2 +- pinot-tools/pom.xml| 2 +- pom.xml| 4 +- 43 files changed, 168 insertions(+), 43 deletions(-) diff --git a/licenses-binary/LICENSE-gpl-2.0.txt b/licenses-binary/LICENSE-gpl-2.0.txt new file mode 100644 index 000..0e102ee --- /dev/null +++ b/licenses-binary/LICENSE-gpl-2.0.txt @@ -0,0 +1,125 @@ +GNU GENERAL PUBLIC LICENSE +Version 2, June 1991 + +Copyright (C) 1989, 1991 Free Software Foundation, Inc. +51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA + +Everyone is permitted to copy and distribute verbatim copies +of this license document, but changing it is not allowed. +Preamble +The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General [...] + +When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. + +To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. + +For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And
[incubator-pinot] 03/04: [maven-release-plugin] prepare release release-0.4.0-rc2
This is an automated email from the ASF dual-hosted git repository.
haibow pushed a commit to branch release-0.4.0-rc2
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit e425f5ae5a77955748cefcee94b92ebcbcd2559f
Author: Haibo Wang
AuthorDate: Thu May 28 21:59:47 2020 -0700
[maven-release-plugin] prepare release release-0.4.0-rc2
---
pinot-broker/pom.xml | 5 ++---
pinot-clients/pinot-java-client/pom.xml | 5 ++---
pinot-clients/pom.xml | 6 ++
pinot-common/pom.xml | 5 ++---
pinot-controller/pom.xml | 5 ++---
pinot-core/pom.xml| 5 ++---
pinot-distribution/pom.xml| 7 +++
pinot-integration-tests/pom.xml | 5 ++---
pinot-minion/pom.xml | 5 ++---
pinot-perf/pom.xml| 5 ++---
.../pinot-batch-ingestion/pinot-batch-ingestion-common/pom.xml| 6 ++
.../pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pom.xml| 6 ++
.../pinot-batch-ingestion/pinot-batch-ingestion-spark/pom.xml | 6 ++
.../pinot-batch-ingestion-standalone/pom.xml | 6 ++
pinot-plugins/pinot-batch-ingestion/pom.xml | 6 ++
.../pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml | 7 +++
.../v0_deprecated/pinot-ingestion-common/pom.xml | 6 ++
.../pinot-batch-ingestion/v0_deprecated/pinot-spark/pom.xml | 7 +++
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pinot-adls/pom.xml| 5 ++---
pinot-plugins/pinot-file-system/pinot-gcs/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pinot-hdfs/pom.xml| 5 ++---
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pom.xml | 6 ++
pinot-plugins/pinot-input-format/pinot-avro-base/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-avro/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-confluent-avro/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-csv/pom.xml| 5 ++---
pinot-plugins/pinot-input-format/pinot-json/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-orc/pom.xml| 6 ++
pinot-plugins/pinot-input-format/pinot-parquet/pom.xml| 5 ++---
pinot-plugins/pinot-input-format/pinot-thrift/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-0.9/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pom.xml | 6 ++
pinot-plugins/pom.xml | 8 +++-
pinot-server/pom.xml | 5 ++---
pinot-spi/pom.xml | 5 ++---
pinot-tools/pom.xml | 5 ++---
pom.xml | 7 +++
42 files changed, 89 insertions(+), 149 deletions(-)
diff --git a/pinot-broker/pom.xml b/pinot-broker/pom.xml
index d9e48e1..2fd885d 100644
--- a/pinot-broker/pom.xml
+++ b/pinot-broker/pom.xml
@@ -19,13 +19,12 @@
under the License.
-->
-http://www.w3.org/2001/XMLSchema-instance;
xmlns="http://maven.apache.org/POM/4.0.0;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
4.0.0
pinot
org.apache.pinot
-${revision}${sha1}
+0.4.0
..
pinot-broker
diff --git a/pinot-clients/pinot-java-client/pom.xml
b/pinot-clients/pinot-java-client/pom.xml
index 6a98e3d..615c5e9 100644
--- a/pinot-clients/pinot-java-client/pom.xml
+++ b/pinot-clients/pinot-java-client/pom.xml
@@ -19,13 +19,12 @@
under the License.
-->
-http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
[incubator-pinot] annotated tag release-0.4.0-rc2 updated (e425f5a -> b4269a6)
This is an automated email from the ASF dual-hosted git repository. haibow pushed a change to annotated tag release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. *** WARNING: tag release-0.4.0-rc2 was modified! *** from e425f5a (commit) to b4269a6 (tag) tagging e425f5ae5a77955748cefcee94b92ebcbcd2559f (commit) by Haibo Wang on Thu May 28 22:00:03 2020 -0700 - Log - [maven-release-plugin] copy for tag release-0.4.0-rc2 --- No new revisions were added by this update. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch release-0.4.0-rc2 updated: [maven-release-plugin] prepare for next development iteration
This is an automated email from the ASF dual-hosted git repository. haibow pushed a commit to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/release-0.4.0-rc2 by this push: new ead32f1 [maven-release-plugin] prepare for next development iteration ead32f1 is described below commit ead32f109fa790ab7c416f7b14f6da5c8aff1efb Author: Haibo Wang AuthorDate: Thu May 28 22:00:09 2020 -0700 [maven-release-plugin] prepare for next development iteration --- pinot-broker/pom.xml | 2 +- pinot-clients/pinot-java-client/pom.xml | 2 +- pinot-clients/pom.xml | 2 +- pinot-common/pom.xml | 2 +- pinot-controller/pom.xml | 2 +- pinot-core/pom.xml| 2 +- pinot-distribution/pom.xml| 2 +- pinot-integration-tests/pom.xml | 2 +- pinot-minion/pom.xml | 2 +- pinot-perf/pom.xml| 2 +- .../pinot-batch-ingestion/pinot-batch-ingestion-common/pom.xml| 2 +- .../pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pom.xml| 2 +- .../pinot-batch-ingestion/pinot-batch-ingestion-spark/pom.xml | 2 +- .../pinot-batch-ingestion/pinot-batch-ingestion-standalone/pom.xml| 2 +- pinot-plugins/pinot-batch-ingestion/pom.xml | 2 +- .../pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml | 2 +- .../v0_deprecated/pinot-ingestion-common/pom.xml | 2 +- pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-spark/pom.xml | 2 +- pinot-plugins/pinot-batch-ingestion/v0_deprecated/pom.xml | 2 +- pinot-plugins/pinot-file-system/pinot-adls/pom.xml| 2 +- pinot-plugins/pinot-file-system/pinot-gcs/pom.xml | 2 +- pinot-plugins/pinot-file-system/pinot-hdfs/pom.xml| 2 +- pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 2 +- pinot-plugins/pinot-file-system/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-avro-base/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-avro/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-confluent-avro/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-csv/pom.xml| 2 +- pinot-plugins/pinot-input-format/pinot-json/pom.xml | 2 +- pinot-plugins/pinot-input-format/pinot-orc/pom.xml| 2 +- pinot-plugins/pinot-input-format/pinot-parquet/pom.xml| 2 +- pinot-plugins/pinot-input-format/pinot-thrift/pom.xml | 2 +- pinot-plugins/pinot-input-format/pom.xml | 2 +- pinot-plugins/pinot-stream-ingestion/pinot-kafka-0.9/pom.xml | 2 +- pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/pom.xml | 2 +- pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/pom.xml | 2 +- pinot-plugins/pinot-stream-ingestion/pom.xml | 2 +- pinot-plugins/pom.xml | 2 +- pinot-server/pom.xml | 2 +- pinot-spi/pom.xml | 2 +- pinot-tools/pom.xml | 2 +- pom.xml | 4 ++-- 42 files changed, 43 insertions(+), 43 deletions(-) diff --git a/pinot-broker/pom.xml b/pinot-broker/pom.xml index 2fd885d..a884630 100644 --- a/pinot-broker/pom.xml +++ b/pinot-broker/pom.xml @@ -24,7 +24,7 @@ pinot org.apache.pinot -0.4.0 +0.5.0-SNAPSHOT .. pinot-broker diff --git a/pinot-clients/pinot-java-client/pom.xml b/pinot-clients/pinot-java-client/pom.xml index 615c5e9..60177d7 100644 --- a/pinot-clients/pinot-java-client/pom.xml +++ b/pinot-clients/pinot-java-client/pom.xml @@ -24,7 +24,7 @@ pinot-clients org.apache.pinot -0.4.0 +0.5.0-SNAPSHOT .. pinot-java-client diff --git a/pinot-clients/pom.xml b/pinot-clients/pom.xml index e05e7ec..902f54f 100644 --- a/pinot-clients/pom.xml +++ b/pinot-clients/pom.xml @@ -24,7 +24,7 @@ pinot org.apache.pinot -0.4.0 +0.5.0-SNAPSHOT .. pinot-clients diff --git a/pinot-common/pom.xml b/pinot-common/pom.xml index db2d28d..7f2a7fc 100644 --- a/pinot-common/pom.xml +++ b/pinot-common/pom.xml @@ -24,7 +24,7 @@ pinot org.apache.pinot -0.4.0 +0.5.0-SNAPSHOT ..
[incubator-pinot] 03/03: [maven-release-plugin] prepare release release-0.4.0-rc2
This is an automated email from the ASF dual-hosted git repository.
haibow pushed a commit to branch release-0.4.0-rc2
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit e425f5ae5a77955748cefcee94b92ebcbcd2559f
Author: Haibo Wang
AuthorDate: Thu May 28 21:59:47 2020 -0700
[maven-release-plugin] prepare release release-0.4.0-rc2
---
pinot-broker/pom.xml | 5 ++---
pinot-clients/pinot-java-client/pom.xml | 5 ++---
pinot-clients/pom.xml | 6 ++
pinot-common/pom.xml | 5 ++---
pinot-controller/pom.xml | 5 ++---
pinot-core/pom.xml| 5 ++---
pinot-distribution/pom.xml| 7 +++
pinot-integration-tests/pom.xml | 5 ++---
pinot-minion/pom.xml | 5 ++---
pinot-perf/pom.xml| 5 ++---
.../pinot-batch-ingestion/pinot-batch-ingestion-common/pom.xml| 6 ++
.../pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pom.xml| 6 ++
.../pinot-batch-ingestion/pinot-batch-ingestion-spark/pom.xml | 6 ++
.../pinot-batch-ingestion-standalone/pom.xml | 6 ++
pinot-plugins/pinot-batch-ingestion/pom.xml | 6 ++
.../pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml | 7 +++
.../v0_deprecated/pinot-ingestion-common/pom.xml | 6 ++
.../pinot-batch-ingestion/v0_deprecated/pinot-spark/pom.xml | 7 +++
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pinot-adls/pom.xml| 5 ++---
pinot-plugins/pinot-file-system/pinot-gcs/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pinot-hdfs/pom.xml| 5 ++---
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pom.xml | 6 ++
pinot-plugins/pinot-input-format/pinot-avro-base/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-avro/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-confluent-avro/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-csv/pom.xml| 5 ++---
pinot-plugins/pinot-input-format/pinot-json/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-orc/pom.xml| 6 ++
pinot-plugins/pinot-input-format/pinot-parquet/pom.xml| 5 ++---
pinot-plugins/pinot-input-format/pinot-thrift/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-0.9/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pom.xml | 6 ++
pinot-plugins/pom.xml | 8 +++-
pinot-server/pom.xml | 5 ++---
pinot-spi/pom.xml | 5 ++---
pinot-tools/pom.xml | 5 ++---
pom.xml | 7 +++
42 files changed, 89 insertions(+), 149 deletions(-)
diff --git a/pinot-broker/pom.xml b/pinot-broker/pom.xml
index d9e48e1..2fd885d 100644
--- a/pinot-broker/pom.xml
+++ b/pinot-broker/pom.xml
@@ -19,13 +19,12 @@
under the License.
-->
-http://www.w3.org/2001/XMLSchema-instance;
xmlns="http://maven.apache.org/POM/4.0.0;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
4.0.0
pinot
org.apache.pinot
-${revision}${sha1}
+0.4.0
..
pinot-broker
diff --git a/pinot-clients/pinot-java-client/pom.xml
b/pinot-clients/pinot-java-client/pom.xml
index 6a98e3d..615c5e9 100644
--- a/pinot-clients/pinot-java-client/pom.xml
+++ b/pinot-clients/pinot-java-client/pom.xml
@@ -19,13 +19,12 @@
under the License.
-->
-http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
[incubator-pinot] 01/03: Single Quote String Literal for function invoker results (#5456)
This is an automated email from the ASF dual-hosted git repository.
haibow pushed a commit to branch release-0.4.0-rc2
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit f26f19b674604dbd7afce9e81954198b245e4338
Author: Xiang Fu
AuthorDate: Wed May 27 21:55:14 2020 -0700
Single Quote String Literal for function invoker results (#5456)
---
.../org/apache/pinot/sql/parsers/CalciteSqlParser.java | 3 +++
.../apache/pinot/sql/parsers/CalciteSqlCompilerTest.java | 6 +++---
.../integration/tests/OfflineClusterIntegrationTest.java | 14 ++
3 files changed, 20 insertions(+), 3 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
index 45b3274..2aec8ec 100644
---
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
+++
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
@@ -676,6 +676,9 @@ public class CalciteSqlParser {
try {
FunctionInvoker invoker = new FunctionInvoker(functionInfo);
Object result = invoker.process(arguments);
+if (result instanceof String) {
+ result = String.format("'%s'", result);
+}
return RequestUtils.getLiteralExpression(result);
} catch (Exception e) {
throw new SqlCompilationException(new
IllegalArgumentException("Unsupported function - " + funcName, e));
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index ac0a5cf..aec7917 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -1534,7 +1534,7 @@ public class CalciteSqlCompilerTest {
Function greaterThan = pinotQuery.getFilterExpression().getFunctionCall();
String today =
greaterThan.getOperands().get(1).getLiteral().getStringValue();
String expectedTodayStr =
-
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z"));
+"'" +
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z")) + "'";
Assert.assertEquals(today, expectedTodayStr);
}
@@ -1557,7 +1557,7 @@ public class CalciteSqlCompilerTest {
Assert.assertTrue(expression.getLiteral() != null);
String today = expression.getLiteral().getStringValue();
String expectedTodayStr =
-
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z"));
+"'" +
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z")) + "'";
Assert.assertEquals(today, expectedTodayStr);
expression =
CalciteSqlParser.compileToExpression("toDateTime(playerName)");
Assert.assertTrue(expression.getFunctionCall() != null);
@@ -1575,7 +1575,7 @@ public class CalciteSqlCompilerTest {
Assert.assertTrue(expression.getFunctionCall() != null);
expression =
CalciteSqlParser.invokeCompileTimeFunctionExpression(expression);
Assert.assertTrue(expression.getLiteral() != null);
-Assert.assertEquals(expression.getLiteral().getFieldValue(), "emaNreyalp");
+Assert.assertEquals(expression.getLiteral().getFieldValue(),
"'emaNreyalp'");
expression = CalciteSqlParser.compileToExpression("count(*)");
Assert.assertTrue(expression.getFunctionCall() != null);
expression =
CalciteSqlParser.invokeCompileTimeFunctionExpression(expression);
diff --git
a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
index f433a3e..223602c 100644
---
a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
+++
b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
@@ -24,6 +24,9 @@ import com.fasterxml.jackson.databind.node.ObjectNode;
import com.google.common.collect.ImmutableList;
import java.io.File;
import java.io.IOException;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
@@ -309,6 +312,17 @@ public class OfflineClusterIntegrationTest extends
BaseClusterIntegrationTestSet
}
@Test
+ public void testTimeFunc()
+ throws Exception {
+String sqlQuery = "SELECT toDateTime(now(), '-MM-dd z') FROM mytable";
+JsonNode response = postSqlQuery(sqlQuery, _brokerBaseApiUrl);
+String todayStr =
response.get("resultTable").get("rows").get(0).get(0).asText();
+String
[incubator-pinot] branch release-0.4.0-rc2 created (now e425f5a)
This is an automated email from the ASF dual-hosted git repository. haibow pushed a change to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at e425f5a [maven-release-plugin] prepare release release-0.4.0-rc2 This branch includes the following new commits: new f26f19b Single Quote String Literal for function invoker results (#5456) new 75f58bc Update release note new e425f5a [maven-release-plugin] prepare release release-0.4.0-rc2 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 02/03: Update release note
This is an automated email from the ASF dual-hosted git repository. haibow pushed a commit to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 75f58bc279988d3818969bd9b8ce850b82202eaa Author: Haibo Wang AuthorDate: Thu May 28 21:15:15 2020 -0700 Update release note --- LICENSE-binary | 155 ++--- NOTICE-binary | 53 +++- 2 files changed, 157 insertions(+), 51 deletions(-) diff --git a/LICENSE-binary b/LICENSE-binary index 5c0e3e4..04faf94 100644 --- a/LICENSE-binary +++ b/LICENSE-binary @@ -211,28 +211,32 @@ com.clearspring.analytics:stream:2.7.0 com.fasterxml.jackson.core:jackson-annotations:2.9.8 com.fasterxml.jackson.core:jackson-core:2.9.8 com.fasterxml.jackson.core:jackson-databind:2.9.8 +com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.9.8 com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.8 +com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.10.0 com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:2.9.8 com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.9.8 com.fasterxml.jackson.module:jackson-module-jaxb-annotations:2.9.8 com.fasterxml.jackson.module:jackson-module-paranamer:2.6.5 com.fasterxml.jackson.module:jackson-module-scala_2.11:2.6.5 +com.fasterxml.woodstox:woodstox-core:5.0.3 com.github.jnr:jffi:1.2.9 com.github.jnr:jnr-constants:0.8.7 com.github.jnr:jnr-ffi:2.0.3 +com.google.api-client:google-api-client com.google.api.grpc:proto-google-common-protos:1.17.0 com.google.api.grpc:proto-google-iam-v1:0.13.0 com.google.api:api-common:1.8.1 com.google.api:gax:1.49.1 com.google.api:gax-httpjson:0.66.1 -com.google.api:google-api-client:1.30.4 -com.google.api:google-api-services-storage:v1-rev20190910-1.30.3.jar +com.google.apis:google-api-services-storage:v1-rev20190910-1.30.3 com.google.auth:google-auth-library-credentials:0.18.0 com.google.auth:google-auth-library-oauth2-http:0.18.0 com.google.auto.value:auto-value-annotations:1.6.6 com.google.cloud:google-cloud-core:1.91.3 com.google.cloud:google-cloud-core-http:1.91.3 -com.google.cloud:google-cloud-storage:1.101.0 +com.google.cloud:google-cloud-nio:0.120.0-alpha +com.google.cloud:google-cloud-storage:1.102.0 com.google.code.findbugs:jsr305:3.0.0 com.google.code.gson:gson:2.2.4 com.google.errorprone:error_prone_annotations:2.3.2 @@ -240,21 +244,20 @@ com.google.guava:guava:20.0 com.google.http-client:google-http-client-appengine:1.32.1 com.google.http-client:google-http-client-jackson2:1.32.1 com.google.http-client:google-http-client:1.32.1 -com.google.inject.extensions:guice-servlet:3.0 -com.google.inject:guice:3.0 com.google.j2objc:j2objc-annotations:1.3 com.google.oauth-client:google-oauth-client:1.30.3 com.google.protobuf:protobuf-java-util:3.10.0 com.google.protobuf:protobuf-java:2.5.0 com.google.protobuf:protobuf-java:3.10.0 com.jamesmurty.utils:java-xmlbuilder:0.4 +com.jamesmurty.utils:java-xmlbuilder:1.0 +com.jayway.jsonpath:json-path:2.4.0 com.lmax:disruptor:3.3.4 com.ning:async-http-client:1.9.21 com.ning:compress-lzf:1.0.3 com.tdunning:t-digest:3.2 -com.twitter:chill-java:0.8.0 -com.twitter:chill_2.11:0.8.0 -com.typesafe:config:1.3.2 +com.typesafe.netty:netty-reactive-streams-http:2.0.4 +com.typesafe.netty:netty-reactive-streams:2.0.4 com.typesafe.scala-logging:scala-logging_2.11:3.9.0 com.yammer.metrics:metrics-core:2.2.0 commons-beanutils:commons-beanutils:1.8.3 @@ -271,15 +274,35 @@ commons-lang:commons-lang:2.6 commons-logging:commons-logging:1.2 commons-net:commons-net:3.1 io.airlift:aircompressor:0.10 +io.confluent:common-config:5.3.1 +io.confluent:common-utils:5.3.1 +io.confluent:kafka-avro-serializer:5.3.1 +io.confluent:kafka-schema-registry-client:5.3.1 io.dropwizard.metrics:metrics-core:3.2.3 -io.dropwizard.metrics:metrics-graphite:3.1.2 -io.dropwizard.metrics:metrics-json:3.1.2 -io.dropwizard.metrics:metrics-jvm:3.1.2 +io.dropwizard.metrics:metrics-graphite:3.2.3 +io.dropwizard.metrics:metrics-json:3.2.3 +io.dropwizard.metrics:metrics-jvm:3.2.3 io.grpc:grpc-context:1.22.1 +io.netty:netty-all:4.1.42.Final +io.netty:netty-buffer:4.1.42.Final +io.netty:netty-codec-http2:4.1.42.Final +io.netty:netty-codec-http:4.1.42.Final +io.netty:netty-codec-socks:4.1.42.Final +io.netty:netty-codec:4.1.42.Final +io.netty:netty-common:4.1.42.Final +io.netty:netty-handler-proxy:4.1.42.Final +io.netty:netty-handler:4.1.42.Final +io.netty:netty-resolver:4.1.42.Final +io.netty:netty-tcnative-boringssl-static:2.0.26.Final +io.netty:netty-transport-native-epoll:linux-x86_64:4.1.42.Final +io.netty:netty-transport-native-unix-common:4.1.42.Final +io.netty:netty-transport:4.1.42.Final io.netty:netty:3.9.6.Final -io.netty:netty-all:4.1.28.Final io.opencensus:opencensus-api:0.24.0 io.opencensus:opencensus-contrib-http-util:0.24.0 +io.projectreactor.addons:reactor-pool:0.1.0.RELEASE +io.projectreactor.netty:reactor-netty:0.9.0.RELEASE
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5459: [Cleanup] Merge RealtimeSegmentOnlineOfflineStateModel and SegmentOnlineOfflineStateModel in CommonConstants
Jackie-Jiang commented on a change in pull request #5459: URL: https://github.com/apache/incubator-pinot/pull/5459#discussion_r432246733 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/RoutingManager.java ## @@ -51,7 +51,7 @@ import org.apache.pinot.common.metrics.BrokerMetrics; import org.apache.pinot.common.request.BrokerRequest; import org.apache.pinot.common.utils.CommonConstants; -import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.RealtimeSegmentOnlineOfflineStateModel; +import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.SegmentOnlineOfflineStateModel; Review comment: @mcvsubbu Changed them to `SegmentStateModel` and `BrokerResourceStateModel`. Also, can you please give me a ship it if it looks good? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5444: Enhance and simplify the filtering
Jackie-Jiang commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r432244794
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/AndDocIdIterator.java
##
@@ -18,112 +18,52 @@
*/
package org.apache.pinot.core.operator.dociditerators;
-import java.util.Arrays;
import org.apache.pinot.core.common.BlockDocIdIterator;
import org.apache.pinot.core.common.Constants;
-// TODO: Optimize this
public final class AndDocIdIterator implements BlockDocIdIterator {
- public final BlockDocIdIterator[] docIdIterators;
- public ScanBasedDocIdIterator[] scanBasedDocIdIterators;
- public final int[] docIdPointers;
- public boolean reachedEnd = false;
- public int currentDocId = -1;
- int currentMax = -1;
- private boolean hasScanBasedIterators;
+ public final BlockDocIdIterator[] _docIdIterators;
- public AndDocIdIterator(BlockDocIdIterator[] blockDocIdIterators) {
-int numIndexBasedIterators = 0;
-int numScanBasedIterators = 0;
-for (int i = 0; i < blockDocIdIterators.length; i++) {
- if (blockDocIdIterators[i] instanceof IndexBasedDocIdIterator) {
-numIndexBasedIterators = numIndexBasedIterators + 1;
- } else if (blockDocIdIterators[i] instanceof ScanBasedDocIdIterator) {
-numScanBasedIterators = numScanBasedIterators + 1;
- }
-}
-// if we have at least one index based then do intersection based on index
based only, and then
-// check if matching docs apply on scan based iterator
-if (numIndexBasedIterators > 0 && numScanBasedIterators > 0) {
- hasScanBasedIterators = true;
- int nonScanIteratorsSize = blockDocIdIterators.length -
numScanBasedIterators;
- this.docIdIterators = new BlockDocIdIterator[nonScanIteratorsSize];
- this.scanBasedDocIdIterators = new
ScanBasedDocIdIterator[numScanBasedIterators];
- int nonScanBasedIndex = 0;
- int scanBasedIndex = 0;
- for (int i = 0; i < blockDocIdIterators.length; i++) {
-if (blockDocIdIterators[i] instanceof ScanBasedDocIdIterator) {
- this.scanBasedDocIdIterators[scanBasedIndex++] =
(ScanBasedDocIdIterator) blockDocIdIterators[i];
-} else {
- this.docIdIterators[nonScanBasedIndex++] = blockDocIdIterators[i];
-}
- }
-} else {
- hasScanBasedIterators = false;
- this.docIdIterators = blockDocIdIterators;
-}
-this.docIdPointers = new int[docIdIterators.length];
-Arrays.fill(docIdPointers, -1);
- }
+ private int _nextDocId = 0;
- @Override
- public int advance(int targetDocId) {
-if (currentDocId == Constants.EOF) {
- return currentDocId;
-}
-if (currentDocId >= targetDocId) {
- return currentDocId;
-}
-// next() method will always increment currentMax by 1.
-currentMax = targetDocId - 1;
-return next();
+ public AndDocIdIterator(BlockDocIdIterator[] docIdIterators) {
+_docIdIterators = docIdIterators;
}
@Override
public int next() {
-if (currentDocId == Constants.EOF) {
- return currentDocId;
-}
-currentMax = currentMax + 1;
-// always increment the pointer to current max, when this is called first
time, every one will
-// be set to start of posting list.
-for (int i = 0; i < docIdIterators.length; i++) {
- docIdPointers[i] = docIdIterators[i].advance(currentMax);
- if (docIdPointers[i] == Constants.EOF) {
-reachedEnd = true;
-currentMax = Constants.EOF;
-break;
- }
- if (docIdPointers[i] > currentMax) {
-currentMax = docIdPointers[i];
-if (i > 0) {
- // we need to advance all pointer since we found a new max
- i = -1;
-}
- }
- if (hasScanBasedIterators && i == docIdIterators.length - 1) {
-// this means we found the docId common to all nonScanBased iterators,
now we need to ensure
-// that its also found in scanBasedIterator, if not matched, we
restart the intersection
-for (ScanBasedDocIdIterator iterator : scanBasedDocIdIterators) {
- if (!iterator.isMatch(currentMax)) {
-i = -1;
-currentMax = currentMax + 1;
-break;
+int maxDocId = _nextDocId;
+int maxDocIdIndex = -1;
+int numDocIdIterators = _docIdIterators.length;
+int index = 0;
+while (index < numDocIdIterators) {
+ if (index == maxDocIdIndex) {
+// Skip the index with the max document id
+index++;
Review comment:
Changed the code a little bit so that it is more clear how index is kept
in bound.
Added AndDocIdIteratorTest and OrDocIdIteratorTest.
I have run all the queries before submitting the PR. Will run them again
before merging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
[incubator-pinot] 02/02: [maven-release-plugin] prepare release release-0.4.0-rc2
This is an automated email from the ASF dual-hosted git repository.
haibow pushed a commit to branch release-0.4.0-rc2
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit b359badceadcfcb3e27634a5ed2e2af6eff8e610
Author: Haibo Wang
AuthorDate: Thu May 28 20:48:59 2020 -0700
[maven-release-plugin] prepare release release-0.4.0-rc2
---
pinot-broker/pom.xml | 5 ++---
pinot-clients/pinot-java-client/pom.xml | 5 ++---
pinot-clients/pom.xml | 6 ++
pinot-common/pom.xml | 5 ++---
pinot-controller/pom.xml | 5 ++---
pinot-core/pom.xml| 5 ++---
pinot-distribution/pom.xml| 7 +++
pinot-integration-tests/pom.xml | 5 ++---
pinot-minion/pom.xml | 5 ++---
pinot-perf/pom.xml| 5 ++---
.../pinot-batch-ingestion/pinot-batch-ingestion-common/pom.xml| 6 ++
.../pinot-batch-ingestion/pinot-batch-ingestion-hadoop/pom.xml| 6 ++
.../pinot-batch-ingestion/pinot-batch-ingestion-spark/pom.xml | 6 ++
.../pinot-batch-ingestion-standalone/pom.xml | 6 ++
pinot-plugins/pinot-batch-ingestion/pom.xml | 6 ++
.../pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml | 7 +++
.../v0_deprecated/pinot-ingestion-common/pom.xml | 6 ++
.../pinot-batch-ingestion/v0_deprecated/pinot-spark/pom.xml | 7 +++
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pinot-adls/pom.xml| 5 ++---
pinot-plugins/pinot-file-system/pinot-gcs/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pinot-hdfs/pom.xml| 5 ++---
pinot-plugins/pinot-file-system/pinot-s3/pom.xml | 6 ++
pinot-plugins/pinot-file-system/pom.xml | 6 ++
pinot-plugins/pinot-input-format/pinot-avro-base/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-avro/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-confluent-avro/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-csv/pom.xml| 5 ++---
pinot-plugins/pinot-input-format/pinot-json/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pinot-orc/pom.xml| 6 ++
pinot-plugins/pinot-input-format/pinot-parquet/pom.xml| 5 ++---
pinot-plugins/pinot-input-format/pinot-thrift/pom.xml | 5 ++---
pinot-plugins/pinot-input-format/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-0.9/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pinot-kafka-base/pom.xml | 6 ++
pinot-plugins/pinot-stream-ingestion/pom.xml | 6 ++
pinot-plugins/pom.xml | 8 +++-
pinot-server/pom.xml | 5 ++---
pinot-spi/pom.xml | 5 ++---
pinot-tools/pom.xml | 5 ++---
pom.xml | 7 +++
42 files changed, 89 insertions(+), 149 deletions(-)
diff --git a/pinot-broker/pom.xml b/pinot-broker/pom.xml
index d9e48e1..2fd885d 100644
--- a/pinot-broker/pom.xml
+++ b/pinot-broker/pom.xml
@@ -19,13 +19,12 @@
under the License.
-->
-http://www.w3.org/2001/XMLSchema-instance;
xmlns="http://maven.apache.org/POM/4.0.0;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
4.0.0
pinot
org.apache.pinot
-${revision}${sha1}
+0.4.0
..
pinot-broker
diff --git a/pinot-clients/pinot-java-client/pom.xml
b/pinot-clients/pinot-java-client/pom.xml
index 6a98e3d..615c5e9 100644
--- a/pinot-clients/pinot-java-client/pom.xml
+++ b/pinot-clients/pinot-java-client/pom.xml
@@ -19,13 +19,12 @@
under the License.
-->
-http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
[incubator-pinot] branch release-0.4.0-rc2 created (now b359bad)
This is an automated email from the ASF dual-hosted git repository. haibow pushed a change to branch release-0.4.0-rc2 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at b359bad [maven-release-plugin] prepare release release-0.4.0-rc2 This branch includes the following new commits: new f26f19b Single Quote String Literal for function invoker results (#5456) new b359bad [maven-release-plugin] prepare release release-0.4.0-rc2 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/02: Single Quote String Literal for function invoker results (#5456)
This is an automated email from the ASF dual-hosted git repository.
haibow pushed a commit to branch release-0.4.0-rc2
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit f26f19b674604dbd7afce9e81954198b245e4338
Author: Xiang Fu
AuthorDate: Wed May 27 21:55:14 2020 -0700
Single Quote String Literal for function invoker results (#5456)
---
.../org/apache/pinot/sql/parsers/CalciteSqlParser.java | 3 +++
.../apache/pinot/sql/parsers/CalciteSqlCompilerTest.java | 6 +++---
.../integration/tests/OfflineClusterIntegrationTest.java | 14 ++
3 files changed, 20 insertions(+), 3 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
index 45b3274..2aec8ec 100644
---
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
+++
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
@@ -676,6 +676,9 @@ public class CalciteSqlParser {
try {
FunctionInvoker invoker = new FunctionInvoker(functionInfo);
Object result = invoker.process(arguments);
+if (result instanceof String) {
+ result = String.format("'%s'", result);
+}
return RequestUtils.getLiteralExpression(result);
} catch (Exception e) {
throw new SqlCompilationException(new
IllegalArgumentException("Unsupported function - " + funcName, e));
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index ac0a5cf..aec7917 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -1534,7 +1534,7 @@ public class CalciteSqlCompilerTest {
Function greaterThan = pinotQuery.getFilterExpression().getFunctionCall();
String today =
greaterThan.getOperands().get(1).getLiteral().getStringValue();
String expectedTodayStr =
-
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z"));
+"'" +
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z")) + "'";
Assert.assertEquals(today, expectedTodayStr);
}
@@ -1557,7 +1557,7 @@ public class CalciteSqlCompilerTest {
Assert.assertTrue(expression.getLiteral() != null);
String today = expression.getLiteral().getStringValue();
String expectedTodayStr =
-
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z"));
+"'" +
Instant.now().atZone(ZoneId.of("UTC")).format(DateTimeFormatter.ofPattern("-MM-dd
z")) + "'";
Assert.assertEquals(today, expectedTodayStr);
expression =
CalciteSqlParser.compileToExpression("toDateTime(playerName)");
Assert.assertTrue(expression.getFunctionCall() != null);
@@ -1575,7 +1575,7 @@ public class CalciteSqlCompilerTest {
Assert.assertTrue(expression.getFunctionCall() != null);
expression =
CalciteSqlParser.invokeCompileTimeFunctionExpression(expression);
Assert.assertTrue(expression.getLiteral() != null);
-Assert.assertEquals(expression.getLiteral().getFieldValue(), "emaNreyalp");
+Assert.assertEquals(expression.getLiteral().getFieldValue(),
"'emaNreyalp'");
expression = CalciteSqlParser.compileToExpression("count(*)");
Assert.assertTrue(expression.getFunctionCall() != null);
expression =
CalciteSqlParser.invokeCompileTimeFunctionExpression(expression);
diff --git
a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
index f433a3e..223602c 100644
---
a/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
+++
b/pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineClusterIntegrationTest.java
@@ -24,6 +24,9 @@ import com.fasterxml.jackson.databind.node.ObjectNode;
import com.google.common.collect.ImmutableList;
import java.io.File;
import java.io.IOException;
+import java.time.Instant;
+import java.time.ZoneId;
+import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
@@ -309,6 +312,17 @@ public class OfflineClusterIntegrationTest extends
BaseClusterIntegrationTestSet
}
@Test
+ public void testTimeFunc()
+ throws Exception {
+String sqlQuery = "SELECT toDateTime(now(), '-MM-dd z') FROM mytable";
+JsonNode response = postSqlQuery(sqlQuery, _brokerBaseApiUrl);
+String todayStr =
response.get("resultTable").get("rows").get(0).get(0).asText();
+String
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #5459: [Cleanup] Merge RealtimeSegmentOnlineOfflineStateModel and SegmentOnlineOfflineStateModel in CommonConstants
mcvsubbu commented on a change in pull request #5459: URL: https://github.com/apache/incubator-pinot/pull/5459#discussion_r432213242 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/RoutingManager.java ## @@ -51,7 +51,7 @@ import org.apache.pinot.common.metrics.BrokerMetrics; import org.apache.pinot.common.request.BrokerRequest; import org.apache.pinot.common.utils.CommonConstants; -import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.RealtimeSegmentOnlineOfflineStateModel; +import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.SegmentOnlineOfflineStateModel; Review comment: I dont think merging BrokerStateModel and SegmentStateModel is a good idea. Broker does not have CONSUMING state. I think the reason these are coded separately (for realtime and offline) is because we did not have CONSUMING state before. By now, the state model in helix should have consuming for all pinot use cases. So, merging these is fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #5444: Enhance and simplify the filtering
mcvsubbu commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r432211637
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/AndDocIdIterator.java
##
@@ -18,112 +18,52 @@
*/
package org.apache.pinot.core.operator.dociditerators;
-import java.util.Arrays;
import org.apache.pinot.core.common.BlockDocIdIterator;
import org.apache.pinot.core.common.Constants;
-// TODO: Optimize this
public final class AndDocIdIterator implements BlockDocIdIterator {
- public final BlockDocIdIterator[] docIdIterators;
- public ScanBasedDocIdIterator[] scanBasedDocIdIterators;
- public final int[] docIdPointers;
- public boolean reachedEnd = false;
- public int currentDocId = -1;
- int currentMax = -1;
- private boolean hasScanBasedIterators;
+ public final BlockDocIdIterator[] _docIdIterators;
- public AndDocIdIterator(BlockDocIdIterator[] blockDocIdIterators) {
-int numIndexBasedIterators = 0;
-int numScanBasedIterators = 0;
-for (int i = 0; i < blockDocIdIterators.length; i++) {
- if (blockDocIdIterators[i] instanceof IndexBasedDocIdIterator) {
-numIndexBasedIterators = numIndexBasedIterators + 1;
- } else if (blockDocIdIterators[i] instanceof ScanBasedDocIdIterator) {
-numScanBasedIterators = numScanBasedIterators + 1;
- }
-}
-// if we have at least one index based then do intersection based on index
based only, and then
-// check if matching docs apply on scan based iterator
-if (numIndexBasedIterators > 0 && numScanBasedIterators > 0) {
- hasScanBasedIterators = true;
- int nonScanIteratorsSize = blockDocIdIterators.length -
numScanBasedIterators;
- this.docIdIterators = new BlockDocIdIterator[nonScanIteratorsSize];
- this.scanBasedDocIdIterators = new
ScanBasedDocIdIterator[numScanBasedIterators];
- int nonScanBasedIndex = 0;
- int scanBasedIndex = 0;
- for (int i = 0; i < blockDocIdIterators.length; i++) {
-if (blockDocIdIterators[i] instanceof ScanBasedDocIdIterator) {
- this.scanBasedDocIdIterators[scanBasedIndex++] =
(ScanBasedDocIdIterator) blockDocIdIterators[i];
-} else {
- this.docIdIterators[nonScanBasedIndex++] = blockDocIdIterators[i];
-}
- }
-} else {
- hasScanBasedIterators = false;
- this.docIdIterators = blockDocIdIterators;
-}
-this.docIdPointers = new int[docIdIterators.length];
-Arrays.fill(docIdPointers, -1);
- }
+ private int _nextDocId = 0;
- @Override
- public int advance(int targetDocId) {
-if (currentDocId == Constants.EOF) {
- return currentDocId;
-}
-if (currentDocId >= targetDocId) {
- return currentDocId;
-}
-// next() method will always increment currentMax by 1.
-currentMax = targetDocId - 1;
-return next();
+ public AndDocIdIterator(BlockDocIdIterator[] docIdIterators) {
+_docIdIterators = docIdIterators;
}
@Override
public int next() {
-if (currentDocId == Constants.EOF) {
- return currentDocId;
-}
-currentMax = currentMax + 1;
-// always increment the pointer to current max, when this is called first
time, every one will
-// be set to start of posting list.
-for (int i = 0; i < docIdIterators.length; i++) {
- docIdPointers[i] = docIdIterators[i].advance(currentMax);
- if (docIdPointers[i] == Constants.EOF) {
-reachedEnd = true;
-currentMax = Constants.EOF;
-break;
- }
- if (docIdPointers[i] > currentMax) {
-currentMax = docIdPointers[i];
-if (i > 0) {
- // we need to advance all pointer since we found a new max
- i = -1;
-}
- }
- if (hasScanBasedIterators && i == docIdIterators.length - 1) {
-// this means we found the docId common to all nonScanBased iterators,
now we need to ensure
-// that its also found in scanBasedIterator, if not matched, we
restart the intersection
-for (ScanBasedDocIdIterator iterator : scanBasedDocIdIterators) {
- if (!iterator.isMatch(currentMax)) {
-i = -1;
-currentMax = currentMax + 1;
-break;
+int maxDocId = _nextDocId;
+int maxDocIdIndex = -1;
+int numDocIdIterators = _docIdIterators.length;
+int index = 0;
+while (index < numDocIdIterators) {
+ if (index == maxDocIdIndex) {
+// Skip the index with the max document id
+index++;
Review comment:
It will be useful to add some comments here as to how we make sure that
`index` never overruns. It took me a couple of iterations to get the test cases
to test this.
On that topic, will be useful to add a unit test for this class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5459: [Cleanup] Merge RealtimeSegmentOnlineOfflineStateModel and SegmentOnlineOfflineStateModel in CommonConstants
Jackie-Jiang commented on a change in pull request #5459: URL: https://github.com/apache/incubator-pinot/pull/5459#discussion_r432207006 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/RoutingManager.java ## @@ -51,7 +51,7 @@ import org.apache.pinot.common.metrics.BrokerMetrics; import org.apache.pinot.common.request.BrokerRequest; import org.apache.pinot.common.utils.CommonConstants; -import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.RealtimeSegmentOnlineOfflineStateModel; +import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.SegmentOnlineOfflineStateModel; Review comment: @mcvsubbu If we want to further clean it up, we can merge the BrokerStateModel and SegmentStateModel and put all instance states (`ONLINE`, `CONSUMING`, `OFFLINE`, `ERROR`) under `CommonConstants.Helix.InstanceState`. Thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] mayankshriv opened a new pull request #5465: Support distinctCountRawThetaSketch aggregation that returns serialized sketch.
mayankshriv opened a new pull request #5465: URL: https://github.com/apache/incubator-pinot/pull/5465 1. Support a variation of theta sketch based distinct count aggregation function that returns serialized bytes of the final aggregated sketch, instead of the actual distinct value. 2. The return value is hex encoded String of the serialized sketch bytes. This can be deserialized at the client side by the library using org.apache.commons.codec.binary as: `Hex.decodeHex(stringValue.toCharArray())`. This is the same as any other byte[] value returned by Pinot. 3. Added unit test for the new aggregation function. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #5459: [Cleanup] Merge RealtimeSegmentOnlineOfflineStateModel and SegmentOnlineOfflineStateModel in CommonConstants
mcvsubbu commented on a change in pull request #5459: URL: https://github.com/apache/incubator-pinot/pull/5459#discussion_r432180426 ## File path: pinot-broker/src/main/java/org/apache/pinot/broker/routing/RoutingManager.java ## @@ -51,7 +51,7 @@ import org.apache.pinot.common.metrics.BrokerMetrics; import org.apache.pinot.common.request.BrokerRequest; import org.apache.pinot.common.utils.CommonConstants; -import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.RealtimeSegmentOnlineOfflineStateModel; +import org.apache.pinot.common.utils.CommonConstants.Helix.StateModel.SegmentOnlineOfflineStateModel; Review comment: can we just call it `SegmentStateModel`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on pull request #5461: Adding Support for SQL CASE Statement
fx19880617 commented on pull request #5461: URL: https://github.com/apache/incubator-pinot/pull/5461#issuecomment-635662402 > very nicely done!. You made it look simple. > > Check the datatypes compatibility of left and right transform functions. > > What happens if the right is just a constant, we use LiteralTransformFunction? Correct, if right side is constant, then it's parsed as LiteralTransformFunction. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 edited a comment on pull request #5461: Adding Support for SQL CASE Statement
fx19880617 edited a comment on pull request #5461: URL: https://github.com/apache/incubator-pinot/pull/5461#issuecomment-635662402 > very nicely done!. You made it look simple. > > Check the datatypes compatibility of left and right transform functions. > > What happens if the right is just a constant, we use LiteralTransformFunction? Correct, constant is represented as LiteralTransformFunction. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch default_pinot_startable_to_use_config_files updated (8626188 -> 84ff2ff)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch default_pinot_startable_to_use_config_files
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 8626188 Adding more config alias
discard 51b3e96 Make Pinot Broker/Server can start by just passing a config
file
add 4270def [TE] endpoints for alerts page filters (#5432)
add 0bdb89c Add broker request info to the error msg in combine operator.
(#5443)
add 41f7ef5 Include trailing empty string in group key split (#5449)
add ac443f4 [TE] fix dockerfile to account for #5428 changes (#5452)
add 008be2d Make Literal transformer return string literals (#5453)
add 3514a17 Add PinotServiceManager to start Pinot components (#5266)
add b69453e Adding support for Protobuf input format (#5293)
add cee2b23 [TE] add mock events loader to pinot-thirdeye docker defaults
(#5454)
add d8ea89c Create PULL_REQUEST_TEMPLATE.md (#5345)
add 4988d72 [TE] add back frontend as submodule (#5457)
add b6cb44c [TE] clean up somce code and disable sla alerts till we have
the ramp feature (#5458)
add 7f10c5c Single Quote String Literal for function invoker results
(#5456)
add 94aad60 Include transformFunction in timeFieldSpec toJsonObject
(#5464)
add 6d92937 Update k8s superset scripts (#5425)
add 41e4604 Make Pinot Broker/Server can start by just passing a config
file
add 84ff2ff Adding more config alias
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (8626188)
\
N -- N -- N
refs/heads/default_pinot_startable_to_use_config_files (84ff2ff)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 28 +
.gitignore |4 +-
.travis/.travis_te_nightly_build.sh|4 +-
docker/images/pinot-thirdeye/Dockerfile|3 +
.../pinot-thirdeye/config/ephemeral/detector.yml | 43 +
.../images/pinot-thirdeye/config/ephemeral/rca.yml | 26 +-
.../config/pinot-quickstart/detector.yml | 43 +
.../pinot-thirdeye/config/pinot-quickstart/rca.yml | 26 +-
kubernetes/helm/superset.yaml | 889
pinot-broker/pom.xml |1 -
.../broker/broker/helix/HelixBrokerStarter.java| 32 +-
.../broker/broker/HelixBrokerStarterTest.java |2 +-
.../pinot/common/metrics/AbstractMetrics.java |4 +
.../apache/pinot/common/utils/CommonConstants.java |3 +
.../apache/pinot/common/utils/ServiceStatus.java | 144 ++-
.../parsers/PinotQuery2BrokerRequestConverter.java |2 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java |3 +
.../apache/pinot/common/data/FieldSpecTest.java| 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 19 +-
pinot-common/src/test/resources/schemaTest.schema | 25 +-
.../apache/pinot/controller/ControllerStarter.java | 28 +-
.../core/operator/CombineGroupByOperator.java |8 +-
.../operator/CombineGroupByOrderByOperator.java| 10 +-
.../pinot/core/operator/CombineOperator.java |6 +-
.../function/LiteralTransformFunction.java | 12 +-
.../aggregation/groupby/GroupKeyGenerator.java |8 +
.../query/aggregation/groupby/GroupKeyTest.java| 66 ++
.../pinot/integration/tests/ClusterTest.java |6 +-
.../tests/OfflineClusterIntegrationTest.java | 14 +
.../tests/ServerStarterIntegrationTest.java|1 +
.../org/apache/pinot/minion/MinionStarter.java | 25 +-
.../v0_deprecated/pinot-hadoop/pom.xml | 11 +
.../hadoop/job/mappers/SegmentCreationMapper.java | 10 +
.../v0_deprecated/pinot-spark/pom.xml | 12 +-
.../spark/jobs/SparkSegmentCreationFunction.java | 10 +
.../{pinot-csv => pinot-protobuf}/pom.xml | 15 +-
.../protobuf/ProtoBufRecordExtractor.java} | 18 +-
.../protobuf/ProtoBufRecordReader.java}| 102 +-
.../protobuf/ProtoBufRecordReaderConfig.java | 19 +-
.../protobuf/ProtoBufRecordReaderTest.java | 129 ++-
.../pinot/plugin/inputformat/protobuf/Sample.java | 1075
.../pinot-protobuf/src/test/resources/sample.desc |8 +
[GitHub] [incubator-pinot] siddharthteotia edited a comment on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia edited a comment on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635650062 Here are the JMH results: cc @kishoreg @Jackie-Jiang Please see the [spreadsheet](https://docs.google.com/spreadsheets/d/1mz_TQe0rXadWPtA_Xov6cXwYrSvUpQB1p1b_ZqROTDQ/edit#gid=497160959) for additional performance numbers obtained via manual instrumentation Benchmark | Score | Mode -- | -- | -- BenchmarkPinotDataBitSet.twoBitBulkWithGaps | 28.537 ops/ms | Throughpput BenchmarkPinotDataBitSet.twoBitBulkWithGapsFast | 44.332 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGaps | 28.262 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGapsFast | 42.449 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGaps | 26.816 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGapsFast | 38.777 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGaps | 21.095 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGapsFast | 38.380 ops/ms | Throughpput  |  |  BenchmarkPinotDataBitSet.twoBitContiguous | 26.188 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitContiguousFast | 15.692 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguous | 26.113 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguousFast | 15.178 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguous | 26.693 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguousFast | 15.726 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguous | 43.968 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguousFast | 21.390 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguous | 32.504 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousFast | 14.614 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguous | 30.794 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousFast | 14.583 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguous | 16.525 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousFast | 10.777 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguous | 54.731 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousFast | 19.312 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguousUnaligned | 32.018 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousUnalignedFast | 14.344 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnaligned | 21.204 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedFast | 15.393 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnaligned | 20.125 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnalignedFast | 14.836 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnaligned | 58.086 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnalignedFast | 22.002 ms/op | Avgt This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia edited a comment on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia edited a comment on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635650062 Here are the JMH results: cc @kishoreg @Jackie-Jiang Please see the spreadsheet for additional performance numbers obtained via manual instrumentation Benchmark | Score | Mode -- | -- | -- BenchmarkPinotDataBitSet.twoBitBulkWithGaps | 28.537 ops/ms | Throughpput BenchmarkPinotDataBitSet.twoBitBulkWithGapsFast | 44.332 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGaps | 28.262 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGapsFast | 42.449 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGaps | 26.816 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGapsFast | 38.777 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGaps | 21.095 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGapsFast | 38.380 ops/ms | Throughpput  |  |  BenchmarkPinotDataBitSet.twoBitContiguous | 26.188 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitContiguousFast | 15.692 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguous | 26.113 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguousFast | 15.178 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguous | 26.693 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguousFast | 15.726 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguous | 43.968 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguousFast | 21.390 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguous | 32.504 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousFast | 14.614 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguous | 30.794 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousFast | 14.583 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguous | 16.525 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousFast | 10.777 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguous | 54.731 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousFast | 19.312 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguousUnaligned | 32.018 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousUnalignedFast | 14.344 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnaligned | 21.204 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedFast | 15.393 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnaligned | 20.125 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnalignedFast | 14.836 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnaligned | 58.086 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnalignedFast | 22.002 ms/op | Avgt This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia edited a comment on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia edited a comment on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635650062 Here are the JMH results: cc @kishoreg @Jackie-Jiang Benchmark | Score | Mode -- | -- | -- BenchmarkPinotDataBitSet.twoBitBulkWithGaps | 28.537 ops/ms | Throughpput BenchmarkPinotDataBitSet.twoBitBulkWithGapsFast | 44.332 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGaps | 28.262 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGapsFast | 42.449 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGaps | 26.816 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGapsFast | 38.777 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGaps | 21.095 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGapsFast | 38.380 ops/ms | Throughpput  |  |  BenchmarkPinotDataBitSet.twoBitContiguous | 26.188 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitContiguousFast | 15.692 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguous | 26.113 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguousFast | 15.178 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguous | 26.693 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguousFast | 15.726 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguous | 43.968 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguousFast | 21.390 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguous | 32.504 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousFast | 14.614 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguous | 30.794 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousFast | 14.583 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguous | 16.525 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousFast | 10.777 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguous | 54.731 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousFast | 19.312 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguousUnaligned | 32.018 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousUnalignedFast | 14.344 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnaligned | 21.204 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedFast | 15.393 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnaligned | 20.125 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnalignedFast | 14.836 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnaligned | 58.086 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnalignedFast | 22.002 ms/op | Avgt This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia edited a comment on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia edited a comment on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635650062 Here are the JMH results: cc @kishoreg @Jackie-Jiang ` Benchmark | Score | Mode -- | -- | -- BenchmarkPinotDataBitSet.twoBitBulkWithGaps | 28.537 ops/ms | Throughpput BenchmarkPinotDataBitSet.twoBitBulkWithGapsFast | 44.332 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGaps | 28.262 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGapsFast | 42.449 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGaps | 26.816 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGapsFast | 38.777 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGaps | 21.095 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGapsFast | 38.380 ops/ms | Throughpput  |  |  BenchmarkPinotDataBitSet.twoBitContiguous | 26.188 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitContiguousFast | 15.692 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguous | 26.113 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguousFast | 15.178 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguous | 26.693 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguousFast | 15.726 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguous | 43.968 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguousFast | 21.390 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguous | 32.504 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousFast | 14.614 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguous | 30.794 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousFast | 14.583 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguous | 16.525 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousFast | 10.777 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguous | 54.731 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousFast | 19.312 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguousUnaligned | 32.018 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousUnalignedFast | 14.344 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnaligned | 21.204 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedFast | 15.393 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnaligned | 20.125 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnalignedFast | 14.836 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnaligned | 58.086 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnalignedFast | 22.002 ms/op | Avgt ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia commented on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635650062 Here are the JMH results: ` Benchmark | Score | Mode -- | -- | -- BenchmarkPinotDataBitSet.twoBitBulkWithGaps | 28.537 ops/ms | Throughpput BenchmarkPinotDataBitSet.twoBitBulkWithGapsFast | 44.332 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGaps | 28.262 ops/ms | Throughpput BenchmarkPinotDataBitSet.fourBitBulkWithGapsFast | 42.449 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGaps | 26.816 ops/ms | Throughpput BenchmarkPinotDataBitSet.eightBitBulkWithGapsFast | 38.777 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGaps | 21.095 ops/ms | Throughpput BenchmarkPinotDataBitSet.sixteenBitBulkWithGapsFast | 38.380 ops/ms | Throughpput  |  |  BenchmarkPinotDataBitSet.twoBitContiguous | 26.188 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitContiguousFast | 15.692 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguous | 26.113 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitContiguousFast | 15.178 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguous | 26.693 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitContiguousFast | 15.726 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguous | 43.968 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitContiguousFast | 21.390 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguous | 32.504 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousFast | 14.614 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguous | 30.794 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousFast | 14.583 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguous | 16.525 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousFast | 10.777 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguous | 54.731 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousFast | 19.312 ms/op | Avgt  |  |  BenchmarkPinotDataBitSet.twoBitBulkContiguousUnaligned | 32.018 ms/op | Avgt BenchmarkPinotDataBitSet.twoBitBulkContiguousUnalignedFast | 14.344 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnaligned | 21.204 ms/op | Avgt BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedFast | 15.393 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnaligned | 20.125 ms/op | Avgt BenchmarkPinotDataBitSet.fourBitBulkContiguousUnalignedFast | 14.836 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnaligned | 58.086 ms/op | Avgt BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnalignedFast | 22.002 ms/op | Avgt ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia edited a comment on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia edited a comment on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635647360 Address the review comments. Please take another look. This is standalone code at this point so want to get this in sooner and put up follow-ups in next couple of days. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #5444: Enhance and simplify the filtering
Jackie-Jiang commented on pull request #5444: URL: https://github.com/apache/incubator-pinot/pull/5444#issuecomment-635648350 Removed the changes not closely related to the enhancement to reduce the size of the change This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia commented on pull request #5409: URL: https://github.com/apache/incubator-pinot/pull/5409#issuecomment-635647360 Address the review comments. Please take another look. This is standalone code at this point so want to get this in sooner and put up follow-ups in next couple of days. Here are the JMH results: `BenchmarkMode Cnt Score Units BenchmarkPinotDataBitSet.twoBitBulkWithGaps thrpt3 28.537 ops/ms BenchmarkPinotDataBitSet.twoBitBulkWithGapsFast thrpt3 44.332 ops/ms BenchmarkPinotDataBitSet.fourBitBulkWithGaps thrpt3 28.262 ops/ms BenchmarkPinotDataBitSet.fourBitBulkWithGapsFast thrpt3 42.449 ops/ms BenchmarkPinotDataBitSet.eightBitBulkWithGaps thrpt3 26.816 ops/ms BenchmarkPinotDataBitSet.eightBitBulkWithGapsFast thrpt3 38.777 ops/ms BenchmarkPinotDataBitSet.sixteenBitBulkWithGaps thrpt3 21.095 ops/ms BenchmarkPinotDataBitSet.sixteenBitBulkWithGapsFast thrpt3 38.380 ops/ms BenchmarkPinotDataBitSet.twoBitContiguousavgt 3 26.188 ms/op BenchmarkPinotDataBitSet.twoBitContiguousFastavgt 3 15.692 ms/op BenchmarkPinotDataBitSet.fourBitContiguous avgt 3 26.113 ms/op BenchmarkPinotDataBitSet.fourBitContiguousFast avgt 3 15.178 ms/op BenchmarkPinotDataBitSet.eightBitContiguous avgt 3 26.693 ms/op BenchmarkPinotDataBitSet.eightBitContiguousFast avgt 3 15.726 ms/op BenchmarkPinotDataBitSet.sixteenBitContiguousavgt 3 43.968 ms/op BenchmarkPinotDataBitSet.sixteenBitContiguousFastavgt 3 21.390 ms/op BenchmarkPinotDataBitSet.twoBitBulkContiguousavgt 3 32.504 ms/op BenchmarkPinotDataBitSet.twoBitBulkContiguousFastavgt 3 14.614 ms/op BenchmarkPinotDataBitSet.fourBitBulkContiguous avgt 3 30.794 ms/op BenchmarkPinotDataBitSet.fourBitBulkContiguousFast avgt 3 14.583 ms/op BenchmarkPinotDataBitSet.eightBitBulkContiguous avgt 3 16.525 ms/op BenchmarkPinotDataBitSet.eightBitBulkContiguousFast avgt 3 10.777ms/op BenchmarkPinotDataBitSet.sixteenBitBulkContiguousavgt 3 54.731 ms/op BenchmarkPinotDataBitSet.sixteenBitBulkContiguousFastavgt 3 19.312 ms/op BenchmarkPinotDataBitSet.twoBitBulkContiguousUnaligned avgt 2 32.018 ms/op BenchmarkPinotDataBitSet.twoBitBulkContiguousUnalignedFast avgt 2 14.344 ms/op BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedavgt 2 21.204 ms/op BenchmarkPinotDataBitSet.eightBitBulkContiguousUnalignedFastavgt 2 15.393 ms/op BenchmarkPinotDataBitSet.fourBitBulkContiguousUnaligned avgt 2 20.125 ms/op BenchmarkPinotDataBitSet.fourBitBulkContiguousUnalignedFast avgt 2 14.836 ms/op BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnaligned avgt 2 58.086 ms/op BenchmarkPinotDataBitSet.sixteenBitBulkContiguousUnalignedFast avgt 2 22.002 ms/op` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #5409: Faster bit unpacking (Part 1)
siddharthteotia commented on a change in pull request #5409:
URL: https://github.com/apache/incubator-pinot/pull/5409#discussion_r432163975
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/io/util/PinotDataBitSetV2.java
##
@@ -0,0 +1,516 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.io.util;
+
+import java.io.Closeable;
+import java.io.IOException;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.core.segment.memory.PinotDataBuffer;
+
+
+public abstract class PinotDataBitSetV2 implements Closeable {
+ private static final int BYTE_MASK = 0xFF;
+ static final int MAX_VALUES_UNPACKED_SINGLE_ALIGNED_READ = 16; // comes from
2-bit encoding
+
+ private static final ThreadLocal THREAD_LOCAL_DICT_IDS =
+ ThreadLocal.withInitial(() -> new
int[DocIdSetPlanNode.MAX_DOC_PER_CALL]);
+
+ protected PinotDataBuffer _dataBuffer;
+ protected int _numBitsPerValue;
+
+ /**
+ * Unpack single dictId at the given docId. This is efficient
+ * because of simplified bitmath.
+ * @param index docId
+ * @return unpacked dictId
+ */
+ public abstract int readInt(int index);
+
+ /**
+ * Unpack dictIds for a contiguous range of docIds represented by startIndex
+ * and length. This uses vectorization as much as possible for all the
aligned
+ * reads and also takes care of the small byte-sized window of unaligned
read.
+ * @param startIndex start docId
+ * @param length length
+ * @param out out array to store the unpacked dictIds
+ */
+ public abstract void readInt(int startIndex, int length, int[] out);
+
+ /**
+ * Unpack dictIds for an array of docIds[] which is not necessarily
+ * contiguous. So there could be gaps in the array:
+ * e.g: [1, 3, 7, 9, 11, 12]
+ * The actual read is done by the previous API since that is efficient
+ * as it exploits contiguity and uses vectorization. However, since
+ * the out[] array has to be correctly populated with the unpacked dictId
+ * for each docId, a post-processing step is needed after the bulk contiguous
+ * read to correctly set the unpacked dictId into the out array throwing away
+ * the unnecessary dictIds unpacked as part of contiguous read
+ * @param docIds docIds array
+ * @param docIdsStartIndex starting index in the docIds array
+ * @param length length to read (number of docIds to read in the array)
+ * @param out out array to store the unpacked dictIds
+ * @param outpos starting index in the out array
+ */
+ public void readInt(int[] docIds, int docIdsStartIndex, int length, int[]
out, int outpos) {
+int startDocId = docIds[docIdsStartIndex];
+int endDocId = docIds[docIdsStartIndex + length - 1];
+int[] dictIds = THREAD_LOCAL_DICT_IDS.get();
+// do a contiguous bulk read
+readInt(startDocId, endDocId - startDocId + 1, dictIds);
+out[outpos] = dictIds[0];
+// set the unpacked dictId correctly. this is needed since there could
+// be gaps and some dictIds may have to be thrown/ignored.
+for (int i = 1; i < length; i++) {
+ out[outpos + i] = dictIds[docIds[docIdsStartIndex + i] - startDocId];
+}
+ }
+
+ public static PinotDataBitSetV2 createBitSet(PinotDataBuffer
pinotDataBuffer, int numBitsPerValue) {
+switch (numBitsPerValue) {
+ case 2:
+return new Bit2Encoded(pinotDataBuffer, numBitsPerValue);
+ case 4:
+return new Bit4Encoded(pinotDataBuffer, numBitsPerValue);
+ case 8:
+return new Bit8Encoded(pinotDataBuffer, numBitsPerValue);
+ case 16:
+return new Bit16Encoded(pinotDataBuffer, numBitsPerValue);
+ case 32:
+return new RawInt(pinotDataBuffer, numBitsPerValue);
+ default:
+throw new UnsupportedOperationException(numBitsPerValue + "not
supported by PinotDataBitSetV2");
+}
+ }
+
+ public static class Bit2Encoded extends PinotDataBitSetV2 {
+Bit2Encoded(PinotDataBuffer dataBuffer, int numBits) {
+ _dataBuffer = dataBuffer;
+ _numBitsPerValue = numBits;
+}
+
+@Override
+public int readInt(int index) {
+ long bitOffset = (long) index * _numBitsPerValue;
+ int byteOffset = (int)
[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #5461: Adding Support for SQL CASE Statement
kishoreg commented on a change in pull request #5461:
URL: https://github.com/apache/incubator-pinot/pull/5461#discussion_r432162832
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/BinaryOperatorTransformFunction.java
##
@@ -0,0 +1,114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.common.DataSource;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * BinaryOperatorTransformFunction abstracts common functions for
binary operators (=, !=, >=, >, <=, <)
+ * The results are in boolean format and stored as an integer array with 1
represents true and 0 represents false.
+ *
+ */
+public abstract class BinaryOperatorTransformFunction extends
BaseTransformFunction {
+
+ protected TransformFunction _leftTransformFunction;
+ protected TransformFunction _rightTransformFunction;
+ protected int[] _results;
+
+ @Override
+ public void init(List arguments, Map
dataSourceMap) {
+// Check that there are more than 1 arguments
+if (arguments.size() != 2) {
Review comment:
also, check that the return types are comparable, for e.g. if one
returns string and other is int, we should throw exception
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/BinaryOperatorTransformFunction.java
##
@@ -0,0 +1,114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.common.DataSource;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * BinaryOperatorTransformFunction abstracts common functions for
binary operators (=, !=, >=, >, <=, <)
+ * The results are in boolean format and stored as an integer array with 1
represents true and 0 represents false.
+ *
+ */
+public abstract class BinaryOperatorTransformFunction extends
BaseTransformFunction {
+
+ protected TransformFunction _leftTransformFunction;
+ protected TransformFunction _rightTransformFunction;
+ protected int[] _results;
+
+ @Override
+ public void init(List arguments, Map
dataSourceMap) {
+// Check that there are more than 1 arguments
+if (arguments.size() != 2) {
+ throw new IllegalArgumentException("Exact 2 arguments are required for
greater transform function");
+}
+_leftTransformFunction = arguments.get(0);
+_rightTransformFunction = arguments.get(1);
+ }
+
+ @Override
+ public TransformResultMetadata getResultMetadata() {
+return INT_SV_NO_DICTIONARY_METADATA;
+ }
+
+ protected void fillResultArray(ProjectionBlock projectionBlock) {
+if (_results == null) {
+ _results = new int[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+}
+FieldSpec.DataType dataType =
[incubator-pinot] branch enhance_quickstart updated (7504fbc -> 5bf2099)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch enhance_quickstart
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 7504fbc Update Quickstart to take tmp dir as a parameter and default
to current dir
add 4270def [TE] endpoints for alerts page filters (#5432)
add 0bdb89c Add broker request info to the error msg in combine operator.
(#5443)
add 41f7ef5 Include trailing empty string in group key split (#5449)
add ac443f4 [TE] fix dockerfile to account for #5428 changes (#5452)
add 008be2d Make Literal transformer return string literals (#5453)
add 3514a17 Add PinotServiceManager to start Pinot components (#5266)
add b69453e Adding support for Protobuf input format (#5293)
add cee2b23 [TE] add mock events loader to pinot-thirdeye docker defaults
(#5454)
add d8ea89c Create PULL_REQUEST_TEMPLATE.md (#5345)
add 4988d72 [TE] add back frontend as submodule (#5457)
add b6cb44c [TE] clean up somce code and disable sla alerts till we have
the ramp feature (#5458)
add 7f10c5c Single Quote String Literal for function invoker results
(#5456)
add 94aad60 Include transformFunction in timeFieldSpec toJsonObject
(#5464)
add 6d92937 Update k8s superset scripts (#5425)
add 5bf2099 Update Quickstart to take tmp dir as a parameter and default
to current dir
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7504fbc)
\
N -- N -- N refs/heads/enhance_quickstart (5bf2099)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 28 +
.gitignore |4 +-
.travis/.travis_te_nightly_build.sh|4 +-
docker/images/pinot-thirdeye/Dockerfile|3 +
.../pinot-thirdeye/config/ephemeral/detector.yml | 43 +
.../images/pinot-thirdeye/config/ephemeral/rca.yml | 26 +-
.../config/pinot-quickstart/detector.yml | 43 +
.../pinot-thirdeye/config/pinot-quickstart/rca.yml | 26 +-
kubernetes/helm/superset.yaml | 889
pinot-broker/pom.xml |1 -
.../broker/broker/helix/HelixBrokerStarter.java| 32 +-
.../broker/broker/HelixBrokerStarterTest.java |2 +-
.../pinot/common/metrics/AbstractMetrics.java |4 +
.../apache/pinot/common/utils/CommonConstants.java |4 +
.../apache/pinot/common/utils/ServiceStatus.java | 144 ++-
.../parsers/PinotQuery2BrokerRequestConverter.java |2 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java |3 +
.../apache/pinot/common/data/FieldSpecTest.java| 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 19 +-
pinot-common/src/test/resources/schemaTest.schema | 25 +-
.../apache/pinot/controller/ControllerStarter.java | 28 +-
.../core/operator/CombineGroupByOperator.java |8 +-
.../operator/CombineGroupByOrderByOperator.java| 10 +-
.../pinot/core/operator/CombineOperator.java |6 +-
.../function/LiteralTransformFunction.java | 12 +-
.../aggregation/groupby/GroupKeyGenerator.java |8 +
.../query/aggregation/groupby/GroupKeyTest.java| 66 ++
.../pinot/integration/tests/ClusterTest.java |6 +-
.../tests/OfflineClusterIntegrationTest.java | 14 +
.../tests/ServerStarterIntegrationTest.java|1 +
.../org/apache/pinot/minion/MinionStarter.java | 25 +-
.../v0_deprecated/pinot-hadoop/pom.xml | 11 +
.../hadoop/job/mappers/SegmentCreationMapper.java | 10 +
.../v0_deprecated/pinot-spark/pom.xml | 12 +-
.../spark/jobs/SparkSegmentCreationFunction.java | 10 +
.../{pinot-csv => pinot-protobuf}/pom.xml | 15 +-
.../protobuf/ProtoBufRecordExtractor.java} | 18 +-
.../protobuf/ProtoBufRecordReader.java}| 102 +-
.../protobuf/ProtoBufRecordReaderConfig.java | 19 +-
.../protobuf/ProtoBufRecordReaderTest.java | 129 ++-
.../pinot/plugin/inputformat/protobuf/Sample.java | 1075
.../pinot-protobuf/src/test/resources/sample.desc |8 +
.../pinot-protobuf/src/test/resources/sample.proto |9 +
pinot-plugins/pinot-input-format/pom.xml |1 +
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #5445: Update Quickstart to take tmp dir as a parameter and default to current dir
fx19880617 commented on a change in pull request #5445: URL: https://github.com/apache/incubator-pinot/pull/5445#discussion_r432157695 ## File path: pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/QuickStartCommand.java ## @@ -34,6 +34,9 @@ @Option(name = "-type", required = false, metaVar = "", usage = "Type of quickstart, supported: STREAM/BATCH/HYBRID") private String _type; + @Option(name = "-tmpDir", required = false, metaVar = "", usage = "Temp Directory to host quickstart data") Review comment: Adding alias for this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch enhance_log4j updated (72c7411 -> 759223d)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch enhance_log4j
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 72c7411 Update log4j2.xml
discard 9b7be2f Update Pinot-admin and quickstart log4j config
add 4270def [TE] endpoints for alerts page filters (#5432)
add 0bdb89c Add broker request info to the error msg in combine operator.
(#5443)
add 41f7ef5 Include trailing empty string in group key split (#5449)
add ac443f4 [TE] fix dockerfile to account for #5428 changes (#5452)
add 008be2d Make Literal transformer return string literals (#5453)
add 3514a17 Add PinotServiceManager to start Pinot components (#5266)
add b69453e Adding support for Protobuf input format (#5293)
add cee2b23 [TE] add mock events loader to pinot-thirdeye docker defaults
(#5454)
add d8ea89c Create PULL_REQUEST_TEMPLATE.md (#5345)
add 4988d72 [TE] add back frontend as submodule (#5457)
add b6cb44c [TE] clean up somce code and disable sla alerts till we have
the ramp feature (#5458)
add 7f10c5c Single Quote String Literal for function invoker results
(#5456)
add 94aad60 Include transformFunction in timeFieldSpec toJsonObject
(#5464)
add 6d92937 Update k8s superset scripts (#5425)
add 832d097 Update Pinot-admin and quickstart log4j config
add 759223d Update log4j2.xml
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (72c7411)
\
N -- N -- N refs/heads/enhance_log4j (759223d)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 28 +
.gitignore |4 +-
.travis/.travis_te_nightly_build.sh|4 +-
docker/images/pinot-thirdeye/Dockerfile|3 +
.../pinot-thirdeye/config/ephemeral/detector.yml | 43 +
.../images/pinot-thirdeye/config/ephemeral/rca.yml | 26 +-
.../config/pinot-quickstart/detector.yml | 43 +
.../pinot-thirdeye/config/pinot-quickstart/rca.yml | 26 +-
kubernetes/helm/superset.yaml | 889
pinot-broker/pom.xml |1 -
.../broker/broker/helix/HelixBrokerStarter.java| 32 +-
.../broker/broker/HelixBrokerStarterTest.java |2 +-
.../pinot/common/metrics/AbstractMetrics.java |4 +
.../apache/pinot/common/utils/CommonConstants.java |4 +
.../apache/pinot/common/utils/ServiceStatus.java | 144 ++-
.../parsers/PinotQuery2BrokerRequestConverter.java |2 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java |3 +
.../apache/pinot/common/data/FieldSpecTest.java| 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 19 +-
pinot-common/src/test/resources/schemaTest.schema | 25 +-
.../apache/pinot/controller/ControllerStarter.java | 28 +-
.../core/operator/CombineGroupByOperator.java |8 +-
.../operator/CombineGroupByOrderByOperator.java| 10 +-
.../pinot/core/operator/CombineOperator.java |6 +-
.../function/LiteralTransformFunction.java | 12 +-
.../aggregation/groupby/GroupKeyGenerator.java |8 +
.../query/aggregation/groupby/GroupKeyTest.java| 66 ++
.../pinot/integration/tests/ClusterTest.java |6 +-
.../tests/OfflineClusterIntegrationTest.java | 14 +
.../tests/ServerStarterIntegrationTest.java|1 +
.../org/apache/pinot/minion/MinionStarter.java | 25 +-
.../v0_deprecated/pinot-hadoop/pom.xml | 11 +
.../hadoop/job/mappers/SegmentCreationMapper.java | 10 +
.../v0_deprecated/pinot-spark/pom.xml | 12 +-
.../spark/jobs/SparkSegmentCreationFunction.java | 10 +
.../{pinot-csv => pinot-protobuf}/pom.xml | 15 +-
.../protobuf/ProtoBufRecordExtractor.java} | 18 +-
.../protobuf/ProtoBufRecordReader.java}| 102 +-
.../protobuf/ProtoBufRecordReaderConfig.java | 19 +-
.../protobuf/ProtoBufRecordReaderTest.java | 129 ++-
.../pinot/plugin/inputformat/protobuf/Sample.java | 1075
.../pinot-protobuf/src/test/resources/sample.desc |8 +
.../pinot-protobuf/src/test/resources/sample.proto |9 +
pinot-plugins/pinot-input-format/pom.xml |1 +
[incubator-pinot] branch master updated: Update k8s superset scripts (#5425)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 6d92937 Update k8s superset scripts (#5425)
6d92937 is described below
commit 6d929373d7c8f911ece32fdedb75133f96cafbaf
Author: Xiang Fu
AuthorDate: Thu May 28 15:13:10 2020 -0700
Update k8s superset scripts (#5425)
---
kubernetes/helm/superset.yaml | 889 +-
1 file changed, 445 insertions(+), 444 deletions(-)
diff --git a/kubernetes/helm/superset.yaml b/kubernetes/helm/superset.yaml
index b77a249..8ba7f29 100644
--- a/kubernetes/helm/superset.yaml
+++ b/kubernetes/helm/superset.yaml
@@ -240,8 +240,8 @@ data:
"css": "",
"dashboard_title": "AirlineStats",
"description": null,
- "json_metadata": "{\"remote_id\": 2, \"import_time\": 1572681302,
\"filter_immune_slices\": [], \"timed_refresh_immune_slices\": [],
\"filter_immune_slice_fields\": {}, \"expanded_slices\": {},
\"refresh_frequency\": 0, \"default_filters\": \"{}\"}",
- "position_json":
"{\"CHART-EPWTG21UAl\":{\"children\":[],\"id\":\"CHART-EPWTG21UAl\",\"meta\":{\"chartId\":2,\"height\":50,\"sliceName\":\"Destination
States Flight Time
Series\",\"width\":4},\"parents\":[\"ROOT_ID\",\"GRID_ID\",\"ROW-DHJKqb0-2\"],\"type\":\"CHART\"},\"CHART-F0aEdk-eR-\":{\"children\":[],\"id\":\"CHART-F0aEdk-eR-\",\"meta\":{\"chartId\":3,\"height\":50,\"sliceName\":\"Flight
Count Per
City\",\"width\":4},\"parents\":[\"ROOT_ID\",\"GRID_ID\",\"ROW-DHJKqb0-2\"],\
[...]
+ "json_metadata": "{\"default_filters\": \"{}\", \"expanded_slices\":
{}, \"import_time\": 1584979678, \"refresh_frequency\": 0, \"remote_id\": 1,
\"timed_refresh_immune_slices\": []}",
+ "position_json": "{\"CHART-EPWTG21UAl\": {\"children\": [], \"id\":
\"CHART-EPWTG21UAl\", \"meta\": {\"chartId\": 2, \"height\": 50, \"sliceName\":
\"Destination States Flight Time Series\", \"width\": 4}, \"parents\":
[\"ROOT_ID\", \"GRID_ID\", \"ROW-DHJKqb0-2\"], \"type\": \"CHART\"},
\"CHART-F0aEdk-eR-\": {\"children\": [], \"id\": \"CHART-F0aEdk-eR-\",
\"meta\": {\"chartId\": 3, \"height\": 50, \"sliceName\": \"Flight Count Per
City\", \"width\": 4}, \"parents\": [\"ROOT_ID [...]
"slices": [
{
"__Slice__": {
@@ -249,7 +249,7 @@ data:
"datasource_name": "airlineStats",
"datasource_type": "table",
"id": 1,
- "params": "{\"adhoc_filters\": [], \"bar_stacked\": false,
\"bottom_margin\": \"auto\", \"color_scheme\": \"bnbColors\", \"columns\": [],
\"contribution\": false, \"datasource\": \"1__table\", \"granularity_sqla\":
null, \"groupby\": [\"DestCityName\"], \"label_colors\": {}, \"metrics\":
[\"count\"], \"order_bars\": false, \"reduce_x_ticks\": false, \"row_limit\":
1, \"show_bar_value\": false, \"show_controls\": false, \"show_legend\":
true, \"slice_id\": 1, \"time_grai [...]
+ "params": "{\"adhoc_filters\": [], \"bar_stacked\": false,
\"bottom_margin\": \"auto\", \"color_scheme\": \"bnbColors\", \"columns\": [],
\"contribution\": false, \"database_name\": \"pinot-quickstart\",
\"datasource\": \"1__table\", \"datasource_name\": \"airlineStats\",
\"granularity_sqla\": null, \"groupby\": [\"DestCityName\"], \"import_time\":
1584979678, \"label_colors\": {}, \"metrics\": [\"count\"], \"order_bars\":
false, \"reduce_x_ticks\": false, \"remote_id\": 1, [...]
"slice_name": "Flight Destination City Count",
"viz_type": "dist_bar"
}
@@ -260,7 +260,7 @@ data:
"datasource_name": "airlineStats",
"datasource_type": "table",
"id": 2,
- "params": "{\"adhoc_filters\": [], \"annotation_layers\": [],
\"bottom_margin\": \"auto\", \"color_scheme\": \"bnbColors\",
\"comparison_type\": \"values\", \"contribution\": false, \"datasource\":
\"1__table\", \"granularity_sqla\": \"SecondsSinceEpoch\", \"groupby\":
[\"DestStateName\"], \"label_colors\": {}, \"left_margin\": \"auto\",
\"line_interpolation\": \"linear\", \"metrics\": [\"count\"], \"order_desc\":
true, \"resample_method\": null, \"resample_rule\": null, \" [...]
+ "params": "{\"adhoc_filters\": [], \"annotation_layers\": [],
\"bottom_margin\": \"auto\", \"color_scheme\": \"bnbColors\",
\"comparison_type\": \"values\", \"contribution\": false, \"database_name\":
\"pinot-quickstart\", \"datasource\": \"1__table\", \"datasource_name\":
\"airlineStats\", \"granularity_sqla\": \"SecondsSinceEpoch\", \"groupby\":
[\"DestStateName\"], \"import_time\": 1584979678, \"label_colors\": {},
\"left_margin\": \"auto\", \"line_interpolation\": \"lin [...]
"slice_name": "Destination States Flight Time Series",
"viz_type": "line"
[GitHub] [incubator-pinot] fx19880617 closed issue #5423: [Docs] Running Pinot in Kubernetes: Load Demo data source issue
fx19880617 closed issue #5423: URL: https://github.com/apache/incubator-pinot/issues/5423 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 merged pull request #5425: Update k8s superset scripts
fx19880617 merged pull request #5425: URL: https://github.com/apache/incubator-pinot/pull/5425 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch ignore-quickstart-files updated (7aa6a60 -> c1517a5)
This is an automated email from the ASF dual-hosted git repository.
kishoreg pushed a change to branch ignore-quickstart-files
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 7aa6a60 Adding files generated by running quickstart to gitignore
add 5223abf Update thirdeye docker image build (#5439)
add 4270def [TE] endpoints for alerts page filters (#5432)
add 0bdb89c Add broker request info to the error msg in combine operator.
(#5443)
add 41f7ef5 Include trailing empty string in group key split (#5449)
add ac443f4 [TE] fix dockerfile to account for #5428 changes (#5452)
add 008be2d Make Literal transformer return string literals (#5453)
add 3514a17 Add PinotServiceManager to start Pinot components (#5266)
add b69453e Adding support for Protobuf input format (#5293)
add cee2b23 [TE] add mock events loader to pinot-thirdeye docker defaults
(#5454)
add d8ea89c Create PULL_REQUEST_TEMPLATE.md (#5345)
add 4988d72 [TE] add back frontend as submodule (#5457)
add b6cb44c [TE] clean up somce code and disable sla alerts till we have
the ramp feature (#5458)
add 7f10c5c Single Quote String Literal for function invoker results
(#5456)
add 94aad60 Include transformFunction in timeFieldSpec toJsonObject
(#5464)
add c1517a5 Adding files generated by running quickstart to gitignore
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (7aa6a60)
\
N -- N -- N refs/heads/ignore-quickstart-files (c1517a5)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 28 +
.gitignore |2 +
.travis/.travis_te_nightly_build.sh|4 +-
docker/images/pinot-thirdeye/Dockerfile|6 +-
docker/images/pinot-thirdeye/bin/start-thirdeye.sh | 10 +-
.../pinot-thirdeye/config/ephemeral/detector.yml | 43 +
.../images/pinot-thirdeye/config/ephemeral/rca.yml | 26 +-
.../config/pinot-quickstart/detector.yml | 43 +
.../pinot-thirdeye/config/pinot-quickstart/rca.yml | 26 +-
pinot-broker/pom.xml |1 -
.../broker/broker/helix/HelixBrokerStarter.java| 32 +-
.../broker/broker/HelixBrokerStarterTest.java |2 +-
.../pinot/common/metrics/AbstractMetrics.java |4 +
.../apache/pinot/common/utils/CommonConstants.java |4 +
.../apache/pinot/common/utils/ServiceStatus.java | 144 ++-
.../parsers/PinotQuery2BrokerRequestConverter.java |2 +-
.../apache/pinot/sql/parsers/CalciteSqlParser.java |3 +
.../apache/pinot/common/data/FieldSpecTest.java| 10 +-
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 19 +-
pinot-common/src/test/resources/schemaTest.schema | 25 +-
.../apache/pinot/controller/ControllerStarter.java | 28 +-
.../core/operator/CombineGroupByOperator.java |8 +-
.../operator/CombineGroupByOrderByOperator.java| 10 +-
.../pinot/core/operator/CombineOperator.java |6 +-
.../function/LiteralTransformFunction.java | 12 +-
.../aggregation/groupby/GroupKeyGenerator.java |8 +
.../query/aggregation/groupby/GroupKeyTest.java| 66 ++
.../pinot/integration/tests/ClusterTest.java |6 +-
.../tests/OfflineClusterIntegrationTest.java | 14 +
.../tests/ServerStarterIntegrationTest.java|1 +
.../org/apache/pinot/minion/MinionStarter.java | 25 +-
.../v0_deprecated/pinot-hadoop/pom.xml | 11 +
.../hadoop/job/mappers/SegmentCreationMapper.java | 10 +
.../v0_deprecated/pinot-spark/pom.xml | 12 +-
.../spark/jobs/SparkSegmentCreationFunction.java | 10 +
.../{pinot-csv => pinot-protobuf}/pom.xml | 15 +-
.../protobuf/ProtoBufRecordExtractor.java} | 18 +-
.../protobuf/ProtoBufRecordReader.java}| 102 +-
.../protobuf/ProtoBufRecordReaderConfig.java | 19 +-
.../protobuf/ProtoBufRecordReaderTest.java | 129 ++-
.../pinot/plugin/inputformat/protobuf/Sample.java | 1075
.../pinot-protobuf/src/test/resources/sample.desc |8 +
.../pinot-protobuf/src/test/resources/sample.proto |9 +
pinot-plugins/pinot-input-format/pom.xml |1 +
.../pinot/server/starter/ServerInstance.java |3 +-
[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #5441: Adding files generated by running quickstart to gitignore
kishoreg commented on a change in pull request #5441: URL: https://github.com/apache/incubator-pinot/pull/5441#discussion_r432151926 ## File path: .gitignore ## @@ -44,4 +44,8 @@ yarn.lock npm-debug.log* yarn-debug.log* -yarn-error.log* \ No newline at end of file +yarn-error.log* + +#quickstart files +examples/ Review comment: This only excludes examples in the root folder. ``` ➜ incubator-pinot git:(ignore-quickstart-files) git status On branch ignore-quickstart-files Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: pinot-tools/src/main/resources/examples/batch/airlineStats/hadoopIngestionJobSpec.yaml ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #5440: Add GenericTransformFunction wrapper for simple ScalarFunctions
kishoreg commented on a change in pull request #5440:
URL: https://github.com/apache/incubator-pinot/pull/5440#discussion_r432148125
##
File path:
pinot-common/src/main/java/org/apache/pinot/common/function/StringFunctions.java
##
@@ -0,0 +1,123 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.function;
+
+import java.util.regex.Pattern;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.pinot.common.function.annotations.ScalarFunction;
+
+
+/**
+ *
+ */
+public class StringFunctions {
Review comment:
java doc
##
File path: pinot-common/pom.xml
##
@@ -248,6 +248,11 @@
org.glassfish.jersey.core
jersey-server
+
+ org.reflections
+ reflections
+ 0.9.11
Review comment:
move version to pom.properties in root pom.xml
##
File path:
pinot-common/src/main/java/org/apache/pinot/common/function/annotations/ScalarFunction.java
##
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.function.annotations;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface ScalarFunction {
Review comment:
add the ability to set name, add javadocs
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #5445: Update Quickstart to take tmp dir as a parameter and default to current dir
kishoreg commented on a change in pull request #5445: URL: https://github.com/apache/incubator-pinot/pull/5445#discussion_r432146659 ## File path: pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/QuickStartCommand.java ## @@ -34,6 +34,9 @@ @Option(name = "-type", required = false, metaVar = "", usage = "Type of quickstart, supported: STREAM/BATCH/HYBRID") private String _type; + @Option(name = "-tmpDir", required = false, metaVar = "", usage = "Temp Directory to host quickstart data") Review comment: should we call -quickstartDir? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch master updated: Include transformFunction in timeFieldSpec toJsonObject (#5464)
This is an automated email from the ASF dual-hosted git repository.
nehapawar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 94aad60 Include transformFunction in timeFieldSpec toJsonObject
(#5464)
94aad60 is described below
commit 94aad60f520d627e257cac6c57c54abeafec37a2
Author: Neha Pawar
AuthorDate: Thu May 28 14:11:08 2020 -0700
Include transformFunction in timeFieldSpec toJsonObject (#5464)
---
.../apache/pinot/common/data/FieldSpecTest.java| 10 ++---
pinot-common/src/test/resources/schemaTest.schema | 25 +++---
.../java/org/apache/pinot/spi/data/FieldSpec.java | 10 ++---
.../org/apache/pinot/spi/data/TimeFieldSpec.java | 1 +
4 files changed, 37 insertions(+), 9 deletions(-)
diff --git
a/pinot-common/src/test/java/org/apache/pinot/common/data/FieldSpecTest.java
b/pinot-common/src/test/java/org/apache/pinot/common/data/FieldSpecTest.java
index 9d29bd4..8063988 100644
--- a/pinot-common/src/test/java/org/apache/pinot/common/data/FieldSpecTest.java
+++ b/pinot-common/src/test/java/org/apache/pinot/common/data/FieldSpecTest.java
@@ -262,7 +262,7 @@ public class FieldSpecTest {
FieldSpec second;
// Single-value boolean type dimension field with default null value.
-String[] dimensionFields = {"\"name\":\"dimension\"",
"\"dataType\":\"BOOLEAN\"", "\"defaultNullValue\":false"};
+String[] dimensionFields = {"\"name\":\"dimension\"",
"\"dataType\":\"BOOLEAN\"", "\"defaultNullValue\":false",
"\"transformFunction\":\"trim(foo)\""};
first =
JsonUtils.stringToObject(getRandomOrderJsonString(dimensionFields),
DimensionFieldSpec.class);
second = JsonUtils.stringToObject(first.toJsonObject().toString(),
DimensionFieldSpec.class);
Assert.assertEquals(first, second, ERROR_MESSAGE);
@@ -276,14 +276,18 @@ public class FieldSpecTest {
// Time field with default value.
String[] timeFields =
-{"\"incomingGranularitySpec\":{\"timeUnitSize\":1,
\"timeType\":\"MILLISECONDS\",\"dataType\":\"LONG\",\"name\":\"incomingTime\"}",
"\"outgoingGranularitySpec\":{\"timeType\":\"SECONDS\",\"dataType\":\"INT\",\"name\":\"outgoingTime\"}",
"\"defaultNullValue\":-1"};
+{"\"incomingGranularitySpec\":{\"timeUnitSize\":1,
\"timeType\":\"MILLISECONDS\",\"dataType\":\"LONG\",\"name\":\"incomingTime\"}",
+
"\"outgoingGranularitySpec\":{\"timeType\":\"SECONDS\",\"dataType\":\"INT\",\"name\":\"outgoingTime\"}",
+"\"defaultNullValue\":-1",
+"\"transformFunction\":\"toEpochDays(millis)\""};
first = JsonUtils.stringToObject(getRandomOrderJsonString(timeFields),
TimeFieldSpec.class);
second = JsonUtils.stringToObject(first.toJsonObject().toString(),
TimeFieldSpec.class);
Assert.assertEquals(first, second, ERROR_MESSAGE);
// DateTime field
String[] dateTimeFields =
-{"\"name\":\"Date\"", "\"dataType\":\"LONG\"",
"\"format\":\"1:MILLISECONDS:EPOCH\"", "\"granularity\":\"5:MINUTES\"",
"\"dateTimeType\":\"PRIMARY\""};
+{"\"name\":\"Date\"", "\"dataType\":\"LONG\"",
"\"format\":\"1:MILLISECONDS:EPOCH\"", "\"granularity\":\"5:MINUTES\"",
+"\"transformFunction\":\"fromEpochDays(daysSinceEpoch)\""};
first = JsonUtils.stringToObject(getRandomOrderJsonString(dateTimeFields),
DateTimeFieldSpec.class);
second = JsonUtils.stringToObject(first.toJsonObject().toString(),
DateTimeFieldSpec.class);
Assert.assertEquals(first, second, ERROR_MESSAGE);
diff --git a/pinot-common/src/test/resources/schemaTest.schema
b/pinot-common/src/test/resources/schemaTest.schema
index 32a83c0..b7c7dcf 100644
--- a/pinot-common/src/test/resources/schemaTest.schema
+++ b/pinot-common/src/test/resources/schemaTest.schema
@@ -8,7 +8,8 @@
{
"name": "longMetric",
"dataType": "LONG",
- "singleValueField": true
+ "singleValueField": true,
+ "transformFunction": "Groovy({foo/1000}, foo)"
},
{
"name": "floatMetric",
@@ -37,7 +38,8 @@
"name": "longDimension",
"dataType": "LONG",
"singleValueField": false,
- "delimiter": null
+ "delimiter": null,
+ "transformFunction": "trim(bar)"
},
{
"name": "floatDimension",
@@ -69,7 +71,24 @@
"timeType": "DAYS",
"name": "outgoing"
},
-"defaultNullValue": 1
+"defaultNullValue": 1,
+"transformFunction": "toEpochDays(millis)"
},
+ "dateTimeFieldSpecs": [
+{
+ "name": "dateTime1",
+ "dataType": "LONG",
+ "format": "1:MILLISECONDS:EPOCH",
+ "granularity": "15:MINUTES",
+ "transformFunction": "fromDateTime(dt, 'MMdd')"
+},
+{
+ "name": "dateTime2",
+ "dataType": "STRING",
+ "format": "1:DAYS:SIMPLE_DATE_FORMAT:MMdd",
+ "granularity": "1:DAYS",
+ "defaultNullValue": "20200101"
+}
+
[GitHub] [incubator-pinot] npawar merged pull request #5464: Include transformFunction in timeFieldSpec toJsonObject
npawar merged pull request #5464: URL: https://github.com/apache/incubator-pinot/pull/5464 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar opened a new pull request #5464: Include transformFunction in timeFieldSpec toJsonObject
npawar opened a new pull request #5464: URL: https://github.com/apache/incubator-pinot/pull/5464 ## Description TransformFunction was being ignored when adding schema via swagger. Worked fine if reading Schema file, or Schema object, hence went undetected in some of the transform field tests which included tiemFieldSpec This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5444: Enhance and simplify the filtering
Jackie-Jiang commented on a change in pull request #5444: URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r432060103 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/AndDocIdSet.java ## @@ -0,0 +1,156 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pinot.core.operator.docidsets; + +import java.util.ArrayList; +import java.util.List; +import org.apache.pinot.common.utils.Pairs.IntPair; +import org.apache.pinot.core.common.BlockDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.AndDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.BitmapBasedDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.RangelessBitmapDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.ScanBasedDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.SortedDocIdIterator; +import org.apache.pinot.core.util.SortedRangeIntersection; +import org.roaringbitmap.buffer.ImmutableRoaringBitmap; +import org.roaringbitmap.buffer.MutableRoaringBitmap; + + +/** + * The FilterBlockDocIdSet to perform AND on all child FilterBlockDocIdSets. + * The AndBlockDocIdSet will construct the BlockDocIdIterator based on the BlockDocIdIterators from the child + * FilterBlockDocIdSets: + * + * + * When there are at least one index-base BlockDocIdIterators (SortedDocIdIterator or BitmapBasedDocIdIterator) and Review comment: It uses a bitmap to perform AND for all index-based BlockDocIdIterators, and use ScanBasedDocIdIterator.applyAnd(docIds) to resolve the scan-based BlockDocIdIterators. After all, there will be one RangelessBitmapDocIdIterator on top of the result bitmap. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #5444: Enhance and simplify the filtering
Jackie-Jiang commented on pull request #5444: URL: https://github.com/apache/incubator-pinot/pull/5444#issuecomment-635539613 > This PR is too big to review effectively IMO. 89 files in total. Can you break it down to 3-4 smaller PRs for review: one covering method removal; 1 or 2 PR each covering a new added major classes (e.g., SortedIndexBasedFilterOperator.java). @chenboat There is no new added major classes. Most of the changes are making the classes compatible with the filter interface change, so it is very hard to break them into multiple PRs. I can make the removed interface in BlockValIterator a separate PR as they are independent of the filtering. This PR is the first step of re-structuring the filtering in Pinot, so for historical reason the name for some interfaces (e.g. BlockDocIdSet) might be confusing. I wouldn't spend too much time renaming and documenting them because they are going to be re-structured in the following steps. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5444: Enhance and simplify the filtering
Jackie-Jiang commented on a change in pull request #5444: URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r432045689 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/AndDocIdSet.java ## @@ -0,0 +1,156 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pinot.core.operator.docidsets; + +import java.util.ArrayList; +import java.util.List; +import org.apache.pinot.common.utils.Pairs.IntPair; +import org.apache.pinot.core.common.BlockDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.AndDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.BitmapBasedDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.RangelessBitmapDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.ScanBasedDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.SortedDocIdIterator; +import org.apache.pinot.core.util.SortedRangeIntersection; +import org.roaringbitmap.buffer.ImmutableRoaringBitmap; +import org.roaringbitmap.buffer.MutableRoaringBitmap; + + +/** + * The FilterBlockDocIdSet to perform AND on all child FilterBlockDocIdSets. + * The AndBlockDocIdSet will construct the BlockDocIdIterator based on the BlockDocIdIterators from the child + * FilterBlockDocIdSets: + * + * + * When there are at least one index-base BlockDocIdIterators (SortedDocIdIterator or BitmapBasedDocIdIterator) and Review comment: It merges all index-based and scan-based BlockDocIdIterators into one bitmap and construct a RangelessBitmapDocIdIterator over it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #5444: Enhance and simplify the filtering
Jackie-Jiang commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r432043014
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/BlockDocIdSet.java
##
@@ -21,6 +21,4 @@
public interface BlockDocIdSet {
BlockDocIdIterator iterator();
Review comment:
The result from the Operator is called Block, and BlockDocIdSet is the
document ids from a Block. It is not structured very well, but it is out of the
scope of this PR. We have plan to clean that up later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar opened a new issue #5463: Show details of failure when schema upload fails
npawar opened a new issue #5463:
URL: https://github.com/apache/incubator-pinot/issues/5463
Noticed today that adding a schema with a field missing in a spec resulted
in just 500 error with message "Failed to add schema". See below example,
`timeType` is missing in the spec.
```
{
"dimensionFieldSpecs": [{
"name": "foo",
"dataType": "STRING"
}
],
"timeFieldSpec": {
"incomingGranularitySpec": {
"dataType": "LONG",
"name": "bar"
}
}
```
Response:
```
{
"code": 500,
"error": "Failed to add new schema test."
}
```
This message is not very helpful.
We need to walk through all possible invalid schema scenarios and handle
them properly.
Tasks:
1) List and validate all cases
2) Return proper message in case of validation failures
3) More test cases to catch these
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #5435: [TE] clean up decprecated/unused code
jihaozh commented on a change in pull request #5435: URL: https://github.com/apache/incubator-pinot/pull/5435#discussion_r432004017 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardApplication.java ## @@ -160,14 +155,12 @@ public void run(ThirdEyeDashboardConfiguration config, Environment env) AnomalyFunctionFactory anomalyFunctionFactory = new AnomalyFunctionFactory(config.getFunctionConfigPath()); Review comment: I think `AnomalyResource` is not used. In `AnomaliesResource`, only the search endpoints listed by Harley is used. `DataResource` is being used, but it shouldn't need `anomalyFunctionFactory` anymore. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch support_case_when_statement updated (fbace8b -> 5667776)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard fbace8b Adding transform function support for case-when-else add 5667776 Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (fbace8b) \ N -- N -- N refs/heads/support_case_when_statement (5667776) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../pinot/core/operator/transform/function/EqualsTransformFunction.java | 2 +- .../transform/function/GreaterThanOrEqualTransformFunction.java | 2 +- .../core/operator/transform/function/GreaterThanTransformFunction.java | 2 +- .../operator/transform/function/LessThanOrEqualTransformFunction.java | 2 +- .../core/operator/transform/function/LessThanTransformFunction.java | 2 +- .../core/operator/transform/function/NotEqualsTransformFunction.java| 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar closed issue #2583: Create an abstraction for stream in realtime
npawar closed issue #2583: URL: https://github.com/apache/incubator-pinot/issues/2583 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on issue #2583: Create an abstraction for stream in realtime
npawar commented on issue #2583: URL: https://github.com/apache/incubator-pinot/issues/2583#issuecomment-635477885 closing this issue, as this was completed a while ago This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] KKcorps opened a new issue #5462: Add MultiValue support to GenericTransformFunction
KKcorps opened a new issue #5462: URL: https://github.com/apache/incubator-pinot/issues/5462 Currently the Generic Transform function only supports the single value scalar functions and not the multivalue ones. This support needs to be added. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on pull request #5455: Use java.io.tmpdir in KafkaStarterUtils
npawar commented on pull request #5455: URL: https://github.com/apache/incubator-pinot/pull/5455#issuecomment-635445884 > Shall we use temp directory or use current directory with better structure? > I remember @kishoreg mentioned some issue users facing about the temp dir? What kind of issues? This code is appending a random suffix, so there shouldn't be an overlap This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request #5435: [TE] clean up decprecated/unused code
xiaohui-sun commented on a change in pull request #5435: URL: https://github.com/apache/incubator-pinot/pull/5435#discussion_r431942838 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardApplication.java ## @@ -160,14 +155,12 @@ public void run(ThirdEyeDashboardConfiguration config, Environment env) AnomalyFunctionFactory anomalyFunctionFactory = new AnomalyFunctionFactory(config.getFunctionConfigPath()); Review comment: @jihaozh can you check which resource is used? I believe most endpoints in AnomalyResource and AnomaliesResource are not used. From Harley's doc only "/anomalies/search/time/" is used, but let's double check with them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] xiaohui-sun commented on a change in pull request #5435: [TE] clean up decprecated/unused code
xiaohui-sun commented on a change in pull request #5435:
URL: https://github.com/apache/incubator-pinot/pull/5435#discussion_r431935559
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/resources/AnomalyResource.java
##
@@ -101,21 +99,18 @@
private MergedAnomalyResultManager anomalyMergedResultDAO;
private AlertConfigManager emailConfigurationDAO;
private MergedAnomalyResultManager mergedAnomalyResultDAO;
- private AutotuneConfigManager autotuneConfigDAO;
private DatasetConfigManager datasetConfigDAO;
private AnomalyFunctionFactory anomalyFunctionFactory;
private AlertFilterFactory alertFilterFactory;
private LoadingCache collectionMaxDataTimeCache;
private static final DAORegistry DAO_REGISTRY = DAORegistry.getInstance();
- public AnomalyResource(AnomalyFunctionFactory anomalyFunctionFactory,
AlertFilterFactory alertFilterFactory,
- AlertFilterAutotuneFactory alertFilterAutotuneFactory) {
+ public AnomalyResource(AnomalyFunctionFactory anomalyFunctionFactory,
AlertFilterFactory alertFilterFactory) {
this.anomalyFunctionDAO = DAO_REGISTRY.getAnomalyFunctionDAO();
this.anomalyMergedResultDAO = DAO_REGISTRY.getMergedAnomalyResultDAO();
Review comment:
The question is duplication. We should remove one of them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch support_case_when_statement updated (ceef961 -> fbace8b)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard ceef961 Adding transform function support for case-when-else add fbace8b Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (ceef961) \ N -- N -- N refs/heads/support_case_when_statement (fbace8b) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../transform/function/EqualsTransformFunction.java| 18 ++ .../function/GreaterThanOrEqualTransformFunction.java | 18 ++ .../function/GreaterThanTransformFunction.java | 18 ++ .../function/LessThanOrEqualTransformFunction.java | 18 ++ .../transform/function/LessThanTransformFunction.java | 18 ++ .../transform/function/NotEqualsTransformFunction.java | 18 ++ 6 files changed, 108 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch support_case_when_statement updated (3e7d64f -> ceef961)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 3e7d64f Adding transform function support for case-when-else add ceef961 Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (3e7d64f) \ N -- N -- N refs/heads/support_case_when_statement (ceef961) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../transform/function/CaseTransformFunction.java | 58 +++--- .../function/LiteralTransformFunction.java | 11 ++-- .../tests/OfflineClusterIntegrationTest.java | 2 +- 3 files changed, 49 insertions(+), 22 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch support_case_when_statement updated (83f81c1 -> 3e7d64f)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 83f81c1 Adding transform function support for case-when-else new 3e7d64f Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (83f81c1) \ N -- N -- N refs/heads/support_case_when_statement (3e7d64f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../tests/OfflineClusterIntegrationTest.java | 31 ++ 1 file changed, 31 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/01: Adding transform function support for case-when-else
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 3e7d64f82cfc51fda4840dc17aa94bd7d0d79dbb
Author: Xiang Fu
AuthorDate: Wed May 27 22:15:59 2020 -0700
Adding transform function support for case-when-else
---
.../common/function/TransformFunctionType.java | 8 +
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 20 +-
.../function/BinaryOperatorTransformFunction.java | 114 +
.../transform/function/CaseTransformFunction.java | 151 ++
.../function/EqualsTransformFunction.java | 41 ++
.../GreaterThanOrEqualTransformFunction.java | 49 ++
.../function/GreaterThanTransformFunction.java | 49 ++
.../function/LessThanOrEqualTransformFunction.java | 49 ++
.../function/LessThanTransformFunction.java| 49 ++
.../function/LiteralTransformFunction.java | 36 +-
.../function/NotEqualsTransformFunction.java | 41 ++
.../function/TransformFunctionFactory.java | 9 +
.../function/BaseTransformFunctionTest.java| 15 +
.../BinaryOperatorTransformFunctionTest.java | 109 +
.../function/CaseTransformFunctionTest.java| 522 +
.../function/EqualsTransformFunctionTest.java | 52 ++
.../GreaterThanOrEqualTransformFunctionTest.java | 52 ++
.../function/GreaterThanTransformFunctionTest.java | 52 ++
.../LessThanOrEqualTransformFunctionTest.java | 52 ++
.../function/LessThanTransformFunctionTest.java| 52 ++
.../function/NotEqualsTransformFunctionTest.java | 52 ++
.../tests/OfflineClusterIntegrationTest.java | 62 ++-
22 files changed, 1614 insertions(+), 22 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
index f8081ce..86d5c8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
@@ -38,7 +38,15 @@ public enum TransformFunctionType {
LN("ln"),
SQRT("sqrt"),
+ EQUALS("equals"),
+ NOT_EQUALS("not_equals"),
+ GREATER_THAN("greater_than"),
+ GREATER_THAN_OR_EQUAL("greater_than_or_equal"),
+ LESS_THAN("less_than"),
+ LESS_THAN_OR_EQUAL("less_than_or_equal"),
+
CAST("cast"),
+ CASE("case"),
JSONEXTRACTSCALAR("jsonExtractScalar"),
JSONEXTRACTKEY("jsonExtractKey"),
TIMECONVERT("timeConvert"),
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index 80ef72b..b590be5 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -52,13 +52,9 @@ public class CalciteSqlCompilerTest {
@Test
public void testCaseWhenStatements() {
PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT OrderID, Quantity,\n"
-+ "CASE\n"
-+ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
-+ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
-+ "ELSE 'The quantity is under 30'\n"
-+ "END AS QuantityText\n"
-+ "FROM OrderDetails");
+"SELECT OrderID, Quantity,\n" + "CASE\n" + "WHEN Quantity > 30
THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n" + "ELSE
'The quantity is under 30'\n"
++ "END AS QuantityText\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
@@ -79,14 +75,8 @@ public class CalciteSqlCompilerTest {
Assert.assertEquals(caseFunc.getOperands().get(4).getLiteral().getFieldValue(),
"The quantity is under 30");
pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT Quantity,\n"
-+ "SUM(CASE\n"
-+ "WHEN Quantity > 30 THEN 3\n"
-+ "WHEN Quantity > 20 THEN 2\n"
-+ "WHEN Quantity > 10 THEN 1\n"
-+ "ELSE 0\n"
-+ "END) AS new_sum_quant\n"
-+ "FROM OrderDetails");
+"SELECT Quantity,\n" + "SUM(CASE\n" + "WHEN Quantity > 30 THEN
3\n" + "WHEN Quantity > 20 THEN 2\n"
++ "WHEN Quantity > 10 THEN 1\n" + "ELSE 0\n" + "END) AS
new_sum_quant\n" + "FROM OrderDetails");
[incubator-pinot] 01/01: Adding transform function support for case-when-else
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 83f81c157d3755aeaa3f26c777960219777a61af
Author: Xiang Fu
AuthorDate: Wed May 27 22:15:59 2020 -0700
Adding transform function support for case-when-else
---
.../common/function/TransformFunctionType.java | 8 +
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 20 +-
.../function/BinaryOperatorTransformFunction.java | 114 +
.../transform/function/CaseTransformFunction.java | 151 ++
.../function/EqualsTransformFunction.java | 41 ++
.../GreaterThanOrEqualTransformFunction.java | 49 ++
.../function/GreaterThanTransformFunction.java | 49 ++
.../function/LessThanOrEqualTransformFunction.java | 49 ++
.../function/LessThanTransformFunction.java| 49 ++
.../function/LiteralTransformFunction.java | 36 +-
.../function/NotEqualsTransformFunction.java | 41 ++
.../function/TransformFunctionFactory.java | 9 +
.../function/BaseTransformFunctionTest.java| 15 +
.../BinaryOperatorTransformFunctionTest.java | 109 +
.../function/CaseTransformFunctionTest.java| 522 +
.../function/EqualsTransformFunctionTest.java | 52 ++
.../GreaterThanOrEqualTransformFunctionTest.java | 52 ++
.../function/GreaterThanTransformFunctionTest.java | 52 ++
.../LessThanOrEqualTransformFunctionTest.java | 52 ++
.../function/LessThanTransformFunctionTest.java| 52 ++
.../function/NotEqualsTransformFunctionTest.java | 52 ++
.../tests/OfflineClusterIntegrationTest.java | 31 +-
22 files changed, 1583 insertions(+), 22 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
index f8081ce..86d5c8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
@@ -38,7 +38,15 @@ public enum TransformFunctionType {
LN("ln"),
SQRT("sqrt"),
+ EQUALS("equals"),
+ NOT_EQUALS("not_equals"),
+ GREATER_THAN("greater_than"),
+ GREATER_THAN_OR_EQUAL("greater_than_or_equal"),
+ LESS_THAN("less_than"),
+ LESS_THAN_OR_EQUAL("less_than_or_equal"),
+
CAST("cast"),
+ CASE("case"),
JSONEXTRACTSCALAR("jsonExtractScalar"),
JSONEXTRACTKEY("jsonExtractKey"),
TIMECONVERT("timeConvert"),
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index 80ef72b..b590be5 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -52,13 +52,9 @@ public class CalciteSqlCompilerTest {
@Test
public void testCaseWhenStatements() {
PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT OrderID, Quantity,\n"
-+ "CASE\n"
-+ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
-+ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
-+ "ELSE 'The quantity is under 30'\n"
-+ "END AS QuantityText\n"
-+ "FROM OrderDetails");
+"SELECT OrderID, Quantity,\n" + "CASE\n" + "WHEN Quantity > 30
THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n" + "ELSE
'The quantity is under 30'\n"
++ "END AS QuantityText\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
@@ -79,14 +75,8 @@ public class CalciteSqlCompilerTest {
Assert.assertEquals(caseFunc.getOperands().get(4).getLiteral().getFieldValue(),
"The quantity is under 30");
pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT Quantity,\n"
-+ "SUM(CASE\n"
-+ "WHEN Quantity > 30 THEN 3\n"
-+ "WHEN Quantity > 20 THEN 2\n"
-+ "WHEN Quantity > 10 THEN 1\n"
-+ "ELSE 0\n"
-+ "END) AS new_sum_quant\n"
-+ "FROM OrderDetails");
+"SELECT Quantity,\n" + "SUM(CASE\n" + "WHEN Quantity > 30 THEN
3\n" + "WHEN Quantity > 20 THEN 2\n"
++ "WHEN Quantity > 10 THEN 1\n" + "ELSE 0\n" + "END) AS
new_sum_quant\n" + "FROM OrderDetails");
[incubator-pinot] branch support_case_when_statement updated (720c968 -> 83f81c1)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 720c968 Adding transform function support for case-when-else new 83f81c1 Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (720c968) \ N -- N -- N refs/heads/support_case_when_statement (83f81c1) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../operator/transform/function/NotEqualsTransformFunction.java | 8 1 file changed, 4 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch support_case_when_statement updated (83d9af5 -> 720c968)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 83d9af5 Adding transform function support for case-when-else new 720c968 Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (83d9af5) \ N -- N -- N refs/heads/support_case_when_statement (720c968) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../function/BinaryOperatorTransformFunction.java | 3 ++- .../transform/function/EqualsTransformFunction.java | 17 + .../function/GreaterThanOrEqualTransformFunction.java | 17 + .../function/GreaterThanTransformFunction.java | 17 + .../function/LessThanOrEqualTransformFunction.java | 17 + .../transform/function/LessThanTransformFunction.java | 17 + .../transform/function/NotEqualsTransformFunction.java | 17 + 7 files changed, 104 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/01: Adding transform function support for case-when-else
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 720c9688b4ebd0bc717864d6f2a8cfeee6fb6813
Author: Xiang Fu
AuthorDate: Wed May 27 22:15:59 2020 -0700
Adding transform function support for case-when-else
---
.../common/function/TransformFunctionType.java | 8 +
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 20 +-
.../function/BinaryOperatorTransformFunction.java | 114 +
.../transform/function/CaseTransformFunction.java | 151 ++
.../function/EqualsTransformFunction.java | 41 ++
.../GreaterThanOrEqualTransformFunction.java | 49 ++
.../function/GreaterThanTransformFunction.java | 49 ++
.../function/LessThanOrEqualTransformFunction.java | 49 ++
.../function/LessThanTransformFunction.java| 49 ++
.../function/LiteralTransformFunction.java | 36 +-
.../function/NotEqualsTransformFunction.java | 41 ++
.../function/TransformFunctionFactory.java | 9 +
.../function/BaseTransformFunctionTest.java| 15 +
.../BinaryOperatorTransformFunctionTest.java | 109 +
.../function/CaseTransformFunctionTest.java| 522 +
.../function/EqualsTransformFunctionTest.java | 52 ++
.../GreaterThanOrEqualTransformFunctionTest.java | 52 ++
.../function/GreaterThanTransformFunctionTest.java | 52 ++
.../LessThanOrEqualTransformFunctionTest.java | 52 ++
.../function/LessThanTransformFunctionTest.java| 52 ++
.../function/NotEqualsTransformFunctionTest.java | 52 ++
.../tests/OfflineClusterIntegrationTest.java | 31 +-
22 files changed, 1583 insertions(+), 22 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
index f8081ce..86d5c8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
@@ -38,7 +38,15 @@ public enum TransformFunctionType {
LN("ln"),
SQRT("sqrt"),
+ EQUALS("equals"),
+ NOT_EQUALS("not_equals"),
+ GREATER_THAN("greater_than"),
+ GREATER_THAN_OR_EQUAL("greater_than_or_equal"),
+ LESS_THAN("less_than"),
+ LESS_THAN_OR_EQUAL("less_than_or_equal"),
+
CAST("cast"),
+ CASE("case"),
JSONEXTRACTSCALAR("jsonExtractScalar"),
JSONEXTRACTKEY("jsonExtractKey"),
TIMECONVERT("timeConvert"),
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index 80ef72b..b590be5 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -52,13 +52,9 @@ public class CalciteSqlCompilerTest {
@Test
public void testCaseWhenStatements() {
PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT OrderID, Quantity,\n"
-+ "CASE\n"
-+ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
-+ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
-+ "ELSE 'The quantity is under 30'\n"
-+ "END AS QuantityText\n"
-+ "FROM OrderDetails");
+"SELECT OrderID, Quantity,\n" + "CASE\n" + "WHEN Quantity > 30
THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n" + "ELSE
'The quantity is under 30'\n"
++ "END AS QuantityText\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
@@ -79,14 +75,8 @@ public class CalciteSqlCompilerTest {
Assert.assertEquals(caseFunc.getOperands().get(4).getLiteral().getFieldValue(),
"The quantity is under 30");
pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT Quantity,\n"
-+ "SUM(CASE\n"
-+ "WHEN Quantity > 30 THEN 3\n"
-+ "WHEN Quantity > 20 THEN 2\n"
-+ "WHEN Quantity > 10 THEN 1\n"
-+ "ELSE 0\n"
-+ "END) AS new_sum_quant\n"
-+ "FROM OrderDetails");
+"SELECT Quantity,\n" + "SUM(CASE\n" + "WHEN Quantity > 30 THEN
3\n" + "WHEN Quantity > 20 THEN 2\n"
++ "WHEN Quantity > 10 THEN 1\n" + "ELSE 0\n" + "END) AS
new_sum_quant\n" + "FROM OrderDetails");
[incubator-pinot] branch support_case_when_statement updated (5a97ea4 -> 83d9af5)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 5a97ea4 Adding transform function support for case-when-else new 83d9af5 Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (5a97ea4) \ N -- N -- N refs/heads/support_case_when_statement (83d9af5) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../transform/function/CaseTransformFunction.java | 22 ++ .../BinaryOperatorTransformFunctionTest.java | 6 ++ 2 files changed, 28 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/01: Adding transform function support for case-when-else
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 83d9af59a23fb3a18c9e41489b4365acff83ffdd
Author: Xiang Fu
AuthorDate: Wed May 27 22:15:59 2020 -0700
Adding transform function support for case-when-else
---
.../common/function/TransformFunctionType.java | 8 +
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 20 +-
.../function/BinaryOperatorTransformFunction.java | 113 +
.../transform/function/CaseTransformFunction.java | 151 ++
.../function/EqualsTransformFunction.java | 24 +
.../GreaterThanOrEqualTransformFunction.java | 32 ++
.../function/GreaterThanTransformFunction.java | 32 ++
.../function/LessThanOrEqualTransformFunction.java | 32 ++
.../function/LessThanTransformFunction.java| 32 ++
.../function/LiteralTransformFunction.java | 36 +-
.../function/NotEqualsTransformFunction.java | 24 +
.../function/TransformFunctionFactory.java | 9 +
.../function/BaseTransformFunctionTest.java| 15 +
.../BinaryOperatorTransformFunctionTest.java | 109 +
.../function/CaseTransformFunctionTest.java| 522 +
.../function/EqualsTransformFunctionTest.java | 52 ++
.../GreaterThanOrEqualTransformFunctionTest.java | 52 ++
.../function/GreaterThanTransformFunctionTest.java | 52 ++
.../LessThanOrEqualTransformFunctionTest.java | 52 ++
.../function/LessThanTransformFunctionTest.java| 52 ++
.../function/NotEqualsTransformFunctionTest.java | 52 ++
.../tests/OfflineClusterIntegrationTest.java | 31 +-
22 files changed, 1480 insertions(+), 22 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
index f8081ce..86d5c8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
@@ -38,7 +38,15 @@ public enum TransformFunctionType {
LN("ln"),
SQRT("sqrt"),
+ EQUALS("equals"),
+ NOT_EQUALS("not_equals"),
+ GREATER_THAN("greater_than"),
+ GREATER_THAN_OR_EQUAL("greater_than_or_equal"),
+ LESS_THAN("less_than"),
+ LESS_THAN_OR_EQUAL("less_than_or_equal"),
+
CAST("cast"),
+ CASE("case"),
JSONEXTRACTSCALAR("jsonExtractScalar"),
JSONEXTRACTKEY("jsonExtractKey"),
TIMECONVERT("timeConvert"),
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index 80ef72b..b590be5 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -52,13 +52,9 @@ public class CalciteSqlCompilerTest {
@Test
public void testCaseWhenStatements() {
PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT OrderID, Quantity,\n"
-+ "CASE\n"
-+ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
-+ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
-+ "ELSE 'The quantity is under 30'\n"
-+ "END AS QuantityText\n"
-+ "FROM OrderDetails");
+"SELECT OrderID, Quantity,\n" + "CASE\n" + "WHEN Quantity > 30
THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n" + "ELSE
'The quantity is under 30'\n"
++ "END AS QuantityText\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
@@ -79,14 +75,8 @@ public class CalciteSqlCompilerTest {
Assert.assertEquals(caseFunc.getOperands().get(4).getLiteral().getFieldValue(),
"The quantity is under 30");
pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT Quantity,\n"
-+ "SUM(CASE\n"
-+ "WHEN Quantity > 30 THEN 3\n"
-+ "WHEN Quantity > 20 THEN 2\n"
-+ "WHEN Quantity > 10 THEN 1\n"
-+ "ELSE 0\n"
-+ "END) AS new_sum_quant\n"
-+ "FROM OrderDetails");
+"SELECT Quantity,\n" + "SUM(CASE\n" + "WHEN Quantity > 30 THEN
3\n" + "WHEN Quantity > 20 THEN 2\n"
++ "WHEN Quantity > 10 THEN 1\n" + "ELSE 0\n" + "END) AS
new_sum_quant\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
[incubator-pinot] 01/01: Adding transform function support for case-when-else
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 5a97ea4025113740e036d68888963f945b0c
Author: Xiang Fu
AuthorDate: Wed May 27 22:15:59 2020 -0700
Adding transform function support for case-when-else
---
.../common/function/TransformFunctionType.java | 8 +
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 20 +-
.../function/BinaryOperatorTransformFunction.java | 113 +
.../transform/function/CaseTransformFunction.java | 129 +
.../function/EqualsTransformFunction.java | 24 +
.../GreaterThanOrEqualTransformFunction.java | 32 ++
.../function/GreaterThanTransformFunction.java | 32 ++
.../function/LessThanOrEqualTransformFunction.java | 32 ++
.../function/LessThanTransformFunction.java| 32 ++
.../function/LiteralTransformFunction.java | 36 +-
.../function/NotEqualsTransformFunction.java | 24 +
.../function/TransformFunctionFactory.java | 9 +
.../function/BaseTransformFunctionTest.java| 15 +
.../BinaryOperatorTransformFunctionTest.java | 103
.../function/CaseTransformFunctionTest.java| 522 +
.../function/EqualsTransformFunctionTest.java | 52 ++
.../GreaterThanOrEqualTransformFunctionTest.java | 52 ++
.../function/GreaterThanTransformFunctionTest.java | 52 ++
.../LessThanOrEqualTransformFunctionTest.java | 52 ++
.../function/LessThanTransformFunctionTest.java| 52 ++
.../function/NotEqualsTransformFunctionTest.java | 52 ++
.../tests/OfflineClusterIntegrationTest.java | 31 +-
22 files changed, 1452 insertions(+), 22 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
index f8081ce..86d5c8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
@@ -38,7 +38,15 @@ public enum TransformFunctionType {
LN("ln"),
SQRT("sqrt"),
+ EQUALS("equals"),
+ NOT_EQUALS("not_equals"),
+ GREATER_THAN("greater_than"),
+ GREATER_THAN_OR_EQUAL("greater_than_or_equal"),
+ LESS_THAN("less_than"),
+ LESS_THAN_OR_EQUAL("less_than_or_equal"),
+
CAST("cast"),
+ CASE("case"),
JSONEXTRACTSCALAR("jsonExtractScalar"),
JSONEXTRACTKEY("jsonExtractKey"),
TIMECONVERT("timeConvert"),
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index 80ef72b..b590be5 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -52,13 +52,9 @@ public class CalciteSqlCompilerTest {
@Test
public void testCaseWhenStatements() {
PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT OrderID, Quantity,\n"
-+ "CASE\n"
-+ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
-+ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
-+ "ELSE 'The quantity is under 30'\n"
-+ "END AS QuantityText\n"
-+ "FROM OrderDetails");
+"SELECT OrderID, Quantity,\n" + "CASE\n" + "WHEN Quantity > 30
THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n" + "ELSE
'The quantity is under 30'\n"
++ "END AS QuantityText\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
@@ -79,14 +75,8 @@ public class CalciteSqlCompilerTest {
Assert.assertEquals(caseFunc.getOperands().get(4).getLiteral().getFieldValue(),
"The quantity is under 30");
pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT Quantity,\n"
-+ "SUM(CASE\n"
-+ "WHEN Quantity > 30 THEN 3\n"
-+ "WHEN Quantity > 20 THEN 2\n"
-+ "WHEN Quantity > 10 THEN 1\n"
-+ "ELSE 0\n"
-+ "END) AS new_sum_quant\n"
-+ "FROM OrderDetails");
+"SELECT Quantity,\n" + "SUM(CASE\n" + "WHEN Quantity > 30 THEN
3\n" + "WHEN Quantity > 20 THEN 2\n"
++ "WHEN Quantity > 10 THEN 1\n" + "ELSE 0\n" + "END) AS
new_sum_quant\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
[incubator-pinot] branch support_case_when_statement updated (c7c941a -> 5a97ea4)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard c7c941a Adding transform function support for case-when-else new 5a97ea4 Adding transform function support for case-when-else This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (c7c941a) \ N -- N -- N refs/heads/support_case_when_statement (5a97ea4) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../transform/function/CaseTransformFunction.java | 11 +- .../function/CaseTransformFunctionTest.java| 344 - 2 files changed, 336 insertions(+), 19 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch support_case_when_statement created (now c7c941a)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch support_case_when_statement in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at c7c941a Adding transform function support for case-when-else This branch includes the following new commits: new 5bf635d Adding case-when-else sql parsing statement new c7c941a Adding transform function support for case-when-else The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] 01/02: Adding case-when-else sql parsing statement
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 5bf635da66507bfbc0a226553a67fd3d6054916b
Author: Xiang Fu
AuthorDate: Tue May 26 16:17:32 2020 -0700
Adding case-when-else sql parsing statement
---
.../apache/pinot/sql/parsers/CalciteSqlParser.java | 20 +++
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 64 ++
2 files changed, 84 insertions(+)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
index 2aec8ec..5e0fe8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
+++
b/pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
@@ -39,6 +39,7 @@ import org.apache.calcite.sql.SqlNumericLiteral;
import org.apache.calcite.sql.SqlOrderBy;
import org.apache.calcite.sql.SqlSelect;
import org.apache.calcite.sql.SqlSelectKeyword;
+import org.apache.calcite.sql.fun.SqlCase;
import org.apache.calcite.sql.parser.SqlParseException;
import org.apache.calcite.sql.parser.SqlParser;
import org.apache.calcite.sql.parser.babel.SqlBabelParserImpl;
@@ -610,6 +611,25 @@ public class CalciteSqlParser {
}
asFuncExpr.getFunctionCall().addToOperands(RequestUtils.getIdentifierExpression(aliasName));
return asFuncExpr;
+ case CASE:
+// CASE WHEN Statement is model as a function with variable length
parameters.
+// Assume N is number of WHEN Statements, total number of parameters
is (2 * N + 1).
+// - N: Convert each WHEN Statement into a function Expression;
+// - N: Convert each THEN Statement into an Expression;
+// - 1: Convert ELSE Statement into an Expression.
+SqlCase caseSqlNode = (SqlCase) node;
+SqlNodeList whenOperands = caseSqlNode.getWhenOperands();
+SqlNodeList thenOperands = caseSqlNode.getThenOperands();
+SqlNode elseOperand = caseSqlNode.getElseOperand();
+Expression caseFuncExpr =
RequestUtils.getFunctionExpression(SqlKind.CASE.name());
+for (SqlNode whenSqlNode : whenOperands.getList()) {
+
caseFuncExpr.getFunctionCall().addToOperands(toExpression(whenSqlNode));
+}
+for (SqlNode thenSqlNode : thenOperands.getList()) {
+
caseFuncExpr.getFunctionCall().addToOperands(toExpression(thenSqlNode));
+}
+
caseFuncExpr.getFunctionCall().addToOperands(toExpression(elseOperand));
+return caseFuncExpr;
case OTHER:
if (node instanceof SqlDataTypeSpec) {
// This is to handle expression like: CAST(col AS INT)
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index aec7917..80ef72b 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -50,6 +50,70 @@ import org.testng.annotations.Test;
public class CalciteSqlCompilerTest {
@Test
+ public void testCaseWhenStatements() {
+PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
+"SELECT OrderID, Quantity,\n"
++ "CASE\n"
++ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
++ "ELSE 'The quantity is under 30'\n"
++ "END AS QuantityText\n"
++ "FROM OrderDetails");
+
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
+
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
+Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
+Assert.assertEquals(asFunc.getOperator(), SqlKind.AS.name());
+Function caseFunc = asFunc.getOperands().get(0).getFunctionCall();
+Assert.assertEquals(caseFunc.getOperator(), SqlKind.CASE.name());
+Assert.assertEquals(caseFunc.getOperandsSize(), 5);
+Function greatThanFunc = caseFunc.getOperands().get(0).getFunctionCall();
+Assert.assertEquals(greatThanFunc.getOperator(),
SqlKind.GREATER_THAN.name());
+
Assert.assertEquals(greatThanFunc.getOperands().get(0).getIdentifier().getName(),
"Quantity");
+
Assert.assertEquals(greatThanFunc.getOperands().get(1).getLiteral().getFieldValue(),
30L);
+Function equalsFunc = caseFunc.getOperands().get(1).getFunctionCall();
+Assert.assertEquals(equalsFunc.getOperator(), SqlKind.EQUALS.name());
+
Assert.assertEquals(equalsFunc.getOperands().get(0).getIdentifier().getName(),
"Quantity");
+
[incubator-pinot] 02/02: Adding transform function support for case-when-else
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch support_case_when_statement
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit c7c941a5b4fe51bfb1b81bd57fb7040ca9413061
Author: Xiang Fu
AuthorDate: Wed May 27 22:15:59 2020 -0700
Adding transform function support for case-when-else
---
.../common/function/TransformFunctionType.java | 8 +
.../pinot/sql/parsers/CalciteSqlCompilerTest.java | 20 +-
.../function/BinaryOperatorTransformFunction.java | 113
.../transform/function/CaseTransformFunction.java | 130 +
.../function/EqualsTransformFunction.java | 24 +++
.../GreaterThanOrEqualTransformFunction.java | 32
.../function/GreaterThanTransformFunction.java | 32
.../function/LessThanOrEqualTransformFunction.java | 32
.../function/LessThanTransformFunction.java| 32
.../function/LiteralTransformFunction.java | 36 +++-
.../function/NotEqualsTransformFunction.java | 24 +++
.../function/TransformFunctionFactory.java | 9 +
.../function/BaseTransformFunctionTest.java| 15 ++
.../BinaryOperatorTransformFunctionTest.java | 103 +++
.../function/CaseTransformFunctionTest.java| 204 +
.../function/EqualsTransformFunctionTest.java | 52 ++
.../GreaterThanOrEqualTransformFunctionTest.java | 52 ++
.../function/GreaterThanTransformFunctionTest.java | 52 ++
.../LessThanOrEqualTransformFunctionTest.java | 52 ++
.../function/LessThanTransformFunctionTest.java| 52 ++
.../function/NotEqualsTransformFunctionTest.java | 52 ++
.../tests/OfflineClusterIntegrationTest.java | 31 +++-
22 files changed, 1135 insertions(+), 22 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
index f8081ce..86d5c8d 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
@@ -38,7 +38,15 @@ public enum TransformFunctionType {
LN("ln"),
SQRT("sqrt"),
+ EQUALS("equals"),
+ NOT_EQUALS("not_equals"),
+ GREATER_THAN("greater_than"),
+ GREATER_THAN_OR_EQUAL("greater_than_or_equal"),
+ LESS_THAN("less_than"),
+ LESS_THAN_OR_EQUAL("less_than_or_equal"),
+
CAST("cast"),
+ CASE("case"),
JSONEXTRACTSCALAR("jsonExtractScalar"),
JSONEXTRACTKEY("jsonExtractKey"),
TIMECONVERT("timeConvert"),
diff --git
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
index 80ef72b..b590be5 100644
---
a/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
+++
b/pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
@@ -52,13 +52,9 @@ public class CalciteSqlCompilerTest {
@Test
public void testCaseWhenStatements() {
PinotQuery pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT OrderID, Quantity,\n"
-+ "CASE\n"
-+ "WHEN Quantity > 30 THEN 'The quantity is greater than 30'\n"
-+ "WHEN Quantity = 30 THEN 'The quantity is 30'\n"
-+ "ELSE 'The quantity is under 30'\n"
-+ "END AS QuantityText\n"
-+ "FROM OrderDetails");
+"SELECT OrderID, Quantity,\n" + "CASE\n" + "WHEN Quantity > 30
THEN 'The quantity is greater than 30'\n"
++ "WHEN Quantity = 30 THEN 'The quantity is 30'\n" + "ELSE
'The quantity is under 30'\n"
++ "END AS QuantityText\n" + "FROM OrderDetails");
Assert.assertEquals(pinotQuery.getSelectList().get(0).getIdentifier().getName(),
"OrderID");
Assert.assertEquals(pinotQuery.getSelectList().get(1).getIdentifier().getName(),
"Quantity");
Function asFunc = pinotQuery.getSelectList().get(2).getFunctionCall();
@@ -79,14 +75,8 @@ public class CalciteSqlCompilerTest {
Assert.assertEquals(caseFunc.getOperands().get(4).getLiteral().getFieldValue(),
"The quantity is under 30");
pinotQuery = CalciteSqlParser.compileToPinotQuery(
-"SELECT Quantity,\n"
-+ "SUM(CASE\n"
-+ "WHEN Quantity > 30 THEN 3\n"
-+ "WHEN Quantity > 20 THEN 2\n"
-+ "WHEN Quantity > 10 THEN 1\n"
-+ "ELSE 0\n"
-+ "END) AS new_sum_quant\n"
-+ "FROM OrderDetails");
+"SELECT Quantity,\n" + "SUM(CASE\n" + "WHEN Quantity > 30 THEN
3\n" + "WHEN Quantity > 20 THEN 2\n"
++ "WHEN Quantity > 10 THEN 1\n" + "ELSE 0\n" + "END) AS
new_sum_quant\n" + "FROM OrderDetails");
[GitHub] [incubator-pinot] harleyjj opened a new pull request #5460: [TE] frontend - harleyjj/packages - remove bower from frontend
harleyjj opened a new pull request #5460: URL: https://github.com/apache/incubator-pinot/pull/5460 ## Description 1) Removes Bower and packages that depend on Bower 2) Uses standard ember-c3 instead of the custom c3 built before, due to Bower dependency This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5444: Enhance and simplify the filtering
chenboat commented on a change in pull request #5444: URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r431615072 ## File path: pinot-core/src/main/java/org/apache/pinot/core/operator/docidsets/AndDocIdSet.java ## @@ -0,0 +1,156 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pinot.core.operator.docidsets; + +import java.util.ArrayList; +import java.util.List; +import org.apache.pinot.common.utils.Pairs.IntPair; +import org.apache.pinot.core.common.BlockDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.AndDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.BitmapBasedDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.RangelessBitmapDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.ScanBasedDocIdIterator; +import org.apache.pinot.core.operator.dociditerators.SortedDocIdIterator; +import org.apache.pinot.core.util.SortedRangeIntersection; +import org.roaringbitmap.buffer.ImmutableRoaringBitmap; +import org.roaringbitmap.buffer.MutableRoaringBitmap; + + +/** + * The FilterBlockDocIdSet to perform AND on all child FilterBlockDocIdSets. + * The AndBlockDocIdSet will construct the BlockDocIdIterator based on the BlockDocIdIterators from the child + * FilterBlockDocIdSets: + * + * + * When there are at least one index-base BlockDocIdIterators (SortedDocIdIterator or BitmapBasedDocIdIterator) and Review comment: does it mean the class performs binary merge of 2 child BlockDocIdSets each time? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on pull request #5444: Enhance and simplify the filtering
chenboat commented on pull request #5444: URL: https://github.com/apache/incubator-pinot/pull/5444#issuecomment-635136647 This PR is too big to review effectively IMO. 89 files in total. Can you break it down to 3-4 smaller PRs for review: one covering method removal; 1 or 2 PR each covering a new added major classes (e.g., SortedIndexBasedFilterOperator.java). > Removed the methods that complicate the code but provide no value: > > * BlockDocIdSet > > * getRaw > * FilterBlockDocIdSet > > * getMinDocId > * getMaxDocId > * setStartDocId > * setEndDocId > * BlockDocIdIterator > > * currentDocId > * ScanBasedDocIdIterator > > * isMatch > > Uniformed the behavior of all filter-related classes to bound the return docIds with numDocs > Simplified the logic of AND/OR handling > Pushed down the AND smart handling logic to BlockDocIdIterator to ensure the best performance: > > * AndDocIdSet might return RangelessBitmapDocIdIterator which can be merged with other iterators > * OrDocIdSet might return BitmapDocIdIterator which can be merged with other iterators > > Tested all the queries (10K PQL, 10K SQL) within the query file using HybridClusterIntegrationTest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5444: Enhance and simplify the filtering
chenboat commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r431606971
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/BlockMultiValIterator.java
##
@@ -18,33 +18,15 @@
*/
package org.apache.pinot.core.common;
-/**
- *
- *
- */
-public abstract class BlockMultiValIterator implements BlockValIterator {
-
- public int nextCharVal(char[] charArray) {
-throw new UnsupportedOperationException();
- }
+public interface BlockMultiValIterator extends BlockValIterator {
- public int nextIntVal(int[] intArray) {
-throw new UnsupportedOperationException();
- }
+ int nextIntVal(int[] intArray);
Review comment:
it is worthwhile to explain how to use this method? what is the input
array?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5444: Enhance and simplify the filtering
chenboat commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r431606505
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/BlockValIterator.java
##
@@ -20,9 +20,23 @@
public interface BlockValIterator {
+ /**
+ * Returns {@code true} if there are more values in the iterator, {@code
false} otherwise.
+ */
boolean hasNext();
+ /**
+ * Returns the next document id to be read.
+ */
+ int getNextDocId();
+
+ /**
+ * Sets the next document id to the given document id.
Review comment:
what is the behavior if the input doc id does not exist?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5444: Enhance and simplify the filtering
chenboat commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r431604976
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/BlockSingleValIterator.java
##
@@ -18,37 +18,17 @@
*/
package org.apache.pinot.core.common;
-/**
- *
- * TODO: Split into two classes, one for iterator over data, another over
dictionary id's.
- */
-public abstract class BlockSingleValIterator implements BlockValIterator {
-
- char nextCharVal() {
-throw new UnsupportedOperationException();
- }
+public interface BlockSingleValIterator extends BlockValIterator {
Review comment:
javadoc
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5444: Enhance and simplify the filtering
chenboat commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r431604799
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/BlockDocIdSet.java
##
@@ -21,6 +21,4 @@
public interface BlockDocIdSet {
BlockDocIdIterator iterator();
Review comment:
can you also explain why it is called block*?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request #5444: Enhance and simplify the filtering
chenboat commented on a change in pull request #5444:
URL: https://github.com/apache/incubator-pinot/pull/5444#discussion_r431604484
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/BlockMultiValIterator.java
##
@@ -18,33 +18,15 @@
*/
package org.apache.pinot.core.common;
-/**
- *
- *
- */
-public abstract class BlockMultiValIterator implements BlockValIterator {
-
- public int nextCharVal(char[] charArray) {
-throw new UnsupportedOperationException();
- }
+public interface BlockMultiValIterator extends BlockValIterator {
Review comment:
javadoc?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
