[GitHub] [incubator-pinot] JackyYangPassion commented on issue #4400: what is the difference between Pinot and Druid
JackyYangPassion commented on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509476704 Pinot can support exactly count distinct in realtime and index-hadoop? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] JackyYangPassion edited a comment on issue #4400: what is the difference between Pinot and Druid
JackyYangPassion edited a comment on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509469809 @kishoreg Firstly :thank you for your attention! I agree with that Druid supports two modes Push (Tranquility) and Pull (Kafka Indexing Service) ;But Kafka Indexing Service also support exactly once semantics. I am very interested in Pinot star-tree index This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] JackyYangPassion commented on issue #4400: what is the difference between Pinot and Druid
JackyYangPassion commented on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509469809 Firstly :thank you for your attention! I agree with that Druid supports two modes Push (Tranquility) and Pull (Kafka Indexing Service) ;But Kafka Indexing Service also support exactly once semantics. I am very interested in Pinot star-tree index This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4404: Added support for Vertica as data source
jihaozh commented on a change in pull request #4404: Added support for Vertica
as data source
URL: https://github.com/apache/incubator-pinot/pull/4404#discussion_r301347009
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/datasource/sql/SqlResponseCacheLoader.java
##
@@ -133,6 +137,35 @@ public SqlResponseCacheLoader(Map
properties) throws Exception {
}
}
+// Init Vertica datasources
+if (properties.containsKey(VERTICA)) {
+ List> verticaMapList =
ConfigUtils.getList(properties.get(VERTICA));
+ for (Map objMap: verticaMapList) {
+Map dbNameToURLMap = (Map)objMap.get(DB);
+String verticaUser = (String)objMap.get(USER);
+String verticaPassword = getPassword(objMap);
+String verticaDriver = (String)objMap.get(DRIVER);
+
+for (Map.Entry entry: dbNameToURLMap.entrySet()) {
+ DataSource dataSource = new DataSource();
+ dataSource.setInitialSize(INIT_CONNECTIONS);
+ dataSource.setMaxActive(MAX_CONNECTIONS);
+ System.out.println(verticaUser);
Review comment:
could you remove the print line?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch master updated: [TE] add a thread pool to run preview tasks (#4405)
This is an automated email from the ASF dual-hosted git repository.
jihao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new ae1ecef [TE] add a thread pool to run preview tasks (#4405)
ae1ecef is described below
commit ae1ecefeaa2d2434028dcca723601bc65120292e
Author: Jihao Zhang
AuthorDate: Mon Jul 8 16:43:06 2019 -0700
[TE] add a thread pool to run preview tasks (#4405)
This PR adds a thread pool to run preview tasks for the dashboard. It adds
the following limits to prevent excessive server usage.
- The number of preview task running at the same time
- The timeout for a preview task
---
.../dashboard/DetectionPreviewConfiguration.java | 50 ++
.../dashboard/ThirdEyeDashboardApplication.java| 2 +-
.../dashboard/ThirdEyeDashboardConfiguration.java | 9
.../thirdeye/detection/yaml/YamlResource.java | 28 +---
.../thirdeye/detection/yaml/YamlResourceTest.java | 3 +-
5 files changed, 85 insertions(+), 7 deletions(-)
diff --git
a/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/DetectionPreviewConfiguration.java
b/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/DetectionPreviewConfiguration.java
new file mode 100644
index 000..94a952f
--- /dev/null
+++
b/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/DetectionPreviewConfiguration.java
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.pinot.thirdeye.dashboard;
+
+import java.util.concurrent.TimeUnit;
+
+
+public class DetectionPreviewConfiguration {
+ // 5 detection previews are running at the same time at most
+ private static final int DEFAULT_PREVIEW_PARALLELISM = 5;
+ // max time allowed for a preview task
+ private static final long DEFAULT_PREVIEW_TIMEOUT =
TimeUnit.MINUTES.toMillis(5);
+
+ private int parallelism = DEFAULT_PREVIEW_PARALLELISM;
+ private long timeout = DEFAULT_PREVIEW_TIMEOUT;
+
+ public int getParallelism() {
+return parallelism;
+ }
+
+ public void setParallelism(int parallelism) {
+this.parallelism = parallelism;
+ }
+
+ public long getTimeout() {
+return timeout;
+ }
+
+ public void setTimeout(long timeout) {
+this.timeout = timeout;
+ }
+}
diff --git
a/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardApplication.java
b/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardApplication.java
index dce1242..adfd6de 100644
---
a/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardApplication.java
+++
b/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardApplication.java
@@ -191,7 +191,7 @@ public class ThirdEyeDashboardApplication
DAO_REGISTRY.getTaskDAO(), DAO_REGISTRY.getAnomalyFunctionDAO()));
env.jersey().register(new DetectionResource());
env.jersey().register(new
DetectionAlertResource(DAO_REGISTRY.getDetectionAlertConfigManager()));
-env.jersey().register(new YamlResource());
+env.jersey().register(new
YamlResource(config.getDetectionPreviewConfig()));
env.jersey().register(new SqlDataSourceResource());
TimeSeriesLoader timeSeriesLoader = new DefaultTimeSeriesLoader(
diff --git
a/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardConfiguration.java
b/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardConfiguration.java
index 4716e03..fe291f4 100644
---
a/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardConfiguration.java
+++
b/thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/dashboard/ThirdEyeDashboardConfiguration.java
@@ -30,6 +30,15 @@ public class ThirdEyeDashboardConfiguration extends
ThirdEyeConfiguration {
AuthConfiguration authConfig;
RootCauseConfiguration rootCause;
List resourceConfig;
+ DetectionPreviewConfiguration
[GitHub] [incubator-pinot] jihaozh merged pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh merged pull request #4405: [TE] add a thread pool to run preview tasks URL: https://github.com/apache/incubator-pinot/pull/4405 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] kishoreg commented on issue #4230: NULL value support for all data types
kishoreg commented on issue #4230: NULL value support for all data types URL: https://github.com/apache/incubator-pinot/issues/4230#issuecomment-509426667 @icefury71 Good point. I think the challenge is to come up with the right filtering condition. Whether the filtering is explicit or implicit is just a syntactic convenience. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh commented on a change in pull request #4405: [TE] add a thread pool to
run preview tasks
URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301336943
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##
@@ -724,13 +735,18 @@ private Response runPreview(long start, long end,
Preconditions.checkNotNull(detectionConfig);
DetectionPipeline pipeline = this.loader.from(this.provider,
detectionConfig, start, end);
- result = pipeline.run();
-
+ future = this.executor.submit(pipeline::run);
+ result = future.get(PREVIEW_TIMEOUT, TimeUnit.MILLISECONDS);
} catch (IllegalArgumentException e) {
return processBadRequestResponse(YamlOperations.PREVIEW.name(),
YamlOperations.RUNNING.name(), payload, e);
} catch (InvocationTargetException e) {
responseMessage.put("message", "Failed to run the preview due to " +
e.getTargetException().getMessage());
return Response.serverError().entity(responseMessage).build();
+} catch (TimeoutException e) {
+ // stop the preview
+ future.cancel(true);
Review comment:
good idea. updated.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh commented on a change in pull request #4405: [TE] add a thread pool to
run preview tasks
URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301334369
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##
@@ -145,6 +154,7 @@ public YamlResource() {
this.evaluationDAO = DAORegistry.getInstance().getEvaluationManager();
this.sessionDAO = DAORegistry.getInstance().getSessionDAO();
this.yaml = new Yaml();
+this.executor = Executors.newFixedThreadPool(PREVIEW_PARALLELISM);
Review comment:
Actually, in this case, we want to use a `newFixedThreadPool`, because we
would like to control the maximum number of preview tasks to keep it from
eating all dashboard resource.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301334391 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java ## @@ -115,6 +119,10 @@ private static final String PROP_SESSION_KEY = "sessionKey"; private static final String PROP_PRINCIPAL_TYPE = "principalType"; private static final String PROP_SERVICE = "SERVICE"; + // 5 detection previews are running at the same time at most + private static final int PREVIEW_PARALLELISM = 5; + // max time allowed for a preview task + private static final long PREVIEW_TIMEOUT = TimeUnit.MINUTES.toMillis(5); Review comment: sure This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
akshayrai commented on a change in pull request #4405: [TE] add a thread pool
to run preview tasks
URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301310354
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##
@@ -724,13 +735,18 @@ private Response runPreview(long start, long end,
Preconditions.checkNotNull(detectionConfig);
DetectionPipeline pipeline = this.loader.from(this.provider,
detectionConfig, start, end);
- result = pipeline.run();
-
+ future = this.executor.submit(pipeline::run);
+ result = future.get(PREVIEW_TIMEOUT, TimeUnit.MILLISECONDS);
} catch (IllegalArgumentException e) {
return processBadRequestResponse(YamlOperations.PREVIEW.name(),
YamlOperations.RUNNING.name(), payload, e);
} catch (InvocationTargetException e) {
responseMessage.put("message", "Failed to run the preview due to " +
e.getTargetException().getMessage());
return Response.serverError().entity(responseMessage).build();
+} catch (TimeoutException e) {
+ // stop the preview
+ future.cancel(true);
Review comment:
would be safer to move this to a finally block?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
akshayrai commented on a change in pull request #4405: [TE] add a thread pool
to run preview tasks
URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301309518
##
File path:
thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java
##
@@ -145,6 +154,7 @@ public YamlResource() {
this.evaluationDAO = DAORegistry.getInstance().getEvaluationManager();
this.sessionDAO = DAORegistry.getInstance().getSessionDAO();
this.yaml = new Yaml();
+this.executor = Executors.newFixedThreadPool(PREVIEW_PARALLELISM);
Review comment:
How about a using a `newCachedThreadPool`? That way the calls won't easily
timeout at peak traffic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] akshayrai commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks
akshayrai commented on a change in pull request #4405: [TE] add a thread pool to run preview tasks URL: https://github.com/apache/incubator-pinot/pull/4405#discussion_r301309653 ## File path: thirdeye/thirdeye-pinot/src/main/java/org/apache/pinot/thirdeye/detection/yaml/YamlResource.java ## @@ -115,6 +119,10 @@ private static final String PROP_SESSION_KEY = "sessionKey"; private static final String PROP_PRINCIPAL_TYPE = "principalType"; private static final String PROP_SERVICE = "SERVICE"; + // 5 detection previews are running at the same time at most + private static final int PREVIEW_PARALLELISM = 5; + // max time allowed for a preview task + private static final long PREVIEW_TIMEOUT = TimeUnit.MINUTES.toMillis(5); Review comment: Can we make this configurable? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #4399: Moving handling of NULL values from RecordReaders to NullValueTransfo…
jackjlli commented on a change in pull request #4399: Moving handling of NULL
values from RecordReaders to NullValueTransfo…
URL: https://github.com/apache/incubator-pinot/pull/4399#discussion_r301306848
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/readers/RecordReaderUtils.java
##
@@ -112,8 +112,9 @@ public static Object convert(FieldSpec fieldSpec,
@Nullable Object value) {
public static Object convertSingleValue(FieldSpec fieldSpec, @Nullable
Object value) {
if (value == null) {
// Do not allow default value for time column
Review comment:
Move this comment or add a description to `NullValueTransformer`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #4399: Moving handling of NULL values from RecordReaders to NullValueTransfo…
jackjlli commented on a change in pull request #4399: Moving handling of NULL
values from RecordReaders to NullValueTransfo…
URL: https://github.com/apache/incubator-pinot/pull/4399#discussion_r301307017
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/readers/RecordReaderUtils.java
##
@@ -112,8 +112,9 @@ public static Object convert(FieldSpec fieldSpec,
@Nullable Object value) {
public static Object convertSingleValue(FieldSpec fieldSpec, @Nullable
Object value) {
if (value == null) {
// Do not allow default value for time column
- assert fieldSpec.getFieldType() != FieldSpec.FieldType.TIME;
- return fieldSpec.getDefaultNullValue();
+// assert fieldSpec.getFieldType() != FieldSpec.FieldType.TIME;
Review comment:
Clean up these lines. Same for the rest commented statements.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] jihaozh opened a new pull request #4405: [TE] add a thread pool to run preview tasks
jihaozh opened a new pull request #4405: [TE] add a thread pool to run preview tasks URL: https://github.com/apache/incubator-pinot/pull/4405 This PR adds a thread pool to run preview tasks for the dashboard. It adds the following limits to prevent excessive server usage. 1. The number of preview task running at the same time 2. The timeout for a preview task This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch add-logic-for-lead-controller-resource updated (a62f1c8 -> e5df01f)
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch add-logic-for-lead-controller-resource in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard a62f1c8 Address PR comments add e5df01f Address PR comments This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a62f1c8) \ N -- N -- N refs/heads/add-logic-for-lead-controller-resource (e5df01f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../pinot/controller/LeadControllerManager.java| 51 -- .../api/resources/PinotTableRestletResource.java | 6 +-- 2 files changed, 50 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #4399: Moving handling of NULL values from RecordReaders to NullValueTransfo…
Jackie-Jiang commented on a change in pull request #4399: Moving handling of
NULL values from RecordReaders to NullValueTransfo…
URL: https://github.com/apache/incubator-pinot/pull/4399#discussion_r301291953
##
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/JsonUtils.java
##
@@ -137,13 +138,15 @@ public static Object extractValue(@Nullable JsonNode
jsonValue, FieldSpec fieldS
}
return values;
} else {
-return new Object[]{fieldSpec.getDefaultNullValue()};
+return null;
Review comment:
Don't know if we should support zero-length array for MV columns, but it is
not the same as null.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #4399: Moving handling of NULL values from RecordReaders to NullValueTransfo…
Jackie-Jiang commented on a change in pull request #4399: Moving handling of
NULL values from RecordReaders to NullValueTransfo…
URL: https://github.com/apache/incubator-pinot/pull/4399#discussion_r301295809
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer/NullValueTransformer.java
##
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.data.recordtransformer;
+
+import java.util.Collection;
+import javax.annotation.Nullable;
+import org.apache.pinot.common.data.FieldSpec;
+import org.apache.pinot.common.data.Schema;
+import org.apache.pinot.core.data.GenericRow;
+
+
+public class NullValueTransformer implements RecordTransformer {
+
+ private final Collection _fieldSpecs;
+ private Schema _schema;
+
+ public NullValueTransformer(Schema schema) {
+_schema = schema;
+_fieldSpecs = _schema.getAllFieldSpecs();
+ }
+
+ @Nullable
+ @Override
+ public GenericRow transform(GenericRow row) {
+for (FieldSpec fieldSpec : _fieldSpecs) {
+ String fieldName = fieldSpec.getName();
+ if (row.getValue(fieldName) == null && fieldSpec.getFieldType() !=
FieldSpec.FieldType.TIME) {
Review comment:
Make the `fieldSpec.getFieldType() != FieldSpec.FieldType.TIME` as a
precondition?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #4399: Moving handling of NULL values from RecordReaders to NullValueTransfo…
Jackie-Jiang commented on a change in pull request #4399: Moving handling of
NULL values from RecordReaders to NullValueTransfo…
URL: https://github.com/apache/incubator-pinot/pull/4399#discussion_r301295587
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/recordtransformer/NullValueTransformer.java
##
@@ -0,0 +1,53 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.data.recordtransformer;
+
+import java.util.Collection;
+import javax.annotation.Nullable;
+import org.apache.pinot.common.data.FieldSpec;
+import org.apache.pinot.common.data.Schema;
+import org.apache.pinot.core.data.GenericRow;
+
+
+public class NullValueTransformer implements RecordTransformer {
+
+ private final Collection _fieldSpecs;
+ private Schema _schema;
+
+ public NullValueTransformer(Schema schema) {
+_schema = schema;
+_fieldSpecs = _schema.getAllFieldSpecs();
+ }
+
+ @Nullable
Review comment:
Remove this annotation as the result will not be null.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] npawar commented on a change in pull request #4396: make kafka version number controlled by config number
npawar commented on a change in pull request #4396: make kafka version number controlled by config number URL: https://github.com/apache/incubator-pinot/pull/4396#discussion_r301288628 ## File path: pinot-integration-tests/pom.xml ## @@ -192,7 +192,7 @@ Review comment: Thanks for this change! Please remove this TODO from the pom.xml in pinot-intergation-tests, pinot-tools, pinot-perf, and put it in the main pom.xml This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch master updated: CompletionConfig for realtime tables (#4367)
This is an automated email from the ASF dual-hosted git repository.
nehapawar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 02f6181 CompletionConfig for realtime tables (#4367)
02f6181 is described below
commit 02f6181e820d2905f3dc20ee7843cef08d0d8ec2
Author: Neha Pawar
AuthorDate: Mon Jul 8 13:18:13 2019 -0700
CompletionConfig for realtime tables (#4367)
Completion config for realtime tables to specify completion related things
such as completion mode (DOWNLOAD, DEFAULT).
---
.../pinot/common/config/CompletionConfig.java | 63
.../SegmentsValidationAndRetentionConfig.java | 30 +---
.../apache/pinot/common/utils/CommonConstants.java | 8 ++
.../pinot/common/config/TableConfigTest.java | 28 +++
.../realtime/LLRealtimeSegmentDataManager.java | 85 +++---
5 files changed, 179 insertions(+), 35 deletions(-)
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/config/CompletionConfig.java
b/pinot-common/src/main/java/org/apache/pinot/common/config/CompletionConfig.java
new file mode 100644
index 000..9eb072d
--- /dev/null
+++
b/pinot-common/src/main/java/org/apache/pinot/common/config/CompletionConfig.java
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.common.config;
+
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import org.apache.pinot.common.utils.EqualityUtils;
+
+
+/**
+ * Class representing configurations related to realtime segment completion.
+ *
+ */
+@JsonIgnoreProperties(ignoreUnknown = true)
+public class CompletionConfig {
+
+ @ConfigKey(value = "completionMode")
+ @ConfigDoc(value = "Mode to use when completing segment. DEFAULT for default
strategy (build segment if segment is equivalent to the committed segment, else
download). DOWNLOAD for always download the segment, never build.", mandatory =
false)
+ private String _completionMode;
+
+ public String getCompletionMode() {
+return _completionMode;
+ }
+
+ public void setCompletionMode(String completionMode) {
+_completionMode = completionMode;
+ }
+
+ @Override
+ public boolean equals(Object o) {
+if (EqualityUtils.isSameReference(this, o)) {
+ return true;
+}
+
+if (EqualityUtils.isNullOrNotSameClass(this, o)) {
+ return false;
+}
+
+CompletionConfig that = (CompletionConfig) o;
+
+return EqualityUtils.isEqual(_completionMode, that._completionMode);
+ }
+
+ @Override
+ public int hashCode() {
+return EqualityUtils.hashCodeOf(_completionMode);
+ }
+}
diff --git
a/pinot-common/src/main/java/org/apache/pinot/common/config/SegmentsValidationAndRetentionConfig.java
b/pinot-common/src/main/java/org/apache/pinot/common/config/SegmentsValidationAndRetentionConfig.java
index e9eeef0..5f917da 100644
---
a/pinot-common/src/main/java/org/apache/pinot/common/config/SegmentsValidationAndRetentionConfig.java
+++
b/pinot-common/src/main/java/org/apache/pinot/common/config/SegmentsValidationAndRetentionConfig.java
@@ -66,6 +66,9 @@ public class SegmentsValidationAndRetentionConfig {
private ReplicaGroupStrategyConfig replicaGroupStrategyConfig;
@NestedConfig
+ private CompletionConfig _completionConfig;
+
+ @NestedConfig
private HllConfig hllConfig;
// Number of replicas per partition of low-level consumers. This config is
used for realtime tables only.
@@ -160,6 +163,14 @@ public class SegmentsValidationAndRetentionConfig {
this.replicaGroupStrategyConfig = replicaGroupStrategyConfig;
}
+ public CompletionConfig getCompletionConfig() {
+return _completionConfig;
+ }
+
+ public void setCompletionConfig(CompletionConfig completionConfig) {
+_completionConfig = completionConfig;
+ }
+
public HllConfig getHllConfig() {
return hllConfig;
}
@@ -222,15 +233,15 @@ public class SegmentsValidationAndRetentionConfig {
SegmentsValidationAndRetentionConfig that =
(SegmentsValidationAndRetentionConfig) o;
-return EqualityUtils.isEqual(retentionTimeUnit, that.retentionTimeUnit) &&
[GitHub] [incubator-pinot] npawar merged pull request #4367: CompletionConfig for realtime tables
npawar merged pull request #4367: CompletionConfig for realtime tables URL: https://github.com/apache/incubator-pinot/pull/4367 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] tysaito opened a new pull request #4404: Added support for Vertica as data source
tysaito opened a new pull request #4404: Added support for Vertica as data source URL: https://github.com/apache/incubator-pinot/pull/4404 I made changes by following https://thirdeye.readthedocs.io/en/latest/contribute_datasource.html. Please let me know if there are any more change which are required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io commented on issue #4368: 3891: Check for validity of segment start/end time during segment generation
codecov-io commented on issue #4368: 3891: Check for validity of segment start/end time during segment generation URL: https://github.com/apache/incubator-pinot/pull/4368#issuecomment-509297732 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/4368?src=pr=h1) Report > Merging [#4368](https://codecov.io/gh/apache/incubator-pinot/pull/4368?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/33f583b2a84a8a22143a26c9cc6b80dc2a81563c?src=pr=desc) will **increase** coverage by `0.06%`. > The diff coverage is `63.41%`. [](https://codecov.io/gh/apache/incubator-pinot/pull/4368?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4368 +/- ## + Coverage 55.7% 55.77% +0.06% Complexity 20 20 Files 1066 1067 +1 Lines 5540355564 +161 Branches 7894 7911 +17 + Hits 3086530991 +126 - Misses2215022183 +33 - Partials 2388 2390 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/4368?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...org/apache/pinot/core/minion/SegmentConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9taW5pb24vU2VnbWVudENvbnZlcnRlci5qYXZh) | `72.28% <100%> (+1.77%)` | `0 <0> (ø)` | :arrow_down: | | [...indexsegment/generator/SegmentGeneratorConfig.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9pbmRleHNlZ21lbnQvZ2VuZXJhdG9yL1NlZ21lbnRHZW5lcmF0b3JDb25maWcuamF2YQ==) | `51.93% <40%> (-0.08%)` | `0 <0> (ø)` | | | [...ment/creator/impl/SegmentColumnarIndexCreator.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2NyZWF0b3IvaW1wbC9TZWdtZW50Q29sdW1uYXJJbmRleENyZWF0b3IuamF2YQ==) | `83.9% <60%> (-2.79%)` | `0 <0> (ø)` | | | [...lix/core/realtime/PinotRealtimeSegmentManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYWx0aW1lL1Bpbm90UmVhbHRpbWVTZWdtZW50TWFuYWdlci5qYXZh) | `66.5% <0%> (-4%)` | `0% <0%> (ø)` | | | [...segment/creator/impl/SegmentDictionaryCreator.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2NyZWF0b3IvaW1wbC9TZWdtZW50RGljdGlvbmFyeUNyZWF0b3IuamF2YQ==) | `79.87% <0%> (-3.91%)` | `0% <0%> (ø)` | | | [...gment/index/readers/ImmutableDictionaryReader.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2luZGV4L3JlYWRlcnMvSW1tdXRhYmxlRGljdGlvbmFyeVJlYWRlci5qYXZh) | `90.47% <0%> (-3.79%)` | `0% <0%> (ø)` | | | [...inot/core/realtime/impl/RealtimeSegmentConfig.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9pbXBsL1JlYWx0aW1lU2VnbWVudENvbmZpZy5qYXZh) | `95.16% <0%> (-1.4%)` | `0% <0%> (ø)` | | | [.../core/segment/index/loader/IndexLoadingConfig.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2luZGV4L2xvYWRlci9JbmRleExvYWRpbmdDb25maWcuamF2YQ==) | `68.88% <0%> (-0.53%)` | `0% <0%> (ø)` | | | [...e/realtime/converter/RealtimeSegmentConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9jb252ZXJ0ZXIvUmVhbHRpbWVTZWdtZW50Q29udmVydGVyLmphdmE=) | `35.82% <0%> (-0.12%)` | `0% <0%> (ø)` | | | ... and [20 more](https://codecov.io/gh/apache/incubator-pinot/pull/4368/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/4368?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/4368?src=pr=footer). Last update
[GitHub] [incubator-pinot] codecov-io commented on issue #4403: Download query result as excel on query console
codecov-io commented on issue #4403: Download query result as excel on query console URL: https://github.com/apache/incubator-pinot/pull/4403#issuecomment-509263147 # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/4403?src=pr=h1) Report > Merging [#4403](https://codecov.io/gh/apache/incubator-pinot/pull/4403?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/49d8fa7907efc4192c3ecd5fef67c33e657c39f5?src=pr=desc) will **increase** coverage by `9.76%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-pinot/pull/4403?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4403 +/- ## + Coverage 55.71% 65.47% +9.76% Complexity 20 20 Files 1067 1067 Lines 5552655526 Branches 7906 7906 + Hits 3093536357+5422 + Misses2219616581-5615 - Partials 2395 2588 +193 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/4403?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ator/transform/function/BaseTransformFunction.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vQmFzZVRyYW5zZm9ybUZ1bmN0aW9uLmphdmE=) | `29.95% <0%> (+0.42%)` | `0% <0%> (ø)` | :arrow_down: | | [...regation/function/customobject/QuantileDigest.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9hZ2dyZWdhdGlvbi9mdW5jdGlvbi9jdXN0b21vYmplY3QvUXVhbnRpbGVEaWdlc3QuamF2YQ==) | `58.18% <0%> (+0.44%)` | `0% <0%> (ø)` | :arrow_down: | | [...g/apache/pinot/common/utils/helix/HelixHelper.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaGVsaXgvSGVsaXhIZWxwZXIuamF2YQ==) | `56.25% <0%> (+0.56%)` | `0% <0%> (ø)` | :arrow_down: | | [...ment/creator/impl/SegmentColumnarIndexCreator.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2NyZWF0b3IvaW1wbC9TZWdtZW50Q29sdW1uYXJJbmRleENyZWF0b3IuamF2YQ==) | `87.5% <0%> (+0.75%)` | `0% <0%> (ø)` | :arrow_down: | | [...r/transform/function/ValueInTransformFunction.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVmFsdWVJblRyYW5zZm9ybUZ1bmN0aW9uLmphdmE=) | `39.2% <0%> (+0.8%)` | `0% <0%> (ø)` | :arrow_down: | | [.../helix/core/realtime/SegmentCompletionManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYWx0aW1lL1NlZ21lbnRDb21wbGV0aW9uTWFuYWdlci5qYXZh) | `70.39% <0%> (+0.87%)` | `0% <0%> (ø)` | :arrow_down: | | [...e/io/writer/impl/MutableOffHeapByteArrayStore.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9pby93cml0ZXIvaW1wbC9NdXRhYmxlT2ZmSGVhcEJ5dGVBcnJheVN0b3JlLmphdmE=) | `86.45% <0%> (+1.04%)` | `0% <0%> (ø)` | :arrow_down: | | [.../pinot/core/segment/index/SegmentMetadataImpl.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2luZGV4L1NlZ21lbnRNZXRhZGF0YUltcGwuamF2YQ==) | `80.81% <0%> (+1.22%)` | `0% <0%> (ø)` | :arrow_down: | | [.../org/apache/pinot/pql/parsers/Pql2AstListener.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QcWwyQXN0TGlzdGVuZXIuamF2YQ==) | `89.3% <0%> (+1.25%)` | `0% <0%> (ø)` | :arrow_down: | | [...g/apache/pinot/common/response/ServerInstance.java](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vcmVzcG9uc2UvU2VydmVySW5zdGFuY2UuamF2YQ==) | `79.72% <0%> (+1.35%)` | `0% <0%> (ø)` | :arrow_down: | | ... and [322 more](https://codecov.io/gh/apache/incubator-pinot/pull/4403/diff?src=pr=tree-more) | | -- [Continue to review full report at
[GitHub] [incubator-pinot] xueyumusic opened a new pull request #4403: Download query result as excel on query console
xueyumusic opened a new pull request #4403: Download query result as excel on query console URL: https://github.com/apache/incubator-pinot/pull/4403 Download query result as excel or csv is a common usage, this PR adds downloading query result as excel or csv or copy on query-console, like:  Other updates: 1. Upgrades jquery version to 1.10.19 2. Updates `select` and `filter` style of `datatables` to keep consist with datatables This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[incubator-pinot] branch kafka_2.0 updated (d9e0316 -> c01cc2e)
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch kafka_2.0 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from d9e0316 Adding support for Kafka 2.0 add c01cc2e fixing the bytes type conversion and adding consumer test No new revisions were added by this update. Summary of changes: .../realtime/impl/kafka2/Kafka2MessageBatch.java | 7 ++- .../Kafka2PartitionLevelConnectionHandler.java | 3 +- ...Kafka2PartitionLevelPartitionLevelConsumer.java | 9 +-- .../kafka2/KafkaPartitionLevelConsumerTest.java| 66 +- 4 files changed, 74 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] kishoreg commented on issue #4400: what is the difference between Pinot and Druid
kishoreg commented on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509098996 Real-time Ingest - Druid supports two modes Push (Tranquility) and Pull (Kafka Indexing Service). Push has the limitation that late arriving events (after a certain window) will be dropped. I am not sure if the limitation applies to the pull mode. https://druid.apache.org/docs/latest/ingestion/stream-ingestion.html Pinot support ingesting data from directly from Kafka. We support two modes - using high level Kafka Consumer or low level Kafka consumer. Using low level you can get exactly once semantics. Pinot does not drop late arriving events. Segment - Pinot & Druid have their own custom columnar format to store data. At a high level, a segment contains forward index (mostly dictionary encoded) and inverted index (optional). Pinot has additional indexing schema known as star-tree index which can speed up Aggregations. https://engineering.linkedin.com/blog/2019/06/star-tree-index--powering-fast-aggregations-on-pinot See slides 21, 28, 35 for performance comparison between Pinot and Druid. https://www.slideshare.net/KishoreGopalakrishna/building-real-time-analytics-applications-using-pinot-a-linkedin-case-study This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] sunny209868 opened a new issue #4402: the most appropriate segment size
sunny209868 opened a new issue #4402: the most appropriate segment size URL: https://github.com/apache/incubator-pinot/issues/4402 Who knows what is the most appropriate segment size？ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] sunny209868 edited a comment on issue #4400: what is the difference between Pinot and Druid
sunny209868 edited a comment on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509092892 > 3Q https://github.com/apache/incubator-pinot/issues/3 this issues can help you @JackyYangPassion This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] sunny209868 edited a comment on issue #4400: what is the difference between Pinot and Druid
sunny209868 edited a comment on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509092892 > 3Q https://github.com/apache/incubator-pinot/issues/3， this issues can help you @JackyYangPassion This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
[GitHub] [incubator-pinot] sunny209868 commented on issue #4400: what is the difference between Pinot and Druid
sunny209868 commented on issue #4400: what is the difference between Pinot and Druid URL: https://github.com/apache/incubator-pinot/issues/4400#issuecomment-509092892 > 3Q https://github.com/apache/incubator-pinot/issues/3，this issues can help you @JackyYangPassion This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
