[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6037: Add list of allowed tables for emitting table level metrics

mcvsubbu commented on a change in pull request #6037: URL: https://github.com/apache/incubator-pinot/pull/6037#discussion_r492959921 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -279,7 +281,10 @@ } public static

[GitHub] [incubator-pinot] timsants opened a new pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

timsants opened a new pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046 ## Description 1. PR for issue [#5507](https://github.com/apache/incubator-pinot/issues/5507). ORC, Thrift, and ProtoBuf readers now convert: - Nested structures to Map -

[GitHub] [incubator-pinot] suvodeep-pyne opened a new pull request #6047: [TE] Remove deprecated docker directory. Use incubator-pinot/docker/images instead

suvodeep-pyne opened a new pull request #6047: URL: https://github.com/apache/incubator-pinot/pull/6047 Removing deprecated docker directory `./incubator-pinot/thirdeye/docker`. The new and updated image is in the `incubator-pinot/docker/images/pinot-thirdeye` directory.

[GitHub] [incubator-pinot] suvodeep-pyne opened a new pull request #6048: [TE] Remove deprecated thirdeye-hadoop module

suvodeep-pyne opened a new pull request #6048: URL: https://github.com/apache/incubator-pinot/pull/6048 This is legacy code and is no longer being used. This is an automated message from the Apache Git Service. To respond to

[incubator-pinot] branch 0.5-upgrade updated (0c0e00b -> 7eb0a74)

This is an automated email from the ASF dual-hosted git repository. tingchen pushed a change to branch 0.5-upgrade in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 0c0e00b [maven-release-plugin] prepare for next development iteration add 7eb0a74 Upgrade

[incubator-pinot] 01/01: [maven-release-plugin] prepare for next development iteration

This is an automated email from the ASF dual-hosted git repository. tingchen pushed a commit to branch 0.5-upgrade in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 0c0e00b5fbf874e21454c160f25a9c056df85127 Author: TING CHEN AuthorDate: Wed Sep 2 17:49:37 2020 -0700

[incubator-pinot] branch 0.5-upgrade created (now 0c0e00b)

This is an automated email from the ASF dual-hosted git repository. tingchen pushed a change to branch 0.5-upgrade in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 0c0e00b [maven-release-plugin] prepare for next development iteration This branch includes the

[GitHub] [incubator-pinot] chenboat opened a new pull request #6045: [maven-release-plugin] prepare for next development iteration after 0.5.0 release

chenboat opened a new pull request #6045: URL: https://github.com/apache/incubator-pinot/pull/6045 ## Description Add a description of your PR here. A good description should include pointers to an issue or design document, etc. ## Upgrade Notes Does this PR prevent a zero

[GitHub] [incubator-pinot] snleee commented on pull request #6049: Add validation for segment partition config column names

snleee commented on pull request #6049: URL: https://github.com/apache/incubator-pinot/pull/6049#issuecomment-696977822 When making a mistake on the column name for partition config, the consumption would stop because it fails to generate the segment. ``` 2020/09/22 07:15:46.135

[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

kishoreg commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r493022384 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] snleee edited a comment on pull request #6049: Add validation for segment partition config column names

snleee edited a comment on pull request #6049: URL: https://github.com/apache/incubator-pinot/pull/6049#issuecomment-696977822 When making a mistake on the column name for partition config, the consumption would stop because it fails to generate the segment. Instead, we should catch this

[GitHub] [incubator-pinot] snleee opened a new pull request #6049: Add validation for segment partition config column names

snleee opened a new pull request #6049: URL: https://github.com/apache/incubator-pinot/pull/6049 Add validation for segment partition config column names This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-pinot] sajjad-moradi commented on a change in pull request #6037: Add list of allowed tables for emitting table level metrics

sajjad-moradi commented on a change in pull request #6037: URL: https://github.com/apache/incubator-pinot/pull/6037#discussion_r493019744 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -279,7 +281,10 @@ } public

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

mayankshriv commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r492853736 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

mayankshriv commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r492853736 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

mayankshriv commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r492853736 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

mayankshriv commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r492853736 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #6026: [TE] Creating a thirdeye-dashboard module to host the dashboard server

suvodeep-pyne commented on pull request #6026: URL: https://github.com/apache/incubator-pinot/pull/6026#issuecomment-696835579 Thanks a lot @vincentchenjl @jihaozh ! This is an automated message from the Apache Git Service.

[GitHub] [incubator-pinot] npawar opened a new pull request #6050: Minion taskExecutor for RealtimeToOfflineSegments task

npawar opened a new pull request #6050: URL: https://github.com/apache/incubator-pinot/pull/6050 https://github.com/apache/incubator-pinot/issues/5753 The minion task executor which receives 1) segments (from realtime tables) 2) a time window And then creates Pinot

[GitHub] [incubator-pinot] npawar commented on pull request #6048: [TE] Remove deprecated thirdeye-hadoop module

npawar commented on pull request #6048: URL: https://github.com/apache/incubator-pinot/pull/6048#issuecomment-697046600 Woohoo! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-pinot] npawar commented on a change in pull request #6038: Adding dependency validation check on Indexing config

npawar commented on a change in pull request #6038: URL: https://github.com/apache/incubator-pinot/pull/6038#discussion_r493103312 ## File path: pinot-core/src/test/java/org/apache/pinot/core/util/TableConfigUtilsTest.java ## @@ -544,55 +545,43 @@ public void

[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

kishoreg commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r493022384 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] apucher merged pull request #6041: remove default javaagent opts in generator.sh script to avoid javaagent port colission

apucher merged pull request #6041: URL: https://github.com/apache/incubator-pinot/pull/6041 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] mayankshriv commented on pull request #6033: Add Broker Reduce Time Log

mayankshriv commented on pull request #6033: URL: https://github.com/apache/incubator-pinot/pull/6033#issuecomment-696497353 @mr-agrwal @fx19880617 We do have metrics for each phase of query in the broker side (including scatter/gather, reduce, etc). Adding just the reduce part in the log

[GitHub] [incubator-pinot] npawar commented on a change in pull request #6038: Adding dependency validation check on Indexing config

npawar commented on a change in pull request #6038: URL: https://github.com/apache/incubator-pinot/pull/6038#discussion_r493103312 ## File path: pinot-core/src/test/java/org/apache/pinot/core/util/TableConfigUtilsTest.java ## @@ -544,55 +545,43 @@ public void

[GitHub] [incubator-pinot] icefury71 commented on a change in pull request #6038: Adding dependency validation check on Indexing config

icefury71 commented on a change in pull request #6038: URL: https://github.com/apache/incubator-pinot/pull/6038#discussion_r493184510 ## File path: pinot-core/src/test/java/org/apache/pinot/core/util/TableConfigUtilsTest.java ## @@ -544,55 +545,43 @@ public void

[GitHub] [incubator-pinot] kishoreg commented on pull request #6033: Add Broker Reduce Time Log

kishoreg commented on pull request #6033: URL: https://github.com/apache/incubator-pinot/pull/6033#issuecomment-696501753 This is the one that uncovered the issue that broker was indeed taking a long time. This will probably help at LinkedIn as well to identify the usecases that can

[GitHub] [incubator-pinot] snleee commented on pull request #6049: Add validation for segment partition config column names

snleee commented on pull request #6049: URL: https://github.com/apache/incubator-pinot/pull/6049#issuecomment-696977822 When making a mistake on the column name for partition config, the consumption would stop because it fails to generate the segment. ``` 2020/09/22 07:15:46.135

[GitHub] [incubator-pinot] snleee edited a comment on pull request #6049: Add validation for segment partition config column names

snleee edited a comment on pull request #6049: URL: https://github.com/apache/incubator-pinot/pull/6049#issuecomment-696977822 When making a mistake on the column name for partition config, the consumption would stop because it fails to generate the segment. Instead, we should catch this

[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #6026: [TE] Creating a thirdeye-dashboard module to host the dashboard server

suvodeep-pyne commented on pull request #6026: URL: https://github.com/apache/incubator-pinot/pull/6026#issuecomment-696835579 Thanks a lot @vincentchenjl @jihaozh ! This is an automated message from the Apache Git Service.

[GitHub] [incubator-pinot] mayankshriv commented on issue #6028: Make BrokerReduceService.reduceOnDataTable Multi Threaded to increase aggregation performance

mayankshriv commented on issue #6028: URL: https://github.com/apache/incubator-pinot/issues/6028#issuecomment-696510899 @mr-agrwal Could you please try the changes in this PR and provide some feedback: https://github.com/apache/incubator-pinot/pull/6044

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

Jackie-Jiang commented on a change in pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#discussion_r493140341 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/RecordExtractor.java ## @@ -27,8 +28,9 @@ * 2) Collections become

[GitHub] [incubator-pinot] sajjad-moradi commented on a change in pull request #6037: Add list of allowed tables for emitting table level metrics

sajjad-moradi commented on a change in pull request #6037: URL: https://github.com/apache/incubator-pinot/pull/6037#discussion_r493019744 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -279,7 +281,10 @@ } public

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6042: Fix built-in virtual columns for immutable segment

Jackie-Jiang merged pull request #6042: URL: https://github.com/apache/incubator-pinot/pull/6042 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #6040: Remove the partition info from the consuming segment ZK metadata

Jackie-Jiang commented on pull request #6040: URL: https://github.com/apache/incubator-pinot/pull/6040#issuecomment-696415308 @mcvsubbu The segment partition pruning for consuming segment will happen on server side instead of broker side. Yes there will be some overhead, and we need to

[GitHub] [incubator-pinot] jihaozh merged pull request #6026: [TE] Creating a thirdeye-dashboard module to host the dashboard server

jihaozh merged pull request #6026: URL: https://github.com/apache/incubator-pinot/pull/6026 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] harleyjj opened a new pull request #6051: [TE] frontend - harleyjj/yaml - show spinner in buttons when submitti…

harleyjj opened a new pull request #6051: URL: https://github.com/apache/incubator-pinot/pull/6051 …ng yaml config for creating or editing detection or subscription group ## Description * Switch yaml config submissions to ember-concurrency * Lever isRunning to track task *

[GitHub] [incubator-pinot] fx19880617 opened a new issue #6052: Tunable znode size for helm chart

fx19880617 opened a new issue #6052: URL: https://github.com/apache/incubator-pinot/issues/6052 Pinot actually requires much larger znode size limit comparing to the default settings. We need to make this setting configurable and put a good default value.

[GitHub] [incubator-pinot] mayankshriv edited a comment on issue #6028: Make BrokerReduceService.reduceOnDataTable Multi Threaded to increase aggregation performance

mayankshriv edited a comment on issue #6028: URL: https://github.com/apache/incubator-pinot/issues/6028#issuecomment-696510899 @mr-agrwal Could you please try the changes in this PR and provide some feedback: https://github.com/apache/incubator-pinot/pull/6044

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6037: Add list of allowed tables for emitting table level metrics

mcvsubbu commented on a change in pull request #6037: URL: https://github.com/apache/incubator-pinot/pull/6037#discussion_r492959921 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -279,7 +281,10 @@ } public static

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6044: Support for multi-threaded Group By reducer for SQL.

mayankshriv commented on a change in pull request #6044: URL: https://github.com/apache/incubator-pinot/pull/6044#discussion_r492853736 ## File path: pinot-common/src/main/java/org/apache/pinot/common/utils/CommonConstants.java ## @@ -161,6 +161,10 @@ public static final

[GitHub] [incubator-pinot] npawar commented on pull request #6048: [TE] Remove deprecated thirdeye-hadoop module

npawar commented on pull request #6048: URL: https://github.com/apache/incubator-pinot/pull/6048#issuecomment-697046600 Woohoo! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6046: Deep Extraction Support for ORC, Thrift, and ProtoBuf Records

Jackie-Jiang commented on a change in pull request #6046: URL: https://github.com/apache/incubator-pinot/pull/6046#discussion_r493140341 ## File path: pinot-spi/src/main/java/org/apache/pinot/spi/data/readers/RecordExtractor.java ## @@ -27,8 +28,9 @@ * 2) Collections become

[GitHub] [incubator-pinot] icefury71 commented on a change in pull request #6038: Adding dependency validation check on Indexing config

icefury71 commented on a change in pull request #6038: URL: https://github.com/apache/incubator-pinot/pull/6038#discussion_r493184510 ## File path: pinot-core/src/test/java/org/apache/pinot/core/util/TableConfigUtilsTest.java ## @@ -544,55 +545,43 @@ public void