[incubator-pinot] branch configurable_spark_job_parallelism updated (6e2cf59 -> 35b7ebc)

2020-09-15 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch configurable_spark_job_parallelism in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 6e2cf59 Adding field 'segmentCreationJobParallelism' to allow users to set

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6012: Configurable segment generation job parallelism for Hadoop and Spark

2020-09-15 Thread GitBox
fx19880617 commented on a change in pull request #6012: URL: https://github.com/apache/incubator-pinot/pull/6012#discussion_r488431110 ## File path: pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-standalone/src/main/resources/segmentCreationAndTarPushJobSpec.yaml

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6012: Configurable segment generation job parallelism for Hadoop and Spark

2020-09-15 Thread GitBox
fx19880617 commented on a change in pull request #6012: URL: https://github.com/apache/incubator-pinot/pull/6012#discussion_r488432050 ## File path:

[GitHub] [incubator-pinot] shahsank3t commented on pull request #5981: Support for Update & Delete in ZooKeeper Browser and added SQL Functions in SQL Editor autocomplete list

2020-09-15 Thread GitBox
shahsank3t commented on pull request #5981: URL: https://github.com/apache/incubator-pinot/pull/5981#issuecomment-692519826 @kishoreg rebased and all checks passed. This is an automated message from the Apache Git Service.

[incubator-pinot] branch master updated: Adding field 'segmentCreationJobParallelism' to allow users to set segment generation job parallelism. Default to the number of input files. (#6012)

2020-09-15 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 054faf7 Adding field

[GitHub] [incubator-pinot] fx19880617 merged pull request #6012: Configurable segment generation job parallelism for Hadoop and Spark

2020-09-15 Thread GitBox
fx19880617 merged pull request #6012: URL: https://github.com/apache/incubator-pinot/pull/6012 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[incubator-pinot] branch master updated: Support for Update & Delete in ZooKeeper Browser and added SQL Functions in SQL Editor autocomplete list (#5981)

2020-09-15 Thread kishoreg
This is an automated email from the ASF dual-hosted git repository. kishoreg pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new 5da3433 Support for Update & Delete

[GitHub] [incubator-pinot] kishoreg merged pull request #5981: Support for Update & Delete in ZooKeeper Browser and added SQL Functions in SQL Editor autocomplete list

2020-09-15 Thread GitBox
kishoreg merged pull request #5981: URL: https://github.com/apache/incubator-pinot/pull/5981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] lgo edited a comment on issue #5814: Pinot's swagger pages does not work on HTTPS-only environments

2020-09-15 Thread GitBox
lgo edited a comment on issue #5814: URL: https://github.com/apache/incubator-pinot/issues/5814#issuecomment-692855728 This has come up against in a different shape after Swagger was bumpted to v3 (#5888). This time, the page does list `HTTP` and `HTTPS` because the Swagger resources

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6017: Table indexing config validation

2020-09-15 Thread GitBox
jackjlli commented on a change in pull request #6017: URL: https://github.com/apache/incubator-pinot/pull/6017#discussion_r488818486 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -65,6 +68,7 @@ public static void

[GitHub] [incubator-pinot] lgo commented on issue #5814: Pinot's swagger pages does not work on HTTPS-only environments

2020-09-15 Thread GitBox
lgo commented on issue #5814: URL: https://github.com/apache/incubator-pinot/issues/5814#issuecomment-692855728 This has come up against in a different shape after Swagger was bumpted to v3 (#5888). This time, the page does list `HTTP` and `HTTPS` because the Swagger resources list those

[GitHub] [incubator-pinot] lgo edited a comment on issue #5814: Pinot's swagger pages does not work on HTTPS-only environments

2020-09-15 Thread GitBox
lgo edited a comment on issue #5814: URL: https://github.com/apache/incubator-pinot/issues/5814#issuecomment-692855728 This has come up against in a different shape after Swagger was bumpted to v3 (#5888). This time, the page does list `HTTP` and `HTTPS` because the Swagger resources

[GitHub] [incubator-pinot] npawar commented on a change in pull request #5934: Segment processing framework

2020-09-15 Thread GitBox
npawar commented on a change in pull request #5934: URL: https://github.com/apache/incubator-pinot/pull/5934#discussion_r488812749 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/processing/utils/SegmentProcessorUtils.java ## @@ -0,0 +1,113 @@ +/** + *

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6017: Table indexing config validation

2020-09-15 Thread GitBox
mcvsubbu commented on a change in pull request #6017: URL: https://github.com/apache/incubator-pinot/pull/6017#discussion_r488791747 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -225,4 +229,54 @@ private static void

[GitHub] [incubator-pinot] npawar commented on a change in pull request #5934: Segment processing framework

2020-09-15 Thread GitBox
npawar commented on a change in pull request #5934: URL: https://github.com/apache/incubator-pinot/pull/5934#discussion_r488812122 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java ## @@ -0,0 +1,148 @@ +/** + *

[incubator-pinot] branch master updated (5da3433 -> 41de9a6)

2020-09-15 Thread nehapawar
This is an automated email from the ASF dual-hosted git repository. nehapawar pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from 5da3433 Support for Update & Delete in ZooKeeper Browser and added SQL Functions in SQL Editor

[GitHub] [incubator-pinot] npawar merged pull request #5934: Segment processing framework

2020-09-15 Thread GitBox
npawar merged pull request #5934: URL: https://github.com/apache/incubator-pinot/pull/5934 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[incubator-pinot] 01/01: Include branch name when publishing pinot release to bintray

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch check-branch-name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit a3bc8b60c3d8c4e4844171d90dd9b3e65d25b4e4 Author: Jack Li(Analytics Engineering) AuthorDate: Tue Sep

[incubator-pinot] branch check-branch-name updated (c5905be -> a3bc8b6)

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch check-branch-name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard c5905be Validate branch name new a3bc8b6 Include branch name when publishing pinot release

[incubator-pinot] 02/04: Fetch configs from cache rather than zookeeper

2020-09-15 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch controller_api_patch in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit f3472898ad5e81f0de3267c4b2333b629fc9d007 Author: Kartik Khare AuthorDate: Tue Sep 15

[incubator-pinot] branch controller_api_patch created (now 36babc3)

2020-09-15 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a change to branch controller_api_patch in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 36babc3 change in constructors of listeners This branch includes the following new

[incubator-pinot] 01/04: Add listeners to cache zookeeper configs for helix

2020-09-15 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch controller_api_patch in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit acd9397d28f1edd400c4b6fcdea8473a770d475e Author: Kartik Khare AuthorDate: Tue Sep 15

[incubator-pinot] 03/04: Add change for Helix Configs

2020-09-15 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch controller_api_patch in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit e30ffb9ecb2be33ad5e1d6161deb838884b08fe1 Author: Kartik Khare AuthorDate: Wed Sep 16

[incubator-pinot] branch check-branch-name updated (6ba644e -> c5905be)

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch check-branch-name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard 6ba644e Validate branch name new c5905be Validate branch name This update added new

[incubator-pinot] 01/01: Validate branch name

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch check-branch-name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit c5905be15f839add19c746cb23e2095d10fd0d1f Author: Jack Li(Analytics Engineering) AuthorDate: Tue Sep

[incubator-pinot] branch master updated: Include branch name when publishing pinot release to bintray (#6019)

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new c42c740 Include branch name when

[GitHub] [incubator-pinot] jackjlli opened a new pull request #6019: Include branch name when publishing pinot release to bintray

2020-09-15 Thread GitBox
jackjlli opened a new pull request #6019: URL: https://github.com/apache/incubator-pinot/pull/6019 ## Description This PR includes branch name in the manifest when publishing pinot release to bintray. As someone mentioned in

[GitHub] [incubator-pinot] KKcorps opened a new pull request #6020: Add Caching in Controller Broker API

2020-09-15 Thread GitBox
KKcorps opened a new pull request #6020: URL: https://github.com/apache/incubator-pinot/pull/6020 Currently, Controller does multiple calls to zookeeper to fetch the list of brokers per tenant and table. This PR aims to cache the instance configs of zookeeper so that the calls don't go

[GitHub] [incubator-pinot] jackjlli merged pull request #6019: Include branch name when publishing pinot release to bintray

2020-09-15 Thread GitBox
jackjlli merged pull request #6019: URL: https://github.com/apache/incubator-pinot/pull/6019 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[incubator-pinot] 01/01: Validate branch name

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch check-branch-name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6ba644e391e795b420f043873fd9e39ec9f1e47d Author: Jack Li(Analytics Engineering) AuthorDate: Tue Sep

[incubator-pinot] branch check-branch-name created (now 6ba644e)

2020-09-15 Thread jlli
This is an automated email from the ASF dual-hosted git repository. jlli pushed a change to branch check-branch-name in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 6ba644e Validate branch name This branch includes the following new commits: new 6ba644e

[incubator-pinot] 04/04: change in constructors of listeners

2020-09-15 Thread kharekartik
This is an automated email from the ASF dual-hosted git repository. kharekartik pushed a commit to branch controller_api_patch in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 36babc3ef54be37809e8b0e14119080a3aada5ca Author: Kartik Khare AuthorDate: Wed Sep 16

[incubator-pinot] branch master updated: Add a length limit of 512 to the properties stored in the segment metadata (#6008)

2020-09-15 Thread jackie
This is an automated email from the ASF dual-hosted git repository. jackie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new b2df8ea Add a length limit of 512 to

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

2020-09-15 Thread GitBox
Jackie-Jiang merged pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework

2020-09-15 Thread GitBox
Jackie-Jiang commented on a change in pull request #6021: URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489090953 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java ## @@ -100,8 +110,11 @@ public

[GitHub] [incubator-pinot] mcvsubbu commented on issue #5960: LLC stopped realtime ingestion due to deadlock caused by out of order Helix state transition events (Pinot 0.4+)

2020-09-15 Thread GitBox
mcvsubbu commented on issue #5960: URL: https://github.com/apache/incubator-pinot/issues/5960#issuecomment-693104377 It is useful add some notes here on this. The reason the semaphore was put in place was because Kafka consumers had an issue with dynamically creating new kafka

[GitHub] [incubator-pinot] npawar commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework

2020-09-15 Thread GitBox
npawar commented on a change in pull request #6021: URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489102318 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java ## @@ -100,8 +110,11 @@ public void

[GitHub] [incubator-pinot] npawar opened a new pull request #6021: List of partitioners in SegmentProcessorFramework

2020-09-15 Thread GitBox
npawar opened a new pull request #6021: URL: https://github.com/apache/incubator-pinot/pull/6021 Changing SegmentPrpcessorFramework config to take List of Partitioners. This is to account for table config's partitioning, which might need to be applied regardless of other partitioning

[GitHub] [incubator-pinot] npawar commented on a change in pull request #6017: Table indexing config validation

2020-09-15 Thread GitBox
npawar commented on a change in pull request #6017: URL: https://github.com/apache/incubator-pinot/pull/6017#discussion_r489027314 ## File path: pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java ## @@ -225,4 +229,54 @@ private static void

[incubator-pinot] branch master updated: [TE] add labeler into yaml (#6007)

2020-09-15 Thread jihao
This is an automated email from the ASF dual-hosted git repository. jihao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new a123c60 [TE] add labeler into yaml

[GitHub] [incubator-pinot] jihaozh merged pull request #6007: [TE] add labeler into yaml

2020-09-15 Thread GitBox
jihaozh merged pull request #6007: URL: https://github.com/apache/incubator-pinot/pull/6007 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] chenboat commented on issue #5960: LLC stopped realtime ingestion due to deadlock caused by out of order Helix state transition events (Pinot 0.4+)

2020-09-15 Thread GitBox
chenboat commented on issue #5960: URL: https://github.com/apache/incubator-pinot/issues/5960#issuecomment-693096957 Thanks. The issue was resolved by now. But it will be great to explicitly exposed this config to Pinot installation. It is also worthwhile to fix the deadlock when

[GitHub] [incubator-pinot] Jackie-Jiang opened a new pull request #6022: Add IN_SUBQUERY support

2020-09-15 Thread GitBox
Jackie-Jiang opened a new pull request #6022: URL: https://github.com/apache/incubator-pinot/pull/6022 ## Description Add `IN_SUBQUERY` transform function to support `IDSET` aggregation function as the subquery. The subquery is handled as a separate query on broker side. E.g. The

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework

2020-09-15 Thread GitBox
mcvsubbu commented on a change in pull request #6021: URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489089813 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java ## @@ -100,8 +110,11 @@ public

[GitHub] [incubator-pinot] lgo opened a new pull request #6023: add `@types/react-router` dep

2020-09-15 Thread GitBox
lgo opened a new pull request #6023: URL: https://github.com/apache/incubator-pinot/pull/6023 ## Description Before, `@types/react-router` was unlisted. Because `@types/react-router-dom` only has `@types/react-router: "*"` defined, it is possible the wrong version is pulled. When

[GitHub] [incubator-pinot] mcvsubbu commented on a change in pull request #6021: List of partitioners in SegmentProcessorFramework

2020-09-15 Thread GitBox
mcvsubbu commented on a change in pull request #6021: URL: https://github.com/apache/incubator-pinot/pull/6021#discussion_r489138013 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/processing/framework/SegmentMapper.java ## @@ -100,8 +110,11 @@ public

[incubator-pinot] branch master updated (a123c60 -> 07666f9)

2020-09-15 Thread xiangfu
This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. from a123c60 [TE] add labeler into yaml (#6007) add 07666f9 bump paranamer (2.6 -> 2.8) (#6024) No new

[GitHub] [incubator-pinot] fx19880617 merged pull request #6024: bump paranamer (2.6 -> 2.8) to fix Spark failures (on 2.4.4 scala 2.12)

2020-09-15 Thread GitBox
fx19880617 merged pull request #6024: URL: https://github.com/apache/incubator-pinot/pull/6024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-pinot] lgo opened a new pull request #6024: bump paranamer (2.6 -> 2.8)

2020-09-15 Thread GitBox
lgo opened a new pull request #6024: URL: https://github.com/apache/incubator-pinot/pull/6024 ## Description Before on paranamer 2.6, there was an incompatibility with certain JDK 8 features as mentioned in https://github.com/paul-hammant/paranamer/issues/17. This seemed to happen