jackjlli opened a new issue #6730:
URL: https://github.com/apache/incubator-pinot/issues/6730
Lead controller resource is design to logically separate the per-table
functionality in pinot cluster.
As lead controller resource feature has been implemented and running for a
while, we'd like to enable this feature to keep the behavior consistent in all
Pinot clusters.
Before enabling the lead controller resource, there is only 1 pinot
controller (which is also the helix lead controller) that does all the pinot
controller periodic tasks and realtime segment completions for all the tables.
The lead helix/pinot controller can be found at the `CONTROLLER/LEADER` ZNode.
After enabling the lead controller resource, helix controllers and pinot
controllers don't have to run on the same hardware. Znode `CONTROLLER/LEADER`
only denotes Helix lead controller. All the controller periodic tasks and
realtime segment completions are distributed to all the available pinot
controllers. The lead Pinot controller for each of the tables can be found by
the following APIs:
1. Get the leaders for all the tables in the cluster
GET /leader/tables
2. Given a table name, return the partition id and lead controller instance
id
GET /leader/tables/{tableName}
Offline segment upload won't bind to any of the pinot controllers; all the
available pinot controllers accepts segment upload since it's neither
controller periodic tasks nor realtime segment completions.
Details can be found in doc:
https://cwiki.apache.org/confluence/display/PINOT/Controller+Separation+between+Helix+and+Pinot
There will be two PRs for this issue:
The first PR will enable lead controller in all Pinot clusters. New onboard
clusters or existing running clusters that start to run with this code will
have lead controller resource enabled.
The second PR will clean up the code that handles the old protocol in the
repo, which will be merged in Apache Pinot Release 0.8.0 or higher. The cleanup
should be done in the following classes:
- Controller: LeadControllerManager.java
- Server: ControllerLeaderLocator.java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
