flyrain commented on code in PR #3990: URL: https://github.com/apache/polaris/pull/3990#discussion_r3249574397
########## site/content/in-dev/unreleased/delegation-service.md: ########## @@ -0,0 +1,116 @@ +--- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +title: Delegation Service +type: docs +weight: 430 +--- + +A Delegation Service (D.S.) is a service that works alongside Polaris, either driving Polaris from outside or running inside the Polaris deployment to do work on its behalf. It can be deployed in one of two modes depending on who runs it and which way the calls flow: + +- **Pull**: the delegation service runs outside Polaris (e.g. a scheduled compaction or snapshot-expiration job) and calls Polaris over REST to fetch policies, table metadata, etc. +- **Push**: the delegation service is co-deployed with Polaris, inside the same security boundary. Polaris invokes it for heavy workloads that would otherwise degrade the Polaris service, such as intensive network calls, large I/O operations, or compute-heavy tasks. The delegation service is hidden behind the Polaris deployment; clients cannot reach it and do not need to know whether one is configured or which implementation is in use. + +The two modes solve different problems. Pull supports external systems that integrate with Polaris. Push lets a Polaris deployment offload internal work (e.g. table file purge on `DROP ... PURGE`, server-side scan planning) without changing the public API. A single deployment can use both. + +``` +Pull mode (delegation service is external): + + ┌──────────────────────┐ REST (pull) ┌────────────┐ + │ Delegation service │ ─────────────────► │ Polaris │ + │ (compute engine, │ │ │ + │ maintenance job) │ └────────────┘ + └──────────────────────┘ + + +Push mode (delegation service is internal, invisible to clients): + + ┌── Polaris deployment ─────────────────┐ + │ │ + ┌────────┐ REST │ ┌──────────┐ internal ┌──────┐ │ + │ Client │ ────────► │ ─►│ Polaris │ ────────────► │ D.S. │ │ + └────────┘ │ └──────────┘ └──────┘ │ + │ │ + └───────────────────────────────────────┘ +``` + +## Pull mode Review Comment: Yes, it is just another REST client. Sounds silly, but the point here is to clarify what Polaris community thoughts on use cases like table maintenance service. There were expectations and discussions of how Polaris community build a TMS. I think the stands here is that Polaris community could do that but in a way everybody else can do that via the public REST APIs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
