Hi Yufei, 1: I do not really know. This is a question about a specific deployment environment.
2: I'm not sure I understand your question. Two endpoints are necessary in cases when the server's view of the network is different from the engine's view. Cheers, Dmitri. On Thu, Jul 31, 2025 at 5:59 PM Yufei Gu <flyrain...@gmail.com> wrote: > Thanks for the explanation. Two questions: > 1. Should the public endpoint used by engines still work with Polaris even > if it co-locates with MinIO server? > 2. Can we set Polaris endpoint directly to the internal address in that > case? Another way to ask this question is that why do we need to keep both > endpoints in the Polaris server? > > Yufei > > > On Thu, Jul 31, 2025 at 11:06 AM Dmitri Bourlatchkov <di...@apache.org> > wrote: > > > Hi Yufei, > > > > The "how" in your question depends on the deployment environment, I > guess. > > There are a lot of variants. > > > > If you wonder whether such a situation is possible in practice, I > > believe it is. An example would be self-hosting non-AWS S3 storage and > > Polaris in a way that Polaris connections go through a certain internal > > network, while connections from query engines running outside of that > > deployment environment go through a different network. This is very > > high-level, of course, since the deployment choices are largely driven by > > specific users' needs. The proposed "endpointInternal" config entry > merely > > expands deployment options that users can choose from. > > > > Cheers, > > Dmitri. > > > > On Thu, Jul 31, 2025 at 1:07 PM Yufei Gu <flyrain...@gmail.com> wrote: > > > > > Hi Dimtri, > > > > > > That generally makes sense to me. For awareness, could you elaborate a > > bit > > > on how the Polaris server and query engines (like Spark, Trino, etc.) > > might > > > access the same object storage (e.g., MinIO) via different DNS > endpoints? > > > > > > Yufei > > > > > > > > > On Thu, Jul 31, 2025 at 4:36 AM Alexandre Dutra <adu...@apache.org> > > wrote: > > > > > > > Hi Dmitri, > > > > > > > > I think your suggestion makes sense. We added something similar in > > > > Nessie long ago, and it is definitely useful. > > > > > > > > I left some comments in the PR. > > > > > > > > Thanks, > > > > Alex > > > > > > > > On Thu, Jul 31, 2025 at 4:12 AM Dmitri Bourlatchkov < > di...@apache.org> > > > > wrote: > > > > > > > > > > Hi All, > > > > > > > > > > I propose to add an `endpointInternal` optional parameter to > > > > > AwsStorageConfigInfo > > > > > in PR [2213]. > > > > > > > > > > The main idea is to support deployment edge cases where Polaris > > Servers > > > > may > > > > > 'see' storage under a different DNS name than query engines. This > use > > > > case > > > > > applies mostly to non-AWS S3 storage (e.g. MinIO). > > > > > > > > > > This change is backward-compatible with existing clients and > deployed > > > > > catalogs. > > > > > > > > > > Thoughts? > > > > > > > > > > [2213] https://github.com/apache/polaris/pull/2213 > > > > > > > > > > Thanks, > > > > > Dmitri. > > > > > > > > > >
