Hi Yufei, The "how" in your question depends on the deployment environment, I guess. There are a lot of variants.
If you wonder whether such a situation is possible in practice, I believe it is. An example would be self-hosting non-AWS S3 storage and Polaris in a way that Polaris connections go through a certain internal network, while connections from query engines running outside of that deployment environment go through a different network. This is very high-level, of course, since the deployment choices are largely driven by specific users' needs. The proposed "endpointInternal" config entry merely expands deployment options that users can choose from. Cheers, Dmitri. On Thu, Jul 31, 2025 at 1:07 PM Yufei Gu <flyrain...@gmail.com> wrote: > Hi Dimtri, > > That generally makes sense to me. For awareness, could you elaborate a bit > on how the Polaris server and query engines (like Spark, Trino, etc.) might > access the same object storage (e.g., MinIO) via different DNS endpoints? > > Yufei > > > On Thu, Jul 31, 2025 at 4:36 AM Alexandre Dutra <adu...@apache.org> wrote: > > > Hi Dmitri, > > > > I think your suggestion makes sense. We added something similar in > > Nessie long ago, and it is definitely useful. > > > > I left some comments in the PR. > > > > Thanks, > > Alex > > > > On Thu, Jul 31, 2025 at 4:12 AM Dmitri Bourlatchkov <di...@apache.org> > > wrote: > > > > > > Hi All, > > > > > > I propose to add an `endpointInternal` optional parameter to > > > AwsStorageConfigInfo > > > in PR [2213]. > > > > > > The main idea is to support deployment edge cases where Polaris Servers > > may > > > 'see' storage under a different DNS name than query engines. This use > > case > > > applies mostly to non-AWS S3 storage (e.g. MinIO). > > > > > > This change is backward-compatible with existing clients and deployed > > > catalogs. > > > > > > Thoughts? > > > > > > [2213] https://github.com/apache/polaris/pull/2213 > > > > > > Thanks, > > > Dmitri. > > >
