Hi Pat,

Remote signing sounds a good idea! Looking forward to a proposal/design doc.

Yufei


On Fri, Aug 1, 2025 at 8:44 AM Pat Patterson <p...@backblaze.com.invalid>
wrote:

> Hi,
>
> I'm Pat Patterson, Chief Technical Evangelist at Backblaze. I've
> been working with Backblaze B2, our S3-compatible cloud object store, and
> Iceberg for a little while now, showing how to use it from Snowflake,
> Trino, DuckDB, etc.
>
> I'm replying here as requested by Dmitri on the "Support for non-AWS S3
> compatible storage with STS" GitHub issue [1]. I think S3 signing would
> work well with Backblaze B2, since we don't currently have an STS. I'm
> happy to help in any way I can - I just left a reply to Alexandre Dutra on
> the "On-Premise S3 & Remote Signing" GitHub issue [2].
>
> [1] https://github.com/apache/polaris/issues/1530#issuecomment-3138005897
> [2] https://github.com/apache/polaris/issues/32#issuecomment-3144991873
>
> Cheers,
>
> Pat
>
> On 2025/07/31 15:35:55 Robert Stupp wrote:
> > Hi,
> >
> > not sure whether exposing the object storage credentials given to
> > Polaris to all clients isn't going to cause a "false impression of
> > security" (aka: "our credentials are vended by Polaris, so we're safe"
> > - nope...).
> > With my "evil user" hat on, I'd try to figure out the configuration
> > option (is it realm-specific?) to tell Polaris to yield its "master"
> > object storage credentials for a few seconds, just long enough so I
> > can gain access to it and have access to all the data.
> >
> > No doubt, there are S3 implementations (software and appliances) that
> > do not support STS, which is admittedly not great. I can imagine that
> > at least some appliance vendors and software projects/products will
> > get STS.
> >
> > For the non-STS use cases, I think S3 signing is the way to go. Sure,
> > it requires one more request, but we can make those requests fast (aka
> > not require any persistence access) as we did in Nessie. With that we
> > could still ensure that clients don't have access to everything,
> > respecting the object-storage level read/write/list privileges.
> >
> > Another option is still to configure the object storage credentials at
> > the clients. It's not great, but it's still an option. Admins can give
> > each client individual credentials to reduce potential risks, being
> > able to revoke access for individual clients, and/or audit those.
> >
> > On Thu, Jul 31, 2025 at 2:51 AM Yufei Gu <fl...@gmail.com> wrote:
> > >
> > > Thanks for raising this, Dmitri!
> > >
> > > For non-STS use cases, some users may be more comfortable without
> > > credential vending. They could configure the storage credentials at the
> > > engines side. Can we first confirm that vending raw credentials are
> really
> > > users asking for?
> > >
> > > If that's the case, raw credential vending should be at least optional,
> > > which could be guarded by feature flags.
> > >
> > > And I didn't see much difference between option 1 and option 2. Both
> > > provide raw credentials and need rotation. Either way is fine with me.
> > >
> > > Yufei
> > >
> > >
> > > On Wed, Jul 30, 2025 at 3:24 PM Dmitri Bourlatchkov <di...@apache.org>
> > > wrote:
> > >
> > > > Hi All,
> > > >
> > > > Recent conversations [1] [2] about non-AWS S3 storage brought up user
> needs
> > > > for operating with S3-compatible storage that does not have STS.
> > > >
> > > > Remote request signing can be used to support those use cases, but it
> is a
> > > > considerable development effort to add to Polaris, plus it has
> different
> > > > performance characteristics than vended credentials.
> > > >
> > > > I propose two short-term options to support users of non-STS S3
> storage.
> > > >
> > > > 1) Add a configuration option to vend the same credentials that
> Polaris has
> > > > to clients.
> > > >
> > > > While this may (rightly) be considered suboptimal from the security
> > > > perspective, this option does give users a choice to operate clients
> > > > without explicitly configuring storage credentials for them. Polaris
> > > > Servers still control the rotation of those credentials.
> > > >
> > > > 2) Add secondary plain credentials for vending to clients. Polaris
> itself
> > > > will use one key/secret pair. Clients will be issued another
> key/secret
> > > > pair. Rotation of the client credentials should be possible to
> implement
> > > > too.
> > > >
> > > > WDYT?
> > > >
> > > > [1]
> https://github.com/apache/polaris/issues/1530#issuecomment-3137374380
> > > > [2] https://github.com/apache/polaris/issues/2207
> > > >
> > > > Thanks,
> > > > Dmitri.
> > > >
> >
>
> --
> This email, including its contents and any attachment(s), may contain
> confidential and/or proprietary information and is solely for the review
> and use of the intended recipient(s). If you have received this email in
> error, please notify the sender and permanently delete this email, its
> content, and any attachment(s).  Any disclosure, copying, or taking of any
> action in reliance on an email received in error is strictly prohibited.
>

Reply via email to