ChenSammi commented on PR #10742:
URL: https://github.com/apache/ozone/pull/10742#issuecomment-5191637358
Thanks @spacemonkd raising this proposal. I see there are three auth
providers proposed in the design,
- KerberosProvider
Mainly this part is moving the Kerberos authentication and delegation
creation function from OzoneManager into this new Custos service. The left part
in OzoneManager is delegation token verification.
- S3SecretProvider
This part plans to move the AWS S3 SigV4 signature HMAC verification
from OzoneManager to Custos.
- OidcProvider
This part has no corresponding support in Ozone.
For KerberosProvider, the biggest challenge I can see is how it can be
integrated within Hadoop ecosystem, since it reinvent a new authentication
flow. In the previous company I worked for, we developed a new private
authentication protocol for Hadoop ecosystem used globally in the company, to
replace Kerberos, to overcome the performance bottleneck and maintenance effort
of Kerberos. The protocol itself implementation is quite quick and
straightforward, after that, we spent close to a year time to integrate the new
authentication flow with all Hadoop ecosystem services, spark, hive, yarn,
hbase, presto, etc. So there is a heavy cost of changing the authentication
way of Hadoop ecosystem, that's why Ozone chooses using Kerberos in the first
place, to avoid extra integration effort. Now if we propose this change in the
upstream, which means we need to have new commits in each popular compute
engine to integrate this new flow. Since it's not an industry standard protocol
or fl
ow, we must have very strong reasons to persuade compute engine code
maintainers to accept the change.
For S3SecretProvider, move the AWS S3 SigV4 signature HMAC verification
from OzoneManager to Custos, will reduce the workload of OzoneManager, that's
the benefit. But in the meanwhile, there is one new hop introduced in the S3
path. Previously, it is s3-client -> s3 gateway -> OM, now it is s3-client ->
s3 gateway -> Custos -> OM. This one new hop will increase the latency,
especially when s3-client tries to get small data or just head operations.
For OidcProvider, there is no detail, so not clear about what's the
expected function.
I also spent some time to explore how the most modern AI training and
inference accessing data. Gemini said for most AI training and inference today,
S3-compatible object storage APIs (HTTP/REST under the hood) are the default,
not the Java/Go native storage clients talking directly to a proprietary
filesystem protocol. Our UCX POC by @sumitagrawl also shows the S3 API is used
in cuObject library. If this is true, then for Ozone, the typical path will be
"Python/S3 SDK -> S3 Gateway -> OM", since we cannot update S3 SDK to make it
support the new authentication flow/protocol, it looks to me that support OIDC
identity(AssumeRoleWithWebIdentity) in OM is the right direction to go, if the
major gap which motivated this proposal is to support OIDC JWT, to enable
Ozone in AI workload ecosystem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
