spacemonkd commented on PR #10742: URL: https://github.com/apache/ozone/pull/10742#issuecomment-5192974627
Thanks @ChenSammi for the detailed inputs. > For KerberosProvider, except the Ozone clients maintained by us, do we need also change the compute engine code to integrate this new authentication flow? such as spark, hive, impala, trino, also yarn? Compute engines (Spark, Hive, Trino, YARN, Impala) do not require any changes. Internally, the Ozone client routes the Kerberos credential through KerberosProvider and receives a `CustosToken`. The engine sees the same standard `Token<>` in Credentials it sees today. > For S3SecretProvider, move the AWS S3 SigV4 signature HMAC verification from OzoneManager to Custos, will reduce the workload of OzoneManager, that's the benefit. But in the meanwhile, there is one new hop introduced in the S3 path. The design is intended to have Custos to be co-located with OM on the same node. This means internal latency is extremely less. The major S3 latency is the network round-trip to the gateway, not the processing inside. So overall having Custos - while it introduces an extra hop - doesn't produce a noticeable impact. Also: - Today OM maintains separate code paths for SigV4, SPNEGO, delegation tokens, and STS. With Custos, all of these collapse into one check i.e. verify the HMAC on a `CustosToken`. - - Multiple Custos nodes allows authentication (HMAC check, credential lookup, identity resolution) to be a horizontally-scalable service. Under high load, for example, thousands of authentication requests can distributed across Custos nodes rather than competing with namespace operations on the single OM leader. > if the major gap which motivated this proposal is to support OIDC JWT, to enable Ozone in AI workload ecosystem. It is not just oriented towards AI workload enablement, apart from this some other benefits are: - Authentication code today is split across the client and OM. Custos makes it a single location; OM then only verifies a small, local HMAC. - Kerberos SASL/GSSAPI is incompatible with HTTP/2 (2015). gRPC clients in Rust, Python, and Go have no viable Kerberos path; Custos gives them one without per-language re-implementation. - The codebase is smaller now. Adding this abstraction today prevents authentication logic from growing further into OM request handlers and the S3 gateway - this is similar to what we face today when we propose pre-ratis execution. As the codebase grows taking decisions which affect the core logic affects a larger surface area. When Ozone is compared to competitors like MinIO, Ceph, VAST etc. they all support OIDC today, though they are oriented towards STS (given the S3 compatibility) the idea is the same: one authentication exchange, one short-lived token, the storage client is unaware of OIDC. With Ozone we have a unique positioning where the filesystem is in parallel to the S3 flow and so we have to tackle FS along side S3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
