ChenSammi commented on PR #10742: URL: https://github.com/apache/ozone/pull/10742#issuecomment-5250569504
> Compute engines (Spark, Hive, Trino, YARN, Impala) do not require any changes. It's good that compute engines doesn't require changes. So there is no further integration efforts with these compute engines. > The design is intended to have Custos to be co-located with OM on the same node. This means internal latency is extremely less. The major S3 latency is the network round-trip to the gateway, not the processing inside. So overall having Custos - while it introduces an extra hop - doesn't produce a noticeable impact. Do you have any data to support this statement that "it doesn't produce a noticeable impact"? Actually I think it will have a noticeable impact on latency for metadata operation. There is an existing case, sort datanode distance based on topology. Datanode topology is the information that only aware by SCM, not OM. So initially sort datanode distance function is provided by SCM and called by OM. Later when Ritesh and Tanvi work on OM performance improvement, they moved the datanode topology distance sort from SCM to OM, to increase the read IOPS of OM. If we move the HMAC verification to new service Custos, it will have latency impact on both read and write operation of OM. Co-located with OM is not a cure, OM and SCM are also co-located in most of cases. > When Ozone is compared to competitors like MinIO, Ceph, VAST etc. they all support OIDC today, though they are oriented towards STS (given the S3 compatibility) the idea is the same: one authentication exchange, one short-lived token, the storage client is unaware of OIDC. I understand the requirement to support OIDC in Ozone, and I agree we should do that. But as I mentioned I did some investigation, S3/HTTP is the dominate way of how storage is used in the AI environment, not the java client, or go/rust client. It leverages Python/S3 SDK, to go through s3g, to access Ozone, which means we can naturally support OIDC in STS. This adding an individual service to do authentication, is quite a big architecture change. The service framework might be stateless, but the authentication provider cannot be stateless too. So all the states management, new service development, configuration and management, HA, new ports requires, all these complexity will be introduced with this new service. I'm not saying that we cannot introduce new service. But we need to evaluate the pros and cons. For now, the two major authentication paths, Kerberos and S3 (OIDC can be naturally supported by STS in future) can be welly supported in current Ozone architecture. Introducing a new service seems overskill to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
