xBis7 opened a new pull request, #4116: URL: https://github.com/apache/ozone/pull/4116
## What changes were proposed in this pull request? Currently, when using the `FairCallQueue`, there is no S3G impersonation, the recorded user for every call is the Ozone super - user which makes the `FairCallQueue` ineffective since it's only reading one user. To fix this, a custom `IdentityProvider` implementation was created for Ozone. In this new class, we are checking if there is an S3 Authentication on the server side and use the user from it to `makeIdentity` else we are using Ozone super-user. Every time a user wants to use the `FairCallQueue`, then that user should also include this in the config file. ``` ipc.9862.identity-provider.impl=org.apache.hadoop.ozone.om.OzoneIdentityProvider ``` Maybe, we should document that somewhere. It would be nice having it as a default option without requiring the user to specify it but it's a bit hard since the code using it, is located in `hadoop-commons`. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-7319 ## How was this patch tested? New unit tests where added for `OzoneIdentityProvider`. Also this patch can be tested manually in a docker environment like so ``` $ cd /hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/compose/ozone ``` edit the `docker-config` file and add the following ``` CORE-SITE.XML_ipc.9862.callqueue.impl=org.apache.hadoop.ipc.FairCallQueue CORE-SITE.XML_ipc.9862.scheduler.impl=org.apache.hadoop.ipc.DecayRpcScheduler CORE-SITE.XML_ipc.9862.identity-provider.impl=org.apache.hadoop.ozone.om.OzoneIdentityProvider CORE-SITE.XML_ipc.9862.scheduler.priority.levels=2 CORE-SITE.XML_ipc.9862.backoff.enable=true CORE-SITE.XML_ipc.9862.faircallqueue.multiplexer.weights=99,1 CORE-SITE.XML_ipc.9862.decay-scheduler.thresholds=90 OZONE-SITE.XML_ozone.om.address=0.0.0.0:9862 ``` then specify an S3G user and create an object ``` $ docker-compose up --scale datanode=3 -d $ docker exec -it ozone_s3g_1 bash bash-4.2$ aws configure AWS Access Key ID [None]: test AWS Secret Access Key [None]: pass Default region name [None]: Default output format [None]: bash-4.2$ aws s3api --endpoint http://localhost:9878 create-bucket --bucket bucket1 { "Location": "http://localhost:9878/bucket1"; } bash-4.2$ aws s3api --endpoint http://localhost:9878 put-object --bucket bucket1 --key dir1/key1 --body README.md ``` If we check the `/jmx` endpoint for OM we can see `Caller(test)` while in master it's always `Caller(hadoop)`. ``` { "name" : "Hadoop:service=OzoneManager,name=DecayRpcSchedulerMetrics2.ipc.9862", "modelerType" : "DecayRpcSchedulerMetrics2.ipc.9862", "tag.Context" : "ipc.9862", "tag.Hostname" : "d8a08ae7ea53", "DecayedCallVolume" : 4, "UniqueCallers" : 1, "Caller(test).Volume" : 4, "Priority.0.AvgResponseTime" : 0.5546875, "Priority.1.AvgResponseTime" : 0.040364583333333336, "Priority.0.CompletedCallVolume" : 0, "Priority.1.CompletedCallVolume" : 0, "CallVolume" : 4 } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
