neils-dev opened a new pull request #2655:
URL: https://github.com/apache/ozone/pull/2655
## What changes were proposed in this pull request?
This patch adds to the s3gateway gRPC feature three main items. CDI filter
extracting user aws authentication and creating user thread context, a cached
ozone client and changes to the OzoneManagerProtocol. The cached ozone client
processes s3 requests through a persistent connection between the s3 gateway
and the om. This single OzoneClient services all s3 requests. S3 http
requests are processed through a CDI interface that includes a CDI filter that
extracts s3 authentication information, `awsAccessId` & `signature`, generates
the `stringToSign` and adds these to an OzoneToken processed through the single
cached OzoneClient. The OzoneClient submits the request to the om through the`
OzoneManagerProtocol` (`OmRequest`) over gRPC OmTransport.
The CDI filter servicing http requests creates a user thread context with a
`UserGroupInformation` structure. Each client side s3 request is processed
through a `UGI doAs` context with user corresponding the request `awsAccessId`.
The cached OzoneClient provides a persistent connection between the client
(s3gateway) and server (Ozone Manager) servicing all s3 requests. This cached
client is created on the first request after the s3gateway is brought up and
runs under CDI ApplicationScope implying destroy only on s3 gateway tear down.
The OzoneManagerProtocol `OmRequest `is modified to include 3 optional
fields for authenticating s3 users on a per request basis - `stringToSign`,
`signature` and `awsAccessId`.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5612
## How was this patch tested?
This patch is tested through unit tests and manually on the docker ozone dev
cluster by submitting s3 bucket requests and validating responses.
1. unit tests - s3 gateway unit tests testing OzoneClient,
AwsSignatureProcessor, StringToSign (s3 authentication), and s3g endpoints.
```
$ cd hadoop-ozone/s3gateway
$ mvn test
```
2, manual test s3 request - ozone cluster processing s3 create bucket
request
`$ cd hadoop-ozone/dist/target/ozone-1.2.0-SNAPSHOT/compose/ozone`
**ENABLE** gRPC Ozone Manager through config.
**_ozone.om.s3.grpc.server_enabled: "true"_**
ADD to docker-compose.yaml or docker-config:
$ vi docker-compose.yaml
`OZONE-SITE.XML_ozone.om.s3.grpc.server_enabled: "true"`
`$ docker-compose up -d --scale datanode=3`
`$ aws s3api --endpoint http://localhost:9878 create-bucket --bucket=bucket1`
_{
"Location": "http://localhost:9878/bucket1";
}_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
