Siyao Meng created HDDS-8058:
--------------------------------
Summary: OM AWSV4AuthValidator throws
ArrayIndexOutOfBoundsException when accessed through S3G with custom curl script
Key: HDDS-8058
URL: https://issues.apache.org/jira/browse/HDDS-8058
Project: Apache Ozone
Issue Type: Bug
Components: OM
Reporter: Siyao Meng
Reproduced in ozonesecure dev docker compose environment on latest master
branch as of today.
{code:title=ArrayIndexOutOfBoundsException}
2023-03-01 12:06:16 2023-03-01 20:06:16,048 [IPC Server handler 39 on default
port 9862] WARN ipc.Server: IPC Server handler 39 on default port 9862, call
Call#5 Retry#15
org.apache.hadoop.ozone.om.protocol.OzoneManagerProtocol.submitRequest from
172.26.0.5:39513
2023-03-01 12:06:16 java.lang.ArrayIndexOutOfBoundsException: Index 2 out of
bounds for length 1
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.AWSV4AuthValidator.getSigningKey(AWSV4AuthValidator.java:77)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.AWSV4AuthValidator.validateRequest(AWSV4AuthValidator.java:103)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.validateS3AuthInfo(OzoneDelegationTokenSecretManager.java:519)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.retrievePassword(OzoneDelegationTokenSecretManager.java:419)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.S3SecurityUtil.validateS3Credential(S3SecurityUtil.java:61)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:166)
2023-03-01 12:06:16 at
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:465)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:578)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
2023-03-01 12:06:16 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
2023-03-01 12:06:16 at
java.base/java.security.AccessController.doPrivileged(Native Method)
2023-03-01 12:06:16 at
java.base/javax.security.auth.Subject.doAs(Subject.java:423)
2023-03-01 12:06:16 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
{code}
Suspecting the tolerance of the validator isn't high enough:
https://github.com/apache/ozone/blob/a6316c832068ecc044de87f4c2671a9ca3b86043/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/security/AWSV4AuthValidator.java#L77
Script used to repro this:
{code:title=s3upload.sh}
#!/usr/bin/env bash
# Usage: ./s3upload.sh bucket1 1.txt
# Ref: https://gist.github.com/PhilipSchmid/1fd2688ace9f51ecaca2788a91fec133
set -o errexit
set -o nounset
set -o pipefail
if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi
bucket=$1
file=$2
host=s3g
# ozone s3 getsecret -e
s3_key='om/[email protected]'
s3_secret='b169aa2ed3e0c6017e0e679143f385861f02592c444a5bafb1d216e9065bcb6a'
resource="/${bucket}/${file}"
content_type="application/octet-stream"
date=`date -R`
_signature="PUT\n\n${content_type}\n${date}\n${resource}"
signature=`echo -en ${_signature} | openssl sha1 -hmac ${s3_secret} -binary |
base64`
echo "Starting upload at $(date -R)"
ls -lh "${file}"
# --fail-with-body
curl -X PUT -T "${file}" \
-H "Host: ${host}" \
-H "Date: ${date}" \
-H "Content-Type: ${content_type}" \
-H "Authorization: AWS ${s3_key}:${signature}" \
-w "%{http_code}" \
http://${host}:9878${resource}
echo "Completed upload at $(date -R)"
{code}
Note:
1. need to install openssl in the docker container to use this: {{sudo yum
install openssl}}
cc [~dteng]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
