[
https://issues.apache.org/jira/browse/HDDS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-8058:
-----------------------------
Description:
Reproduced in ozonesecure dev docker compose environment on latest master
branch as of today.
{code:title=ArrayIndexOutOfBoundsException}
2023-03-01 12:06:16 2023-03-01 20:06:16,048 [IPC Server handler 39 on default
port 9862] WARN ipc.Server: IPC Server handler 39 on default port 9862, call
Call#5 Retry#15
org.apache.hadoop.ozone.om.protocol.OzoneManagerProtocol.submitRequest from
172.26.0.5:39513
2023-03-01 12:06:16 java.lang.ArrayIndexOutOfBoundsException: Index 2 out of
bounds for length 1
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.AWSV4AuthValidator.getSigningKey(AWSV4AuthValidator.java:77)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.AWSV4AuthValidator.validateRequest(AWSV4AuthValidator.java:103)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.validateS3AuthInfo(OzoneDelegationTokenSecretManager.java:519)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.retrievePassword(OzoneDelegationTokenSecretManager.java:419)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.S3SecurityUtil.validateS3Credential(S3SecurityUtil.java:61)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:166)
2023-03-01 12:06:16 at
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:465)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:578)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
2023-03-01 12:06:16 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
2023-03-01 12:06:16 at
java.base/java.security.AccessController.doPrivileged(Native Method)
2023-03-01 12:06:16 at
java.base/javax.security.auth.Subject.doAs(Subject.java:423)
2023-03-01 12:06:16 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
{code}
Suspecting the tolerance of the validator isn't high enough:
https://github.com/apache/ozone/blob/a6316c832068ecc044de87f4c2671a9ca3b86043/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/security/AWSV4AuthValidator.java#L77
It is not gracefully handling the case where the request is not fully
conforming to the standard, where {{strToSign}} in this case is empty.
Script used to repro this:
{code:title=s3upload.sh}
#!/usr/bin/env bash
# Usage: ./s3upload.sh bucket1 1.txt
# Ref: https://gist.github.com/PhilipSchmid/1fd2688ace9f51ecaca2788a91fec133
set -o errexit
set -o nounset
set -o pipefail
if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi
bucket=$1
file=$2
host=s3g
# ozone s3 getsecret -e
s3_key='om/[email protected]'
s3_secret='b169aa2ed3e0c6017e0e679143f385861f02592c444a5bafb1d216e9065bcb6a'
resource="/${bucket}/${file}"
content_type="application/octet-stream"
date=`date -R`
_signature="PUT\n\n${content_type}\n${date}\n${resource}"
signature=`echo -en ${_signature} | openssl sha1 -hmac ${s3_secret} -binary |
base64`
echo "Starting upload at $(date -R)"
ls -lh "${file}"
# --fail-with-body
curl -X PUT -T "${file}" \
-H "Host: ${host}" \
-H "Date: ${date}" \
-H "Content-Type: ${content_type}" \
-H "Authorization: AWS ${s3_key}:${signature}" \
-w "%{http_code}" \
http://${host}:9878${resource}
echo "Completed upload at $(date -R)"
{code}
Note:
1. need to install openssl in the docker container to use this: {{sudo yum
install openssl}}
cc [~dteng]
was:
Reproduced in ozonesecure dev docker compose environment on latest master
branch as of today.
{code:title=ArrayIndexOutOfBoundsException}
2023-03-01 12:06:16 2023-03-01 20:06:16,048 [IPC Server handler 39 on default
port 9862] WARN ipc.Server: IPC Server handler 39 on default port 9862, call
Call#5 Retry#15
org.apache.hadoop.ozone.om.protocol.OzoneManagerProtocol.submitRequest from
172.26.0.5:39513
2023-03-01 12:06:16 java.lang.ArrayIndexOutOfBoundsException: Index 2 out of
bounds for length 1
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.AWSV4AuthValidator.getSigningKey(AWSV4AuthValidator.java:77)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.AWSV4AuthValidator.validateRequest(AWSV4AuthValidator.java:103)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.validateS3AuthInfo(OzoneDelegationTokenSecretManager.java:519)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.retrievePassword(OzoneDelegationTokenSecretManager.java:419)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.security.S3SecurityUtil.validateS3Credential(S3SecurityUtil.java:61)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:166)
2023-03-01 12:06:16 at
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
2023-03-01 12:06:16 at
org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:465)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:578)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
2023-03-01 12:06:16 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
2023-03-01 12:06:16 at
java.base/java.security.AccessController.doPrivileged(Native Method)
2023-03-01 12:06:16 at
java.base/javax.security.auth.Subject.doAs(Subject.java:423)
2023-03-01 12:06:16 at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
2023-03-01 12:06:16 at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
{code}
Suspecting the tolerance of the validator isn't high enough:
https://github.com/apache/ozone/blob/a6316c832068ecc044de87f4c2671a9ca3b86043/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/security/AWSV4AuthValidator.java#L77
Script used to repro this:
{code:title=s3upload.sh}
#!/usr/bin/env bash
# Usage: ./s3upload.sh bucket1 1.txt
# Ref: https://gist.github.com/PhilipSchmid/1fd2688ace9f51ecaca2788a91fec133
set -o errexit
set -o nounset
set -o pipefail
if [[ "${TRACE-0}" == "1" ]]; then
set -o xtrace
fi
bucket=$1
file=$2
host=s3g
# ozone s3 getsecret -e
s3_key='om/[email protected]'
s3_secret='b169aa2ed3e0c6017e0e679143f385861f02592c444a5bafb1d216e9065bcb6a'
resource="/${bucket}/${file}"
content_type="application/octet-stream"
date=`date -R`
_signature="PUT\n\n${content_type}\n${date}\n${resource}"
signature=`echo -en ${_signature} | openssl sha1 -hmac ${s3_secret} -binary |
base64`
echo "Starting upload at $(date -R)"
ls -lh "${file}"
# --fail-with-body
curl -X PUT -T "${file}" \
-H "Host: ${host}" \
-H "Date: ${date}" \
-H "Content-Type: ${content_type}" \
-H "Authorization: AWS ${s3_key}:${signature}" \
-w "%{http_code}" \
http://${host}:9878${resource}
echo "Completed upload at $(date -R)"
{code}
Note:
1. need to install openssl in the docker container to use this: {{sudo yum
install openssl}}
cc [~dteng]
> OM AWSV4AuthValidator throws ArrayIndexOutOfBoundsException when accessed
> through S3G with custom curl script
> -------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-8058
> URL: https://issues.apache.org/jira/browse/HDDS-8058
> Project: Apache Ozone
> Issue Type: Bug
> Components: OM
> Reporter: Siyao Meng
> Priority: Major
>
> Reproduced in ozonesecure dev docker compose environment on latest master
> branch as of today.
> {code:title=ArrayIndexOutOfBoundsException}
> 2023-03-01 12:06:16 2023-03-01 20:06:16,048 [IPC Server handler 39 on default
> port 9862] WARN ipc.Server: IPC Server handler 39 on default port 9862, call
> Call#5 Retry#15
> org.apache.hadoop.ozone.om.protocol.OzoneManagerProtocol.submitRequest from
> 172.26.0.5:39513
> 2023-03-01 12:06:16 java.lang.ArrayIndexOutOfBoundsException: Index 2 out of
> bounds for length 1
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.security.AWSV4AuthValidator.getSigningKey(AWSV4AuthValidator.java:77)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.security.AWSV4AuthValidator.validateRequest(AWSV4AuthValidator.java:103)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.validateS3AuthInfo(OzoneDelegationTokenSecretManager.java:519)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.security.OzoneDelegationTokenSecretManager.retrievePassword(OzoneDelegationTokenSecretManager.java:419)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.security.S3SecurityUtil.validateS3Credential(S3SecurityUtil.java:61)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:166)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server.processCall(ProtobufRpcEngine.java:465)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:578)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1043)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:971)
> 2023-03-01 12:06:16 at
> java.base/java.security.AccessController.doPrivileged(Native Method)
> 2023-03-01 12:06:16 at
> java.base/javax.security.auth.Subject.doAs(Subject.java:423)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> 2023-03-01 12:06:16 at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:2976)
> {code}
> Suspecting the tolerance of the validator isn't high enough:
> https://github.com/apache/ozone/blob/a6316c832068ecc044de87f4c2671a9ca3b86043/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/security/AWSV4AuthValidator.java#L77
> It is not gracefully handling the case where the request is not fully
> conforming to the standard, where {{strToSign}} in this case is empty.
> Script used to repro this:
> {code:title=s3upload.sh}
> #!/usr/bin/env bash
> # Usage: ./s3upload.sh bucket1 1.txt
> # Ref: https://gist.github.com/PhilipSchmid/1fd2688ace9f51ecaca2788a91fec133
> set -o errexit
> set -o nounset
> set -o pipefail
> if [[ "${TRACE-0}" == "1" ]]; then
> set -o xtrace
> fi
> bucket=$1
> file=$2
> host=s3g
> # ozone s3 getsecret -e
> s3_key='om/[email protected]'
> s3_secret='b169aa2ed3e0c6017e0e679143f385861f02592c444a5bafb1d216e9065bcb6a'
> resource="/${bucket}/${file}"
> content_type="application/octet-stream"
> date=`date -R`
> _signature="PUT\n\n${content_type}\n${date}\n${resource}"
> signature=`echo -en ${_signature} | openssl sha1 -hmac ${s3_secret} -binary |
> base64`
> echo "Starting upload at $(date -R)"
> ls -lh "${file}"
> # --fail-with-body
> curl -X PUT -T "${file}" \
> -H "Host: ${host}" \
> -H "Date: ${date}" \
> -H "Content-Type: ${content_type}" \
> -H "Authorization: AWS ${s3_key}:${signature}" \
> -w "%{http_code}" \
> http://${host}:9878${resource}
> echo "Completed upload at $(date -R)"
> {code}
> Note:
> 1. need to install openssl in the docker container to use this: {{sudo yum
> install openssl}}
> cc [~dteng]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
