Duong created HDDS-9446:
---------------------------
Summary: S3 default GRPC transport doesn't utilize enough
parallelism on OM server-side
Key: HDDS-9446
URL: https://issues.apache.org/jira/browse/HDDS-9446
Project: Apache Ozone
Issue Type: Improvement
Reporter: Duong
GRPC is enabled by default as the transport between S3->OM, as per HDDS-7309.
We did a S3 GET stress test using warp in a cluster and realized that with GRPC
transport, S3 can't invoke enough parallelism in OM. We tested with more than
300 warp client. threads spread on 15 S3 instances, but OM doesn't use up to
more than 15 threads concurrently (we use the metric added by HDDS-9424 to
access).
{code:java}
warp get --warp-client=1.1.1.{34...49} --host=localhost:9879 --tls --insecure
--access-key=hdfs/[email protected]
--secret-key=ada0758fe2d12a55dbea8ff6b7df52c377bf46a8e2886d6d17fae5b77a7c4380
--obj.size=1kb --concurrent=20 --bucket bucket-obs {code}
And this has caused a degradation (30%) in S3->OM performance.
It's also verified that switching back to HadoopRPC3 did utilize 100 threads in
OM RPC server and brought back the performance before.
I think this is purely a problem with GRPC default configs.
We need to either tune GRPC default configs, or make HadoopRPC the default
transport.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
