For reference, with parallel writes using the S3 Go API (via hsbench:
https://github.com/markhpc/hsbench), I was recently doing about 600ish
MB/s to a single RGW instance from one client. RadosGW used around 3ish
HW threads from a 2016 era Xeon to do that. Didn't try single-file
tests in
With this options I still see around 38-40MB/s for my 16gb test file. So far my
testing is mostly synthetic, I’m going to be using some programs like GitLab
and Sonatype Nexus that store their data in object storage. At work I deal with
real S3 and regular see upload speeds in the 100s of MB/s
With s5cmd and its defaults I got around 127MB/s for a single 16gb test file.
Is there any way to make s5cmd give feedback while it’s running. At first I
didn’t think it was working because it just sat there for a while.
Thanks
Shawn
On Feb 10, 2023, at 8:45 AM, Matt Benjamin wrote:
Hi
> The problem I’m seeing is after setting up RadosGW I can only upload to “S3”
> at around 25MBs with the official AWS CLI. Using s3cmd is slightly better at
> around 45MB/s. I’m going directly to the RadosGW instance with no load
> balancers in between and no ssl enabled. Just trying to figure
Hi Shawn,
To get another S3 upload baseline, I'd recommend doing some upload testing
with s5cmd [1].
1. https://github.com/peak/s5cmd
Matt
On Fri, Feb 10, 2023 at 9:38 AM Shawn Weeks
wrote:
> Good morning everyone, been running a small Ceph cluster with Proxmox for
> a while now and I’ve
