Benoit Tellier created JAMES-4222:
-------------------------------------
Summary: Improve Ceph + Rados Gateway support
Key: JAMES-4222
URL: https://issues.apache.org/jira/browse/JAMES-4222
Project: James Server
Issue Type: Improvement
Components: s3
Reporter: Benoit Tellier
h3. Why ?
Rados uses omap (KVRock backed metadata) to support listing.
It splits data into buckets that each owns its own metadata db. Limit of
objects per shard is BTW limited so holding large count of objects (like james
needs) involves a large count of shards.
S3 ordered listing is done with 1000 item page and eg 2000 shard bucket results
in a scatter-gather of over 2 million items.
Larger deployment with more mail=more objects=more shards...
James relies on listing for the GC. Thus we need to enhance this.
h3. How?
Rados allows to add a query parameter for unordered listing. This skips the
scatter gaver alltogether.
?allow-unordered=true
The idea would be to rely on S3 driver ExecutionInterceptor, and add it if and
only if "-D+james.s3.rados.allow.unorder=true"
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
