This is an automated email from the ASF dual-hosted git repository. btellier pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 9f73a00386d6e1f0591a190f79103244b2285e35 Author: Matthieu Baechler <[email protected]> AuthorDate: Tue Feb 11 11:45:29 2020 +0100 JAMES-3028 ADR about using S3-over-Swift --- src/adr/0041-replace-jcloud-with-s3.md | 43 ++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/src/adr/0041-replace-jcloud-with-s3.md b/src/adr/0041-replace-jcloud-with-s3.md new file mode 100644 index 0000000..7c38220 --- /dev/null +++ b/src/adr/0041-replace-jcloud-with-s3.md @@ -0,0 +1,43 @@ +# 41. Replace JCloud-based ObjectStorage backend with AWS-S3 SDK backend + +Date: 2020-02-11 + +## Status + +Proposed + +Adoption needs to be backed by some performance tests. + +## Context + +James has an interface called blob-store that allows to choose how Blobs are stored between several implementations. + +For Distributed product, the default backend is Cassandra because it's fast and it doesn't require yet another +server. + +When storage cost concerns are more important than raw performances, James can store Blobs in S3 or Swift +Object Storage technologies. + +Object Storage is known to be cheap and yet offers guarantee about durability. + +When we implemented Object Storage blob-store in the past, we decided to implement it with +[Apache jcloud library](https://jclouds.apache.org/) because we wanted to target both S3 and Swift API. + +However, we found that our implementation was complex and it didn't fit the Reactive style we now have in the codebase. +It also contribute negatively to the project build time. + +At the same time, we figured out Swift was providing a good compatibility layer for S3 and that we may drop +our Swift code without dropping Swift support at the same time. + +## Decision + +* Use AWS S3 v2 SDK to implement a Reactive S3 blob-store (`blobstore-s3`). +* Replace current `blob-store-objectstorage` with `blobstore-s3` +* Run load tests to ensure there's no penalty when switching from Swift to S3 over Swift + +## Consequences + +* We have to document carefully how to transition from `blob-store-objectstorage` with `blobstore-s3` in the +configuration and how to handle existing data + +* We need to work on a new implementation of blob encryption. \ No newline at end of file --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
