Thanks cbodley for the clarification. I'll definitely look more into rclone Gilles. I was actually putting together a POC with that too, in case my understanding of the cloud sync module was wrong. Thanks for the heads-up on the other stuff too :)
Best Regards, Jimmy On April 11, 2024 5:29:30 PM EDT, Casey Bodley <[email protected]> wrote: >unfortunately, this cloud sync module only exports data from ceph to a >remote s3 endpoint, not the other way around: > >"This module syncs zone data to a remote cloud service. The sync is >unidirectional; data is not synced back from the remote zone." > >i believe that rclone supports copying from one s3 endpoint to >another. does anyone have experience with that? > >On Thu, Apr 11, 2024 at 4:45 PM James McClune <[email protected]> wrote: >> >> Hello Ceph User Community, >> >> I currently have a large Amazon S3 environment with terabytes of data >> spread over dozens of buckets. I'm looking to migrate from Amazon S3 to an >> on-site Ceph cluster using the RGW. I'm trying to figure out the >> most efficient way to achieve this. Looking through the documentation, I >> found articles related to the cloud sync module, released in Mimic ( >> https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/). I also watched >> a video on the cloud sync module as well. It *sounds* like this is the >> functionality I'm looking for. >> >> Given I'm moving away from Amazon S3, I'm really just looking for a one-way >> replication between the buckets (i.e. Provide an Amazon S3 access >> key/secret which is associated to the buckets and the same for the Ceph >> environment, so object data can be replicated one-to-one, without creating >> ad-hoc tooling). Once the data is replicated from S3 to Ceph, I plan on >> modifying my boto connection objects to use the new Ceph environment. Is >> what I'm describing feasible with the cloud sync module? Just looking for >> some affirmation, given I'm not well versed in Ceph's RGW, especially >> around multi-site configurations. >> >> Thanks, >> Jimmy >> _______________________________________________ >> ceph-users mailing list -- [email protected] >> To unsubscribe send an email to [email protected] >> > _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
