Hi Alberto, I didn't look into boto implementation, but I suspect that COPY Object API does NOT work between different S3-like systems. The actual interface definition of the API is [1] and source bucket/key is just a string in the x-amz-copy-source header. The request went into the system that includes rk02.ejemplo.com in your example, but it did not know anything about source bucket/key because it does not have the bucket/key. Object contents should be transferred in some way, e.g. GET Object from source and PUT Object (or Multipart Upload for large objects) to target system.
[1] http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html Thanks, Shino 2015-11-16 21:28 GMT+09:00 Alberto Ayllon <aayl...@qdqmedia.com>: > Hello. > > Thanks for your help Dmitri. > > Perhaps this is not the correct place to ask this, but maybe someone had > have the same problem. > > I build a test environment with two nodes of RIAK-CS, not in cluster, and > I'm trying to move objects from one to another, to a bucket with the same > name (testbucket). > I'm using BOTO, and the copy_key method, but it fails, I guess that problem > is in the HEAD call, this call checks if the key exists in the destination > bucket before make the PUT, that copy the key. > > Here is the code I'm using. > > from boto.s3.key import Key > from boto.s3.connection import S3Connection > from boto.s3.connection import OrdinaryCallingFormat > > apikey04='GFR3O0HFPXQ-BWSXEMAG' > secretkey04='eIiigR4Rov2O2kxuSHNW7WPoJE2KmrtMpzzqlg==' > > apikey02='J0TT_C9MJPWPGHW-KEWY' > secretkey02='xcLOt3ANqyNJ0kAjP8Mxx68qr7kgyXG3eqJuMA==' > cf=OrdinaryCallingFormat() > > conn04=S3Connection(aws_access_key_id=apikey04,aws_secret_access_key=secretkey04, > > is_secure=False,host='rk04.ejemplo.com',port=8080,calling_format=cf) > > conn02=S3Connection(aws_access_key_id=apikey02,aws_secret_access_key=secretkey02, > > is_secure=False,host='rk02.ejemplo.com',port=8080,calling_format=cf) > > bucket04=conn04.get_bucket('testbucket') > bucket02=conn02.get_bucket('testbucket') > > rs04 = bucket04.list() > > for k in rs04: > print k.name > bucket02.copy_key(k.key, bucket04, k.key) > > > When this script is executed it returns: > > Traceback (most recent call last): > File "s3_connect_2.py", line 38, in <module> > bucket02.copy_key(k.key, bucket04, k.key) > File > "/home/alberto/.virtualenvs/boto/local/lib/python2.7/site-packages/boto/s3/bucket.py", > line 888, in copy_key > response.reason, body) > boto.exception.S3ResponseError: S3ResponseError: 404 Not Found > <?xml version="1.0" > encoding="UTF-8"?><Error><Code>NoSuchKey</Code><Message>The specified key > does not exist.</Message><Resource>/<Bucket: testbucket>$ > > > > The idea is copy all keys in testbucket from rk04.ejemplo.com, to testbucket > in test02.ejemplo.com, maybe someone can help me. > > > Thanks a lot. > > > 2015-11-13 17:06 GMT+01:00 Dmitri Zagidulin <dzagidu...@basho.com>: >> >> Hi Alberto, >> >> From what I understand, the state of the art in terms of migration of >> objects from Amazon S3 to Riak CS is -- writing migration scripts. >> Either as shell scripts (using s3cmd), or language-specific libraries like >> boto (or even just the S3 SDKs). >> And the scripts would consist of: >> 1) get a list of the buckets you want to migrate >> 2) List the keys in those buckets >> 3) Migrate each object from AWS to CS. >> >> You're right that mounting buckets as filesystems is a (distant) >> possibility, but we have not seen much successful use of those (though if >> anybody's made that work, let us know). >> >> >> >> On Thu, Nov 12, 2015 at 12:40 PM, Alberto Ayllon <aayl...@qdqmedia.com> >> wrote: >>> >>> Hello. >>> >>> I'm new using Riak and Riak-cs, I have installed a Riak-cs cluster with 4 >>> nodes and it works fine, >>> >>> Here is my question, the company where I work has some buckets in Amazon >>> s3, and I would like migrate objects from these buckets to our Riak-cs >>> installation, as far as I know I can do it using S3FUSE or S3BACKER, >>> mounting buckets as a filesystem, but would like avoid mount it as >>> filesystem. I tried it with boto python library, using the copy_key method, >>> but it doesn't work. >>> >>> Has anybody try with success synchronize buckets from AS3 to Riak-CS? >>> >>> Thanks. >>> >>> P:D: Excuse for my English. >>> >>> _______________________________________________ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >> > > > _______________________________________________ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
