Take a look at stream2es https://github.com/elasticsearch/stream2es
On Friday, June 6, 2014 2:13:06 PM UTC-3, Brian Lamb wrote: > > I should also point out that I had to edit a file in the metadata-snapshot > file to change around the s3 keys and bucket name to match what development > was expecting. > > On Friday, June 6, 2014 1:11:57 PM UTC-4, Brian Lamb wrote: >> >> Hi all, >> >> I want to do a one time copy of the data on my production elastic search >> instance to my development elastic search instance. Both are managed by AWS >> if that makes this easier. Here is what I tried: >> >> On production: >> >> curl -XPUT 'http://localhost:9200/_snapshot/my_s3_repository' -d '{ >> "type": "s3", >> "settings": { >> "access_key": "productionAccessKey", >> "bucket": "productionBucketName", >> "region": "region", >> "secret_key": "productionSecretKey" >> } >> }' >> curl -XPUT " >> http://localhost:9200/_snapshot/my_s3_repository/snapshot_2014_06_02"; >> >> What this does is upload the instance to a production level s3 bucket. >> >> Then in the aws console, I copy all of it to a development level s3 >> bucket. >> >> Next on development: >> >> curl -XPUT 'http://localhost:9200/_snapshot/my_s3_repository' -d '{ >> "type": "s3", >> "settings": { >> "access_key": "developmentAccessKey", >> "bucket": "developmentBucketName", >> "region": "region", >> "secret_key": "developmentSecretKey" >> } >> }' >> curl -XPOST " >> http://localhost:9200/_snapshot/my_s3_repository/snapshot_2014_06_02/_restore >> " >> >> This gives me the following message: >> >> $ curl -XPOST " >> http://localhost:9200/_snapshot/my_s3_repository/snapshot_2014_06_02/_restore?pretty=true >> " >> { >> "error" : "SnapshotException[[my_s3_repository:snapshot_2014_06_02] >> failed to get snapshots]; nested: IOException[Failed to get >> [snapshot-snapshot_2014_06_02]]; nested: AmazonS3Exception[Status Code: >> 404, AWS Service: Amazon S3, AWS Request ID: RequestId, AWS Error Code: >> NoSuchKey, AWS Error Message: The specified key does not exist.]; ", >> "status" : 500 >> } >> >> Also, when I try to get the snapshots, I get the following: >> >> $ curl -XGET "localhost:9200/_snapshot/_status?pretty=true" >> { >> "snapshots" : [ ] >> } >> >> This leads me to believe that I am not connecting the snapshot correctly >> but I'm not sure what I am doing incorrectly. Regenerating the index on >> development is not really a possibility as it took a few months to generate >> the index the first time around. If there is a better way to do this, I'm >> all for it. >> >> Thanks, >> >> Brian Lamb >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/52168c96-30ea-4527-b287-676e757b1e6a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
