RE: Ceph Snapshot Backups
{\rtf1\ansi\fbidis\ansicpg1252\deff0\deftab720\fromtext{\fonttbl{\f0\fswiss\fcharset0 Times New Roman;}{\f1\fswiss\fcharset2 symbol;}} {\colortbl;\red192\green192\blue192;} {\*\generator Microsoft Exchange Server;} {\*\formatConverter converted from text;} \viewkind5\viewscale100 {\*\bkmkstart BM_BEGIN}\pard\plain\f0{\fs20 Hello community,\line \line is there anyone out there using CloudStack in combination with Ceph?\line What are your experiences or use cases with snapshot's?\line \line Your feedback is appreciate.\line \line Regards,\line Ingo\line \line -Original Message-\line From: Stock, Alexander \line Sent: Dienstag, 2. April 2019 11:08\line To: users@cloudstack.apache.org\line Subject: Ceph Snapshot Backups\line \line Hi Everyone,\line \line \line some time ago we evaluated the snapshot features from Cloudstack for the combination Ceph + KVM + Cloudstack and found some space for improvements.\line Our goal is to provide snapshot backup for our customers which is easy to handle over the Cloudstack API and provides incremental/differential features.\line \line We recognised that Cloudstack uses qemu-convert to extract the data from the raw device.\line This means that every time a backup gets triggered 100 % of the used data gets transferred to the secondary storage and not just the delta.\line For big volumes (500GB to TB volumes) which are highly used, the backup process can last very long and uses much space on the secondary side.\line >From our side it would be nice to have a differential and/or incremental >backup feature for our Ceph volumes.\line \line So we took a look around and found this as a possible solution: https://smex12-5-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2fwww.backy2.com=7432fd1e-8f5e-46b9-baa4-273ddd62e3c4=b99d0007cfff65e3b95b9dd26b369f7ce0a2b6f1-a893b29523305f5294e5c68afec4a2a6cb35626a This is a python tool which can do differential backups from Ceph snapshots and store them on a filesystem or in an S3 Storage.\line Sadly there is no REST API for this tool which makes the management of it a little bit inconvenient.\line We were wondering if this tool, some parts of its functionality or the main approach of it could be integrated into Cloudstack to improve the snapshot functionality.\line \line I checked the Github issues and did not find any similar tickets for that.\line So maybe here are more people who needs this functionality or have already some solutions for this.\line \line BR\line Alexander\line \line }}
Re: Ceph Snapshot Backups
Cool. Bin gespannt ob sich jemand meldet. VG Ingo > Am 02.04.2019 um 11:14 schrieb Stock, Alexander > : > > Hi Everyone, > > > some time ago we evaluated the snapshot features from Cloudstack for the > combination Ceph + KVM + Cloudstack and found some space for improvements. > Our goal is to provide snapshot backup for our customers which is easy to > handle over the Cloudstack API and provides incremental/differential features. > > We recognised that Cloudstack uses qemu-convert to extract the data from the > raw device. > This means that every time a backup gets triggered 100 % of the used data > gets transferred to the secondary storage and not just the delta. > For big volumes (500GB to TB volumes) which are highly used, the backup > process can last very long and uses much space on the secondary side. > From our side it would be nice to have a differential and/or incremental > backup feature for our Ceph volumes. > > So we took a look around and found this as a possible solution: > http://www.backy2.com/ > This is a python tool which can do differential backups from Ceph snapshots > and store them on a filesystem or in an S3 Storage. > Sadly there is no REST API for this tool which makes the management of it a > little bit inconvenient. > We were wondering if this tool, some parts of its functionality or the main > approach of it could be integrated into Cloudstack to improve the snapshot > functionality. > > I checked the Github issues and did not find any similar tickets for that. > So maybe here are more people who needs this functionality or have already > some solutions for this. > > BR > Alexander >
Ceph Snapshot Backups
Hi Everyone, some time ago we evaluated the snapshot features from Cloudstack for the combination Ceph + KVM + Cloudstack and found some space for improvements. Our goal is to provide snapshot backup for our customers which is easy to handle over the Cloudstack API and provides incremental/differential features. We recognised that Cloudstack uses qemu-convert to extract the data from the raw device. This means that every time a backup gets triggered 100 % of the used data gets transferred to the secondary storage and not just the delta. For big volumes (500GB to TB volumes) which are highly used, the backup process can last very long and uses much space on the secondary side. >From our side it would be nice to have a differential and/or incremental >backup feature for our Ceph volumes. So we took a look around and found this as a possible solution: http://www.backy2.com/ This is a python tool which can do differential backups from Ceph snapshots and store them on a filesystem or in an S3 Storage. Sadly there is no REST API for this tool which makes the management of it a little bit inconvenient. We were wondering if this tool, some parts of its functionality or the main approach of it could be integrated into Cloudstack to improve the snapshot functionality. I checked the Github issues and did not find any similar tickets for that. So maybe here are more people who needs this functionality or have already some solutions for this. BR Alexander