Re: [Users] Snapshot merging and the effect on underlying LV metadata
A very neat solution.I like that. Thank you for sharing. -Original Message- From: R P Herrold [mailto:herr...@owlriver.com] Sent: 27 February 2014 19:20 To: Davis, Richard Cc: 'users@ovirt.org' Subject: Snapshot merging and the effect on underlying LV metadata -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 27 Feb 2014, Davis, Richard wrote: I am being told that unless the Wipe After Delete option is set on a vDisk, any subsequent snapshot merging of the related VM will not delete LV metadata (or any data!) from the volume created by the snapshot. Is this correct ? I'm kinda hoping not ! It is my belief a depetion cannot be relied upon to have happened in all cases. Some options flag sets in lvm ** do ** persist old data, and so our security practice at PMman to treat data on removed LV's as though it persists There are published reports that instances on other public cloud providers have been deployed with 'non-wiped' drives in the 'slack space'. Why run the reputational risk? When we reclaim a LV, we perform a 'renaming' that permits to spot 'dirty' and 'scratched' instances needing wiping. [we also fill a new VG / PV with LV's indicating it needs wiping, as we do not wish to expose content if a drive is pulled and then re-used after testing when SMART errors appeared, but do not stand up to disqualify a drive] Later a cron driven process, sensitive to IO load runs. It builds a list of candidates over a day old, using 'find' and the LV name series showing it is dirty and scratched. Then in turn by LV found, it fires off a sub-task (when load is low), which in turn performs a 'niced' 'shred' operation on that LV, followed by the 'shred 'zeroing' operation. When load is too high, it sleeps for a couple of minutes, and re-tries fragment: $_shredCmd = ionice -c 3 shred -n \ .$_num_passes. -z .$_working_lvm; Only when that sub-process has completed do we 'rename' and later 'remove' a given LV, to let its space re-enter the assignment pool - -- Russ herrold -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEARECAAYFAlMPkAMACgkQMRh1QZtklkSamQCgnVqEo2Kmzq9Ao8T0BCYhBTyn aToAoIaOVGkxX3EsVghMxOtgE3RiUr9G =rm/K -END PGP SIGNATURE- This email is confidential and should not be used by anyone who is not the original intended recipient. PGDS cannot accept liability for statements made which are clearly the sender's own and not made on behalf of the company. In addition no statement should be construed as giving investment advice within or outside the United Kingdom. PGDS (UK ONE) LIMITED, Laurence Pountney Hill, London, EC4R 0HH. Incorporated and registered in England and Wales. Registered Office as above. Registered number 1967719. PGDS is the trading name of certain subsidiaries of Prudential plc (registered in England, number 1397169), whose registered office is at Laurence Pountney Hill London EC4R OHH, some of whose subsidiaries are authorised and regulated, as applicable, by the Prudential Regulation Authority and the Financial Conduct Authority. Prudential plc is not affiliated in any manner with Prudential Financial, Inc, a company whose principal place of business is in the United States of America. A list of other Prudential companies together with their registered statutory details can be found in 'About Prudential' on http://www.prudential.co.uk An e-mail reply to this address may be subject to interception or monitoring for operational reasons or for lawful business practices. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Snapshot merging and the effect on underlying LV metadata
In lvm2 version 2.02.105 lvconvert gained a --splitsnapshot option to allow people to wipe snapshot content before releasing the extents for reallocation. --splitsnapshot Separates SnapshotLogicalVolume from its origin. The volume that is split off contains the chunks that differ from the ori- gin along with the metadata describing them. This volume can be wiped and then destroyed with lvremove. The inverse of --snap- shot. Alasdair ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] Snapshot merging and the effect on underlying LV metadata
On Fri, 28 Feb 2014, Alasdair G Kergon wrote: In lvm2 version 2.02.105 lvconvert gained a --splitsnapshot option to allow people to wipe snapshot content before releasing the extents for reallocation. --splitsnapshot Separates SnapshotLogicalVolume from its origin. The volume that is split off contains the chunks that differ from the ori- gin along with the metadata describing them. This volume can be wiped and then destroyed with lvremove. The inverse of --snap- shot. Nice to know ... we use the snapshot feature heavilyin our virtualization, but as: CentOS 6 is at lvm2-2.02.100-8.el6.x86_64, and C 5 at lvm2-2.02.88-12.el5, we will need to wait a bit before relying on its presence. Any chance of a re-basing / refresh / backport at least into RHEL 6 (we have only one Xen oriented dom0 at this point on C5)? Thanks --Russ herrold ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Snapshot merging and the effect on underlying LV metadata
On Fri, Feb 28, 2014 at 02:45:28PM -0500, R P Herrold wrote: Nice to know ... we use the snapshot feature heavilyin our virtualization, but as: CentOS 6 is at lvm2-2.02.100-8.el6.x86_64, and C 5 at lvm2-2.02.88-12.el5, we will need to wait a bit before relying on its presence. Any chance of a re-basing / refresh / backport at least into RHEL 6 (we have only one Xen oriented dom0 at this point on C5)? It will be in RHEL6.6. Alasdair ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] Snapshot merging and the effect on underlying LV metadata
Hi I am being told that unless the Wipe After Delete option is set on a vDisk, any subsequent snapshot merging of the related VM will not delete LV metadata (or any data!) from the volume created by the snapshot. Is this correct ? I'm kinda hoping not ! Richard Davis Technical Specialist PGDS Midrange (UK) - Solaris Linux This email is confidential and should not be used by anyone who is not the original intended recipient. PGDS cannot accept liability for statements made which are clearly the sender's own and not made on behalf of the company. In addition no statement should be construed as giving investment advice within or outside the United Kingdom. PGDS (UK ONE) LIMITED, Laurence Pountney Hill, London, EC4R 0HH. Incorporated and registered in England and Wales. Registered Office as above. Registered number 1967719. PGDS is the trading name of certain subsidiaries of Prudential plc (registered in England, number 1397169), whose registered office is at Laurence Pountney Hill London EC4R OHH, some of whose subsidiaries are authorised and regulated, as applicable, by the Prudential Regulation Authority and the Financial Conduct Authority. Prudential plc is not affiliated in any manner with Prudential Financial, Inc, a company whose principal place of business is in the United States of America. A list of other Prudential companies together with their registered statutory details can be found in 'About Prudential' on http://www.prudential.co.uk An e-mail reply to this address may be subject to interception or monitoring for operational reasons or for lawful business practices. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] Snapshot merging and the effect on underlying LV metadata
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 27 Feb 2014, Davis, Richard wrote: I am being told that unless the Wipe After Delete option is set on a vDisk, any subsequent snapshot merging of the related VM will not delete LV metadata (or any data!) from the volume created by the snapshot. Is this correct ? I'm kinda hoping not ! It is my belief a depetion cannot be relied upon to have happened in all cases. Some options flag sets in lvm ** do ** persist old data, and so our security practice at PMman to treat data on removed LV's as though it persists There are published reports that instances on other public cloud providers have been deployed with 'non-wiped' drives in the 'slack space'. Why run the reputational risk? When we reclaim a LV, we perform a 'renaming' that permits to spot 'dirty' and 'scratched' instances needing wiping. [we also fill a new VG / PV with LV's indicating it needs wiping, as we do not wish to expose content if a drive is pulled and then re-used after testing when SMART errors appeared, but do not stand up to disqualify a drive] Later a cron driven process, sensitive to IO load runs. It builds a list of candidates over a day old, using 'find' and the LV name series showing it is dirty and scratched. Then in turn by LV found, it fires off a sub-task (when load is low), which in turn performs a 'niced' 'shred' operation on that LV, followed by the 'shred 'zeroing' operation. When load is too high, it sleeps for a couple of minutes, and re-tries fragment: $_shredCmd = ionice -c 3 shred -n \ .$_num_passes. -z .$_working_lvm; Only when that sub-process has completed do we 'rename' and later 'remove' a given LV, to let its space re-enter the assignment pool - -- Russ herrold -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.14 (GNU/Linux) iEYEARECAAYFAlMPkAMACgkQMRh1QZtklkSamQCgnVqEo2Kmzq9Ao8T0BCYhBTyn aToAoIaOVGkxX3EsVghMxOtgE3RiUr9G =rm/K -END PGP SIGNATURE- ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
