Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Xen]

Zdenek Kabelac schreef op 28-03-2018 0:17:


Hi

This is why users do open BZ if they would like to see some 
enhancement.


Normally cache is integral part of a volume - so it's partially
missing - whole volume is considered to be garbage.

But in 'writethrough' mode there could be likely possible better 
recovery.


Of course this case needs usability without --force.

So please open  RFE BZ for this case.


It goes into the mess I usually get myself into; if you "dd copy" the 
disk containing the origin volume before uncaching it, and then go to 
some live session where you only have the new backup copy, but you want 
to clean up its LVM,


then you now must fix the VGs in isolation of the cache; I suppose this 
is just the wrong order of doing things, but as part of a backup you 
don't really want to uncache first, as that requires more work to get it 
back to normal after.


So you end up in a situation where the new origin copy has a reference 
to the cache disk --- all of this assumes writethrough mode --- and you 
need to clear that reference.


However, you cannot, or should not, attach the cache disk again; it 
might get effected, and you don't want that, you want it to remain in 
its pristine state.


Therefore, you are now left with the task of removing the cache from the 
VG, because you cannot actually run vgimportclone while the cache disk 
is missing.


The obvious solution is to *also* clone the cache disk and then run 
operations on the combined set, but this might not be possible.


Therefore, all that was left was:

  vgreduce --remove-missing --force
  cd /etc/lvm/archive
  cp  /etc/lvm/backup/
  cd /etc/lvm/backup
  vi 
  " remove cache PV, and change origin to regular linear volume, and add
  " the visible tag

  vgcfgrestore 

  # presto, origin is restored as regular volume without the cache

  vgimportclone -i  

  # now have distinct volume group, VG UUID and PV UUID

So the problem is making dd backups of origin, perhaps dd backups should 
be avoided, but for some purposes (such as system migration) file copies 
are just


more work

in general, and can complicate things as well, for instance if there are 
NTFS partitions or whatnot.


And disk images can be nice to have, in any case.

This was the use case basically.

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Gang He]

Hi Zdenek,


>>> 
> Dne 27.3.2018 v 07:55 Gang He napsal(a):
>> Hi Fran,
>> 
>> 
>
>>> On 26 March 2018 at 08:04, Gang He  wrote:
 It looks like each PV includes a copy meta data for VG, but if some PV has
>>> changed (e.g. removed, or moved to another VG),
 the remained PV should have a method to check the integrity when each
>>> startup (activated?), to avoid such inconsistent problem automatically.
>>>
>>> Your workflow is strange. What are you trying to accomplish here?
>> I just reproduced a problem from the customer, since they did virtual disk 
> migration from one virtual machine  to another one.
>> According to your comments, this does not look like a LVM code problem,
>> the problem can be considered as LVM administer misoperation?
>> 
>> Thanks
>> Gang
> 
> 
> Ahh, so welcome Eric's replacement  :)
Yes, thank for your inputs.

> 
> Yes - this use scenario was improper usage of lvm2 - and lvm2 has cached the 
> user before he could ruing his data any further...
> 
> 
> Regards
> 
> 
> Zdenek


___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Zdenek Kabelac]

Dne 27.3.2018 v 12:27 Xen napsal(a):

Zdenek Kabelac schreef op 27-03-2018 12:22:


IMO 'vgreduce --removemissing'  doesn't look to me like a real rocket science.


Yeah I don't wanna get into it.

--force didn't work very well when the missing PV was a cache PV as it removed 
the entire cached volume including the origin.



Hi

This is why users do open BZ if they would like to see some enhancement.

Normally cache is integral part of a volume - so it's partially missing - 
whole volume is considered to be garbage.


But in 'writethrough' mode there could be likely possible better recovery.

Of course this case needs usability without --force.

So please open  RFE BZ for this case.


Regards

Zdenek

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Zdenek Kabelac]

Dne 27.3.2018 v 11:12 Xen napsal(a):

Gang He schreef op 27-03-2018 7:55:


I just reproduced a problem from the customer, since they did virtual
disk migration from one virtual machine  to another one.
According to your comments, this does not look like a LVM code problem,
the problem can be considered as LVM administer misoperation?


Counterintuitively, you must remove the PV from the VG before you remove the 
(physical) disk from the system.


Yes that is something you can often forget doing, but as it stands resolving 
the situation often becomes a lot harder when you do it in reverse.


Ie. removing the disk first and then removing the PV from the VG is a lot 
harder.




IMO 'vgreduce --removemissing'  doesn't look to me like a real rocket science.

Of course if the space on a missing PV has been in-use with some LVs - it gets
a bit more complex - but still as long as you don't care about data adding
'--force' option should usually help  (although there can be some room for
improvement -  but every coding takes its time - and there are not to many 
hands working on this...)


Regards

Zdenek

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Xen]

Gang He schreef op 27-03-2018 7:55:


I just reproduced a problem from the customer, since they did virtual
disk migration from one virtual machine  to another one.
According to your comments, this does not look like a LVM code problem,
the problem can be considered as LVM administer misoperation?


Counterintuitively, you must remove the PV from the VG before you remove 
the (physical) disk from the system.


Yes that is something you can often forget doing, but as it stands 
resolving the situation often becomes a lot harder when you do it in 
reverse.


Ie. removing the disk first and then removing the PV from the VG is a 
lot harder.


___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Zdenek Kabelac]

Dne 27.3.2018 v 07:55 Gang He napsal(a):

Hi Fran,





On 26 March 2018 at 08:04, Gang He  wrote:

It looks like each PV includes a copy meta data for VG, but if some PV has

changed (e.g. removed, or moved to another VG),

the remained PV should have a method to check the integrity when each

startup (activated?), to avoid such inconsistent problem automatically.

Your workflow is strange. What are you trying to accomplish here?

I just reproduced a problem from the customer, since they did virtual disk 
migration from one virtual machine  to another one.
According to your comments, this does not look like a LVM code problem,
the problem can be considered as LVM administer misoperation?

Thanks
Gang



Ahh, so welcome Eric's replacement  :)

Yes - this use scenario was improper usage of lvm2 - and lvm2 has cached the 
user before he could ruing his data any further...



Regards


Zdenek

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Gang He]

Hi Fran,


>>> 
> On 26 March 2018 at 08:04, Gang He  wrote:
>> It looks like each PV includes a copy meta data for VG, but if some PV has 
> changed (e.g. removed, or moved to another VG),
>> the remained PV should have a method to check the integrity when each 
> startup (activated?), to avoid such inconsistent problem automatically.
> 
> Your workflow is strange. What are you trying to accomplish here?
I just reproduced a problem from the customer, since they did virtual disk 
migration from one virtual machine  to another one.
According to your comments, this does not look like a LVM code problem, 
the problem can be considered as LVM administer misoperation?

Thanks
Gang   

> 
> Your steps in 5 should be:
> 
> vgreduce vg01 /dev/vdc /dev/vdc
> pvremove /dev/vdc /dev/vdd
> 
> That way you ensure there's no leftover metadata in the PVs (specially
> if you need to attach those disks to a different system)
> 
> Again a usecase to understand your workflow would be beneficial...
> 
> Cheers
> 
> fran
> 
> ___
> linux-lvm mailing list
> linux-lvm@redhat.com 
> https://www.redhat.com/mailman/listinfo/linux-lvm 
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Fran Garcia]

On 26 March 2018 at 08:04, Gang He  wrote:
>> Gang He schreef op 23-03-2018 9:30:
>>
>>> 6) attach disk2 to VM2(tb0307-nd2), the vg on VM2 looks abnormal.
>>> tb0307-nd2:~ # pvs
>>>   WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
>>> found or rejected by a filter.
>>>   PV VG  Fmt  Attr PSize  PFree
>>>   /dev/vdc   vg2 lvm2 a--  20.00g 20.00g
>>>   /dev/vdd   vg1 lvm2 a--  20.00g 20.00g
>>>   [unknown]  vg1 lvm2 a-m  20.00g 20.00g
>>
>> This is normal because /dev/vdd contains metadata for vg1 which includes
>> now missing disk /dev/vdc   as the PV is no longer the same.
>
> It looks like each PV includes a copy meta data for VG, but if some PV has 
> changed (e.g. removed, or moved to another VG),
> the remained PV should have a method to check the integrity when each startup 
> (activated?), to avoid such inconsistent problem automatically.

Your workflow is strange. What are you trying to accomplish here?

Your steps in 5 should be:

vgreduce vg01 /dev/vdc /dev/vdc
pvremove /dev/vdc /dev/vdd

That way you ensure there's no leftover metadata in the PVs (specially
if you need to attach those disks to a different system)

Again a usecase to understand your workflow would be beneficial...

Cheers

Fran

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Zdenek Kabelac]

Dne 26.3.2018 v 08:04 Gang He napsal(a):

Hi Xen,





Gang He schreef op 23-03-2018 9:30:


6) attach disk2 to VM2(tb0307-nd2), the vg on VM2 looks abnormal.
tb0307-nd2:~ # pvs
   WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
found or rejected by a filter.
   PV VG  Fmt  Attr PSize  PFree
   /dev/vdc   vg2 lvm2 a--  20.00g 20.00g
   /dev/vdd   vg1 lvm2 a--  20.00g 20.00g
   [unknown]  vg1 lvm2 a-m  20.00g 20.00g


This is normal because /dev/vdd contains metadata for vg1 which includes
now missing disk /dev/vdc   as the PV is no longer the same.





tb0307-nd2:~ # vgs
   WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
found or rejected by a filter.
   VG  #PV #LV #SN Attr   VSize  VFree
   vg1   2   0   0 wz-pn- 39.99g 39.99g
   vg2   1   0   0 wz--n- 20.00g 20.00g


This is normal because you haven't removed /dev/vdc from vg1 on
/dev/vdd, since it was detached while you operated on its vg.



7) reboot VM2, the result looks worse (vdc disk belongs to two vg).
tb0307-nd2:/mnt/shared # pvs
   PV VG  Fmt  Attr PSize  PFree
   /dev/vdc   vg1 lvm2 a--  20.00g 0
   /dev/vdc   vg2 lvm2 a--  20.00g 10.00g
   /dev/vdd   vg1 lvm2 a--  20.00g  9.99g


When you removed vdd when it was not attached, the VG1 metadata on vdd
was not altered. The metadata resides on both disks, so you had
inconsistent metadata between both disks because you operated on the
shared volume group while one device was missing.

You also did not recreate PV on /dev/vdc so it has the same UUID as when
it was part of VG1, this is why VG1 when VDD is booted will still try to
include /dev/vdc because it was never removed from the volume group on
VDD.

So the state of affairs is:

/dev/vdc contains volume group info for VG2 and includes only /dev/vdc

/dev/vdd contains volume group info for VG1, and includes both /dev/vdc
and /dev/vdd by UUID for its PV, however, it is a bug that it should
include /dev/vdc even though the VG UUID is now different (and the name
as well).

It looks like each PV includes a copy meta data for VG, but if some PV has 
changed (e.g. removed, or moved to another VG),
the remained PV should have a method to check the integrity when each startup 
(activated?), to avoid such inconsistent problem automatically.





Hi

I'm not really sure what are you trying to achieve - are you 'validating' that 
you cannot foolish lvm2 too easily or something else ?


Simple case is when you have a VG on 2 PV disks. Both PV hold full metadata 
for a VG. There are numerous other case - i.e. you can have 1000PVs in single 
VG then any update of metadata would require to update 1000 disks - for this 
case you can select lower number metadata copies - i.e. randomly or 
user-selected  PVs only hold VG metadata and rest of PV are without metadata.

The less metadata copies - the less secure it is, but update is faster...


There are no metadata for use stored in your filesystem - VG metadata are 
always recorded in PV metadata area.



1.) So when you 1st. remove device and then you run 'pvremove' on this missing 
disk, it's kind of pointless operation.


2.) lvm2 command will not let you 'easily' remove PV which is in use by some 
LV in your VG


3.) lvm2 supports 2 commands:
  'vgreduce --removemissing'(try to make consistent VG when PV is lost)
  'vgextend --restoeremissing'  (restore missing PV back into VG)


Regards

Zdenek

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Fran Garcia]

On 26 March 2018 at 08:04, Gang He  wrote:
> It looks like each PV includes a copy meta data for VG, but if some PV has 
> changed (e.g. removed, or moved to another VG),
> the remained PV should have a method to check the integrity when each startup 
> (activated?), to avoid such inconsistent problem automatically.

Your workflow is strange. What are you trying to accomplish here?

Your steps in 5 should be:

vgreduce vg01 /dev/vdc /dev/vdc
pvremove /dev/vdc /dev/vdd

That way you ensure there's no leftover metadata in the PVs (specially
if you need to attach those disks to a different system)

Again a usecase to understand your workflow would be beneficial...

Cheers

fran

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Gang He]

Hi Xen,


>>> 
> Gang He schreef op 23-03-2018 9:30:
> 
>> 6) attach disk2 to VM2(tb0307-nd2), the vg on VM2 looks abnormal.
>> tb0307-nd2:~ # pvs
>>   WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
>> found or rejected by a filter.
>>   PV VG  Fmt  Attr PSize  PFree
>>   /dev/vdc   vg2 lvm2 a--  20.00g 20.00g
>>   /dev/vdd   vg1 lvm2 a--  20.00g 20.00g
>>   [unknown]  vg1 lvm2 a-m  20.00g 20.00g
> 
> This is normal because /dev/vdd contains metadata for vg1 which includes 
> now missing disk /dev/vdc   as the PV is no longer the same.
> 
> 
> 
> 
>> tb0307-nd2:~ # vgs
>>   WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
>> found or rejected by a filter.
>>   VG  #PV #LV #SN Attr   VSize  VFree
>>   vg1   2   0   0 wz-pn- 39.99g 39.99g
>>   vg2   1   0   0 wz--n- 20.00g 20.00g
> 
> This is normal because you haven't removed /dev/vdc from vg1 on 
> /dev/vdd, since it was detached while you operated on its vg.
> 
> 
>> 7) reboot VM2, the result looks worse (vdc disk belongs to two vg).
>> tb0307-nd2:/mnt/shared # pvs
>>   PV VG  Fmt  Attr PSize  PFree
>>   /dev/vdc   vg1 lvm2 a--  20.00g 0
>>   /dev/vdc   vg2 lvm2 a--  20.00g 10.00g
>>   /dev/vdd   vg1 lvm2 a--  20.00g  9.99g
> 
> When you removed vdd when it was not attached, the VG1 metadata on vdd 
> was not altered. The metadata resides on both disks, so you had 
> inconsistent metadata between both disks because you operated on the 
> shared volume group while one device was missing.
> 
> You also did not recreate PV on /dev/vdc so it has the same UUID as when 
> it was part of VG1, this is why VG1 when VDD is booted will still try to 
> include /dev/vdc because it was never removed from the volume group on 
> VDD.
> 
> So the state of affairs is:
> 
> /dev/vdc contains volume group info for VG2 and includes only /dev/vdc
> 
> /dev/vdd contains volume group info for VG1, and includes both /dev/vdc 
> and /dev/vdd by UUID for its PV, however, it is a bug that it should 
> include /dev/vdc even though the VG UUID is now different (and the name 
> as well).
It looks like each PV includes a copy meta data for VG, but if some PV has 
changed (e.g. removed, or moved to another VG),
the remained PV should have a method to check the integrity when each startup 
(activated?), to avoid such inconsistent problem automatically.

Thanks
Gang 

> 
> Regardless, from vdd's perspective /dev/vdc is still part of VG1.
> 
> ___
> linux-lvm mailing list
> linux-lvm@redhat.com 
> https://www.redhat.com/mailman/listinfo/linux-lvm 
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/


___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Can't work normally after attaching disk volumes originally in a VG on another machine

[Xen]

Gang He schreef op 23-03-2018 9:30:


6) attach disk2 to VM2(tb0307-nd2), the vg on VM2 looks abnormal.
tb0307-nd2:~ # pvs
  WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
found or rejected by a filter.
  PV VG  Fmt  Attr PSize  PFree
  /dev/vdc   vg2 lvm2 a--  20.00g 20.00g
  /dev/vdd   vg1 lvm2 a--  20.00g 20.00g
  [unknown]  vg1 lvm2 a-m  20.00g 20.00g


This is normal because /dev/vdd contains metadata for vg1 which includes 
now missing disk /dev/vdc   as the PV is no longer the same.






tb0307-nd2:~ # vgs
  WARNING: Device for PV JJOL4H-kc0j-jyTD-LDwl-71FZ-dHKM-YoFtNV not
found or rejected by a filter.
  VG  #PV #LV #SN Attr   VSize  VFree
  vg1   2   0   0 wz-pn- 39.99g 39.99g
  vg2   1   0   0 wz--n- 20.00g 20.00g


This is normal because you haven't removed /dev/vdc from vg1 on 
/dev/vdd, since it was detached while you operated on its vg.




7) reboot VM2, the result looks worse (vdc disk belongs to two vg).
tb0307-nd2:/mnt/shared # pvs
  PV VG  Fmt  Attr PSize  PFree
  /dev/vdc   vg1 lvm2 a--  20.00g 0
  /dev/vdc   vg2 lvm2 a--  20.00g 10.00g
  /dev/vdd   vg1 lvm2 a--  20.00g  9.99g


When you removed vdd when it was not attached, the VG1 metadata on vdd 
was not altered. The metadata resides on both disks, so you had 
inconsistent metadata between both disks because you operated on the 
shared volume group while one device was missing.


You also did not recreate PV on /dev/vdc so it has the same UUID as when 
it was part of VG1, this is why VG1 when VDD is booted will still try to 
include /dev/vdc because it was never removed from the volume group on 
VDD.


So the state of affairs is:

/dev/vdc contains volume group info for VG2 and includes only /dev/vdc

/dev/vdd contains volume group info for VG1, and includes both /dev/vdc 
and /dev/vdd by UUID for its PV, however, it is a bug that it should 
include /dev/vdc even though the VG UUID is now different (and the name 
as well).


Regardless, from vdd's perspective /dev/vdc is still part of VG1.

___
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/