Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread Chris Laprise

On 7/29/19 12:17 PM, Thomas Kerin wrote:
Sorry, I mean, lvs does show them, I'm just wondering what it'll take to 
show them as active again.


Normally, the following command should force a volume to become active:

lvchange -kn -ay qubes_dom0/volume




That directory seems to just have files from today!


Avoid them unless the date-time was from when the system was (recently) 
still working.



--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB  4AB3 1DC4 D106 F07F 1886

--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/5d1e0473-634f-8f1e-5ed8-a01dff9ffc06%40posteo.net.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread thomas . kerin


My current problem is this error (triggerable by running `lvm lvchange -ay 
-v qubes_dom0/vm-vault-personal-docs-private`)
It happens with some other affected vms.

Activating logical volume qubes_dom0/vm-vault-personal-docs-private 
exclusively.
activation/volume_list configuration setting not defined: Checking only 
host tags for qubes_dom0/vm-vault-personal-docs-private.
Loading qubes_dom0-pool00_tdata table (253:2)
Suppressed qubes_dom0-pool00_tdata (253:2) identical table reload.
Loading qubes_dom0-pool00_tmeta table (253:1)
Suppressed qubes_dom0-pool00_tmeta (253:1) identical table reload.
Loading qubes_dom0-pool00-tpool table (253:3)
Suppressed qubes_dom0-pool00-tpool (253:3) identical table reload.
Creating qubes_dom0-vm--vault--personal--docs--private
Loading qubes_dom0-vm--vault--personal--docs--private table (253:52)
  device-mapper: reload ioctl on (253:52) failed: No data available
Removing qubes_dom0-vm--vault--personal--docs--private (253:52)


Btw, I've scanned through the files in /etc/lvm/archive. I wasn't sure if I 
should follow your advice there as that command requires --force to work 
with thin provisioning and I've seen warnings about this online. I see the 
files contain references to the volumes that lvs doesnt show.

I tested thin_check on qubes_dom0/pool00_meta0 volume (created by lvmodify 
--recover qubes_dom0), and get the following:
examining superblock
examining devices tree
   missing devices: [1, 747]
  too few entries in btree_node: 41, expected at least 42(max 
entries=126)

Running thin_check on meta1 and meta2 (created by running lvchange 
--recover qubes_dom0 a further 2 times) doesn't yield anything major:
examining superblock
examining devices tree
examining mapping tree
checking space map counts

I've followed a procedure to get a metadata snapshot (I couldn't directly 
access _tmeta using normal tools): https://serverfault.com/a/971620
and used thin_dump on the _meta0, _meta1, and _meta2 volumes created by 
`lvchange --recover`.

I diff'd the tmeta file against the others, and it seems only the first 
line is different?
1c1
< 
---
> 

so the pools tmeta has nr_data_blocks = 0.

Maybe my data is still there but the metadata is wrong?

On Monday, 29 July 2019 16:37:26 UTC, thoma...@gmail.com wrote:

> Oh, forgive me, no not all vms are present in lvs. 

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/ac1978a4-68da-4960-bd69-8c4fdadd27ce%40googlegroups.com.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread thomas . kerin
Oh, forgive me, no not all vms are present in lvs. 

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/1a8a95a3-25a9-42c4-8051-60d5342245b4%40googlegroups.com.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread Thomas Kerin
Thanks Chris for your response!

On Mon, 29 Jul 2019, 4:25 pm Chris Laprise,  wrote:

> On 7/29/19 10:19 AM, thomas.ke...@gmail.com wrote:
> > Thanks for your response.
> >
> > I have some which don't show as active - it's looking like some data
> loss..
> >
> > Something I am getting when I run
> > Lvconvert --repair qubes_dom0/pool00
> >
> >
> > WARNING: Sum of all thin volume sizes (2.67TiB) exceeds the size of thin
> pools and the size of whole volume group (931.02GiB)
> >
> > Is this something I can fix perhaps?
>
> This is normal. Thin provisioning usually involves over-provisioning,
> and that's what you're seeing. Most of our Qubes systems display this
> warning when using lvm commands.
>
>
Understood. Thanks!


> >
> > Also, I have some large volumes which are present. I've considered
> trying to remove them, but I might hold off until I get data off the active
> volumes first..
> >
> > I've run across the thin_dump / thin_check / thin_repair commands. It
> seems they're used under the hood by lvconvert --repair to check thin
> volumes.
> >
> > Is there a way to relate those dev_ids back to the thin volumes lvm
> can't seem to find?
>
> If 'lvs' won't show them, then I don't know precisely how. A long time
> ago, I think I used 'vgcfgrestore /etc/lvm/archive/' to
> resolve this kind of issue.
>
>
> Sorry, I mean, lvs does show them, I'm just wondering what it'll take to
show them as active again.

That directory seems to just have files from today!

I also recommend seeking help from the wider Linux community, since this
> is a basic Linux storage issue.
>
> I have spent the morning researching, and found a few posts on redhat.com
and some other sites describing how to repair the metadata.

The most common reason seems to be overflowing the metadata partition,
though mine is currently around 37%

Others (one qubes user) encountered this after power failure. I shut down
cleanly as far as I can tell this was a routine reboot..

And of course, a reminder there mishaps are a good reason to do the
> following:
>
> 1. After installation, at least double the size of your pool00 tmeta
> volume.
>
> 2. Perform regular backups (I'm working on a tool that can backup lvs
> much quicker than the Qubes backup tool).
>
I definitely agree with both, although seems unlikely to have been point
one in this case.

I'm fairly sure the main disk has about 50% free also

Backups are evidently a must.. I've screwed up qubes installs before, but
never lost data until maybe now. I know lvm was only adopted in R4.0,
everything else has been going so well with this install, but I had only
just recovered and organized several old disks worth of data so I'll be
gutted if I lost it and won't know why :/

I see a few people posting on the GitHub qubes-issues repo, one guy says 3
people in the past month have had this issue (or at least the same symptoms)

>
> --
>
> Chris Laprise, tas...@posteo.net
> https://github.com/tasket
> https://twitter.com/ttaskett
> PGP: BEE2 20C5 356E 764A 73EB  4AB3 1DC4 D106 F07F 1886
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/CAHv%2Btb7994bBVc-XZWBADke6r0znUC2b5sGaLfs%2BoVFOy0A_yA%40mail.gmail.com.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread Chris Laprise

On 7/29/19 10:19 AM, thomas.ke...@gmail.com wrote:

Thanks for your response.

I have some which don't show as active - it's looking like some data loss..

Something I am getting when I run
Lvconvert --repair qubes_dom0/pool00


WARNING: Sum of all thin volume sizes (2.67TiB) exceeds the size of thin pools 
and the size of whole volume group (931.02GiB)

Is this something I can fix perhaps?


This is normal. Thin provisioning usually involves over-provisioning, 
and that's what you're seeing. Most of our Qubes systems display this 
warning when using lvm commands.




Also, I have some large volumes which are present. I've considered trying to 
remove them, but I might hold off until I get data off the active volumes 
first..

I've run across the thin_dump / thin_check / thin_repair commands. It seems 
they're used under the hood by lvconvert --repair to check thin volumes.

Is there a way to relate those dev_ids back to the thin volumes lvm can't seem 
to find?


If 'lvs' won't show them, then I don't know precisely how. A long time 
ago, I think I used 'vgcfgrestore /etc/lvm/archive/' to 
resolve this kind of issue.


I also recommend seeking help from the wider Linux community, since this 
is a basic Linux storage issue.


And of course, a reminder there mishaps are a good reason to do the 
following:


1. After installation, at least double the size of your pool00 tmeta volume.

2. Perform regular backups (I'm working on a tool that can backup lvs 
much quicker than the Qubes backup tool).


--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB  4AB3 1DC4 D106 F07F 1886

--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/11508ea4-fed4-e09c-875c-37026e6197a4%40posteo.net.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread thomas . kerin
Thanks for your response.

I have some which don't show as active - it's looking like some data loss..

Something I am getting when I run 
Lvconvert --repair qubes_dom0/pool00


WARNING: Sum of all thin volume sizes (2.67TiB) exceeds the size of thin pools 
and the size of whole volume group (931.02GiB)

Is this something I can fix perhaps?

Also, I have some large volumes which are present. I've considered trying to 
remove them, but I might hold off until I get data off the active volumes 
first..

I've run across the thin_dump / thin_check / thin_repair commands. It seems 
they're used under the hood by lvconvert --repair to check thin volumes.

Is there a way to relate those dev_ids back to the thin volumes lvm can't seem 
to find?

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/1e4c4985-d981-4fc8-b701-0b2636f52134%40googlegroups.com.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-29 Thread Chris Laprise

On 7/28/19 9:47 PM, 'awokd' via qubes-users wrote:

'awokd' via qubes-users:

thomas.ke...@gmail.com:
I've just encountered this issue, and I thought my problems were over 
once I found this post..


Fyi, previously lvscan on my system shown root, pool00, and every 
volume but swap as inactive


I followed your instructions, but the system still fails to boot. 
I've run 'vgchange -ay' and o saw the following printed a number of 
times.


device-mapper: table 253:6: thin: Couldn't open thin internal device
   device-mapper: reload ioctl on (253:6) failed: no data available


I ran 'lvscan' again, and this time some VMS were marked active, but 
a number (root,various -back volumes, several -root volumes, etc)


Really terrified everything is gone as I had just recovered from a 
backup while my hardware got fixed, but I don't have the backup anymore.


Can't tell which post you're replying to, but I get the idea. The 
volumes you are most concerned about all end in --private. If you've 
gotten them to the point where they show as active, you can make a 
subdir and "sudo mount /dev/mapper/qubes_dom0-vm--work--private 
subdir" for example, copy out the contents, umount subdir and move on 
to the next. You can ignore --root volumes, since installing default 
templates will recreate. If you can't get the --private volumes you 
want to show as active, I'm afraid recovering those is beyond me.


Also, if you can't get a --private volume active, try its 
--private--##--back equivalent.


Did you run "lvm lvconvert --repair qubes_dom0/pool00"? I think that 
would be one of the first things you do when the underlying thin device 
fails.


If it needs additional space, you could delete the swap lv, then re-add 
it later.


--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB  4AB3 1DC4 D106 F07F 1886

--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/cb4a0b84-3b9f-7439-b807-d1c56bc11680%40posteo.net.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-28 Thread 'awokd' via qubes-users

'awokd' via qubes-users:

thomas.ke...@gmail.com:
I've just encountered this issue, and I thought my problems were over 
once I found this post..


Fyi, previously lvscan on my system shown root, pool00, and every 
volume but swap as inactive


I followed your instructions, but the system still fails to boot. I've 
run 'vgchange -ay' and o saw the following printed a number of times.


device-mapper: table 253:6: thin: Couldn't open thin internal device
   device-mapper: reload ioctl on (253:6) failed: no data available


I ran 'lvscan' again, and this time some VMS were marked active, but a 
number (root,various -back volumes, several -root volumes, etc)


Really terrified everything is gone as I had just recovered from a 
backup while my hardware got fixed, but I don't have the backup anymore.


Can't tell which post you're replying to, but I get the idea. The 
volumes you are most concerned about all end in --private. If you've 
gotten them to the point where they show as active, you can make a 
subdir and "sudo mount /dev/mapper/qubes_dom0-vm--work--private subdir" 
for example, copy out the contents, umount subdir and move on to the 
next. You can ignore --root volumes, since installing default templates 
will recreate. If you can't get the --private volumes you want to show 
as active, I'm afraid recovering those is beyond me.


Also, if you can't get a --private volume active, try its 
--private--##--back equivalent.


--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/dc3ad51f-e268-33cc-fc69-075ea47177db%40danwin1210.me.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-28 Thread 'awokd' via qubes-users

thomas.ke...@gmail.com:

I've just encountered this issue, and I thought my problems were over once I 
found this post..

Fyi, previously lvscan on my system shown root, pool00, and every volume but 
swap as inactive

I followed your instructions, but the system still fails to boot. I've run 
'vgchange -ay' and o saw the following printed a number of times.

device-mapper: table 253:6: thin: Couldn't open thin internal device
   device-mapper: reload ioctl on (253:6) failed: no data available


I ran 'lvscan' again, and this time some VMS were marked active, but a number 
(root,various -back volumes, several -root volumes, etc)

Really terrified everything is gone as I had just recovered from a backup while 
my hardware got fixed, but I don't have the backup anymore.

Can't tell which post you're replying to, but I get the idea. The 
volumes you are most concerned about all end in --private. If you've 
gotten them to the point where they show as active, you can make a 
subdir and "sudo mount /dev/mapper/qubes_dom0-vm--work--private subdir" 
for example, copy out the contents, umount subdir and move on to the 
next. You can ignore --root volumes, since installing default templates 
will recreate. If you can't get the --private volumes you want to show 
as active, I'm afraid recovering those is beyond me.


--
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/906bb465-1ec6-418e-6ee3-73c1b80fb724%40danwin1210.me.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2019-07-28 Thread thomas . kerin
I've just encountered this issue, and I thought my problems were over once I 
found this post..

Fyi, previously lvscan on my system shown root, pool00, and every volume but 
swap as inactive 

I followed your instructions, but the system still fails to boot. I've run 
'vgchange -ay' and o saw the following printed a number of times.

device-mapper: table 253:6: thin: Couldn't open thin internal device
  device-mapper: reload ioctl on (253:6) failed: no data available


I ran 'lvscan' again, and this time some VMS were marked active, but a number 
(root,various -back volumes, several -root volumes, etc)

Really terrified everything is gone as I had just recovered from a backup while 
my hardware got fixed, but I don't have the backup anymore.

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/30c815fb-299c-4902-bdc3-279e13e6c48f%40googlegroups.com.


Re: [qubes-users] fixing LVM corruption, question about LVM locking type in Qubes

2018-02-13 Thread 'awokd' via qubes-users
On Tue, February 13, 2018 6:36 pm, qtpie wrote:
> Summary: fixing LVM corruption, question about LVM locking


> Questions:
> - what is the standard LVM locking type for a Qubes install? (in Fedora
> it is 1). - if not 1 should I change it back?

On my fresh install of rc4, it's set to 1. Per the comment in there,
that's the "standard mode".

> - if 1 what caused it to be changed?

Possibly when something detected LVM corruption?

> - is powerfailure the probable cause of this issue?

I think that's a reasonable guess but haven't seen a power failure cause
LVM corruption before. Not like a have a fleet of servers running it,
though.


-- 
You received this message because you are subscribed to the Google Groups 
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-users+unsubscr...@googlegroups.com.
To post to this group, send email to qubes-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-users/7ac3628b01ffe55038da50fba66c05f5.squirrel%40tt3j2x4k5ycaa5zt.onion.
For more options, visit https://groups.google.com/d/optout.