Re: Does debian installer use volume names for LVM? (was: Re: trixie update/upgrade strangeness)
On Sun, Oct 15, 2023 at 10:32 AM Max Nikulin wrote: > > I am curious if debian installer uses volume names in /etc/fstab when > LVM is involved (either guided or manual partitioning). I'm pretty sure it does, I checked a few of my machines that I'm reasonably sure I haven't modified too much, and the mounts from LVM are indeed specified as /dev/mapper/. in /etc/fstab. I don't know what grub does, though.
Does debian installer use volume names for LVM? (was: Re: trixie update/upgrade strangeness)
On 12/10/2023 09:55, Jeffrey Walton wrote: On Wed, Oct 11, 2023 at 10:49 PM Andy Smith wrote: - they were using LVM - they'd taken a snapshot of their root fs - they were finding and mounting their root fs by fs UUID - snapshot obviously had same fs UUID - the kernel was finding the snapshot first at boot ... I can't help but wonder how many hundreds or thousands of man hours were wasted because LVM was not designed to avoid the problem by default. It is not responsibility of block-level device to deal with UUID of file systems. I would expect that manuals recommending UUID or file system labels have warnings concerning LVM. I am curious if debian installer uses volume names in /etc/fstab when LVM is involved (either guided or manual partitioning).
Re: trixie update/upgrade strangeness
Hello, On Thu, Oct 12, 2023 at 10:08:06AM -0500, David Wright wrote: > It's not clear to me what's meant in this thread by snapshot. Neither is it yet clear if the OP knows or if my description of a snapshot is the same as what the OP is experiencing, so I can only tell you what I meant. > If you mean snapshot in the way it's been used in several recent > threads here (something that takes up no room to start with, but > gradually grows as the filesystem is modified, ie Copy-On-Write), > then I don't see how you can mount such an object on rebooting. > Is that possible? Yes. An LVM snapshot LV is a new block device that can be used in exactly the same way as an existing LVM LV. Since it is a copy [on write] of the origin, if there is a filesystem on the origin block device then the snapshot will have the same FS label and FS UUID as the one on the origin device. If your initramfs is set up to activate an LVM volume group then every LV inside it will appear as an available block device to the kernel. It is possible for the step which mounts things by label or UUID to find the one on the snapshot before it finds the one on the origin device. This will mount and be writeable by default, at which point you have a snapshot that diverges from the origin. To avoid this sort of thing, many people prefer to refer to LVs by their LVM path, e.g. /dev/vgname/lvname rather than label or UUID. In the case of my customer, they took a snapshot while doing an upgrade and then forgot about it. Nearly a year later they had cause to reboot, and were confused as to why everything seemed about a year old. By then they had forgotten that the snapshot even existed. Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting
Re: trixie update/upgrade strangeness
Hello, On Thu, Oct 12, 2023 at 08:21:23AM -0500, Mark Copper wrote: > Yes, something like that. I should have mentioned it wasn't just files > disappearing but also files that had been moved were back in their previous > location. This "snapshot" appears to have been taken at the time of my > upgrade from bookworm to trixie. > > The whole system is behaving strangely so I had better save the data that > remains. So let's work out what has happened, not just nuke it and accept that everything since the mysterious snapshot has been lost. Show us the output of # mount # cat /etc/fstab # lsblk If we don't do this, you won't know how it happened and risk having it happen again. You say "Yes, something like that" but does that mean that you KNOW it is the result of some sort of snapshot (what sort? Do you recall actually doing it?) or are you simply saying that what I describe is similar to what you see, but you know not how? Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting
Re: trixie update/upgrade strangeness
On Wed 11 Oct 2023 at 22:55:03 (-0400), Jeffrey Walton wrote: > On Wed, Oct 11, 2023 at 10:49 PM Andy Smith wrote: > > On Wed, Oct 11, 2023 at 04:25:58PM -0500, Mark Copper wrote: > > > suddenly, all files created on /dev/sda1 for the last 27 days have > > > disappeared, even files I edited this morning before restarting. Like the > > > clock was turned back a month. The disk, a new 18 TB, passed the test > > > > This happened to a customer of mine once, and they asked me to look > > into it. When I did I found that: > > > > - they were using LVM > > - they'd taken a snapshot of their root fs > > - they were finding and mounting their root fs by fs UUID > > - snapshot obviously had same fs UUID > > - the kernel was finding the snapshot first at boot > > > > Could anything like that be happening? > > That's devious. > > I can't help but wonder how many hundreds or thousands of man hours > were wasted because LVM was not designed to avoid the problem by > default. It's not clear to me what's meant in this thread by snapshot. If you mean snapshot in the way it's been used in several recent threads here (something that takes up no room to start with, but gradually grows as the filesystem is modified, ie Copy-On-Write), then I don't see how you can mount such an object on rebooting. Is that possible? If you mean snapshot as a full copy of a filesystem at a point in time (like 27 days ago), then step N+1 is to either remove the copy from the system (a backup is of dubious value when stored in the same machine), or change it's LABEL/UUID/…, or both. That change is just as essential whether the copy was created by means of a "snapshot" or just a simple dd. Cheers, David.
Re: trixie update/upgrade strangeness
On Wed, Oct 11, 2023, 9:49 PM Andy Smith wrote: > Hello, > > On Wed, Oct 11, 2023 at 04:25:58PM -0500, Mark Copper wrote: > > suddenly, all files created on /dev/sda1 for the last 27 days have > > disappeared, even files I edited this morning before restarting. Like the > > clock was turned back a month. The disk, a new 18 TB, passed the test > > This happened to a customer of mine once, and they asked me to look > into it. When I did I found that: > > - they were using LVM > - they'd taken a snapshot of their root fs > - they were finding and mounting their root fs by fs UUID > - snapshot obviously had same fs UUID > - the kernel was finding the snapshot first at boot > > Could anything like that be happening? > > Thanks, > Andy > > -- > https://bitfolk.com/ -- No-nonsense VPS hosting > Yes, something like that. I should have mentioned it wasn't just files disappearing but also files that had been moved were back in their previous location. This "snapshot" appears to have been taken at the time of my upgrade from bookworm to trixie. The whole system is behaving strangely so I had better save the data that remains. Thanks for responding. >
Re: trixie update/upgrade strangeness
On Wed, Oct 11, 2023 at 10:49 PM Andy Smith wrote: > > On Wed, Oct 11, 2023 at 04:25:58PM -0500, Mark Copper wrote: > > suddenly, all files created on /dev/sda1 for the last 27 days have > > disappeared, even files I edited this morning before restarting. Like the > > clock was turned back a month. The disk, a new 18 TB, passed the test > > This happened to a customer of mine once, and they asked me to look > into it. When I did I found that: > > - they were using LVM > - they'd taken a snapshot of their root fs > - they were finding and mounting their root fs by fs UUID > - snapshot obviously had same fs UUID > - the kernel was finding the snapshot first at boot > > Could anything like that be happening? That's devious. I can't help but wonder how many hundreds or thousands of man hours were wasted because LVM was not designed to avoid the problem by default. Jeff
Re: trixie update/upgrade strangeness
Hello, On Wed, Oct 11, 2023 at 04:25:58PM -0500, Mark Copper wrote: > suddenly, all files created on /dev/sda1 for the last 27 days have > disappeared, even files I edited this morning before restarting. Like the > clock was turned back a month. The disk, a new 18 TB, passed the test This happened to a customer of mine once, and they asked me to look into it. When I did I found that: - they were using LVM - they'd taken a snapshot of their root fs - they were finding and mounting their root fs by fs UUID - snapshot obviously had same fs UUID - the kernel was finding the snapshot first at boot Could anything like that be happening? Thanks, Andy -- https://bitfolk.com/ -- No-nonsense VPS hosting
