Re: setup-storage fails on blank disk

[Holger Parplies]

Hi,

Andreas Heinlein wrote on 2018-01-04 14:47:21 +0100 [Re: setup-storage fails on 
blank disk]:
> Am 03.01.2018 um 17:28 schrieb Holger Parplies:
> > [...]
> > My point: am I missing something, and there is some obscure benefit of
> > putting an LVM container within an extended-partition-container
> > [...]
> > does using 'primary' instead of 'logical' for all three partitions change
> > anything concerning the error you are experiencing?
> > [...]
> yes, you are right - in some way, this *is* a legacy windows issue that
> has developed over time.
> [...]
> On the other hand, up to now we had no problems with that, so no urge to
> change anything. If you think it might help, I could try changing this.

it's not so much that I'm suspecting "that's the problem", it's one thing
I would try out, because it seems rather simple to do so, and because it
*might* make a difference. If it doesn't, it's a cause we can rule out. If
it does, it's a point of knowledge we've gained which might help others in
the future.

You could always reinstall with logical partitions after that to keep your
systems identical ...

Regards,
Holger

Re: setup-storage fails on blank disk

[Derek Poon]

Me too.  I have been using the following script, placed in 
hooks/partition.LINUX, as a workaround.


#!/bin/bash

##
# If setup-storage encounters a disk to be partitioned, but the disk
# contains no partition table at all, it will give up.  To prevent
# such failures, this hook creates an empty partition table of the
# type specified in the setup-storage configuration file on any disk
# whose first 512 bytes are all null.
##

. /usr/lib/fai/subroutines

# Prints the absolute path of the disk configuration file, chosen based on
# the last applicable class to have a configuration file in
# /var/lib/fai/config/disk_config.
disk_config_file() {
echo $classes | tr ' ' '\n' | tac | while read class ; do
if [ -f "$FAI/disk_config/$class" ]; then
echo "$FAI/disk_config/$class"
break
fi
done
}

# Output contains one line per disk, with the device followed by the partition
# table type that it is supposed to have:
#
# /dev/sda gpt
# /dev/sda gpt-bios
# /dev/sdc msdos
want_partition_tables() {
config_file=`disk_config_file`
if [ -z "$config_file" ]; then
# No suitable setup-storage config file
exit 1
fi
awk -v disks="`/usr/lib/fai/fai-disk-info | sort`" '
BEGIN {
split(disks, dev);
}

$1 == "disk_config" && $2 ~ "disk.*" {
sub("^disk", "", $2);
for (i = 3; i <= NF; i++) {
if (split($i, optionpair, ":")) {
if (optionpair[1] == "sameas") {
sub("^disk", "", optionpair[2]);
partition_table[$2] = partition_table[optionpair[2]];
if (dev[$2]) print "/dev/" dev[$2], partition_table[$2];
next;
} else if (optionpair[1] == "disklabel") {
partition_table[$2] = optionpair[2];
if (dev[$2]) print "/dev/" dev[$2], partition_table[$2];
next;
}
}
}

# If no disklabel option is given, setup-storage uses "msdos" as
# the default.
if (dev[$2]) print "/dev/" dev[$2], "msdos";
}
' "$config_file"
}

# Returns 0 if the first 512 bytes of the specified device are all null.
has_partition_table() {
local dev="$1"
perl -e "
open D, '<', '$dev' and
read D, \$buf, 512 and
\$buf ne \"\\000\" x 512 or
exit 1
"
}

# For every disk in the setup-storage configuration, if there is no partition
# table apparent in the first 512 bytes, then we will create one of the
# appropriate type.
ensure_partition_tables_exist() {
set -o pipefail
want_partition_tables |
while read dev table ; do
if ! has_partition_table "$dev" ; then
[ "$table" = "gpt-bios" ] && table=gpt
echo "$dev has no partition table; making one of type $table"
parted --script "$dev" mktable "$table"
fi
done
}

ensure_partition_tables_exist



> On Jan 4, 2018, at 7:12 AM, Brian Kroth  wrote:
> 
> Ages ago I recall having a similar problem to that as well.  I think it was a 
> bug with the preserver_reinstall flag not working correctly with LVM setups 
> on fresh disks.  Pretty sure I ended up writing a hack that executed early on 
> in the classes phase to figure out if the disk was lacking a label (partition 
> table) and then creating one and setting the reinstall flag to force the disk 
> layout down.  Unfortunately I don't easily have access to that code anymore, 
> else I'd share it.  Also, it may no longer be applicable.  setup-storage has 
> changed a lot since then.
> 
> Cheers,
> Brian

Re: setup-storage fails on blank disk

[Brian Kroth]

Ages ago I recall having a similar problem to that as well.  I think it was
a bug with the preserver_reinstall flag not working correctly with LVM
setups on fresh disks.  Pretty sure I ended up writing a hack that executed
early on in the classes phase to figure out if the disk was lacking a label
(partition table) and then creating one and setting the reinstall flag to
force the disk layout down.  Unfortunately I don't easily have access to
that code anymore, else I'd share it.  Also, it may no longer be
applicable.  setup-storage has changed a lot since then.

Cheers,
Brian

On Thu, Jan 4, 2018 at 8:41 AM, Justin Cattle <j...@ocado.com> wrote:

> Hi,
>
>
> Are you 100% sure you tried it on a "fresh" disk that was really clean?
> It defiantly feels like there is some metadata or something remaining in
> some blocks on the disk.
>
> When FAI fails, are you able check for things like md info, dm info, lvm
> info and the like?
>
> You may have to do some dmsetup remove, vgremove or pvremove.
> wiptefs is also a good utility at clearing metadata, run it against any
> partitions before removing them.
> Then remove partitions with dd, put some zeros on the first few Mg of the
> disk.
>
>
>
>
> Cheers,
> Just
>
> On 4 January 2018 at 13:47, Andreas Heinlein <aheinl...@gmx.com> wrote:
>
>> Am 03.01.2018 um 17:28 schrieb Holger Parplies:
>> > Hi,
>> >
>> > Andreas Heinlein wrote on 2018-01-03 13:53:40 +0100 [setup-storage
>> fails on blank disk]:
>> >> [...]
>> >> I have encountered a problem with setup-storage which occurs only when
>> >> the disk is blank, i.e. wiped with nwipe/dban or brand new. It then
>> >> fails on creating the LVM; running 'pvcreate' returns 'cannot open
>> >> /dev/sda5 exclusively'.
>> > this is probably unrelated, but is there any reason to put the LVM PV
>> inside
>> > a "logical" volume? DOS extended partitions seem to be the worst hack
>> ever
>> > invented to get around a limitation in a bad design, yet they repeatedly
>> > and apparently unnecessarily pop up in quoted disk_configs:
>> >
>> >> [...]
>> >> This is your disk_config file:
>> >> # generic disk configuration for one small disk
>> >> # disk size from 500Mb up to what you can buy today
>> >> #
>> >> #   [extra
>> options]
>> >>
>> >> disk_config disk1 disklabel:msdos bootable:1 preserve_lazy:6
>> align-at:1M fstabkey:uuid
>> >> primary  /boot  300  ext4rw  createopts="-O
>> ^64bit,^metadata_csum"
>> >> logical  -  29500-3  -   -
>> >> logical  /media/daten  1024- ext4acl createopts="-O
>> ^64bit,^metadata_csum -L Daten"
>> > I count three partitions, which would work perfectly with primary
>> partitions
>> > (furthermore, you are using LVM to have an arbitrary number of named and
>> > dynamic "volumes" (i.e. partitions) anyway, so if you needed more, LVM
>> would
>> > be the superior mechanism; of course, your specific requirements may
>> vary).
>> > Ok, you are preserving a logical partition, so in this particular case
>> you'd
>> > actually need to stick with logical partitions, but the partition in
>> question
>> > is ext4, not FAT-based, so it doesn't appear to be a legacy Windoze
>> issue.
>> >
>> > My point: am I missing something, and there is some obscure benefit of
>> putting
>> > an LVM container within an extended-partition-container (such as hiding
>> it
>> > from something), or is it simply a common misconception that you for
>> some
>> > reason cannot or should not put an LVM PV (or even several individual
>> native
>> > Linux partitions - such as /, /var and /tmp) into primary partitions -
>> > assuming you only need upto four of them (and, obviously, assuming you
>> are
>> > still using MSDOS partition tables)?
>> >
>> > Or, differently: for a *blank disk*, you obviously won't be preserving
>> sda6,
>> > and you probably aren't referencing it by partition number
>> ("fstabkey:uuid"),
>> > so does using 'primary' instead of 'logical' for all three partitions
>> change
>> > anything concerning the error you are experiencing?
>> >
>> > Hope that helps someone (perhaps me ;-) ...
>> >
>> > Regards,
>> > Holger
>> Hello,
>>
>> yes, you are right - in some way, this *is* a legacy windows i

Re: setup-storage fails on blank disk

[Justin Cattle]

Hi,


Are you 100% sure you tried it on a "fresh" disk that was really clean?
It defiantly feels like there is some metadata or something remaining in
some blocks on the disk.

When FAI fails, are you able check for things like md info, dm info, lvm
info and the like?

You may have to do some dmsetup remove, vgremove or pvremove.
wiptefs is also a good utility at clearing metadata, run it against any
partitions before removing them.
Then remove partitions with dd, put some zeros on the first few Mg of the
disk.




Cheers,
Just

On 4 January 2018 at 13:47, Andreas Heinlein <aheinl...@gmx.com> wrote:

> Am 03.01.2018 um 17:28 schrieb Holger Parplies:
> > Hi,
> >
> > Andreas Heinlein wrote on 2018-01-03 13:53:40 +0100 [setup-storage fails
> on blank disk]:
> >> [...]
> >> I have encountered a problem with setup-storage which occurs only when
> >> the disk is blank, i.e. wiped with nwipe/dban or brand new. It then
> >> fails on creating the LVM; running 'pvcreate' returns 'cannot open
> >> /dev/sda5 exclusively'.
> > this is probably unrelated, but is there any reason to put the LVM PV
> inside
> > a "logical" volume? DOS extended partitions seem to be the worst hack
> ever
> > invented to get around a limitation in a bad design, yet they repeatedly
> > and apparently unnecessarily pop up in quoted disk_configs:
> >
> >> [...]
> >> This is your disk_config file:
> >> # generic disk configuration for one small disk
> >> # disk size from 500Mb up to what you can buy today
> >> #
> >> #   [extra
> options]
> >>
> >> disk_config disk1 disklabel:msdos bootable:1 preserve_lazy:6
> align-at:1M fstabkey:uuid
> >> primary  /boot  300  ext4rw  createopts="-O
> ^64bit,^metadata_csum"
> >> logical  -  29500-3  -   -
> >> logical  /media/daten  1024- ext4acl createopts="-O
> ^64bit,^metadata_csum -L Daten"
> > I count three partitions, which would work perfectly with primary
> partitions
> > (furthermore, you are using LVM to have an arbitrary number of named and
> > dynamic "volumes" (i.e. partitions) anyway, so if you needed more, LVM
> would
> > be the superior mechanism; of course, your specific requirements may
> vary).
> > Ok, you are preserving a logical partition, so in this particular case
> you'd
> > actually need to stick with logical partitions, but the partition in
> question
> > is ext4, not FAT-based, so it doesn't appear to be a legacy Windoze
> issue.
> >
> > My point: am I missing something, and there is some obscure benefit of
> putting
> > an LVM container within an extended-partition-container (such as hiding
> it
> > from something), or is it simply a common misconception that you for some
> > reason cannot or should not put an LVM PV (or even several individual
> native
> > Linux partitions - such as /, /var and /tmp) into primary partitions -
> > assuming you only need upto four of them (and, obviously, assuming you
> are
> > still using MSDOS partition tables)?
> >
> > Or, differently: for a *blank disk*, you obviously won't be preserving
> sda6,
> > and you probably aren't referencing it by partition number
> ("fstabkey:uuid"),
> > so does using 'primary' instead of 'logical' for all three partitions
> change
> > anything concerning the error you are experiencing?
> >
> > Hope that helps someone (perhaps me ;-) ...
> >
> > Regards,
> > Holger
> Hello,
>
> yes, you are right - in some way, this *is* a legacy windows issue that
> has developed over time. In fact, the preserved partition once was a FAT
> partition as long as we had dualboot installations on these machines. We
> finally removed the dualboot two or three years ago and chose to format
> this partition ext4 instead. Why we didn't move to primary partitions or
> put it within the LVM at that time - I don't know.
>
> On the other hand, up to now we had no problems with that, so no urge to
> change anything. If you think it might help, I could try changing this.
>
> Bye,
> Andreas
>

-- 


Notice:  This email is confidential and may contain copyright material of 
members of the Ocado Group. Opinions and views expressed in this message 
may not necessarily reflect the opinions and views of the members of the 
Ocado Group. 

 

If you are not the intended recipient, please notify us immediately and 
delete all copies of this message. Please note that it is your 
responsibility to scan this message for viruses. 

 

Fetch and Sizzle are trading names of Speciality Stores Limited and Fabled 
is a trading name of Marie Claire Beauty Limited, both members of the Ocado 
Group.

 

References to the “Ocado Group” are to Ocado Group plc (registered in 
England and Wales with number 7098618) and its subsidiary undertakings (as 
that expression is defined in the Companies Act 2006) from time to time. 
 The registered office of Ocado Group plc is Buildings One & Two, Trident 
Place, Mosquito Way, Hatfield, Hertfordshire, AL10 9UL.

Re: setup-storage fails on blank disk

[Andreas Heinlein]

Am 03.01.2018 um 17:28 schrieb Holger Parplies:
> Hi,
>
> Andreas Heinlein wrote on 2018-01-03 13:53:40 +0100 [setup-storage fails on 
> blank disk]:
>> [...]
>> I have encountered a problem with setup-storage which occurs only when
>> the disk is blank, i.e. wiped with nwipe/dban or brand new. It then
>> fails on creating the LVM; running 'pvcreate' returns 'cannot open
>> /dev/sda5 exclusively'.
> this is probably unrelated, but is there any reason to put the LVM PV inside
> a "logical" volume? DOS extended partitions seem to be the worst hack ever
> invented to get around a limitation in a bad design, yet they repeatedly
> and apparently unnecessarily pop up in quoted disk_configs:
>
>> [...]
>> This is your disk_config file:
>> # generic disk configuration for one small disk
>> # disk size from 500Mb up to what you can buy today
>> #
>> #   [extra options]
>>
>> disk_config disk1 disklabel:msdos bootable:1 preserve_lazy:6 align-at:1M 
>> fstabkey:uuid 
>> primary  /boot  300  ext4rw  createopts="-O 
>> ^64bit,^metadata_csum"
>> logical  -  29500-3  -   -
>> logical  /media/daten  1024- ext4acl createopts="-O 
>> ^64bit,^metadata_csum -L Daten"
> I count three partitions, which would work perfectly with primary partitions
> (furthermore, you are using LVM to have an arbitrary number of named and
> dynamic "volumes" (i.e. partitions) anyway, so if you needed more, LVM would
> be the superior mechanism; of course, your specific requirements may vary).
> Ok, you are preserving a logical partition, so in this particular case you'd
> actually need to stick with logical partitions, but the partition in question
> is ext4, not FAT-based, so it doesn't appear to be a legacy Windoze issue.
>
> My point: am I missing something, and there is some obscure benefit of putting
> an LVM container within an extended-partition-container (such as hiding it
> from something), or is it simply a common misconception that you for some
> reason cannot or should not put an LVM PV (or even several individual native
> Linux partitions - such as /, /var and /tmp) into primary partitions -
> assuming you only need upto four of them (and, obviously, assuming you are
> still using MSDOS partition tables)?
>
> Or, differently: for a *blank disk*, you obviously won't be preserving sda6,
> and you probably aren't referencing it by partition number ("fstabkey:uuid"),
> so does using 'primary' instead of 'logical' for all three partitions change
> anything concerning the error you are experiencing?
>
> Hope that helps someone (perhaps me ;-) ...
>
> Regards,
> Holger
Hello,

yes, you are right - in some way, this *is* a legacy windows issue that
has developed over time. In fact, the preserved partition once was a FAT
partition as long as we had dualboot installations on these machines. We
finally removed the dualboot two or three years ago and chose to format
this partition ext4 instead. Why we didn't move to primary partitions or
put it within the LVM at that time - I don't know.

On the other hand, up to now we had no problems with that, so no urge to
change anything. If you think it might help, I could try changing this.

Bye,
Andreas

Re: setup-storage fails on blank disk

[Thomas Lange]

> On Wed, 3 Jan 2018 13:53:40 +0100, Andreas Heinlein  
> said:

> I have encountered a problem with setup-storage which occurs only when
> the disk is blank, i.e. wiped with nwipe/dban or brand new.
This is very strange. I would only expect problems, if the disk is not wiped.


> It then
> fails on creating the LVM; running 'pvcreate' returns 'cannot open
> /dev/sda5 exclusively'. I have attached the fai.log with all the details.
> When I reboot the machine, which now has a partition table in place,
> everything works fine. Same for reinstalling machines which were already
> installed with earlier versions of FAI.
I took your disk_config file from the fai.log and did an installation
with FAI 5.5, kernel 4.9 and dracut 045+132-1 inside the nfsroot. Then
I started a new kvm machine with a fresh, empty disk. Everything works
fine. No problems.

So I cannot reproduce your error.
-- 
regards Thomas

setup-storage fails on blank disk

[Andreas Heinlein]

Hello,

I have encountered a problem with setup-storage which occurs only when
the disk is blank, i.e. wiped with nwipe/dban or brand new. It then
fails on creating the LVM; running 'pvcreate' returns 'cannot open
/dev/sda5 exclusively'. I have attached the fai.log with all the details.
When I reboot the machine, which now has a partition table in place,
everything works fine. Same for reinstalling machines which were already
installed with earlier versions of FAI.
I can't exactly tell which FAI version we had in use when we last set up
a brand new machine - might be 5.3, might be even earlier, so the error
may already exist for a while.

Thanks,
Andreas
 -
   Fully Automatic Installation  -  FAI

   5.5   (c) 1999-2017
   Thomas Lange  
 -

Calling task_confdir
Kernel currently running: Linux 4.9.0-4-amd64 x86_64 GNU/Linux
Kernel parameters: BOOT_IMAGE=vmlinuz-4.9.0-4-amd64 initrd=initrd.img-4.9.0-4-amd64 ip=dhcp root=/srv/fai/nfsroot rootovl FAI_FLAGS=verbose,createvt,sshd FAI_ACTION=install quiet FAI_CONFIG_SRC=nfs://***/srv/fai/config
Reading /tmp/fai/boot.log
FAI_FLAGS: verbose createvt sshd
Setting SERVER=***. Value extracted from FAI_CONFIG_SRC.
Can't connect to monserver on *** port 4711. Monitoring disabled.
FAI_CONFIG_SRC is set to nfs://***/srv/fai/config
Configuration space ***:/srv/fai/config mounted to /var/lib/fai/config
Source hook: setup.DEFAULT.sh
setup.DEFAULT.sh OK.
Calling task_setup
FAI_FLAGS: verbose createvt sshd
 3 Jan 13:08:06 ntpdate[991]: step time server *** offset -0.064882 sec
Press ctrl-c to interrupt FAI and to get a shell
Starting FAI execution - 20180103_130806
Calling task_defclass
fai-class: Defining classes.
Executing /var/lib/fai/config/class/01-classes.
01-classes   OK.
Executing /var/lib/fai/config/class/10-base-classes.
10-base-classes  OK.
Executing /var/lib/fai/config/class/20-hwdetect.sh.
Loading kernel module mptspi
Loading kernel module dm-mod
Loading kernel module md-mod
Loading kernel module aes
Loading kernel module dm-crypt
2: eth0:  mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:5e:c0:3e brd ff:ff:ff:ff:ff:ff
inet 172.16.9.236/24 brd 172.16.9.255 scope global eth0
inet6 fe80::a00:27ff:fe5e:c03e/64 scope link 
New disklist: sda
20-hwdetect.sh   OK.
Executing /var/lib/fai/config/class/40-parse-profiles.sh.
40-parse-profiles.sh OK.
Executing /var/lib/fai/config/class/41-warning.sh.
41-warning.shOK.
Executing /var/lib/fai/config/class/50-host-classes.
50-host-classes  OK.
Executing /var/lib/fai/config/class/58-hardware.
58-hardware  OK.
Executing /var/lib/fai/config/class/60-misc.
60-misc  OK.
List of all classes:  DEFAULT LINUX AMD64 DHCPC PRECISE PRECISE_HOMI GERMAN FAICLIENT GRUB_PC ... LAST
Calling task_defvar
Executing PRECISE.var
++ FAI_ALLOW_UNSIGNED=1
++ CONSOLEFONT=
++ KEYMAP=us-latin1
++ UTC=yes
++ TIMEZONE=Europe/Berlin
++ ROOTPW=X
++ MODULESLIST='usbkbd ehci-hcd ohci-hcd uhci-hcd usbhid psmouse fuse'
++ SUDO_FORCE_REMOVE=yes
++ MAXPACKAGES=1
++ UCF_FORCE_CONFFOLD=1
Executing GERMAN.var
++ KEYMAP=de-latin1-nodeadkeys
Defining variables from additional.var
++ disklist='sda '
Loading keymap(s) de-latin1-nodeadkeys ...done.
Calling task_action
FAI_ACTION: install
Performing FAI installation. All data may be overwritten!
Calling task_install
Calling task_partition
Starting setup-storage 2.1
Using config file: /var/lib/fai/config/disk_config/PRECISE
Parted could not read a disk label (new disk?)
Executing: parted -s /dev/sda mklabel msdos
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
Creating directory "/run/lock/lvm"
No volume groups found.
Executing: parted -s /dev/sda mklabel msdos
Executing: parted -s /dev/sda mkpart primary "ext3" 1048576B 315621375B
Executing: parted -s /dev/sda set 1 boot on
Executing: parted -s /dev/sda mkpart extended "" 315621376B 42949672959B
Executing: parted -s /dev/sda mkpart logical "" 316669952B 31380733951B
Executing: parted -s /dev/sda mkpart logical "ext3" 31381782528B 42949672959B
Executing: mkfs.ext4 -O ^64bit,^metadata_csum /dev/sda1
Executing: mkfs.ext4 -O ^64bit,^metadata_csum -L Daten /dev/sda6
Executing: parted -s /dev/sda set 5 lvm on
Executing: pvcreate -ff -y  /dev/sda5
(STDERR)   WARNING: Failed to connect to lvmetad. Falling back to device scanning.
(STDERR)   Can't open /dev/sda5 exclusively.  Mounted filesystem?
Command had non-zero exit code

This is your disk_config file:
# generic disk configuration for one small disk
# disk size from 500Mb up to what you can buy today
#
#   [extra options]

disk_config disk1 disklabel:msdos bootable:1 preserve_lazy:6 align-at:1M fstabkey:uuid 
primary  /boot	   300	ext4	rw	createopts="-O