[gentoo-user] Re: MBR & GPT dual compliant format

[James]

R0b0t1  gmail.com> writes:

> On Jul 22, 2016 5:43 PM, "Neil Bothwick"  digimed.co.uk> wrote:
> > I take it this is a limitation of Apple's firmware as I have set up a
> > number of uUEFI systems and never had to do this.

> It is.


There is another document that talks in depth about the issue, although
it was centric to using gpt disk on a bios world that was slowly moving
to efi [1].


[1] http://mjg59.dreamwidth.org/8035.html

Here is the essence::
"But most BIOSes (and most older operating systems) don't understand GPT, so
plugging in a GPT-partitioned disk would result in the system believing that
the drive was uninitialised. This is avoided by specifying a protective MBR.
This is a valid MBR partition table with a single partition covering the
entire disk (or the first 2.2TB of the disk if it's larger than that) and
the partition type set to 0xee ("GPT Protective"). GPT-unaware BIOSes and
operating systems will see a partition they don't understand and simply
ignore it."


I do not know how to set up a 'protective MBR', that's my issue. This
reference goes on to talk about how the code was written for parted but
never made the permanent status. It sure would fix a lot of installation
issues among many different distros. An excellent read, if anyone has the
time. Me, I'm going to use this method::

1. First, write an example of what the partition table should look like.

2. Figure out the separate tools & sequences to achieve the final result.

3. Document the steps so they are clearly available for our community.

4. Hope that one of the devs/hackers spins a patched version of a "parted"
formatting tool to achieve this ability, system-rescue seems  to be the best
home. Or if a patched parted only lives in an overlay, that would ease quite
a lot of pain for many folks as in my research experience, setting up the
disk partitioning schemes is the toughest part of an installation these
days. This duality of disk usage  is critical to my cluster testing schema.
I'll  also have a variety of bootstap codes to deal with from various
embedded systems, in addition to commonly purchased hardware platforms, so
extending the formatting to other forms of storage, in a consistent and
generic way, provides an even greater appeal.

>From the same doc::
"It violates the spec and it confuses the majority of partitioning tools. I
wrote some code to make parted do it at one point, but I don't believe it
was ever merged. It's very difficult to make it work well. "

They discuss also some of the MAC family of issues and explain why macs
still suffer from this malaise. I hope that code is still around


Thanks for all the advice and help.
James

Re: [gentoo-user] nfsv4 issues

[Adam Carter]

> >> I don't use systemd on Gentoo but for the nfs-utils upstream-shipped
> >> systemd units that I think that Gentoo's using, you have to re-run
> >> nfs-config.service - or run the script that it calls - in order to
> >> update the "/run/sysconfig/nfs-utils" environment file that's sourced
> >> by the nfs-server.service unit.
> >
> > In /usr/lib/systemd/system/nfs-server.service
> > [Service]
> > EnvironmentFile=/etc/conf.d/nfs
>
> Sorry. Looking at the ebuild, there's:
>
> 
> rm "${D}$(systemd_get_unitdir)"/nfs-config.service || die
> sed -i -r \
> -e "/^EnvironmentFile=/s:=.*:=${EPREFIX}/etc/conf.d/nfs:" \
> -e '/^(After|Wants)=nfs-config.service$/d' \
> -e 's:/usr/sbin/rpc.statd:/sbin/rpc.statd:' \
> "${D}$(systemd_get_unitdir)"/* || die
> 
>
> so the upstream "nfs-config.service" waltz is avoided.
>
> But that means that the variables in "/etc/conf.d/nfs" aren't renamed.
> So the openrc nfs script uses "${OPTS_RPC_NFSD}", which is defined,
> and the systemd service uses "$RPCNFSDARGS", which isn't.
>

I've added $RPCNFSDARGS to /etc/conf.d/nfs, restarted, and the nproc
setting works.

>
> >> Does "/var/lib/nfs/v4recovery/" exist?
> >
> > No
> > # ls /var/lib/nfs/
> > etab export-lock rmtab rpc_pipefs sm sm.bak state xtab
>
> IIRC, it's needed to avoid this delay. I thought that I'd saved a url
> about this but I can't find it.
>
> Do you have a syslog message about "stable storage"? "man nfsdcltrack".
>

There's no message about stable storage, but there's this;
kernel: [578030.628415] NFSD: the nfsdcld client tracking upcall will be
removed in 3.10. Please transition to using nfsdcltrack.

# which nfsdcltrack
which: no nfsdcltrack in
(/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/5.4.0:/usr/lib64/subversion/bin:/opt/vmware/bin)
# qlist nfs | grep nfsdcltrack
#


> The openrc script has
>
> 
> mkdir_nfsdirs() {
> local d
> for d in v4recovery v4root ; do
> d="/var/lib/nfs/${d}"
> [ ! -d "${d}" ] && mkdir -p "${d}"
> done
> }
> 
>
> but systemd doesn't have anything equivalent. On RHEL and Ubuntu,
> "/var/lib/nfs/v4recovery/" is created at installation time. Perhaps
> the Gentoo ebuild should do the same or should ship a
> "/usr/lib/tmpfiles.d/var-lib-nfs.conf" to create it at boot if it
> doesn't exist.
>
> I've added the directory, and after restarting syslog now has new entries;
kernel: [912267.948883] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4
state recovery directory
kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery
directory

I will test shortly and report back - thanks!

Re: [gentoo-user] MBR & GPT dual compliant format

[R0b0t1]

On Jul 22, 2016 5:43 PM, "Neil Bothwick"  wrote:
> I take it this is a limitation of Apple's firmware as I have set up a
> number of uUEFI systems and never had to do this.
>

It is.

Re: [gentoo-user] MBR & GPT dual compliant format

[Neil Bothwick]

On Fri, 22 Jul 2016 14:53:47 -0500, R0b0t1 wrote:

> You need to set the bootable flag in the protective MBR.
> 
> I had to use gdisk and fdisk to make a partition that was bootable by
> Apple's EFI. The proper setting does not seem to exist in gdisk, even
> though gdisk can read it (oversight by the author?).

I take it this is a limitation of Apple's firmware as I have set up a
number of uUEFI systems and never had to do this.


-- 
Neil Bothwick

Men who go out with flat chested woman have reasons for feeling down


pgp6pqZs3yldo.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] MBR & GPT dual compliant format

[R0b0t1]

You need to set the bootable flag in the protective MBR.

I had to use gdisk and fdisk to make a partition that was bootable by
Apple's EFI. The proper setting does not seem to exist in gdisk, even
though gdisk can read it (oversight by the author?).

[gentoo-user] MBR & GPT dual compliant format

[James]

Hello,

I thought I'd post a new thread on this issue. My goals is to have a single
default partition scheme on a sata disk that allows me to use either
Bios(mbr) or EFI(gpt) systems on these drives. Also  the goal is to keep the
partition scheme unchanged (boot;root;swap;'usr/local') but be able to set
up different file systems  and distributed file systems on these drives to
facilitate testing a wide variety of cluster architectures. I'd sure
appreciate some 'thinking outside the box' ideas for these mostly 2T sata
drives. I intend to only use one bootloader (grub legacy) but that is 
not a fixed limitation. /usr/local will be the only storage if one of these
drives is used for secondary or additional capacity, thus preserving the
partition scheme.

Here is the essences of what Neil posted before::

You can use gdisk and a GPT whetheryou are using BIO or EFI. The 
difference is in your first partition. For EFI it must be type EF00 and 
formatted with FAT. For BIOS booting you need to start the disk with 
a small BIOS compatibility partition of type EF02. This is 1M here and you
don't format or use it, it just has to be there. And.

I'm not sure it can be done. BIOS needs an EF00 partition at the start.
EFI calls for an EF00 partition, which is recommended at the start but I
don't think it's compulsory that it is there. I have heard of people
using sda2 as the ESP where sda1 is a Windows rescue partition. So you
may get away with

p1 EF02 partition
p2 EF00 partition, formatted as FAT and mounted at /boot
root and swap partitions as you see fit.

You could try it and see, but I'm not sure it could be guaranteed to work
on all EFI hardware, although it should work on all BIOS hardware

I'd use gdisk to set the partition type to get the partition table listing
to include :: (EF02, EF00)?

This doc seems to suggest there is a way to configure such disks [1], as do
other docs I have read, but do not give explicit examples just how to do this.

So is at [1]:
https://wiki.manjaro.org/index.php?title=Some_basics_of_MBR_
v/s_GPT_and_BIOS_v/s_UEFI


Now, I should use GPT (gdisk) and label and setup the disk complete with a
gpt labeled table, but preserve the MBR for legacy booting of most bios
based systems? If that is correct, then here is what the partition scheme
could look like, for a 2T drive::

Number  Start (sector)End (sector)  Size   Code  Name
   12048  411647   200.0 MiB   EFO2  FAT
   2  411648   270747647   128.9 GiB   8200  Linux swap
   3   270747648  1859022847   757.3 GiB   8300  Linux filesystem
   4  1859022848  3907022847   976.6 GiB   8300  Linux filesystem

Maybe someone can edit this table and show me an example (no worries on
boundaries or sizes) and include a few sentences to explain and guide me
on this effort?


curiously,
James

Re: [gentoo-user] nfsv4 issues

[Tom H]

On Wed, Jul 20, 2016 at 10:51 PM, Adam Carter  wrote:


>> I don't use systemd on Gentoo but for the nfs-utils upstream-shipped
>> systemd units that I think that Gentoo's using, you have to re-run
>> nfs-config.service - or run the script that it calls - in order to
>> update the "/run/sysconfig/nfs-utils" environment file that's sourced
>> by the nfs-server.service unit.
>
> In /usr/lib/systemd/system/nfs-server.service
> [Service]
> EnvironmentFile=/etc/conf.d/nfs

Sorry. Looking at the ebuild, there's:


rm "${D}$(systemd_get_unitdir)"/nfs-config.service || die
sed -i -r \
-e "/^EnvironmentFile=/s:=.*:=${EPREFIX}/etc/conf.d/nfs:" \
-e '/^(After|Wants)=nfs-config.service$/d' \
-e 's:/usr/sbin/rpc.statd:/sbin/rpc.statd:' \
"${D}$(systemd_get_unitdir)"/* || die


so the upstream "nfs-config.service" waltz is avoided.

But that means that the variables in "/etc/conf.d/nfs" aren't renamed.
So the openrc nfs script uses "${OPTS_RPC_NFSD}", which is defined,
and the systemd service uses "$RPCNFSDARGS", which isn't.


>> Does "/var/lib/nfs/v4recovery/" exist?
>
> No
> # ls /var/lib/nfs/
> etab export-lock rmtab rpc_pipefs sm sm.bak state xtab

IIRC, it's needed to avoid this delay. I thought that I'd saved a url
about this but I can't find it.

Do you have a syslog message about "stable storage"? "man nfsdcltrack".

The openrc script has


mkdir_nfsdirs() {
local d
for d in v4recovery v4root ; do
d="/var/lib/nfs/${d}"
[ ! -d "${d}" ] && mkdir -p "${d}"
done
}


but systemd doesn't have anything equivalent. On RHEL and Ubuntu,
"/var/lib/nfs/v4recovery/" is created at installation time. Perhaps
the Gentoo ebuild should do the same or should ship a
"/usr/lib/tmpfiles.d/var-lib-nfs.conf" to create it at boot if it
doesn't exist.

Re: [gentoo-user] Re: GPT newbee needs some help

[Helmut Jarausch]

On 07/22/2016 10:49:35 AM, Dmitry Bogun wrote:

Look like you don't have gpt support in kernel.


Many thanks Dmitry,
that was the problem.

Since I have a somewhat older mother board with no UEFI support, I  
couldn't image why I need the

EFI GUID Partition support
setting for my kernel.
I have set this now and the new kernel does see my partitions.

Many thanks again,
Helmut

Re: [gentoo-user] Re: GPT newbee needs some help

[Dmitry Bogun]

Look like you don't have gpt support in kernel.

Post output from command "gunzip -c /proc/config.gz | grep '_PARTITION\>'"

> On Jul 22, 2016, at 11:37 AM, Helmut Jarausch  wrote:
> 
> On 07/22/2016 10:28:35 AM, Neil Bothwick wrote:
>> On Fri, 22 Jul 2016 10:04:58 +0200, Helmut Jarausch wrote:
>> > I have zeroed the first 8 MB and then I used gdisk
>> > gdisk still notes that there is a backup GPT. I opted to created a new
>> > blank GPT.
>> > Then I created 4 partitions.
>> > I have used the w(rite) command before exiting gdisk.
>> > Starting gdisk again, it shows the 4 partitions.
>> You still haven't showed us the output from gdisk -l /dev/sde
> 
> OK, here it is:
> 
> gdisk -l /dev/sde
> GPT fdisk (gdisk) version 1.0.1
> 
> Partition table scan:
>  MBR: protective
>  BSD: not present
>  APM: not present
>  GPT: present
> 
> Found valid GPT with protective MBR; using GPT.
> Disk /dev/sde: 9767541167 sectors, 4.5 TiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): A072CFE0-0651-441C-8BA2-8527623BA142
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 9767541133
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 2014 sectors (1007.0 KiB)
> 
> Number  Start (sector)End (sector)  Size   Code  Name
>   12048  3145730047   1.5 TiB 8300  Linux filesystem
>   2  3145730048  5293213695   1024.0 GiB  8300  Linux filesystem
>   3  5293213696  7440697343   1024.0 GiB  8300  Linux filesystem
>   4  7440697344  9767541133   1.1 TiB 8300  Linux filesystem
> 

Re: [gentoo-user] Re: GPT newbee needs some help

[Helmut Jarausch]

On 07/22/2016 10:28:35 AM, Neil Bothwick wrote:

On Fri, 22 Jul 2016 10:04:58 +0200, Helmut Jarausch wrote:

> I have zeroed the first 8 MB and then I used gdisk
> gdisk still notes that there is a backup GPT. I opted to created a  
new

> blank GPT.
> Then I created 4 partitions.
> I have used the w(rite) command before exiting gdisk.
> Starting gdisk again, it shows the 4 partitions.

You still haven't showed us the output from gdisk -l /dev/sde


OK, here it is:

gdisk -l /dev/sde
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sde: 9767541167 sectors, 4.5 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): A072CFE0-0651-441C-8BA2-8527623BA142
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 9767541133
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)

Number  Start (sector)End (sector)  Size   Code  Name
   12048  3145730047   1.5 TiB 8300  Linux  
filesystem
   2  3145730048  5293213695   1024.0 GiB  8300  Linux  
filesystem
   3  5293213696  7440697343   1024.0 GiB  8300  Linux  
filesystem
   4  7440697344  9767541133   1.1 TiB 8300  Linux  
filesystem

Re: [gentoo-user] Re: GPT newbee needs some help

[Neil Bothwick]

On Fri, 22 Jul 2016 10:04:58 +0200, Helmut Jarausch wrote:

> I have zeroed the first 8 MB and then I used gdisk
> gdisk still notes that there is a backup GPT. I opted to created a new  
> blank GPT.
> Then I created 4 partitions.
> I have used the w(rite) command before exiting gdisk.
> Starting gdisk again, it shows the 4 partitions.

You still haven't showed us the output from gdisk -l /dev/sde


-- 
Neil Bothwick

I'm writing a book. I've got the page numbers done.


pgpDN2ryqDs5u.pgp
Description: OpenPGP digital signature

Re: [gentoo-user] Re: GPT newbee needs some help

[Helmut Jarausch]

Thanks to all of you who have tried to help.
Unfortunately, I am still lost.

I just want to run Gentoo on my system, and the new drive is just for  
backup, i.e. it needn't be bootable.


I have zeroed the first 8 MB and then I used gdisk
gdisk still notes that there is a backup GPT. I opted to created a new  
blank GPT.

Then I created 4 partitions.
I have used the w(rite) command before exiting gdisk.
Starting gdisk again, it shows the 4 partitions.
But, after a 'sync' command I detached and re-attached the drive and I  
get partially strange

output from dmesg :


[ 2225.690410] usb 9-1: New USB device found, idVendor=0bc2,  
idProduct=ab34
[ 2225.690418] usb 9-1: New USB device strings: Mfr=2, Product=3,  
SerialNumber=1

[ 2225.690423] usb 9-1: Product: Backup+  Desk
[ 2225.690426] usb 9-1: Manufacturer: Seagate
[ 2225.690430] usb 9-1: SerialNumber: NA7EV58E
[ 2225.692007] usb-storage 9-1:1.0: USB Mass Storage device detected
[ 2225.692167] scsi host8: usb-storage 9-1:1.0
[ 2226.693728] scsi 8:0:0:0: Direct-Access Seagate  Backup+   
Desk040B PQ: 0 ANSI: 6

[ 2226.694322] sd 8:0:0:0: Attached scsi generic sg5 type 0
[ 2226.696829] sd 8:0:0:0: [sde] Spinning up disk...
16 # dmesg | tail
[ 2240.741282] sd 8:0:0:0: [sde] Very big device. Trying to use READ  
CAPACITY(16).
[ 2240.741400] sd 8:0:0:0: [sde] 9767541167 512-byte logical blocks:  
(5.00 TB/4.55 TiB)

[ 2240.741405] sd 8:0:0:0: [sde] 2048-byte physical blocks
[ 2240.791085] sd 8:0:0:0: [sde] Write Protect is off
[ 2240.791096] sd 8:0:0:0: [sde] Mode Sense: 4f 00 00 00
[ 2240.791853] sd 8:0:0:0: [sde] Write cache: enabled, read cache:  
enabled, doesn't support DPO or FUA
[ 2240.792897] sd 8:0:0:0: [sde] Very big device. Trying to use READ  
CAPACITY(16).

[ 2240.832400]  sde: sde1
[ 2240.832943] sd 8:0:0:0: [sde] Very big device. Trying to use READ  
CAPACITY(16).

[ 2240.835581] sd 8:0:0:0: [sde] Attached SCSI disk
17 # dmesg | tail
[ 2240.741282] sd 8:0:0:0: [sde] Very big device. Trying to use READ  
CAPACITY(16).
[ 2240.741400] sd 8:0:0:0: [sde] 9767541167 512-byte logical blocks:  
(5.00 TB/4.55 TiB)

[ 2240.741405] sd 8:0:0:0: [sde] 2048-byte physical blocks
[ 2240.791085] sd 8:0:0:0: [sde] Write Protect is off
[ 2240.791096] sd 8:0:0:0: [sde] Mode Sense: 4f 00 00 00
[ 2240.791853] sd 8:0:0:0: [sde] Write cache: enabled, read cache:  
enabled, doesn't support DPO or FUA
[ 2240.792897] sd 8:0:0:0: [sde] Very big device. Trying to use READ  
CAPACITY(16).

[ 2240.832400]  sde: sde1
[ 2240.832943] sd 8:0:0:0: [sde] Very big device. Trying to use READ  
CAPACITY(16).

[ 2240.835581] sd 8:0:0:0: [sde] Attached SCSI disk

and only  /dev/sde and /dev/sde1 are visible.

Where are the other partitions - or am I missing a kernel option.
That is my first drive bigger than 3 TB.

Many thanks for your help,
Helmut