Re: DISKLABEL_EI option for system with MBR

[Rin Okuyama]

At the moment, there is no definite objection to the patch. I would
like to commit it in this weekend. Since this option is potentially
dangerous, as pointed out by Michael, it will be enabled only for
ALL on amd64 and i386, and commented out for other kernel
configuration files by default.

Thanks,
rin

Re: DISKLABEL_EI option for system with MBR

[Rin Okuyama]

On 2017/02/16 5:47, Michael van Elst wrote:

On Wed, Feb 15, 2017 at 09:27:16AM +, David Brownlee wrote:

On 12 February 2017 at 11:57, Rin Okuyama  wrote:

Michael, Martin, thank you for letting me know about wedge(4).
It is exactly what I need! It is more portable than my patch.
I withdraw the patch and the PR.


I think that DISKLABEL_EI would still be a good idea - as it would
make other endian disklabels Just Work for people (including easy
fstab usage)


I agree. It's not 100% compatible, but useful enough to allow some
confusion.


David, Michael, thank you for your comments. I agree that it would be
still useful, but also potentially dangerous. How about turning on it
only for ALL on x86, and commented out for other kernel configuration
files?

Then, going back to Michael's question:

On 2017/02/12 17:13, Michael van Elst wrote:

Saying that, why limit it to platforms using MBR?


This is because routines that read and write disklabels are almost MD.
The only exceptions are these for ports using MBR, that are provided in
kern/subr_disk_mbr.c. The target of my patch is this file. If someone
needs, she or he can easily port it to other ports. It would be nice if
we can refactor routines into kern/subr_disk.c. But it probably makes
codes unnecessarily complicated.

Thanks,
rin

Re: DISKLABEL_EI option for system with MBR

[Michael van Elst]

On Wed, Feb 15, 2017 at 09:27:16AM +, David Brownlee wrote:
> On 12 February 2017 at 11:57, Rin Okuyama  wrote:
> > Michael, Martin, thank you for letting me know about wedge(4).
> > It is exactly what I need! It is more portable than my patch.
> > I withdraw the patch and the PR.
> 
> I think that DISKLABEL_EI would still be a good idea - as it would
> make other endian disklabels Just Work for people (including easy
> fstab usage)

I agree. It's not 100% compatible, but useful enough to allow some
confusion.


-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: DISKLABEL_EI option for system with MBR

[David Holland]

On Tue, Feb 14, 2017 at 04:10:11PM -0500, Mouse wrote:
 > > [B]ased on things I saw when hacking lfs last year (all of which got
 > > fixed), I wouldn't rely on FFS_EI until someone gives it a good
 > > thorough audit, preferably with some kind of automated checking tool.
 > 
 > What sort of things?  (If you care to say, of course.)  I've been using
 > FFS_EI semi-routinely (as in, I build ~all my kernels with it, and,
 > while I don't want it all the time, when I do I haven't hesitated) and
 > have never seen any issues I can ascribe to it.

I found places in ufs/ where the byte swaps were missing. This will in
general fail silently unless you happen to read through such a path
and notice you got the wrong thing... or write through such a path and
notice that the volume has been corrupted.

I totally forget what they were, but probably it can be dug out of the
source-changes archive.

In any event because of the large number of code sites that touch
things that need to be swapped, without some kind of automated
crosscheck it's practically a given that some of them are wrong.

 > I haven't been trying to stress it, though.  Also, depending on what
 > vintage you're talking about, it may have got broken - or broken worse
 > - in between the versions I use and the versions you looked at.

Indeed.

-- 
David A. Holland
dholl...@netbsd.org

Re: DISKLABEL_EI option for system with MBR

[David Brownlee]

On 12 February 2017 at 11:57, Rin Okuyama  wrote:
> Michael, Martin, thank you for letting me know about wedge(4).
> It is exactly what I need! It is more portable than my patch.
> I withdraw the patch and the PR.

I think that DISKLABEL_EI would still be a good idea - as it would
make other endian disklabels Just Work for people (including easy
fstab usage)

Re: DISKLABEL_EI option for system with MBR

[Manuel Bouyer]

On Tue, Feb 14, 2017 at 04:10:11PM -0500, Mouse wrote:
> > [B]ased on things I saw when hacking lfs last year (all of which got
> > fixed), I wouldn't rely on FFS_EI until someone gives it a good
> > thorough audit, preferably with some kind of automated checking tool.
> 
> What sort of things?  (If you care to say, of course.)  I've been using
> FFS_EI semi-routinely (as in, I build ~all my kernels with it, and,
> while I don't want it all the time, when I do I haven't hesitated) and
> have never seen any issues I can ascribe to it.

also there are some atf tests for it

-- 
Manuel Bouyer 
 NetBSD: 26 ans d'experience feront toujours la difference
--

Re: DISKLABEL_EI option for system with MBR

[Mouse]

> [B]ased on things I saw when hacking lfs last year (all of which got
> fixed), I wouldn't rely on FFS_EI until someone gives it a good
> thorough audit, preferably with some kind of automated checking tool.

What sort of things?  (If you care to say, of course.)  I've been using
FFS_EI semi-routinely (as in, I build ~all my kernels with it, and,
while I don't want it all the time, when I do I haven't hesitated) and
have never seen any issues I can ascribe to it.

I haven't been trying to stress it, though.  Also, depending on what
vintage you're talking about, it may have got broken - or broken worse
- in between the versions I use and the versions you looked at.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: DISKLABEL_EI option for system with MBR

[Rin Okuyama]

Michael, Martin, thank you for letting me know about wedge(4).
It is exactly what I need! It is more portable than my patch.
I withdraw the patch and the PR.

Thanks,
rin

Re: DISKLABEL_EI option for system with MBR

[Martin Husemann]

On Sun, Feb 12, 2017 at 11:23:52AM +0900, Rin Okuyama wrote:
> Currently, we have FFS_EI and LFS_EI kernel options, that enable us to
> mount UFS partitions written in the different byte order. However,
> unfortunately, the system does not recognize disklabels written in the
> different byte order.

Not objecting to the patch...

I think you can use dkscan_bsdlabel(8) to create wedges from the wrong
endian disklabel and then mount the wedges. Haven't tested that though
(and it is not automatically handled by rc.d either, so not a permanent
solution).

Martin

Re: DISKLABEL_EI option for system with MBR

[Michael van Elst]

dholland-t...@netbsd.org (David Holland) writes:

>On Sun, Feb 12, 2017 at 11:06:35AM +0800, Paul Goyette wrote:
> > 5. Does the current FFS_EI allow for creation of opposite-endian
> >file systems?  I don't see any endian option for newfs(8).

>Not that I know of. Also, based on things I saw when hacking lfs last
>year (all of which got fixed), I wouldn't rely on FFS_EI until someone
>gives it a good thorough audit, preferably with some kind of automated
>checking tool.

FFS_EI works fine. I regularly exchange disk images between amd64 (host,
little endian) and amiga (emulated guest, big endian). Obviously that's
no audit.

You could re-run the atf filesystem tests on opposite endian images.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: DISKLABEL_EI option for system with MBR

[Michael van Elst]

p...@whooppee.com (Paul Goyette) writes:

>4. Where on the physical device is the wedge configuration saved,
>anyway?)

There is no special wedge configuration on the disk. Wedges are
generated from labels. There is code to handle GPT, MBR and BSD disklabel,
the latter is not configured in GENERIC.

If you enable BSD disklabel discovery for wedges, it will accept
labels in either endianess.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: DISKLABEL_EI option for system with MBR

[Michael van Elst]

rokuy...@rk.phys.keio.ac.jp (Rin Okuyama) writes:

>The patch attached to tech-kern/51208 adds DISKLABEL_EI option for ports
>using MBR. By this option, the system can mount the disk (image) written
>in the different byte order. This should be useful for cross building.
>For bi-endian architectures, like evbarm, the little and big endian
>partitions can share a same disk by this option.

I agree, but this may also confuse people when suddenly an existing
disklabel gets recognized.

N.B. the same is true when you enable wedges for bsdlabel. The discovery
code detects labels with either endianess.

Saying that, why limit it to platforms using MBR?

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: DISKLABEL_EI option for system with MBR

[David Holland]

On Sun, Feb 12, 2017 at 11:06:35AM +0800, Paul Goyette wrote:
 > 5. Does the current FFS_EI allow for creation of opposite-endian
 >file systems?  I don't see any endian option for newfs(8).

Not that I know of. Also, based on things I saw when hacking lfs last
year (all of which got fixed), I wouldn't rely on FFS_EI until someone
gives it a good thorough audit, preferably with some kind of automated
checking tool.

-- 
David A. Holland
dholl...@netbsd.org

Re: DISKLABEL_EI option for system with MBR

[Rin Okuyama]

Thank you very much for your comments.

On 2017/02/12 12:06, Paul Goyette wrote:

I don't really have any objections, but I would like to clarify the scope of 
what is being proposed.

1. Does this allow only processing of disk labels that are (a) already
   written in opposite-endian format?  Or does it also (b) provide a
   mechanism to write _new_ labels in opposite-endian format, too?

   If the answer is (a), is this sufficient?  Or would (b) be a
   desirable feature?


Neither (a) nor (b). With ioctls, new labels are written in the native
byte order. When we update labels, their byte order is kept. We can
write labels in any byte order by using -F option for disklabel(8)
(bypass ioctls and treat disks as regular files) combined with "-B le"
or "-B be" options.


2. Does this affect (c) only the disklabel(8) stuff?  Or does it also
   (d) affect the MBR itself (the stuff handled by fdisk(8))?

   Again, is (c) sufficient, or should we also do (d)?


The answer is (c). MBR stuff is always written in little endian.


3. Do wedges still work?  Does the patch allow opposite-endian for
   the wedge configuration data?


This patch does not take special care of wedges. I don't know much of
wedges, but opposite-endian for the wedge configuration data should
be supported by sys/dev/dkwedge if needed.


And a couple of side questions:

4. Where on the physical device is the wedge configuration saved,
   anyway?)


Sorry, I don't know.


5. Does the current FFS_EI allow for creation of opposite-endian
   file systems?  I don't see any endian option for newfs(8).


We can use "newfs -B le" or "newfs -B be".

I would appreciate if someone could give comments on wedges.

Thanks,
rin

DISKLABEL_EI option for system with MBR

[Rin Okuyama]

Hi,

I'd like to add DISKLABEL_EI kernel option, cf tech-kern/51208:

  http://gnats.netbsd.org/51208

Currently, we have FFS_EI and LFS_EI kernel options, that enable us to
mount UFS partitions written in the different byte order. However,
unfortunately, the system does not recognize disklabels written in the
different byte order.

The patch attached to tech-kern/51208 adds DISKLABEL_EI option for ports
using MBR. By this option, the system can mount the disk (image) written
in the different byte order. This should be useful for cross building.
For bi-endian architectures, like evbarm, the little and big endian
partitions can share a same disk by this option.

The patch seems almost trivial; if we encounter the disklabel written in
the different byte order, swap its order. The main patch is only ~250
lines including the manpage (The PR is very long since it also contains
the patch for kernel configuration files; it adds DISKLABEL_EI option
commented out except for ALL on amd64 and i386). Currently, this option
is restricted to ports with MBR. However, porting it to other ports may
also be trivial if someone needs.

I'd like to commit this patch if there's no objection (and enable it for
GENERIC kernels on i386 and amd64). Any comments?

Thanks,
rin