Re: Amiga RDB partition support for disks >= 2 TB
On 20180628 04:38, Martin Steigerwald wrote:
Martin Steigerwald - 28.06.18, 13:30:
jdow - 28.06.18, 12:00:
On 20180628 01:16, Martin Steigerwald wrote:
[…]
That brings to the fore an interesting question. Why bother with
RDBs
over 2TB unless you want a disk with one single partition? This
Win10
monster I am using has a modest BIOS driver partition for the OS
and
a giant data partition. That smaller partition would easily work
with
any RDB/Filesystem combination since 2.0. So there are some good
workarounds that are probably "safer" and at least as flexible as
RDBs, one Linux has used for a very long time, too.
Well, my use case was simple:
I had this 2 TB disk and I choose to share it as a backup disk for
Linux *and* AmigaOS 4.x on that Sam440ep I still have next to me
desk here.
EEK! The hair on my neck is standing up straight! Have you heard
of SAMBA? The linux mail server firewall etc machine has an extra
4TB
disk on it as a backup for the other systems, although a piddly 4TB
is small when I save the entire 3G RAID system I have. It's a proof
of concept so A full backup on a 1gig Ethernet still takes a
long time. But backing up even an 18GB disk on an Amiga via
100Base-t isn't too bad. And disk speeds of the era being what they
were it's about all you can do anyway.
Heh, the thing worked just fine in Amiga OS 4. I got away with it
without an issue, until I plugged the disk to my Linux laptop and
wrote data onto the Linux file system. Mind you, I think in that
partition marked LNX\0 I even created a Linux LVM with pvcreate. Do
you call that insane? Well it probably is. :)
And as an Amiga user I could just return to you: I clicked it, it did
not warn, so all is good :)
But yeah, as mentioned I researched the topic before. And I think
there
has not even been an overflow within the RDB:
The raw, theoretical limit on the maximum device capacity is about
2^105 bytes:
32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512
bytes/sector for the HD size in struct RigidDiskBlock
http://wiki.amigaos.net/wiki/RDB_(Amiga_Rigid_Disk_Block)
Confirmed by:
The .ADF (Amiga Disk File) format FAQ:
http://lclevy.free.fr/adflib/adf_info.html#p6
But what do I write, you know the RDB format :)
So just do the calculation in 96 Bit and you all are set :)
For sectors.
Or
3*32+9 (for 512 bytes per sector) = 105 bits
for bytes.
Now that is a reason for 128 Bit CPUs :).
Muuhahaha.
And if you make the logical block size 65536 you need 128 bits or 10^something
big = 2^128.
{^_-}
Re: Amiga RDB partition support for disks >= 2 TB
On 20180628 04:38, Martin Steigerwald wrote:
Martin Steigerwald - 28.06.18, 13:30:
jdow - 28.06.18, 12:00:
On 20180628 01:16, Martin Steigerwald wrote:
[…]
That brings to the fore an interesting question. Why bother with
RDBs
over 2TB unless you want a disk with one single partition? This
Win10
monster I am using has a modest BIOS driver partition for the OS
and
a giant data partition. That smaller partition would easily work
with
any RDB/Filesystem combination since 2.0. So there are some good
workarounds that are probably "safer" and at least as flexible as
RDBs, one Linux has used for a very long time, too.
Well, my use case was simple:
I had this 2 TB disk and I choose to share it as a backup disk for
Linux *and* AmigaOS 4.x on that Sam440ep I still have next to me
desk here.
EEK! The hair on my neck is standing up straight! Have you heard
of SAMBA? The linux mail server firewall etc machine has an extra
4TB
disk on it as a backup for the other systems, although a piddly 4TB
is small when I save the entire 3G RAID system I have. It's a proof
of concept so A full backup on a 1gig Ethernet still takes a
long time. But backing up even an 18GB disk on an Amiga via
100Base-t isn't too bad. And disk speeds of the era being what they
were it's about all you can do anyway.
Heh, the thing worked just fine in Amiga OS 4. I got away with it
without an issue, until I plugged the disk to my Linux laptop and
wrote data onto the Linux file system. Mind you, I think in that
partition marked LNX\0 I even created a Linux LVM with pvcreate. Do
you call that insane? Well it probably is. :)
And as an Amiga user I could just return to you: I clicked it, it did
not warn, so all is good :)
But yeah, as mentioned I researched the topic before. And I think
there
has not even been an overflow within the RDB:
The raw, theoretical limit on the maximum device capacity is about
2^105 bytes:
32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512
bytes/sector for the HD size in struct RigidDiskBlock
http://wiki.amigaos.net/wiki/RDB_(Amiga_Rigid_Disk_Block)
Confirmed by:
The .ADF (Amiga Disk File) format FAQ:
http://lclevy.free.fr/adflib/adf_info.html#p6
But what do I write, you know the RDB format :)
So just do the calculation in 96 Bit and you all are set :)
For sectors.
Or
3*32+9 (for 512 bytes per sector) = 105 bits
for bytes.
Now that is a reason for 128 Bit CPUs :).
Muuhahaha.
And if you make the logical block size 65536 you need 128 bits or 10^something
big = 2^128.
{^_-}
Re: Amiga RDB partition support for disks >= 2 TB
Martin Steigerwald - 28.06.18, 13:30: > jdow - 28.06.18, 12:00: > > On 20180628 01:16, Martin Steigerwald wrote: > […] > > > >> That brings to the fore an interesting question. Why bother with > > >> RDBs > > >> over 2TB unless you want a disk with one single partition? This > > >> Win10 > > >> monster I am using has a modest BIOS driver partition for the OS > > >> and > > >> a giant data partition. That smaller partition would easily work > > >> with > > >> any RDB/Filesystem combination since 2.0. So there are some good > > >> workarounds that are probably "safer" and at least as flexible as > > >> RDBs, one Linux has used for a very long time, too. > > > > > > Well, my use case was simple: > > > > > > I had this 2 TB disk and I choose to share it as a backup disk for > > > Linux *and* AmigaOS 4.x on that Sam440ep I still have next to me > > > desk here. > > > > EEK! The hair on my neck is standing up straight! Have you heard > > of SAMBA? The linux mail server firewall etc machine has an extra > > 4TB > > disk on it as a backup for the other systems, although a piddly 4TB > > is small when I save the entire 3G RAID system I have. It's a proof > > of concept so A full backup on a 1gig Ethernet still takes a > > long time. But backing up even an 18GB disk on an Amiga via > > 100Base-t isn't too bad. And disk speeds of the era being what they > > were it's about all you can do anyway. > > Heh, the thing worked just fine in Amiga OS 4. I got away with it > without an issue, until I plugged the disk to my Linux laptop and > wrote data onto the Linux file system. Mind you, I think in that > partition marked LNX\0 I even created a Linux LVM with pvcreate. Do > you call that insane? Well it probably is. :) > > And as an Amiga user I could just return to you: I clicked it, it did > not warn, so all is good :) > > But yeah, as mentioned I researched the topic before. And I think > there > has not even been an overflow within the RDB: > > The raw, theoretical limit on the maximum device capacity is about > > 2^105 bytes: > > > > 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512 > > bytes/sector for the HD size in struct RigidDiskBlock > > http://wiki.amigaos.net/wiki/RDB_(Amiga_Rigid_Disk_Block) > > Confirmed by: > > The .ADF (Amiga Disk File) format FAQ: > http://lclevy.free.fr/adflib/adf_info.html#p6 > > But what do I write, you know the RDB format :) > > So just do the calculation in 96 Bit and you all are set :) For sectors. Or 3*32+9 (for 512 bytes per sector) = 105 bits for bytes. > Now that is a reason for 128 Bit CPUs :). > > Muuhahaha. -- Martin
Re: Amiga RDB partition support for disks >= 2 TB
Martin Steigerwald - 28.06.18, 13:30: > jdow - 28.06.18, 12:00: > > On 20180628 01:16, Martin Steigerwald wrote: > […] > > > >> That brings to the fore an interesting question. Why bother with > > >> RDBs > > >> over 2TB unless you want a disk with one single partition? This > > >> Win10 > > >> monster I am using has a modest BIOS driver partition for the OS > > >> and > > >> a giant data partition. That smaller partition would easily work > > >> with > > >> any RDB/Filesystem combination since 2.0. So there are some good > > >> workarounds that are probably "safer" and at least as flexible as > > >> RDBs, one Linux has used for a very long time, too. > > > > > > Well, my use case was simple: > > > > > > I had this 2 TB disk and I choose to share it as a backup disk for > > > Linux *and* AmigaOS 4.x on that Sam440ep I still have next to me > > > desk here. > > > > EEK! The hair on my neck is standing up straight! Have you heard > > of SAMBA? The linux mail server firewall etc machine has an extra > > 4TB > > disk on it as a backup for the other systems, although a piddly 4TB > > is small when I save the entire 3G RAID system I have. It's a proof > > of concept so A full backup on a 1gig Ethernet still takes a > > long time. But backing up even an 18GB disk on an Amiga via > > 100Base-t isn't too bad. And disk speeds of the era being what they > > were it's about all you can do anyway. > > Heh, the thing worked just fine in Amiga OS 4. I got away with it > without an issue, until I plugged the disk to my Linux laptop and > wrote data onto the Linux file system. Mind you, I think in that > partition marked LNX\0 I even created a Linux LVM with pvcreate. Do > you call that insane? Well it probably is. :) > > And as an Amiga user I could just return to you: I clicked it, it did > not warn, so all is good :) > > But yeah, as mentioned I researched the topic before. And I think > there > has not even been an overflow within the RDB: > > The raw, theoretical limit on the maximum device capacity is about > > 2^105 bytes: > > > > 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512 > > bytes/sector for the HD size in struct RigidDiskBlock > > http://wiki.amigaos.net/wiki/RDB_(Amiga_Rigid_Disk_Block) > > Confirmed by: > > The .ADF (Amiga Disk File) format FAQ: > http://lclevy.free.fr/adflib/adf_info.html#p6 > > But what do I write, you know the RDB format :) > > So just do the calculation in 96 Bit and you all are set :) For sectors. Or 3*32+9 (for 512 bytes per sector) = 105 bits for bytes. > Now that is a reason for 128 Bit CPUs :). > > Muuhahaha. -- Martin
Re: Amiga RDB partition support for disks >= 2 TB
jdow - 28.06.18, 12:00: > On 20180628 01:16, Martin Steigerwald wrote: […] > >> That brings to the fore an interesting question. Why bother with > >> RDBs > >> over 2TB unless you want a disk with one single partition? This > >> Win10 > >> monster I am using has a modest BIOS driver partition for the OS > >> and > >> a giant data partition. That smaller partition would easily work > >> with > >> any RDB/Filesystem combination since 2.0. So there are some good > >> workarounds that are probably "safer" and at least as flexible as > >> RDBs, one Linux has used for a very long time, too. > > > > Well, my use case was simple: > > > > I had this 2 TB disk and I choose to share it as a backup disk for > > Linux *and* AmigaOS 4.x on that Sam440ep I still have next to me > > desk here. > EEK! The hair on my neck is standing up straight! Have you heard > of SAMBA? The linux mail server firewall etc machine has an extra 4TB > disk on it as a backup for the other systems, although a piddly 4TB > is small when I save the entire 3G RAID system I have. It's a proof > of concept so A full backup on a 1gig Ethernet still takes a > long time. But backing up even an 18GB disk on an Amiga via > 100Base-t isn't too bad. And disk speeds of the era being what they > were it's about all you can do anyway. Heh, the thing worked just fine in Amiga OS 4. I got away with it without an issue, until I plugged the disk to my Linux laptop and wrote data onto the Linux file system. Mind you, I think in that partition marked LNX\0 I even created a Linux LVM with pvcreate. Do you call that insane? Well it probably is. :) And as an Amiga user I could just return to you: I clicked it, it did not warn, so all is good :) But yeah, as mentioned I researched the topic before. And I think there has not even been an overflow within the RDB: > The raw, theoretical limit on the maximum device capacity is about > 2^105 bytes: > > 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512 > bytes/sector for the HD size in struct RigidDiskBlock http://wiki.amigaos.net/wiki/RDB_(Amiga_Rigid_Disk_Block) Confirmed by: The .ADF (Amiga Disk File) format FAQ: http://lclevy.free.fr/adflib/adf_info.html#p6 But what do I write, you know the RDB format :) So just do the calculation in 96 Bit and you all are set :) Now that is a reason for 128 Bit CPUs :). Muuhahaha. Ciao, -- Martin
Re: Amiga RDB partition support for disks >= 2 TB
jdow - 28.06.18, 12:00: > On 20180628 01:16, Martin Steigerwald wrote: […] > >> That brings to the fore an interesting question. Why bother with > >> RDBs > >> over 2TB unless you want a disk with one single partition? This > >> Win10 > >> monster I am using has a modest BIOS driver partition for the OS > >> and > >> a giant data partition. That smaller partition would easily work > >> with > >> any RDB/Filesystem combination since 2.0. So there are some good > >> workarounds that are probably "safer" and at least as flexible as > >> RDBs, one Linux has used for a very long time, too. > > > > Well, my use case was simple: > > > > I had this 2 TB disk and I choose to share it as a backup disk for > > Linux *and* AmigaOS 4.x on that Sam440ep I still have next to me > > desk here. > EEK! The hair on my neck is standing up straight! Have you heard > of SAMBA? The linux mail server firewall etc machine has an extra 4TB > disk on it as a backup for the other systems, although a piddly 4TB > is small when I save the entire 3G RAID system I have. It's a proof > of concept so A full backup on a 1gig Ethernet still takes a > long time. But backing up even an 18GB disk on an Amiga via > 100Base-t isn't too bad. And disk speeds of the era being what they > were it's about all you can do anyway. Heh, the thing worked just fine in Amiga OS 4. I got away with it without an issue, until I plugged the disk to my Linux laptop and wrote data onto the Linux file system. Mind you, I think in that partition marked LNX\0 I even created a Linux LVM with pvcreate. Do you call that insane? Well it probably is. :) And as an Amiga user I could just return to you: I clicked it, it did not warn, so all is good :) But yeah, as mentioned I researched the topic before. And I think there has not even been an overflow within the RDB: > The raw, theoretical limit on the maximum device capacity is about > 2^105 bytes: > > 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512 > bytes/sector for the HD size in struct RigidDiskBlock http://wiki.amigaos.net/wiki/RDB_(Amiga_Rigid_Disk_Block) Confirmed by: The .ADF (Amiga Disk File) format FAQ: http://lclevy.free.fr/adflib/adf_info.html#p6 But what do I write, you know the RDB format :) So just do the calculation in 96 Bit and you all are set :) Now that is a reason for 128 Bit CPUs :). Muuhahaha. Ciao, -- Martin
Re: Amiga RDB partition support for disks >= 2 TB
On 20180628 01:16, Martin Steigerwald wrote:
Dear Joanne.
jdow - 28.06.18, 08:39:
Anything done to RDBs for Linux must remain 100.000% compatible with
existing Amiga equipment. Otherwise, what's the point of bothering to
use RDBs?
Done to, in the sense of written to: Yes. I completely agree. But that
is for amiga-fdisk and parted. And for partitioning tools on native OS.
Design changes, too.
[…]
That brings to the fore an interesting question. Why bother with RDBs
over 2TB unless you want a disk with one single partition? This Win10
monster I am using has a modest BIOS driver partition for the OS and
a giant data partition. That smaller partition would easily work with
any RDB/Filesystem combination since 2.0. So there are some good
workarounds that are probably "safer" and at least as flexible as
RDBs, one Linux has used for a very long time, too.
Well, my use case was simple:
I had this 2 TB disk and I choose to share it as a backup disk for Linux
*and* AmigaOS 4.x on that Sam440ep I still have next to me desk here.
EEK! The hair on my neck is standing up straight! Have you heard of SAMBA?
The linux mail server firewall etc machine has an extra 4TB disk on it as a
backup for the other systems, although a piddly 4TB is small when I save the
entire 3G RAID system I have. It's a proof of concept so A full backup on a
1gig Ethernet still takes a long time. But backing up even an 18GB disk on
an Amiga via 100Base-t isn't too bad. And disk speeds of the era being what they
were it's about all you can do anyway.
{o.o}
Re: Amiga RDB partition support for disks >= 2 TB
On 20180628 01:16, Martin Steigerwald wrote:
Dear Joanne.
jdow - 28.06.18, 08:39:
Anything done to RDBs for Linux must remain 100.000% compatible with
existing Amiga equipment. Otherwise, what's the point of bothering to
use RDBs?
Done to, in the sense of written to: Yes. I completely agree. But that
is for amiga-fdisk and parted. And for partitioning tools on native OS.
Design changes, too.
[…]
That brings to the fore an interesting question. Why bother with RDBs
over 2TB unless you want a disk with one single partition? This Win10
monster I am using has a modest BIOS driver partition for the OS and
a giant data partition. That smaller partition would easily work with
any RDB/Filesystem combination since 2.0. So there are some good
workarounds that are probably "safer" and at least as flexible as
RDBs, one Linux has used for a very long time, too.
Well, my use case was simple:
I had this 2 TB disk and I choose to share it as a backup disk for Linux
*and* AmigaOS 4.x on that Sam440ep I still have next to me desk here.
EEK! The hair on my neck is standing up straight! Have you heard of SAMBA?
The linux mail server firewall etc machine has an extra 4TB disk on it as a
backup for the other systems, although a piddly 4TB is small when I save the
entire 3G RAID system I have. It's a proof of concept so A full backup on a
1gig Ethernet still takes a long time. But backing up even an 18GB disk on
an Amiga via 100Base-t isn't too bad. And disk speeds of the era being what they
were it's about all you can do anyway.
{o.o}
Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?)
Dear Joanne.
jdow - 28.06.18, 08:39:
> Anything done to RDBs for Linux must remain 100.000% compatible with
> existing Amiga equipment. Otherwise, what's the point of bothering to
> use RDBs?
Done to, in the sense of written to: Yes. I completely agree. But that
is for amiga-fdisk and parted. And for partitioning tools on native OS.
[…]
> That brings to the fore an interesting question. Why bother with RDBs
> over 2TB unless you want a disk with one single partition? This Win10
> monster I am using has a modest BIOS driver partition for the OS and
> a giant data partition. That smaller partition would easily work with
> any RDB/Filesystem combination since 2.0. So there are some good
> workarounds that are probably "safer" and at least as flexible as
> RDBs, one Linux has used for a very long time, too.
Well, my use case was simple:
I had this 2 TB disk and I choose to share it as a backup disk for Linux
*and* AmigaOS 4.x on that Sam440ep I still have next to me desk here.
As AmigaOS up to my knowledge does not have GPT support and MBR with 2
TB disk is asking for even more trouble and is not supported in any
sensible partitioning tool on AmigaOS and I choose to use Media Toolbox,
I went with RDB as I thought it is nicely supported in Linux, which it
is, apart from the overflowing issue.
So I did it this way.
> As I have said, for the RDB parser fix the famndool thing. Do fix it
> right in such a manner that if somebody compiles it against a version
> with no 64 bit device code it will throw a proper overflow error and
> protect the user. Users are dumb. We like to think of ourselves as
Sure, if the Linux kernel can´t handle it due to missing CONFIG_LBDAF or
so… by all means *bail* out. Without even a kernel option to parse this
anyway. No overflowing. Period. That is what I wrote from the beginning.
> smart. Let's try to be smart about this where we can so fingers can't
> point back at us rather than the fool that made some other error.
I completely agree.
> And do remember, I am merely (and vociferously) advising rather than
> dictating. You don't need my approval to proceed. I may want my name
> noted as an early contributor only. Meanwhile I spit out ideas as
> they come to me. One or more of them might be good. And offering
> alternatives is better than simply saying "No" most of the time.
>
> If people are using RDBs for TB level disks I doubt they can remember
> which is the left shoe when they are getting dressed in the morning
> before going out in the yard to beat some dead horses. Or else maybe
Heh, I remembered my shoes back then. And I informed myself about what I
was doing.
> they just want to see how far they can flog the m68k architecture as
> a mental challenge. In that case, taking it too seriously could hurt.
And well, yes… I wanted to see how far I can get away with it :)
> Note that I am mostly ignoring m68k Linux. People using that are hard
> core. People using x86/x64 Linux aren't such hard core folks. And I
> bet most of them want to read the disks so they can copy stuff to
> Amiga Forever or WinUAE running on other architectures. So TB is not
> likely to be an issue for them, either.
Yes.
Print a warning and be done with it in the RDB parser code.
Put a big fat warning into amiga-fdisk and parted!
Thanks,
Martin
> {^_^}
>
> On 20180627 22:43, Michael Schmitz wrote:
> > Joanne,
> >
> > Linux on m68k has supported lseek64 (or llseek) for a long time
> > (from glibc version 2.1 according to what I found). About the only
> > area where we are limited by 32 bits is the virtual memory size.
> >
> > I'm not proposing to modify the RDB format definition, though an
> > extension to store 64 bit offsets separate from the 32 bit ones
> > would be one way to make certain such disks are safe to use on 3.1
> > and earlier versions of AmigaOS. (Another one would be to modify
> > the disk drivers on older versions to do the offset calculation in
> > 64 bit, and check for overflow just as we do here. Not sure whether
> > that's feasible. And as you so eloquently describe, we can't rely
> > on users listening.)
> >
> > Either way, we need the cooperation of partitioning tool writers to
> > ensure partition information that is prone to overflows is never
> > stored in the 32 bit, classic RDB. That appears to have failed
> > already, as Martin's experience illustrates.
> >
> > I'm only concerned with fixing the (dangerous) but in the Linux
> > partition format parser for RDB. Refusing to use any partitions
> > that will cause havoc on old AmigaOS versions is all I can do to
> > try and get the users' attention.
> >
> > Your warning makes me wonder whether the log message should just say
> > 'report this bug to linux-m...@vger.kernel.org' to at least try and
> > educate any that respond about the dangers of their partitioning
> > scheme before telling them about the override option. Problem with
> > that is, in about three years no one will remember any of this ...
Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?)
Dear Joanne.
jdow - 28.06.18, 08:39:
> Anything done to RDBs for Linux must remain 100.000% compatible with
> existing Amiga equipment. Otherwise, what's the point of bothering to
> use RDBs?
Done to, in the sense of written to: Yes. I completely agree. But that
is for amiga-fdisk and parted. And for partitioning tools on native OS.
[…]
> That brings to the fore an interesting question. Why bother with RDBs
> over 2TB unless you want a disk with one single partition? This Win10
> monster I am using has a modest BIOS driver partition for the OS and
> a giant data partition. That smaller partition would easily work with
> any RDB/Filesystem combination since 2.0. So there are some good
> workarounds that are probably "safer" and at least as flexible as
> RDBs, one Linux has used for a very long time, too.
Well, my use case was simple:
I had this 2 TB disk and I choose to share it as a backup disk for Linux
*and* AmigaOS 4.x on that Sam440ep I still have next to me desk here.
As AmigaOS up to my knowledge does not have GPT support and MBR with 2
TB disk is asking for even more trouble and is not supported in any
sensible partitioning tool on AmigaOS and I choose to use Media Toolbox,
I went with RDB as I thought it is nicely supported in Linux, which it
is, apart from the overflowing issue.
So I did it this way.
> As I have said, for the RDB parser fix the famndool thing. Do fix it
> right in such a manner that if somebody compiles it against a version
> with no 64 bit device code it will throw a proper overflow error and
> protect the user. Users are dumb. We like to think of ourselves as
Sure, if the Linux kernel can´t handle it due to missing CONFIG_LBDAF or
so… by all means *bail* out. Without even a kernel option to parse this
anyway. No overflowing. Period. That is what I wrote from the beginning.
> smart. Let's try to be smart about this where we can so fingers can't
> point back at us rather than the fool that made some other error.
I completely agree.
> And do remember, I am merely (and vociferously) advising rather than
> dictating. You don't need my approval to proceed. I may want my name
> noted as an early contributor only. Meanwhile I spit out ideas as
> they come to me. One or more of them might be good. And offering
> alternatives is better than simply saying "No" most of the time.
>
> If people are using RDBs for TB level disks I doubt they can remember
> which is the left shoe when they are getting dressed in the morning
> before going out in the yard to beat some dead horses. Or else maybe
Heh, I remembered my shoes back then. And I informed myself about what I
was doing.
> they just want to see how far they can flog the m68k architecture as
> a mental challenge. In that case, taking it too seriously could hurt.
And well, yes… I wanted to see how far I can get away with it :)
> Note that I am mostly ignoring m68k Linux. People using that are hard
> core. People using x86/x64 Linux aren't such hard core folks. And I
> bet most of them want to read the disks so they can copy stuff to
> Amiga Forever or WinUAE running on other architectures. So TB is not
> likely to be an issue for them, either.
Yes.
Print a warning and be done with it in the RDB parser code.
Put a big fat warning into amiga-fdisk and parted!
Thanks,
Martin
> {^_^}
>
> On 20180627 22:43, Michael Schmitz wrote:
> > Joanne,
> >
> > Linux on m68k has supported lseek64 (or llseek) for a long time
> > (from glibc version 2.1 according to what I found). About the only
> > area where we are limited by 32 bits is the virtual memory size.
> >
> > I'm not proposing to modify the RDB format definition, though an
> > extension to store 64 bit offsets separate from the 32 bit ones
> > would be one way to make certain such disks are safe to use on 3.1
> > and earlier versions of AmigaOS. (Another one would be to modify
> > the disk drivers on older versions to do the offset calculation in
> > 64 bit, and check for overflow just as we do here. Not sure whether
> > that's feasible. And as you so eloquently describe, we can't rely
> > on users listening.)
> >
> > Either way, we need the cooperation of partitioning tool writers to
> > ensure partition information that is prone to overflows is never
> > stored in the 32 bit, classic RDB. That appears to have failed
> > already, as Martin's experience illustrates.
> >
> > I'm only concerned with fixing the (dangerous) but in the Linux
> > partition format parser for RDB. Refusing to use any partitions
> > that will cause havoc on old AmigaOS versions is all I can do to
> > try and get the users' attention.
> >
> > Your warning makes me wonder whether the log message should just say
> > 'report this bug to linux-m...@vger.kernel.org' to at least try and
> > educate any that respond about the dangers of their partitioning
> > scheme before telling them about the override option. Problem with
> > that is, in about three years no one will remember any of this ...
Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?)
Michael Schmitz - 28.06.18, 07:43: > Joanne, > > Linux on m68k has supported lseek64 (or llseek) for a long time (from > glibc version 2.1 according to what I found). About the only area > where we are limited by 32 bits is the virtual memory size. > > I'm not proposing to modify the RDB format definition, though an > extension to store 64 bit offsets separate from the 32 bit ones would > be one way to make certain such disks are safe to use on 3.1 and > earlier versions of AmigaOS. (Another one would be to modify the disk > drivers on older versions to do the offset calculation in 64 bit, and > check for overflow just as we do here. Not sure whether that's > feasible. And as you so eloquently describe, we can't rely on users > listening.) I think that would be up to upstream developers to decide. That would be AmigaOS developers and/or MorphOS, AROS developers. I could try to at least point them to this discussion here. Whether they choose to take the time to look at it? I don´t know. They are developing that stuff in their spare time. > Either way, we need the cooperation of partitioning tool writers to > ensure partition information that is prone to overflows is never > stored in the 32 bit, classic RDB. That appears to have failed > already, as Martin's experience illustrates. Heh :) But yeah, I´d say the damage is done already. > The raw, theoretical limit on the maximum device capacity is about > 2^105 bytes: > > 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512 > bytes/sector for the HD size in struct RigidDiskBlock http://wiki.amigaos.net/wiki/RDB Of course that only holds true for as long as >32 bit math is used :). So yeah. I wonder how many Amiga users may try to use such a large disk with a native operating system that does not support this. NSD64 and TD64 are at least 10 years old, if not older. Just see the dates in: http://aminet.net/search?query=nsdpatch http://aminet.net/package/disk/misc/td64patch Forget about 10 years. 20 years are more accurate. This stuff is *ancient*. Officially support is in AmigaOS since version 3.5, which has been released more than 20 years ago as well: https://en.wikipedia.org/wiki/AmigaOS_version_history#AmigaOS_3.5,_3.9 Even AmigaOS 4.0 is older than 10 years already. In any case, it is the responsibility of the tool that creates or change the RDB to take care of warning the user or using a new format. Here the kernel just reads that which already exists in the wild. > I'm only concerned with fixing the (dangerous) but in the Linux > partition format parser for RDB. Refusing to use any partitions that > will cause havoc on old AmigaOS versions is all I can do to try and > get the users' attention. Exactly. And I do think, that no tool on Linux can create these kind of RDBs, and if they can… a big fat warning belongs into that tool. Not even Media Toolbox warns about that currently, so yes… adding an additional warning in the kernel may be helpful. However without refusing to parse this, the user may never notice the warning. So that is an argument for refusing to parse this without a kernel option. I still think that is too much hand-holding for the few native OS users out there. Cause in order to see the warning, the user would have to stuff the disk into a Linux machine anyway. So the user is still perfectly able to create such an RDB on AmigaOS 4.x or even AmigaOS 3.5/3.9 or even earlier, and then stuff the disk into a machine with AmigaOS 1.3 and probably let it go boom there. Now, how many users who do not know about these limits will ever put the disk into a Linux machine, receive the warning and then be saved from data destruction? I bet: None. It is just as unlikely as it can get. :) We are talking about maybe a few thousand Amiga users here. And there would be even the question how many of them will try to use disks larger than 2 TiB. I bet there will be some, but I bet there won´t be many. So it would be way more important to warn in Media Toolbox, amiga-fdisk, parted and whatever other partitioning tool users may use. I´d still leave in the warning anyway, but I think the kernel option… is over-protecting users. Anyway your call and I perfectly get it, in case you choose to be on a 100% safe side. You are submitting the patch. > Your warning makes me wonder whether the log message should just say > 'report this bug to linux-m...@vger.kernel.org' to at least try and > educate any that respond about the dangers of their partitioning > scheme before telling them about the override option. Problem with > that is, in about three years no one will remember any of this ... > > Cheers, > > Michael > > Am 28.06.2018 um 15:44 schrieb jdow: > > Michael, as long as m68k only supports int fseek( int ) 4 GB * block > > size is your HARD limit. Versions that support __int64 fseek64( > > __int64 ) can work with larger disks. RDBs could include RDSK and a > > new set of other blocks
Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?)
Michael Schmitz - 28.06.18, 07:43: > Joanne, > > Linux on m68k has supported lseek64 (or llseek) for a long time (from > glibc version 2.1 according to what I found). About the only area > where we are limited by 32 bits is the virtual memory size. > > I'm not proposing to modify the RDB format definition, though an > extension to store 64 bit offsets separate from the 32 bit ones would > be one way to make certain such disks are safe to use on 3.1 and > earlier versions of AmigaOS. (Another one would be to modify the disk > drivers on older versions to do the offset calculation in 64 bit, and > check for overflow just as we do here. Not sure whether that's > feasible. And as you so eloquently describe, we can't rely on users > listening.) I think that would be up to upstream developers to decide. That would be AmigaOS developers and/or MorphOS, AROS developers. I could try to at least point them to this discussion here. Whether they choose to take the time to look at it? I don´t know. They are developing that stuff in their spare time. > Either way, we need the cooperation of partitioning tool writers to > ensure partition information that is prone to overflows is never > stored in the 32 bit, classic RDB. That appears to have failed > already, as Martin's experience illustrates. Heh :) But yeah, I´d say the damage is done already. > The raw, theoretical limit on the maximum device capacity is about > 2^105 bytes: > > 32 bit rdb_Cylinders * 32 bit rdb_Heads * 32 bit rdb_Sectors * 512 > bytes/sector for the HD size in struct RigidDiskBlock http://wiki.amigaos.net/wiki/RDB Of course that only holds true for as long as >32 bit math is used :). So yeah. I wonder how many Amiga users may try to use such a large disk with a native operating system that does not support this. NSD64 and TD64 are at least 10 years old, if not older. Just see the dates in: http://aminet.net/search?query=nsdpatch http://aminet.net/package/disk/misc/td64patch Forget about 10 years. 20 years are more accurate. This stuff is *ancient*. Officially support is in AmigaOS since version 3.5, which has been released more than 20 years ago as well: https://en.wikipedia.org/wiki/AmigaOS_version_history#AmigaOS_3.5,_3.9 Even AmigaOS 4.0 is older than 10 years already. In any case, it is the responsibility of the tool that creates or change the RDB to take care of warning the user or using a new format. Here the kernel just reads that which already exists in the wild. > I'm only concerned with fixing the (dangerous) but in the Linux > partition format parser for RDB. Refusing to use any partitions that > will cause havoc on old AmigaOS versions is all I can do to try and > get the users' attention. Exactly. And I do think, that no tool on Linux can create these kind of RDBs, and if they can… a big fat warning belongs into that tool. Not even Media Toolbox warns about that currently, so yes… adding an additional warning in the kernel may be helpful. However without refusing to parse this, the user may never notice the warning. So that is an argument for refusing to parse this without a kernel option. I still think that is too much hand-holding for the few native OS users out there. Cause in order to see the warning, the user would have to stuff the disk into a Linux machine anyway. So the user is still perfectly able to create such an RDB on AmigaOS 4.x or even AmigaOS 3.5/3.9 or even earlier, and then stuff the disk into a machine with AmigaOS 1.3 and probably let it go boom there. Now, how many users who do not know about these limits will ever put the disk into a Linux machine, receive the warning and then be saved from data destruction? I bet: None. It is just as unlikely as it can get. :) We are talking about maybe a few thousand Amiga users here. And there would be even the question how many of them will try to use disks larger than 2 TiB. I bet there will be some, but I bet there won´t be many. So it would be way more important to warn in Media Toolbox, amiga-fdisk, parted and whatever other partitioning tool users may use. I´d still leave in the warning anyway, but I think the kernel option… is over-protecting users. Anyway your call and I perfectly get it, in case you choose to be on a 100% safe side. You are submitting the patch. > Your warning makes me wonder whether the log message should just say > 'report this bug to linux-m...@vger.kernel.org' to at least try and > educate any that respond about the dangers of their partitioning > scheme before telling them about the override option. Problem with > that is, in about three years no one will remember any of this ... > > Cheers, > > Michael > > Am 28.06.2018 um 15:44 schrieb jdow: > > Michael, as long as m68k only supports int fseek( int ) 4 GB * block > > size is your HARD limit. Versions that support __int64 fseek64( > > __int64 ) can work with larger disks. RDBs could include RDSK and a > > new set of other blocks
Re: Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?)
Changing subject, so that there is at least a chance for someone to find
this discussions with a search engine :)
Joanne,
jdow - 28.06.18, 04:57:
> The issue is what happens when one of those disks appears on a 3.1
> system. {^_^}
That is right, so I think the warning about 64 bit support in native OS
is okay, but that issue already exists *without* Linux.
Remember, I created that RDB with Media Toolbox on AmigaOS 4.0. I did
not even use Linux to create that beast :). If it booms in AmigaOS < 4
without NSD64, TD64 or SCSI direct, that would happen with or without
the warning in Linux, even without the disk ever have been seen by a
Linux kernel.
I´d say the warning about support in native OS does not harm, even when
it is about educating Amiga users who, in case they use that large
drives – and I pretty much bet, some of them do –, better know the
limitations beforehand.
I do think the extra kernel option does not make all that much sense,
but I accept it anyway.
Cause if you argue like that, what would need fixing is AmigaOS < 4
without NSD64, TD64 or SCSI direct, but then that is what NSD64 and TD64
was made for more than 10 years ago.
Of course, if a partitioning tool for Linux ever allows to create such
an RDB, it makes sense to add a big fat warning about that. As… I think
would make sense to have in Media Toolbox and AmigaOS partitioning
tools.
However Linux here just reads the RDB, so I´d personally go with the
warning about support in native OS, but spare myself the extra kernel
option stuff.
It is Michael´s call tough, as he submits the patch. And if he chooses
to be on a safer side than this, that is fine with me.
Thanks,
Martin
> On 20180627 01:03, Martin Steigerwald wrote:
> > Dear Joanne.
> >
> > jdow - 27.06.18, 08:24:
> >> You allergic to using a GPT solution? It will get away from some of
> >> the evils that RDB has inherent in it because they are also
> >> features?
> >> (Loading a filesystem or DriveInit code from RDBs is just asking
> >> for
> >> a nearly impossible to remove malware infection.) Furthermore, any
> >> 32
> >> bit system that sees an RDSK block is going to try to translate it.
> >> If you add a new RDB format you are going to get bizarre and
> >> probably
> >> quite destructive results from the mistake. Fail safe is a rather
> >> good notion, methinks.
> >>
> >> Personally I figure this is all rather surreal. 2TG of junk on an
> >> Amiga system seems utterly outlandish to me. You cited another
> >> overflow potential. There are at least three we've identified, I
> >> believe. Are you 100% sure there are no more? The specific one you
> >> mention of translating RDB to Linux has a proper solution in the
> >> RDB
> >> reader. It should recover such overflow errors in the RDB as it can
> >> with due care and polish. It should flag any other overflow error
> >> it
> >> detects within the RDBs and return an error such as to leave the
> >> disk
> >> unmounted or mounted read-only if you feel like messing up a poor
> >> sod's backups. The simple solution is to read each of the variables
> >> with the nominal RDB size and convert it to uint64_t before
> >> calculating byte indices.
> >>
> >> However, consider my inputs as advice from an adult who has seen
> >> the
> >> Amiga Elephant so to speak. I am not trying to assert any control.
> >> Do
> >> as you wish; but, I would plead with you to avoid ANY chance you
> >> can
> >> for the user to make a bonehead stupid move and lose all his
> >> treasured disk archives. Doing otherwise is very poor form.
> >
> > I am pretty confident that larger than 2 TiB disks are fully
> > supported within AmigaOS 4, as I outlined in my other mail.
> >
> > So with all due respect: I used a larger than 2 TiB disk in AmigaOS
> > 4 in 2012 already *just* fine. I even found I had the same
> > questions back then, and researched it. Which lead to this official
> > article back then:
> >
> > http://wiki.amigaos.net/wiki/RDB
> >
> > I am also pretty sure that AmigaOS still uses RDB as partitioning
> > format. They support MBR. I don´t think AmigaOS 4.1 supports GPT.
> > Whether to implement that of course is the decision of AmigaOS 4
> > development team. I am no longer a member of it since some time.
> >
> > Linux m68k should already be able to use disks in GPT format, but
> > you
> > likely won´t be able to read them in AmigaOS, unless there is some
> > third party support for it meanwhile.
> >
> > Thanks,
> > Martin
> >
> >> {o.o} Joanne "Said enough, she has" Dow
> >>
> >> On 20180626 18:07, Michael Schmitz wrote:
> >>> Joanne,
> >>>
> >>> As far as I have been able to test, the change is backwards
> >>> compatible (RDB partitions from an old disk 80 GB disk are still
> >>> recognized OK). That"s only been done on an emulator though.
> >>>
> >>> Your advice about the dangers of using RDB disks that would have
> >>> failed the current Linux RDB parser on legacy 32 bit systems is
> >>> well
> >>> taken though.
Re: Amiga RDB partition support for disks >= 2 TB (was: Re: moving affs + RDB partition support to staging?)
Changing subject, so that there is at least a chance for someone to find
this discussions with a search engine :)
Joanne,
jdow - 28.06.18, 04:57:
> The issue is what happens when one of those disks appears on a 3.1
> system. {^_^}
That is right, so I think the warning about 64 bit support in native OS
is okay, but that issue already exists *without* Linux.
Remember, I created that RDB with Media Toolbox on AmigaOS 4.0. I did
not even use Linux to create that beast :). If it booms in AmigaOS < 4
without NSD64, TD64 or SCSI direct, that would happen with or without
the warning in Linux, even without the disk ever have been seen by a
Linux kernel.
I´d say the warning about support in native OS does not harm, even when
it is about educating Amiga users who, in case they use that large
drives – and I pretty much bet, some of them do –, better know the
limitations beforehand.
I do think the extra kernel option does not make all that much sense,
but I accept it anyway.
Cause if you argue like that, what would need fixing is AmigaOS < 4
without NSD64, TD64 or SCSI direct, but then that is what NSD64 and TD64
was made for more than 10 years ago.
Of course, if a partitioning tool for Linux ever allows to create such
an RDB, it makes sense to add a big fat warning about that. As… I think
would make sense to have in Media Toolbox and AmigaOS partitioning
tools.
However Linux here just reads the RDB, so I´d personally go with the
warning about support in native OS, but spare myself the extra kernel
option stuff.
It is Michael´s call tough, as he submits the patch. And if he chooses
to be on a safer side than this, that is fine with me.
Thanks,
Martin
> On 20180627 01:03, Martin Steigerwald wrote:
> > Dear Joanne.
> >
> > jdow - 27.06.18, 08:24:
> >> You allergic to using a GPT solution? It will get away from some of
> >> the evils that RDB has inherent in it because they are also
> >> features?
> >> (Loading a filesystem or DriveInit code from RDBs is just asking
> >> for
> >> a nearly impossible to remove malware infection.) Furthermore, any
> >> 32
> >> bit system that sees an RDSK block is going to try to translate it.
> >> If you add a new RDB format you are going to get bizarre and
> >> probably
> >> quite destructive results from the mistake. Fail safe is a rather
> >> good notion, methinks.
> >>
> >> Personally I figure this is all rather surreal. 2TG of junk on an
> >> Amiga system seems utterly outlandish to me. You cited another
> >> overflow potential. There are at least three we've identified, I
> >> believe. Are you 100% sure there are no more? The specific one you
> >> mention of translating RDB to Linux has a proper solution in the
> >> RDB
> >> reader. It should recover such overflow errors in the RDB as it can
> >> with due care and polish. It should flag any other overflow error
> >> it
> >> detects within the RDBs and return an error such as to leave the
> >> disk
> >> unmounted or mounted read-only if you feel like messing up a poor
> >> sod's backups. The simple solution is to read each of the variables
> >> with the nominal RDB size and convert it to uint64_t before
> >> calculating byte indices.
> >>
> >> However, consider my inputs as advice from an adult who has seen
> >> the
> >> Amiga Elephant so to speak. I am not trying to assert any control.
> >> Do
> >> as you wish; but, I would plead with you to avoid ANY chance you
> >> can
> >> for the user to make a bonehead stupid move and lose all his
> >> treasured disk archives. Doing otherwise is very poor form.
> >
> > I am pretty confident that larger than 2 TiB disks are fully
> > supported within AmigaOS 4, as I outlined in my other mail.
> >
> > So with all due respect: I used a larger than 2 TiB disk in AmigaOS
> > 4 in 2012 already *just* fine. I even found I had the same
> > questions back then, and researched it. Which lead to this official
> > article back then:
> >
> > http://wiki.amigaos.net/wiki/RDB
> >
> > I am also pretty sure that AmigaOS still uses RDB as partitioning
> > format. They support MBR. I don´t think AmigaOS 4.1 supports GPT.
> > Whether to implement that of course is the decision of AmigaOS 4
> > development team. I am no longer a member of it since some time.
> >
> > Linux m68k should already be able to use disks in GPT format, but
> > you
> > likely won´t be able to read them in AmigaOS, unless there is some
> > third party support for it meanwhile.
> >
> > Thanks,
> > Martin
> >
> >> {o.o} Joanne "Said enough, she has" Dow
> >>
> >> On 20180626 18:07, Michael Schmitz wrote:
> >>> Joanne,
> >>>
> >>> As far as I have been able to test, the change is backwards
> >>> compatible (RDB partitions from an old disk 80 GB disk are still
> >>> recognized OK). That"s only been done on an emulator though.
> >>>
> >>> Your advice about the dangers of using RDB disks that would have
> >>> failed the current Linux RDB parser on legacy 32 bit systems is
> >>> well
> >>> taken though.
