Re: Ability to further support 32bit architectures

[Aurelien Jarno]

On 2024-01-11 18:38, Aurelien Jarno wrote:
> On 2024-01-11 13:24, Adrian Bunk wrote:
> > On Thu, Jan 11, 2024 at 11:28:19AM +0100, Bastian Blank wrote:
> > >...
> > > On Thu, Jan 11, 2024 at 09:48:34AM +, Dimitri John Ledkov wrote:
> > > > Disabling debug symbols, enabling debug symbol zstd compression, using
> > > > split debug symbols (disabled BTF usage) should help here.
> > > 
> > > Okay, maybe more workarounds exist.  But none of them look really
> > > promising.
> > >...
> > 
> > gcc being a memory hog on for C++ code is a hard problem,
> > and debug symbols for C++ code can be a problem since
> > they might be > 1 GB for some binaries.
> > 
> > But gcc needing more than 4 GB for a small C kernel driver is not
> > a problem for the "Ability to further support 32bit architectures",
> > that's a gcc bug that should be reported upstream just like you wouldn't
> > suggest dropping amd64 if gcc would ICE on one kernel driver on that
> > architecture.
> 
> Or maybe just blame the kernel instead:
> https://lore.kernel.org/lkml/CAHk-=whkGHOmpM_1kNgzX1UDAs10+UuALcpeEWN29EE0m-my=w...@mail.gmail.com/

And following the suggestion in that thread, I have just sent a patch fixing 
the issue:
https://lore.kernel.org/lkml/20240113183334.1690740-1-aurel...@aurel32.net/T/#u


-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://aurel32.net

Re: Ability to further support 32bit architectures

[Rene Engelhard]

Hi,

Am 13.01.24 um 13:59 schrieb rhys:

No.

You are AGAIN assuming what I am talking about.

Maybe because of how you write...

I know the difference between a 32-bit processor and a 64-bit processor.


Obviously you don't. Or at least are not aware about consequences.


Since you still offer 32bit machines of which Debian has enough of. (64 
bit kernel probably but it doesn't matter) where it does not matter at all.



You ignore the stated fact in this thread that on a 32bit processor one 
process can't get more than 3GB or even less of RAM (regardless of what 
memory extension stuff exists).


In  https://lists.debian.org/debian-kernel/2024/01/msg00191.html (where 
the quoting is a problem, one doesn't know what is yours and what not) 
you ignored what YunQiang said in the post you replied to:


https://lists.debian.org/debian-kernel/2024/01/msg00189.html

Quoting again just for you:

"[...] It is about some limitation of 32bit.
2 examples for it:
   1. if we use 32bit value for time, it will overflow in 2038, then
your time will be shown as 1900.
   https://en.wikipedia.org/wiki/Year_2038_problem
   2. A single process (or maybe APP, not precisely), can only use UP
to 4GiB RAM.
   In fact on most system the value is less than 4GiB:
  on intel32, it is 3GiB
  on mips32, it is 2GiB
   But currently, it is not enough, for example, when we build a
big APP, it will need much more RAM.
   The RAM does install in your Rack, but you can NOT use it.
  https://en.wikipedia.org/wiki/3_GB_barrier

"

And THAT (2.) is the problem here for linking big applications/compile 
units. Not the lack of machines.


And that is a limitation of *the architecture*.

Putting more "32bit machines" on it do not change anything of that 
except that there were more machines which cannot build big stuff.



Regards,


Rene

Re: Ability to further support 32bit architectures

[rhys]

> * Thank you for your offering, but Debian is never in lack of
> x86/x86_64/amd64/intel/amd/whatever_you_name_it hardware for package building.
> In fact, we now have some of the very powerful machines.

If you're sure.  Working 32-bit systems are not as common as they once were, 
and sometimes cross-compliling isn't quite the same (though that's usually a 
matter of optimization rather than function).

I'll leave the offer open in case the need arises.  My system is an Intel Atom 
N270.  I keep it around for some simple services but you're welcome to use it 
if it helps.

--J

Re: Ability to further support 32bit architectures

[rhys]

No.

You are AGAIN assuming what I am talking about.

I know the difference between a 32-bit processor and a 64-bit processor.

If you're not going to answer my question, kindly don't answer at all.

--J

> On Jan 12, 2024, at 21:40, YunQiang Su  wrote:
> 
> rhys  于2024年1月13日周六 11:27写道：
>> 
>> Let me try again, following up on the previous thread, but removing most of 
>> the irrelevant history.
>> 
>> If I have a 32-bit Intel system that is currently supported on bookworm 
>> (currently running bullseye, but I can upgrade it), is that of use to anyone 
>> as a native build platform for 32-bit binary packages for Debian?
>> 
> You are yet another person who is confused by the name "i386" vs "amd64".
> AMD64 is just the named due to that X86 is extended to X86-64 by AMD *first*.
> It means that "Intel 64" or "EM64T" is almost same with "AMD64".
> 
> So, you, the Intel CPU user,  should use "AMD64", if you don't clearly
> know that your
> Intel CPU is 32bit only.
> 
> For more clear, for Debian, "AMD64" is equal to X86-64.
> https://en.wikipedia.org/wiki/X86-64
> 
> -- 
> YunQiang Su

Re: Ability to further support 32bit architectures

[Boyuan Yang]

Hi,

在 2024-01-12星期五的 21:26 -0600，rhys写道：
> Let me try again, following up on the previous thread, but removing most of
> the irrelevant history.

Let me try to strike this message down to avoid the discussion from shifting
further to an unknown direction.

> If I have a 32-bit Intel system that is currently supported on bookworm
> (currently running bullseye, but I can upgrade it), is that of use to anyone
> as a native build platform for 32-bit binary packages for Debian?

* Thank you for your offering, but Debian is never in lack of
x86/x86_64/amd64/intel/amd/whatever_you_name_it hardware for package building.
In fact, we now have some of the very powerful machines.

* The original problem described by
https://lists.debian.org/debian-devel/2024/01/msg00134.html is a limitation
imposed by the choice of running compiler in a 32-bit native x86 environment
(chroot or real OS, doesn't matter). This limitation cannot be mitigated by
hardware upgrade.

* The original problem described by
https://lists.debian.org/debian-devel/2024/01/msg00134.html can be mitigated
through cross-compilation or various build memory reduction methods. The
related discussion is already available in
https://lists.debian.org/debian-devel/2024/01/msg00135.html and later emails
in this thread. These mitigations are not related to hardware.

Thanks,
Boyuan Yang


signature.asc
Description: This is a digitally signed message part

Re: Ability to further support 32bit architectures

[rhys]

Let me try again, following up on the previous thread, but removing most of the 
irrelevant history.

If I have a 32-bit Intel system that is currently supported on bookworm 
(currently running bullseye, but I can upgrade it), is that of use to anyone as 
a native build platform for 32-bit binary packages for Debian?

--J

> On Jan 12, 2024, at 12:06, Alan Corey  wrote:
> 
> Are you forgetting that 64 bit is slower?  In the arm world where it's easily 
> switchable 64 bit is pokey when you don't need it.
> 

Re: Ability to further support 32bit architectures

[YunQiang Su]

rhys  于2024年1月13日周六 11:27写道：
>
> Let me try again, following up on the previous thread, but removing most of 
> the irrelevant history.
>
> If I have a 32-bit Intel system that is currently supported on bookworm 
> (currently running bullseye, but I can upgrade it), is that of use to anyone 
> as a native build platform for 32-bit binary packages for Debian?
>
You are yet another person who is confused by the name "i386" vs "amd64".
AMD64 is just the named due to that X86 is extended to X86-64 by AMD *first*.
It means that "Intel 64" or "EM64T" is almost same with "AMD64".

So, you, the Intel CPU user,  should use "AMD64", if you don't clearly
know that your
Intel CPU is 32bit only.

For more clear, for Debian, "AMD64" is equal to X86-64.
https://en.wikipedia.org/wiki/X86-64

-- 
YunQiang Su

Re: Ability to further support 32bit architectures

[Alan Corey]

Are you forgetting that 64 bit is slower?  In the arm world where it's
easily switchable 64 bit is pokey when you don't need it.

On Fri, Jan 12, 2024, 12:54 PM  wrote:

>
>
> Sent from my mobile device.
>
> --
> *From:* YunQiang Su 
> *Sent:* Friday, January 12, 2024 10:11
> *To:* r...@neoquasar.org
> *Cc:* noloa...@gmail.com; debian-ker...@lists.debian.org;
> debian-...@lists.debian.org; debian-de...@lists.debian.org;
> debian-release@lists.debian.org
> *Subject:* Re: Ability to further support 32bit architectures
>
>  于2024年1月12日周五 23:59写道：
> >
> > Keeping in mind that I am new to this arena...
> >
> > I have some Intel systems - both 64-bit and 32-bit - that I might be
> able to use as build platforms.
> >
>
> I guess all of your hardwares are 64bit. You setup different OS on them.
>
> No. I have multiple 32-bit systems, one of which is Intel.
>
> > What does the Debian team need from me to be able to use these systems?
> >
>
> It's not about performance of hardware. It is about some limitation of
> 32bit.
> 2 examples for it:
>1. if we use 32bit value for time, it will overflow in 2038, then
> your time will be shown as 1900.
>https://en.wikipedia.org/wiki/Year_2038_problem
>2. A single process (or maybe APP, not precisely), can only use UP
> to 4GiB RAM.
>In fact on most system the value is less than 4GiB:
>   on intel32, it is 3GiB
>   on mips32, it is 2GiB
>But currently, it is not enough, for example, when we build a
> big APP, it will need much more RAM.
>The RAM does install in your Rack, but you can NOT use it.
>   https://en.wikipedia.org/wiki/3_GB_barrier
>
> > I can't guarantee they'll be FAST, but I'll do what I can to make them
> EFFECTIVE.
> >
>
> If you are really need 32bit system. Maybe you can say out why you
> *REALLY* need it.
> For most users, the suggestion is: upgrade to 64bit.
>
>
> That's not at all what I was asking or talking about.
>
> --J
>

Re: Ability to further support 32bit architectures

[rhys]

Sent from my mobile device.


From: YunQiang Su 
Sent: Friday, January 12, 2024 10:11
To: r...@neoquasar.org
Cc: noloa...@gmail.com; debian-ker...@lists.debian.org; 
debian-...@lists.debian.org; debian-de...@lists.debian.org; 
debian-release@lists.debian.org
Subject: Re: Ability to further support 32bit architectures

 于2024年1月12日周五 23:59写道： 
> 
> Keeping in mind that I am new to this arena... 
> 
> I have some Intel systems - both 64-bit and 32-bit - that I might be able to 
> use as build platforms. 
> 

I guess all of your hardwares are 64bit. You setup different OS on them.

No. I have multiple 32-bit systems, one of which is Intel. 

> What does the Debian team need from me to be able to use these systems? 
> 

It's not about performance of hardware. It is about some limitation of 32bit. 
2 examples for it: 
   1. if we use 32bit value for time, it will overflow in 2038, then 
your time will be shown as 1900. 
   https://en.wikipedia.org/wiki/Year_2038_problem 
   2. A single process (or maybe APP, not precisely), can only use UP 
to 4GiB RAM. 
   In fact on most system the value is less than 4GiB: 
  on intel32, it is 3GiB 
  on mips32, it is 2GiB 
   But currently, it is not enough, for example, when we build a 
big APP, it will need much more RAM. 
   The RAM does install in your Rack, but you can NOT use it. 
  https://en.wikipedia.org/wiki/3_GB_barrier 

> I can't guarantee they'll be FAST, but I'll do what I can to make them 
> EFFECTIVE. 
> 

If you are really need 32bit system. Maybe you can say out why you 
*REALLY* need it. 
For most users, the suggestion is: upgrade to 64bit. 


That's not at all what I was asking or talking about. 

--J

Re: Ability to further support 32bit architectures

[rhys]

Keeping in mind that I am new to this arena...

I have some Intel systems - both 64-bit and 32-bit - that I might be able to 
use as build platforms. 

What does the Debian team need from me to be able to use these systems?

I can't guarantee they'll be FAST, but I'll do what I can to make them 
EFFECTIVE.

--J

Sent from my mobile device.

From: Jeffrey Walton 
Sent: Thursday, January 11, 2024 09:48
To: debian-ker...@lists.debian.org; debian-...@lists.debian.org; 
debian-de...@lists.debian.org; debian-release@lists.debian.org
Subject: Re: Ability to further support 32bit architectures

On Thu, Jan 11, 2024 at 5:45 AM Bastian Blank  wrote: 
> 
> On Thu, Jan 11, 2024 at 09:48:34AM +, Dimitri John Ledkov wrote: 
> > Disabling debug symbols, enabling debug symbol zstd compression, using 
> > split debug symbols (disabled BTF usage) should help here. 
> 
> Okay, maybe more workarounds exist.  But none of them look really 
> promising. 

Also see <https://wiki.debian.org/ReduceBuildMemoryOverhead>. 

> > Separately, I wish we had cross-builders available, and cross-build 
> > i386/armhf kernels from amd64/arm64 and thus having access to 64-bit 
> > compiler. 
> 
> Real cross-builders would use some fast amd64/arm64/ppc64el (and for 
> amd64 also reasonably cheap) machines to build all other architectures. 

Jeff 

Re: Ability to further support 32bit architectures

[YunQiang Su]

 于2024年1月12日周五 23:59写道：
>
> Keeping in mind that I am new to this arena...
>
> I have some Intel systems - both 64-bit and 32-bit - that I might be able to 
> use as build platforms.
>

I guess all of your hardwares are 64bit. You setup different OS on them.

> What does the Debian team need from me to be able to use these systems?
>

It's not about performance of hardware. It is about some limitation of 32bit.
2 examples for it:
   1. if we use 32bit value for time, it will overflow in 2038, then
your time will be shown as 1900.
   https://en.wikipedia.org/wiki/Year_2038_problem
   2. A single process (or maybe APP, not precisely), can only use UP
to 4GiB RAM.
   In fact on most system the value is less than 4GiB:
  on intel32, it is 3GiB
  on mips32, it is 2GiB
   But currently, it is not enough, for example, when we build a
big APP, it will need much more RAM.
   The RAM does install in your Rack, but you can NOT use it.
  https://en.wikipedia.org/wiki/3_GB_barrier

> I can't guarantee they'll be FAST, but I'll do what I can to make them 
> EFFECTIVE.
>

If you are really need 32bit system. Maybe you can say out why you
*REALLY* need it.
For most users, the suggestion is: upgrade to 64bit.

> --J
>

Re: Ability to further support 32bit architectures

[Aurelien Jarno]

On 2024-01-11 13:24, Adrian Bunk wrote:
> On Thu, Jan 11, 2024 at 11:28:19AM +0100, Bastian Blank wrote:
> >...
> > On Thu, Jan 11, 2024 at 09:48:34AM +, Dimitri John Ledkov wrote:
> > > Disabling debug symbols, enabling debug symbol zstd compression, using
> > > split debug symbols (disabled BTF usage) should help here.
> > 
> > Okay, maybe more workarounds exist.  But none of them look really
> > promising.
> >...
> 
> gcc being a memory hog on for C++ code is a hard problem,
> and debug symbols for C++ code can be a problem since
> they might be > 1 GB for some binaries.
> 
> But gcc needing more than 4 GB for a small C kernel driver is not
> a problem for the "Ability to further support 32bit architectures",
> that's a gcc bug that should be reported upstream just like you wouldn't
> suggest dropping amd64 if gcc would ICE on one kernel driver on that
> architecture.

Or maybe just blame the kernel instead:
https://lore.kernel.org/lkml/CAHk-=whkGHOmpM_1kNgzX1UDAs10+UuALcpeEWN29EE0m-my=w...@mail.gmail.com/

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://aurel32.net

Re: Ability to further support 32bit architectures

[Jeffrey Walton]

On Thu, Jan 11, 2024 at 5:45 AM Bastian Blank  wrote:
>
> On Thu, Jan 11, 2024 at 09:48:34AM +, Dimitri John Ledkov wrote:
> > Disabling debug symbols, enabling debug symbol zstd compression, using
> > split debug symbols (disabled BTF usage) should help here.
>
> Okay, maybe more workarounds exist.  But none of them look really
> promising.

Also see .

> > Separately, I wish we had cross-builders available, and cross-build
> > i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
> > compiler.
>
> Real cross-builders would use some fast amd64/arm64/ppc64el (and for
> amd64 also reasonably cheap) machines to build all other architectures.

Jeff

Re: Ability to further support 32bit architectures

[Adrian Bunk]

On Thu, Jan 11, 2024 at 11:28:19AM +0100, Bastian Blank wrote:
>...
> On Thu, Jan 11, 2024 at 09:48:34AM +, Dimitri John Ledkov wrote:
> > Disabling debug symbols, enabling debug symbol zstd compression, using
> > split debug symbols (disabled BTF usage) should help here.
> 
> Okay, maybe more workarounds exist.  But none of them look really
> promising.
>...

gcc being a memory hog on for C++ code is a hard problem,
and debug symbols for C++ code can be a problem since
they might be > 1 GB for some binaries.

But gcc needing more than 4 GB for a small C kernel driver is not
a problem for the "Ability to further support 32bit architectures",
that's a gcc bug that should be reported upstream just like you wouldn't
suggest dropping amd64 if gcc would ICE on one kernel driver on that
architecture.

> Bastian

cu
Adrian

Re: Ability to further support 32bit architectures

[Bastian Blank]

On Thu, Jan 11, 2024 at 10:50:31AM +0100, John Paul Adrian Glaubitz wrote:
> > As it is now, we will not be able to provide a kernel for maybe all
> > 32bit architectures for Trixie.
> I don't think that this would be a reasonable decision. We're preparing to 
> switch
> 32-bit architectures over to time64_t. Disabling 32-bit kernel builds would 
> make
> this whole work moot.

This is completely unrelated.  Userland != kernel.  And people already
talked about only supporting userland for those architectures.

> FWIW, both m68k and powerpc are not affected by this bug, the powerpc build 
> fails
> because of a packaging problem.

Actually powerpc fails for exactly the same reason:

| cc1: out of memory allocating 135266296 bytes after a total of 244908032 bytes
| make[9]: *** [/<>/scripts/Makefile.build:248: 
drivers/media/pci/solo6x10/solo6x10-p2m.o] Error 1

https://buildd.debian.org/status/fetch.php?pkg=linux=powerpc=6.7-1%7Eexp1=1704796355=0

Bastian

-- 
Earth -- mother of the most beautiful women in the universe.
-- Apollo, "Who Mourns for Adonais?" stardate 3468.1

Re: Ability to further support 32bit architectures

[Bastian Blank]

Hi

On Thu, Jan 11, 2024 at 09:48:34AM +, Dimitri John Ledkov wrote:
> Disabling debug symbols, enabling debug symbol zstd compression, using
> split debug symbols (disabled BTF usage) should help here.

Okay, maybe more workarounds exist.  But none of them look really
promising.

> Separately, I wish we had cross-builders available, and cross-build
> i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
> compiler.

Real cross-builders would use some fast amd64/arm64/ppc64el (and for
amd64 also reasonably cheap) machines to build all other architectures.

Bastian

-- 
You're dead, Jim.
-- McCoy, "Amok Time", stardate 3372.7

Re: Ability to further support 32bit architectures

[John Paul Adrian Glaubitz]

Hello Dimitri,

On Thu, 2024-01-11 at 09:48 +, Dimitri John Ledkov wrote:
> Separately, I wish we had cross-builders available, and cross-build
> i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
> compiler.

Helmut Grohne is actually working towards cross-builders and I think there
is a chance we might see these in the foreseeable future.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Ability to further support 32bit architectures

[Dimitri John Ledkov]

Hi,

On Thu, 11 Jan 2024 at 09:42, Bastian Blank  wrote:
>
> Hi
>
> Linux 6.7 fails to build on at least i386 and armhf.  Even it now
> manages to make the compiler fail to allocate memory:
> | cc1: out of memory allocating 135266296 bytes after a total of 235675648 
> bytes
>
> Right now both fail on the same driver, so a short team workaround would
> be to disable it.  But we need a long term fix, and quickly.
>
> As it is now, we will not be able to provide a kernel for maybe all
> 32bit architectures for Trixie.
>
> Bastian
>

Disabling debug symbols, enabling debug symbol zstd compression, using
split debug symbols (disabled BTF usage) should help here.

Separately, I wish we had cross-builders available, and cross-build
i386/armhf kernels from amd64/arm64 and thus having access to 64-bit
compiler.

I am experiencing the same issue with the armhf kernels on my infrastructure.

-- 
Dimitri

Sent from Ubuntu Pro
https://ubuntu.com/pro

Re: Ability to further support 32bit architectures

[John Paul Adrian Glaubitz]

Hi!

On Thu, 2024-01-11 at 10:25 +0100, Bastian Blank wrote:
> Linux 6.7 fails to build on at least i386 and armhf.  Even it now
> manages to make the compiler fail to allocate memory:
> > cc1: out of memory allocating 135266296 bytes after a total of 235675648 
> > bytes
> 
> Right now both fail on the same driver, so a short team workaround would
> be to disable it.  But we need a long term fix, and quickly.

The long term fix would be fixing this driver so a single compilation unit 
doesn't
take several gigabytes of RAM.

> As it is now, we will not be able to provide a kernel for maybe all
> 32bit architectures for Trixie.

I don't think that this would be a reasonable decision. We're preparing to 
switch
32-bit architectures over to time64_t. Disabling 32-bit kernel builds would make
this whole work moot.

FWIW, both m68k and powerpc are not affected by this bug, the powerpc build 
fails
because of a packaging problem.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913