Re: Options for 64-bit time_t support on 32-bit architectures

[Florian Weimer]

* Simon McVittie:

> On Fri, 19 Jul 2019 at 15:13:00 +0300, Adrian Bunk wrote:
>> Remaining usecases of i386 will be old binaries, some old Linux binaries 
>> but especially old software (including many games) running in Wine.
>> Old Linux binaries will still need the old 32bit time_t.
>
> Based on background from my contributions to the Steam Runtime:
>
> I don't have numbers, but you might be surprised how many Linux-supporting
> games are 32-bit. The Steam client itself is currently also 32-bit
> (with some 64-bit subprocesses); this is somewhat deliberate, to act as
> a canary for whether 32-bit code works at all, particularly when combined
> with graphics.
>
> The Steam Runtime (a LD_LIBRARY_PATH library bundle used to run Steam and
> Steam games) is built on an increasingly ancient version of Ubuntu, but
> it tries to use newer libraries of the same SONAME from the host system
> where available, which they often will be, because people who install
> Steam probably also install Wine, which has 32-bit dependencies. If those
> libraries have an incompatible ABI involving 64-bit time_t, and it is used
> at the ABI "surface" between a host-system library and a Steam Runtime
> library or the game, then 32-bit games, and the Steam client itself,
> will crash.

We could in theory bump soname for these libraries, but that has the
unfortunate side effect that it will likely leak to 64-bit
architectures, creating more work for everyone.

I don't see a good way to maintain those libraries with a single-ABI
approach.  So if that's an important use case, it would be a fairly
strong case against it, I think.

Re: Options for 64-bit time_t support on 32-bit architectures

[Aurelien Jarno]

On 2019-07-19 15:13, Adrian Bunk wrote:
> There are two current release architectures where it is at least 
> imaginable that they will still be around closer to the year 2038:
> i386 and armhf

With the current way packages are built, i.e. natively on the same
architecture, I don't see 32-bit architectures being able to survive
more than 2 more release cycles. Even with LTS and extended LTS, this is
still far from 2038.

The toolchain on 32-bit architectures reaches the 4GB address space
limit more and more often and people have to deploy tricks to workaround
that. We might want to use cross-compilers on those architectures, but
nowadays it's much more than only GCC.

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: Options for 64-bit time_t support on 32-bit architectures

[Adam Borowski]

On Fri, Jul 19, 2019 at 11:19:23PM +0300, Adrian Bunk wrote:
> On Fri, Jul 19, 2019 at 07:13:28PM +0200, Florian Weimer wrote:
> > Similar to the LFS support, with the
> > additional property that binaries built in either mode should continue
> > to work on kernels which predate support for the *_time64 system
> > calls.
> 
> Debian does not support running on kernels older than the one in the
> previous stable release.

Stretch supports running on Squeeze's kernel (-3 releases), Buster supports
running on Wheezy's kernel (-3 releases).

> E.g. Qt in Debian 9 unconditionally uses the getrandom syscall that is 
> not in kernel 3.16 in Debian 7.

Individual packages may fail, yeah.  No one is really going to run GUI stuff
with an ancient kernel.  It's really widespread in hosting scenarios, though
-- and for big servery stuff, the running kernel might be still-supported
2.6.18 while 3.10 still gets newest shiniest stuff.  (But not getrandom,
dammit!).

The time64 syscalls got added only in 5.1, which is not even in Buster.

And 32-bit hardware notoriously needs ancient vendor kernels.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ At least spammers get it right: "Hello beautiful!".
⠈⠳⣄

Re: Options for 64-bit time_t support on 32-bit architectures

[Aurelien Jarno]

On 2019-07-19 23:19, Adrian Bunk wrote:
> Support for the Intel Quark was dropped when the i386 baseline was 
> raised from 586 to 686 in stretch, so that's already irrelevant for
> the Debian i386 port.

Intel Quark has never been supported by Debian due to errata #71538,
which requires to omit the LOCK prefix in the compiled code.

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: Options for 64-bit time_t support on 32-bit architectures

[Florian Weimer]

* Adrian Bunk:

> On Fri, Jul 19, 2019 at 07:13:28PM +0200, Florian Weimer wrote:
>> * Adrian Bunk:
>>...
>> For comparison, the original plan was to provide a macro, perhaps
>> -D_TIME_BITS=32 and -D_TIME_BITS=64, to select at build time which ABI
>> set is used (“dual ABI”).
>
> To me this would sound like more trouble than a clear break,
> similar to the mostly working dual OpenSSL 1.0 and 1.1 support
> in stretch.

Could be.  But it would enable keeping i386 at the old ABI while still
building the distribution with newer glibc versions with current
kernel headers (the libc-alpha discussion is evolving regarding the
precise nature of the enablement approach).

Other 32-bit architectures could opt to do the transition now.

>> Similar to the LFS support, with the
>> additional property that binaries built in either mode should continue
>> to work on kernels which predate support for the *_time64 system
>> calls.
>
> Debian does not support running on kernels older than the one in the
> previous stable release.
>
> E.g. Qt in Debian 9 unconditionally uses the getrandom syscall that is 
> not in kernel 3.16 in Debian 7.

The 64-bit system calls arrived in Linux 5.1, so I think the fallback
will be needed for quite some time.

Re: Options for 64-bit time_t support on 32-bit architectures

[Adrian Bunk]

On Fri, Jul 19, 2019 at 07:13:28PM +0200, Florian Weimer wrote:
> * Adrian Bunk:
>...
> For comparison, the original plan was to provide a macro, perhaps
> -D_TIME_BITS=32 and -D_TIME_BITS=64, to select at build time which ABI
> set is used (“dual ABI”).

To me this would sound like more trouble than a clear break,
similar to the mostly working dual OpenSSL 1.0 and 1.1 support
in stretch.

"Mostly working" as in Apache segfaulting depending on the load order of 
its modules - it is really hard to get all special cases like plugins 
sorted out correctly in such setups.

> Similar to the LFS support, with the
> additional property that binaries built in either mode should continue
> to work on kernels which predate support for the *_time64 system
> calls.

Debian does not support running on kernels older than the one in the
previous stable release.

E.g. Qt in Debian 9 unconditionally uses the getrandom syscall that is 
not in kernel 3.16 in Debian 7.

>...
> > For i386 the last newly released 32bit-only hardware were some early
> > Intel Atoms 10 years ago, and when the AMD Geode goes out of production
> > soon there might be no hardware in production left.
> > There are still surprisingly many people using Debian on 32bit-only
> > hardware, but in 20 years this will have changed.
> 
> You have thankfully edited out the Intel Quark. 8-)
>...

Support for the Intel Quark was dropped when the i386 baseline was 
raised from 586 to 686 in stretch, so that's already irrelevant for
the Debian i386 port.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Options for 64-bit time_t support on 32-bit architectures

[Simon McVittie]

On Fri, 19 Jul 2019 at 15:13:00 +0300, Adrian Bunk wrote:
> Remaining usecases of i386 will be old binaries, some old Linux binaries 
> but especially old software (including many games) running in Wine.
> Old Linux binaries will still need the old 32bit time_t.

Based on background from my contributions to the Steam Runtime:

I don't have numbers, but you might be surprised how many Linux-supporting
games are 32-bit. The Steam client itself is currently also 32-bit
(with some 64-bit subprocesses); this is somewhat deliberate, to act as
a canary for whether 32-bit code works at all, particularly when combined
with graphics.

The Steam Runtime (a LD_LIBRARY_PATH library bundle used to run Steam and
Steam games) is built on an increasingly ancient version of Ubuntu, but
it tries to use newer libraries of the same SONAME from the host system
where available, which they often will be, because people who install
Steam probably also install Wine, which has 32-bit dependencies. If those
libraries have an incompatible ABI involving 64-bit time_t, and it is used
at the ABI "surface" between a host-system library and a Steam Runtime
library or the game, then 32-bit games, and the Steam client itself,
will crash.

The Steam Runtime also relies on the host system for the OpenGL stack
(in practice Mesa or proprietary NVIDIA drivers), and for glibc itself.

In practice, many of the 32-bit games are not ever going to be recompiled
against a new ABI; the games are no longer developed or actively
supported, and their developers might no longer even be still in business.

Outside the Steam ecosystem, 32-bit games typically rely on host-system
libraries for things like SDL, X11 libraries, audio libraries and graphics
format libraries. It's unfortunate that GTK is one of the libraries
with time_t in its ABI, because GTK 2 is a fairly common choice for
game launcher/frontend programs.

smcv

Re: Options for 64-bit time_t support on 32-bit architectures

[Florian Weimer]

* Adrian Bunk:

> [ only speaking for myself ]
>
> On Thu, Jul 18, 2019 at 11:05:53PM +0200, Florian Weimer wrote:
>>...
>> The consequence is that in order to build 32-bit-time_t libraries
>> (Gtk, for example), an old glibc needs to be kept around.  In
>> practice, it would probably mean that it is impossible to maintain a
>> set of 32-bit-time_t libraries in a classic distribution build
>> environment (with a unified buildroot and native builds).
>>...
>> Do you want to build 32-bit libraries (besides glibc) which are
>> compatible with legacy applications, with a 32-bit time_t, in the
>> future?  Or is a world where time_t is pretty much always 64 bit
>> something that would be acceptable?
>
> So this is an ABI-incompatible change that would result in new Debian 
> architectures, similar to arm (OABI), armel (EABI softfp) and armhf 
> (EABI hardfp) being different Debian architectures for 32bit little 
> endian ARM?

Not quite.  glibc would still be able to run binaries from the old ABI
and the new ABI.  But under the proposal, you would have to use an old
glibc (missing new system call wrappers etc.) if you want to build
libraries that provide interfaces involving 32-bit time_t.

So in practice, it would likely mean a new Debian architecture, or a
de-facto ABI bump for i386 and armhfp.

For comparison, the original plan was to provide a macro, perhaps
-D_TIME_BITS=32 and -D_TIME_BITS=64, to select at build time which ABI
set is used (“dual ABI”).  Similar to the LFS support, with the
additional property that binaries built in either mode should continue
to work on kernels which predate support for the *_time64 system
calls.  The should also use the vDSO as before.  All these
requirements make an implementation quite hairy, hence the desire for
simplification.

> There are two current release architectures where it is at least 
> imaginable that they will still be around closer to the year 2038:
> i386 and armhf

Right.

> For i386 the last newly released 32bit-only hardware were some early
> Intel Atoms 10 years ago, and when the AMD Geode goes out of production
> soon there might be no hardware in production left.
> There are still surprisingly many people using Debian on 32bit-only
> hardware, but in 20 years this will have changed.

You have thankfully edited out the Intel Quark. 8-)

> Remaining usecases of i386 will be old binaries, some old Linux binaries 
> but especially old software (including many games) running in Wine.
> Old Linux binaries will still need the old 32bit time_t.
> Which options are viable from a Wine point of view?

I talked to a Wine developer in the office, and Wine doesn't directly
expose the time_t ABI to Windows binaries (which isn't suprising).
It's also already been ported to 32-bit systems with a 64-bit time_t.
I expect that this is not a determining factor.

> For armhf new hardware might be available long enough to come close
> to the year 2038, this might require a new architecture at some point.

The push for a 64-bit time_t definitely comes from the embedded 32-bit
processor direction.

For glibc, providing a dual ABI configurable at build time one or all
32-bit architectures probably does not make much of a different in
terms of overall effort.  This means that if we need to produce the
dual ABI for i386, armhfp will likely get it as well.

Re: Options for 64-bit time_t support on 32-bit architectures

[Adrian Bunk]

[ only speaking for myself ]

On Thu, Jul 18, 2019 at 11:05:53PM +0200, Florian Weimer wrote:
>...
> The consequence is that in order to build 32-bit-time_t libraries
> (Gtk, for example), an old glibc needs to be kept around.  In
> practice, it would probably mean that it is impossible to maintain a
> set of 32-bit-time_t libraries in a classic distribution build
> environment (with a unified buildroot and native builds).
>...
> Do you want to build 32-bit libraries (besides glibc) which are
> compatible with legacy applications, with a 32-bit time_t, in the
> future?  Or is a world where time_t is pretty much always 64 bit
> something that would be acceptable?

So this is an ABI-incompatible change that would result in new Debian 
architectures, similar to arm (OABI), armel (EABI softfp) and armhf 
(EABI hardfp) being different Debian architectures for 32bit little 
endian ARM?

There are two current release architectures where it is at least 
imaginable that they will still be around closer to the year 2038:
i386 and armhf

For i386 the last newly released 32bit-only hardware were some early
Intel Atoms 10 years ago, and when the AMD Geode goes out of production
soon there might be no hardware in production left.
There are still surprisingly many people using Debian on 32bit-only
hardware, but in 20 years this will have changed.

Remaining usecases of i386 will be old binaries, some old Linux binaries 
but especially old software (including many games) running in Wine.
Old Linux binaries will still need the old 32bit time_t.
Which options are viable from a Wine point of view?

For armhf new hardware might be available long enough to come close
to the year 2038, this might require a new architecture at some point.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Options for 64-bit time_t support on 32-bit architectures

[Florian Weimer]

There is an effort under way to enhance glibc so that it can use the
Y2038 support in the kernel.  The result will be that more 32-bit
architectures can use a 64-bit time_t.  (Currently, it's x86-64 x32
only.)

Originally, the plan was to support both ABIs in glibc for building
new applications, similar to what is currently possible with
-D_FILE_OFFSET_BITS=64 for changing the size of off_t.  However, this
turned out to be difficult to implement, and so far, no one has posted
patches which appear to be reasonably correct and complete.

The latest proposal is a single-ABI mode for development:

  

Old binaries with a 32-bit time_t will continue to run, but new
binaries built against a current glibc will always use a 64-bit time_t
under this approach.

The consequence is that in order to build 32-bit-time_t libraries
(Gtk, for example), an old glibc needs to be kept around.  In
practice, it would probably mean that it is impossible to maintain a
set of 32-bit-time_t libraries in a classic distribution build
environment (with a unified buildroot and native builds).

I do not have a strong opinion about this because I personally do not
care about 32-bit architectures at all (sorry).  I would like to
solicit Debian's feedback on this matter.

Do you want to build 32-bit libraries (besides glibc) which are
compatible with legacy applications, with a 32-bit time_t, in the
future?  Or is a world where time_t is pretty much always 64 bit
something that would be acceptable?