Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Arnd Bergmann]

[some mailing lists appear to have classified the earlier mail as spam,
 it was quite long and contained a lot of links. See
 https://lists.debian.org/debian-arm/2020/03/msg00032.html for the
 start of the thread if you did not get that]

On Wed, Mar 11, 2020 at 3:37 PM Lukasz Majewski  wrote:
> >   - stat()/fstat()/lstat(), nanosleep(), wait3()/wait4(), ppoll_chk()
> > are some of the other interfaces that take a time_t based
> > argument and need to grow a time64 version to avoid an ABI
> > mismatch.
>
> The stat() and friends will use statx internally, which supports 64 bit
> time from the outset.
> Unfortunately, it hasn't been yet converted.
>
> As statx was added in 4.1 (IIRC) - after the minimal supported Linux
> kernel version is bumped to this version (from 3.2 as now) it all will
> be fixed.

The problem I had with these was not on the kernel API side (I
still have CONFIG_COMPAT_32BIT_TIME enabled for now) but
on the application side. In particular, the 'struct stat' definition
(when __USE_XOPEN2K8 is defined) contains

 struct timespec st_atim;

and similar fields that are interpreted using 64-bit time_t in the
application including the header, but with 32-bit time_t inside of
the ___fxstat64() implementation in glibc. The problem is caused
by the mismatched ABI, not by the time_t overflow, in the same
way that happens in the nptl library callers and in the other interfaces
I mentioned.
This is also what seems to cause most of the testcase failures
when the tests are built with __TIME_BITS=64.

> >   - The timeval prototype appears to be broken, as it's missing
> > padding on architectures without native alignment of __time64
> > (e.g. i386) and on all big-endian architectures.
> >
>
> You mean the one "exported" to the system or one, which is internal to
> glibc (from ./include/time.h)?

I mean in the installed headers, where I get (after preprocessing)

typedef long long __time64_t;
typedef long __suseconds_t;
struct timeval
{
  __time64_t tv_sec;
  __suseconds_t tv_usec;
};

On i386 and m68k, this leads to a 12 byte structure when the kernel
interfaces expect a 16 byte structure. All other 32-bit architectures add
four byte padding at the end, so the size is correct, but on big-endian
systems, kernel also expects padding *before* tv_usec, in the same
way as the timespec definition does. IIRC the best way to handle this
is with a 64-bit suseconds_t, e.g.  by adding a __suseconds64_t
defined the same way as __time64_t.

> > I have spent more time on this now than I had planned, and don't
> > expect to do further work on it anytime soon, but I hope my summary
> > is useful to others that are going to need this later.  I can
> > obviously share my patches and build artifacts if anyone needs them.
>
> Could you upload them to any server? (kernel.org or github)?

I have uploaded the modified debian-glibc and rebootstrap
sources to https://git.linaro.org/people/arnd/ now, this should
be all that's needed to recreate the build, using these steps:

- build an x86-64 debian glibc-2.31 package (binary plus source)
  based on the glibc package data
- create a pbuilder instance with that available as a source to apt
- log into the pbuilder and run the modified bootstrap.sh according
  to information on the pbuilder web page.

The binary packages I created are not as useful, as they would
not work with any build of glibc, neither the version I built, nor
any fixed one. I could find a way to send that to you in private,
but it's hundreds of megabytes.

Unfortunately I lost the build logs during a crash yesterday.

> > There are two additional approaches that would likely get a Debian
> > bootstrap further, but that I have not tried as they were previously
> > dismissed:
> >
> > * Adding a time64 armhf as a separate (incompatible) target in glibc
> >   that defines __TIMESIZE==64 and a 64-bit __time_t would avoid
> >   most of the remaining ABI issues and put armhf-time64 in the same
> >   category as riscv32 and arc, but this idea was so far rejected by
> > the glibc maintainers.
>
> As fair as I know riscv32 and arc will use generic syscall interface.
> The arm32 bit doesn't support it - so the code from those two
> aforementioned ports will not be used.

The differences between the generic syscall interface and the arm
version are much smaller than ABI differences between the ABIs
for 32-bit __time_t and the __time_t == __time64_t version.

In particular, as such a new port could mandate a minimum kernel
of v5.1, it could just use all the time64 syscalls, the split sysvipc
and statx as a baseline.

  Arnd
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Rich Felker]

On Wed, 11 Mar 2020 13:52:00 +01000, Arnd Bergmann wrote:
> As discussed before, I tried using the rebootstrap tool [1] to see what
> problems come up once the entire distro gets rebuilt.  Based on Lukasz'
> recommendation, I tried the 'y2038_edge' branch with his experimental
> glibc  patches [2], using commit c2de7ee9461 dated 2020-02-17.
> 
> Here is a rough summary of what I tried, what worked, and what problems
> I ran into:
> 
> [...]
> 
> * Actually building a time64 version of glibc turned out to be
>   harder, including some issues discussed on the libc mailing list[5]:
> 
>   - Always setting -D_TIME_BITS=64 in the global compiler flags for
> the distro breaks both the native 64-bit (x86_64) build and the
> 32-bit build, as glibc itself expects to be built without this.

This seems like a small issue, but glibc should probably either remove
it from CFLAGS in the build system or at least catch it at configure
time and error out, so that it's not confusing when it breaks.

>   - Removing the time32 symbols from the glibc shared object did not
> work as they are still used (a lot) internally, and by the testsuite.

That they're used internally sounds like a major problem; anywhere
they're being used internally potentially has hidden Y2038 bugs. This
is also why I'm concerned about glibc's approach of not building
itself with _TIME_BITS=64, and just undefining it or doing something
else in the wrapper files for the legacy time32 symbols.

>   - I tried converting all the internal symbols to use the time64
> variants with the correct types (e.g. __clock_gettime64() instead
> of __clock_gettime()), but then ran into a lot of APIs that take
> timespec/timeval/... arguments and pass them down into internal
> functions. These seem to all be bugs that require adding a time64
> version of the external ABI.

This also sounds bad. The set of functions that need time64 versions
has little to do with the syscalls that needed changing, and rather is
a matter of which functions have time_t-derived types in their public
interfaces. I think it would be useful to compare current glibc
patches against musl's "nm -D libc.so | grep time64", which has 63
lines. There may be more functions glibc needs to have time64 versions
of because of of additional functionality it supports, but if it's
lacking any of the ones musl has, that's probably indicative of a bug.
I'm attaching the list.

>   - The nptl and sunrpc portions have numerous interfaces with
> 'timeval' or 'timespec' arguments that each cause an ABI break.

nptl is essential but I think sunrpc is pure legacy ABI and not
intended to be linkable in the future.

>   - stat()/fstat()/lstat(), nanosleep(), wait3()/wait4(), ppoll_chk()
> are some of the other interfaces that take a time_t based
> argument and need to grow a time64 version to avoid an ABI mismatch.

And this requires a decision whether to keep the __xstat framework
with a new _STAT_VER or make a new symbol.

> I have spent more time on this now than I had planned, and don't expect
> to do further work on it anytime soon, but I hope my summary is useful
> to others that are going to need this later.  I can obviously share
> my patches and build artifacts if anyone needs them. There are two
> additional approaches that would likely get a Debian bootstrap further,
> but that I have not tried as they were previously dismissed:

It's really amazing how much time you put into this. Thank you!!

Rich
__adjtime64
__adjtimex_time64
__aio_suspend_time64
__clock_adjtime64
__clock_getres_time64
__clock_gettime64
__clock_nanosleep_time64
__clock_settime64
__cnd_timedwait_time64
__ctime64
__ctime64_r
__difftime64
__dlsym_time64
__fstat_time64
__fstatat_time64
__ftime64
__futimens_time64
__futimes_time64
__futimesat_time64
__getitimer_time64
__getrusage_time64
__gettimeofday_time64
__gmtime64
__gmtime64_r
__localtime64
__localtime64_r
__lstat_time64
__lutimes_time64
__mktime64
__mq_timedreceive_time64
__mq_timedsend_time64
__mtx_timedlock_time64
__nanosleep_time64
__ppoll_time64
__pselect_time64
__pthread_cond_timedwait_time64
__pthread_mutex_timedlock_time64
__pthread_rwlock_timedrdlock_time64
__pthread_rwlock_timedwrlock_time64
__pthread_timedjoin_np_time64
__recvmmsg_time64
__sched_rr_get_interval_time64
__select_time64
__sem_timedwait_time64
__semtimedop_time64
__setitimer_time64
__settimeofday_time64
__sigtimedwait_time64
__stat_time64
__stime64
__thrd_sleep_time64
__time64
__timegm_time64
__timer_gettime64
__timer_settime64
__timerfd_gettime64
__timerfd_settime64
__timespec_get_time64
__utime64
__utimensat_time64
__utimes_time64
__wait3_time64
__wait4_time64
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Arnd Bergmann]

On Fri, Mar 13, 2020 at 9:22 PM Rich Felker  wrote:
>
> On Wed, 11 Mar 2020 13:52:00 +01000, Arnd Bergmann wrote:
> > As discussed before, I tried using the rebootstrap tool [1] to see what
> > problems come up once the entire distro gets rebuilt.  Based on Lukasz'
> > recommendation, I tried the 'y2038_edge' branch with his experimental
> > glibc  patches [2], using commit c2de7ee9461 dated 2020-02-17.
> >
> > Here is a rough summary of what I tried, what worked, and what problems
> > I ran into:
> >
> > [...]
> >
> > * Actually building a time64 version of glibc turned out to be
> >   harder, including some issues discussed on the libc mailing list[5]:
> >
> >   - Always setting -D_TIME_BITS=64 in the global compiler flags for
> > the distro breaks both the native 64-bit (x86_64) build and the
> > 32-bit build, as glibc itself expects to be built without this.
>
> This seems like a small issue, but glibc should probably either remove
> it from CFLAGS in the build system or at least catch it at configure
> time and error out, so that it's not confusing when it breaks.

Right, that would make sense. For the test suite though, I guess
it would actually need to run each test case that references
time_t both ways.

> >   - Removing the time32 symbols from the glibc shared object did not
> > work as they are still used (a lot) internally, and by the testsuite.
>
> That they're used internally sounds like a major problem; anywhere
> they're being used internally potentially has hidden Y2038 bugs. This
> is also why I'm concerned about glibc's approach of not building
> itself with _TIME_BITS=64, and just undefining it or doing something
> else in the wrapper files for the legacy time32 symbols.

I thought this was the long-term plan. Working on the ABI first and
then changing the implementation may help speed up the timeline
before distro-level work can start, but OTOH removing all the 32-bit
codepaths from the implementation first makes it more likely to find
all relevant bits.

> >   - The nptl and sunrpc portions have numerous interfaces with
> > 'timeval' or 'timespec' arguments that each cause an ABI break.
>
> nptl is essential but I think sunrpc is pure legacy ABI and not
> intended to be linkable in the future.

That would be helpful, but what does it mean for distro packages
that link against it today?
codesearch.debian.org e.g. finds nfs-utls, nis, libtirpc, ntirpc
and nfswatch including . Can these just use a
replacement that is built with 64-bit time_t then?

 Arnd
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Rich Felker]

On Mon, Mar 16, 2020 at 03:28:43PM +0100, Arnd Bergmann wrote:
> On Fri, Mar 13, 2020 at 9:22 PM Rich Felker  wrote:
> >
> > On Wed, 11 Mar 2020 13:52:00 +01000, Arnd Bergmann wrote:
> > > As discussed before, I tried using the rebootstrap tool [1] to see what
> > > problems come up once the entire distro gets rebuilt.  Based on Lukasz'
> > > recommendation, I tried the 'y2038_edge' branch with his experimental
> > > glibc  patches [2], using commit c2de7ee9461 dated 2020-02-17.
> > >
> > > Here is a rough summary of what I tried, what worked, and what problems
> > > I ran into:
> > >
> > > [...]
> > >
> > > * Actually building a time64 version of glibc turned out to be
> > >   harder, including some issues discussed on the libc mailing list[5]:
> > >
> > >   - Always setting -D_TIME_BITS=64 in the global compiler flags for
> > > the distro breaks both the native 64-bit (x86_64) build and the
> > > 32-bit build, as glibc itself expects to be built without this.
> >
> > This seems like a small issue, but glibc should probably either remove
> > it from CFLAGS in the build system or at least catch it at configure
> > time and error out, so that it's not confusing when it breaks.
> 
> Right, that would make sense. For the test suite though, I guess
> it would actually need to run each test case that references
> time_t both ways.

Indeed.

> > >   - Removing the time32 symbols from the glibc shared object did not
> > > work as they are still used (a lot) internally, and by the testsuite.
> >
> > That they're used internally sounds like a major problem; anywhere
> > they're being used internally potentially has hidden Y2038 bugs. This
> > is also why I'm concerned about glibc's approach of not building
> > itself with _TIME_BITS=64, and just undefining it or doing something
> > else in the wrapper files for the legacy time32 symbols.
> 
> I thought this was the long-term plan. Working on the ABI first and
> then changing the implementation may help speed up the timeline
> before distro-level work can start, but OTOH removing all the 32-bit
> codepaths from the implementation first makes it more likely to find
> all relevant bits.

In my experience it was easiest to do *with* the aid of the public
header redirections applying internally to libc as well. I don't
really understand how glibc is trying to make this easier by avoiding
that.

> > >   - The nptl and sunrpc portions have numerous interfaces with
> > > 'timeval' or 'timespec' arguments that each cause an ABI break.
> >
> > nptl is essential but I think sunrpc is pure legacy ABI and not
> > intended to be linkable in the future.
> 
> That would be helpful, but what does it mean for distro packages
> that link against it today?
> codesearch.debian.org e.g. finds nfs-utls, nis, libtirpc, ntirpc
> and nfswatch including . Can these just use a
> replacement that is built with 64-bit time_t then?

libtirpc is the replacement. I wasn't aware if uses libc-provided rpc
headers (presumably only if they exist, since folks are using it fine
on musl) but even if so I think the types will automatically update
when time_t changes. Of course that leaves the libtirpc ABI dependent
on which time_t is used.

Rich
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Arnd Bergmann]

On Mon, Mar 16, 2020 at 3:47 PM Rich Felker  wrote:

> libtirpc is the replacement. I wasn't aware if uses libc-provided rpc
> headers (presumably only if they exist, since folks are using it fine
> on musl) but even if so I think the types will automatically update
> when time_t changes. Of course that leaves the libtirpc ABI dependent
> on which time_t is used.

Ok, makes sense. I suppose it just provides a header with the same
name then.

  Arnd
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Arnd Bergmann]

On Mon, Mar 16, 2020 at 11:12 PM Lukasz Majewski  wrote:
> > On Fri, Mar 13, 2020 at 9:22 PM Rich Felker  wrote:
> > > >   - Removing the time32 symbols from the glibc shared object did
> > > > not work as they are still used (a lot) internally, and by the
> > > > testsuite.
> > >
> > > That they're used internally sounds like a major problem; anywhere
> > > they're being used internally potentially has hidden Y2038 bugs.
> > > This is also why I'm concerned about glibc's approach of not
> > > building itself with _TIME_BITS=64, and just undefining it or doing
> > > something else in the wrapper files for the legacy time32 symbols.
> >
> > I thought this was the long-term plan. Working on the ABI first and
> > then changing the implementation may help speed up the timeline
> > before distro-level work can start, but OTOH removing all the 32-bit
> > codepaths from the implementation first makes it more likely to find
> > all relevant bits.
>
> If I understood the question correctly - the problem is with having
> glibc ABI consistent. This requires having 64 bit types for relevant
> functions. For example the __clock_settime64 accepts struct
> __timespec64 parameter which:
>
> - Is aliased to "normal" struct timespec on machines with
>   __WORDSIZE==64 (x32 is a special case)
>
> - The struct __timespec64 is used on 32 bit machines
>
> As a result the glibc is ready to handle 64 bit time always (with
> clock_settime on __WORDSIZE==64 or clock_settime64 otherwise), as
> exported struct timespec fields size vary depending on the machine for
> which glibc is built.

I think we all understand the need to duplicate each interface that
passes a data type derived from time_t, and how the aliasing works,

The point above is purely for the internal implementation. The approach
that I have picked for the kernel and Rich did for musl was that
internal code never sees the old __time_t definition for any data
structure or function call, those are only used to define the wrappers
for 32-bit architectures that provide the legacy interfaces.

  Arnd
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Ben Hutchings]

On Mon, 2020-03-16 at 16:02 +0100, Arnd Bergmann wrote:
> On Mon, Mar 16, 2020 at 3:47 PM Rich Felker  wrote:
> 
> > libtirpc is the replacement. I wasn't aware if uses libc-provided rpc
> > headers (presumably only if they exist, since folks are using it fine
> > on musl) but even if so I think the types will automatically update
> > when time_t changes. Of course that leaves the libtirpc ABI dependent
> > on which time_t is used.
> 
> Ok, makes sense. I suppose it just provides a header with the same
> name then.

* nfs-utils build-depends on libtirpc-dev, and isn't using the glibc
SunRPC headers except for .  libtirpc's 
specifically avoids declaring things that are also declared in glibc's
.

* ntirpc is a different port of the SunRPC code, used by nfs-ganesha.

* nis and nfswatch really are using the glibc SunRPC headers.

Ben.

-- 
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
 Manchester, M1 2HF, United Kingdom

___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Florian Weimer]

* Ben Hutchings:

> On Mon, 2020-03-16 at 16:02 +0100, Arnd Bergmann wrote:
>> On Mon, Mar 16, 2020 at 3:47 PM Rich Felker  wrote:
>> 
>> > libtirpc is the replacement. I wasn't aware if uses libc-provided rpc
>> > headers (presumably only if they exist, since folks are using it fine
>> > on musl) but even if so I think the types will automatically update
>> > when time_t changes. Of course that leaves the libtirpc ABI dependent
>> > on which time_t is used.
>> 
>> Ok, makes sense. I suppose it just provides a header with the same
>> name then.
>
> * nfs-utils build-depends on libtirpc-dev, and isn't using the glibc
> SunRPC headers except for .  libtirpc's 
> specifically avoids declaring things that are also declared in glibc's
> .
>
> * ntirpc is a different port of the SunRPC code, used by nfs-ganesha.
>
> * nis and nfswatch really are using the glibc SunRPC headers.

Which part of NIS?  There's a new upstream for libnsl
 and the NSS module
.  (There is a nisplus module
as well.)

All these use libtirpc and support IPv6 in addition to IPv4.  As far
as I know, it is possible to build a full NIS stack without relying on
any of the legacy glibc code.

(I don't know about nfswatch.)
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Ben Hutchings]

On Fri, 2020-03-20 at 00:09 +0100, Florian Weimer wrote:
> * Ben Hutchings:
> 
> > On Mon, 2020-03-16 at 16:02 +0100, Arnd Bergmann wrote:
> > > On Mon, Mar 16, 2020 at 3:47 PM Rich Felker  wrote:
> > > 
> > > > libtirpc is the replacement. I wasn't aware if uses libc-provided rpc
> > > > headers (presumably only if they exist, since folks are using it fine
> > > > on musl) but even if so I think the types will automatically update
> > > > when time_t changes. Of course that leaves the libtirpc ABI dependent
> > > > on which time_t is used.
> > > 
> > > Ok, makes sense. I suppose it just provides a header with the same
> > > name then.
> > 
> > * nfs-utils build-depends on libtirpc-dev, and isn't using the glibc
> > SunRPC headers except for .  libtirpc's 
> > specifically avoids declaring things that are also declared in glibc's
> > .
> > 
> > * ntirpc is a different port of the SunRPC code, used by nfs-ganesha.
> > 
> > * nis and nfswatch really are using the glibc SunRPC headers.
> 
> Which part of NIS?  There's a new upstream for libnsl
>  and the NSS module
> ;.  (There is a nisplus module
> as well.)

This is Debian's "nis" source package, which is a bundle of yp-tools,
ypserv, and ypbind-mt from the same upstream author.  It's unmaintained
and has lots of bug reports in Debian.

> All these use libtirpc and support IPv6 in addition to IPv4.  As far
> as I know, it is possible to build a full NIS stack without relying on
> any of the legacy glibc code.
> 
> (I don't know about nfswatch.)

The upstream for that is . 
The current Fedora package is patched to use libtirpc.

Ben.

-- 
Ben Hutchings, Software Developer Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
 Manchester, M1 2HF, United Kingdom

___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Steve McIntyre]

Hey Arnd,

Catching up on this thread a little late, sorry... :-/

On Wed, Mar 11, 2020 at 01:52:00PM +0100, Arnd Bergmann wrote:
>As discussed before, I tried using the rebootstrap tool [1] to see what
>problems come up once the entire distro gets rebuilt.  Based on Lukasz'
>recommendation, I tried the 'y2038_edge' branch with his experimental
>glibc  patches [2], using commit c2de7ee9461 dated 2020-02-17.
>
>Here is a rough summary of what I tried, what worked, and what problems
>I ran into:
>
>* Building a Debian package from this was fairly straightforward, using
>  the 2.31 branch in the package git repository[3] after replacing the
>  debian/patches/git-updates.diff file with one generated from [2] and
>  disabling the hurd patches because of conflicts.
>
>* After installing the modified x86 glibc package, I ran into a runtime
>  bug in [4], which needs to pass AT_FDCWD instead of 0 to avoid
>  ENOTDIR errors.
>
>* Bootstrapping a regular time32 Debian armhf with this libc took me
>  a few days to get right, but that was mostly for getting familiar
>  with rebootstrap and running into known issues unrelated to time64
>  or the glibc changes.

Cool!



>* There is an open question regarding the name of the Debian
>  architecture. For my experiments, I kept using the 'armhf' name
>  unmodified, though there seems to be a general feeling that using a
>  different name would be required to address the broad incompatibilities
>  between time32 and time64 versions of all the libraries in the
>  distro. Gradually changing them won't work because of the timeline and
>  the number of affected libraries. However, the new name of the distro
>  also implies having a distinct target triplet, which must then be known
>  by glibc along with everything else using config.guess/config.sub. I
>  expect this topic to require a lot more discussion.

ACK. I'm about to prod on this again.

>* Continuing with the rebootstrap build despite the known glibc issues
>  and the open question on the architecture name went surprisingly
>  well, only two out of the 152 source packages I built had
>  compile-time problems:
>
>  - building the final gcc failed in libsanitizer, which has
>compile-time checks to ensure some libc data structures have the
>expected layout. It noticed that 'struct timeb' and 'struct dirent'
>are different based on _TIME_BITS and _FILE_OFFSET_BITS. I disabled
>the checks to be able to continue. To this properly, the library
>has to learn about the new data structures as well. I opened a
>bug report against the library[7].
>
>  - libpreludecpp12 failed to build because of checks for changes
>in the exported functions, which are different with time64.
>I disabled the checks. Once we have agreed on a new debian
>architecture name, the symbols can be made arch specific.

Yup.

>* After everything was built, I tried installing the packages into
>  a chroot with qemu-debootstrap, which failed because I had
>  configured the glibc to assume it's running on a new kernel
>  while the qemu-user binary I had lacks the new syscalls.
>  I believe this is fixed in upstream qemu, but did not try that.
>
>* Trying to install again I used a clean debian-arm64 installation
>  running in qemu-system-aarch64, and attempted installing the
>  armhf packages using a regular debootstrap, running the 32-bit
>  binaries in compat mode of a recent arm64 kernel. This partially
>  worked and I could chroot into the system and use a shell, but
>  ultimately the debootstrap did not complete because of errors.
>  I saw that 'tar' had failed because of the stat() glibc ABI mismatch
>  breaking its private gnulib fdutimens() implementation, and this is
>  where I gave up.

Nod. :-/ I think it's time that somebody else picked up from you here.

>I have spent more time on this now than I had planned, and don't expect
>to do further work on it anytime soon, but I hope my summary is useful
>to others that are going to need this later.  I can obviously share
>my patches and build artifacts if anyone needs them. There are two
>additional approaches that would likely get a Debian bootstrap further,
>but that I have not tried as they were previously dismissed:
>
>* Adding a time64 armhf as a separate (incompatible) target in glibc
>  that defines __TIMESIZE==64 and a 64-bit __time_t would avoid
>  most of the remaining ABI issues and put armhf-time64 in the same
>  category as riscv32 and arc, but this idea was so far rejected by the
>  glibc maintainers. Depending on how hard this turns out to be,
>  it could be used to get to the point of self-hosting though, and
>  help find time64 related bugs in the rest of the distro.

OK. I'm thinking it's probably not worth it?

>* Doing the bootstrap using a musleabihf target instead of gnueabihf
>  would avoid the current issues internal to glibc-y2038, but instead
>  lead to new problems with packages that do not currently work with
>  musl. Adelie Linux has shown that

Re: [Y2038] Trying Debian/armhf rebootstrap with time64

[Arnd Bergmann]

On Mon, Mar 23, 2020 at 7:21 PM Steve McIntyre  wrote:
> On Wed, Mar 11, 2020 at 01:52:00PM +0100, Arnd Bergmann wrote:
> >* Adding a time64 armhf as a separate (incompatible) target in glibc
> >  that defines __TIMESIZE==64 and a 64-bit __time_t would avoid
> >  most of the remaining ABI issues and put armhf-time64 in the same
> >  category as riscv32 and arc, but this idea was so far rejected by the
> >  glibc maintainers. Depending on how hard this turns out to be,
> >  it could be used to get to the point of self-hosting though, and
> >  help find time64 related bugs in the rest of the distro.
>
> OK. I'm thinking it's probably not worth it?

This depends on the timeline of Lukasz' work. My feeling is that there is still
quite a bit to be done before it's worth trying the Debian bootstrap again.

If you or someone else wants to continue where I stopped with the Debian
rebuilding without waiting for the complete glibc port, adding a new armhf
target to glibc on top of the current glibc-y2038 tree is probably a quicker
way to get something that builds and boots. I don't know how much work
exactly there would be for this approach, but my feeling is that it's not that
much after looking at the kind of problems I ran into, and at the state of
the riscv32 port that uses the same approach.

Arnd
___
Y2038 mailing list
Y2038@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/y2038