bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Pádraig Brady]

On 29/04/2022 00:56, Paul Eggert wrote:

On 4/27/22 09:42, Pádraig Brady wrote:

Marking this as done in the coreutils bug tracker,
now that this is being tracked in glibc.


This could also be worked around Gnulib for the benefit of 32-bit apps
running with unpatched glibc 2.34 and 2.35, or glibc older than 2.34.
Not sure it's worth the trouble, though, as the fixed glibc should be
universal pretty much everywhere before the year 2038 rolls around.


Yes I was thinking about that,
and also thought it not necessary to deal with in gnulib.

thanks,
Pádraig.

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Paul Eggert]

On 4/27/22 09:42, Pádraig Brady wrote:

Marking this as done in the coreutils bug tracker,
now that this is being tracked in glibc.


This could also be worked around Gnulib for the benefit of 32-bit apps 
running with unpatched glibc 2.34 and 2.35, or glibc older than 2.34. 
Not sure it's worth the trouble, though, as the fixed glibc should be 
universal pretty much everywhere before the year 2038 rolls around.

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Adhemerval Zanella]

On 27/04/2022 13:42, Pádraig Brady wrote:
> Marking this as done in the coreutils bug tracker,
> now that this is being tracked in glibc.
> 
> thank you!
> 
> On 27/04/2022 13:46, Adhemerval Zanella wrote:
> 
>>> On 21/04/2022 14:36, Pádraig Brady wrote:
 That suggests the kernel (statx) returns fine,
 but glibc's fstatat64 is returning the EOVERFLOW.
> 
>> It seems to be a glibc missing support indeed.  The coreutils issues 
>> indicates
>> that lchmodat failed somehow:
>>
>>    if (lchmodat (dst_dirfd, dst_relname, dst_mode | S_IRWXU) != 
>> 0)
>>  {
>>    error (0, errno, _("setting permissions for %s"),
>>   quoteaf (dst_name));
>>    goto un_backup;
>>  }
>>
>> And lchmodat is a gnulib wrapper for fchmodat:
>>
>> CHMODAT_INLINE int
>> lchmodat (int fd, char const *file, mode_t mode)
>> {
>>    return fchmodat (fd, file, mode, AT_SYMLINK_NOFOLLOW);
>> }
>>
>> And since Linux fchmodat syscall does not provide a 'flag' argument (to
>> handle AT_SYMLINK_NOFOLLOW), glibc emulates it through opening a procfs file
>> descriptor, issuing fstatat to check if it is link (since some kernels and
>> filesystem it returns in inconsistent results), and then issue chmod.
>>
>> However, the glibc internal fstat does not use the 64-bit version, which
>> then results in EOVERFLOW.
>>
>> I have opened https://sourceware.org/bugzilla/show_bug.cgi?id=29097 and I 
>> will
>> fix it upstream and backport to 2.34 and 2.35.
> 

It is now fixed on master, 2.35, and 2.34 branches.

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Adhemerval Zanella]

On 26/04/2022 09:44, Arnaud Panaïotis wrote:
> 
> On 21/04/2022 22:59, Pádraig Brady wrote:
>> On 21/04/2022 18:41, Adhemerval Zanella wrote:
>>>
>>>
>>> On 21/04/2022 12:25, Pádraig Brady wrote:
 On 21/04/2022 15:53, Arnaud Panaïotis wrote:
>
> On 21/04/2022 14:36, Pádraig Brady wrote:
>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>>> On 19/04/2022 16:00, Pádraig Brady wrote:
 On 19/04/2022 08:47, Arnaud Panaïotis wrote:
> Hello,
>
> I did not received any feedback from this request right now. Have you
> made any progress on this subject ?
>
> Please let me know the progress for this, or contact me for additional
> information if needed.
>
> I'd like to have a ticket link to follow the advancement of this issue
> (if possible). I'm available to test a patch if you are able to 
> provide
> me one.
>
> Best regards,
>
> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>
>> Hello,
>>
>> I'm working for a client to generate embedded 32-bits Linux Kernel
>> working after y2038 issue.
>>
>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>
>> The Kernel pass y2038 but I found an issue with cp:
>> After analysis, the error occurs when trying to move an empty folder
>> without all user mode rights.
>>
>> Here how to reproduce:
>>
>> # mkdir -p test/test1 folder
>> # chmod u-w test/test1
>> # date -s "2040-04-02"
>> # cp -a test/* folder/
>> cp: setting permissions for 'folder/test1' : Value too large for 
>> defined data type
>>
>> Note: The folder is copied before the error occurs. The copy works
>> fine before y2038.
>>
>>
>> The issue comes from coreutils-9.0/src/cp.c
>>
>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>
>> FYI I had a previous issue while calling lstat function from
>>  which is included in lib/lchmod.c. I used /usr/bin/stat
>> as a workaround.
>>
>>
>> Keep me in touch if you need more information.

 The original mail seems to not have hit the lists, sorry.

 The error suggests that the fstatat() done within lchmod()
 is using a 32 bit time_t component of the stat structure.
 Your kernel is new enough to support the 64 bit equivalent,
 but you don't mention glibc. Can you ensure you're using
 at least glibc 2.34, which added support for the 64 bit variants.

 coreutils is configured by default to enable use
 of the 64 bit variants where available, and you've confirmed
 this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are 
 defined.

 An strace of the cp command would be useful to confirm the problematic 
 syscall.
>>
>> That suggests the kernel (statx) returns fine,
>> but glibc is returning the EOVERFLOW.
>> That suggests fstatat() rather than fstatat64() is being used
>> (inferring that by comparing
>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>
>> So why is fstatat() being used if compiling with
>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>
>> You might confirm what statat is being called with:
>> nm cp | grep U.*statat
>>
>> The redirect from fstatat() in code to the appropriate 64 bit interface
>> should be done with asm redirects and so immune to any undef etc.
>> that gnulib may be doing.
>>
>> So in summary please look at how fstatat() is being referenced
>> on your system (by gnulib).
>>
>> thanks,
>> Pádraig
>
> Hi,
>
> I had to add -D option to nm to avoid "no symbols" error.
>
>   U __fstatat64_time64@GLIBC_2.34
>
> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.

 That looks correct.

 So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
 You might get to the issue more quickly by installing debug symbols for
 your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args

>>>
>>> Assuming a default config option (without --enable-kernel) the
>>> _fstatat64_time64 should first try statx and then the old fstatat64
>>> if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
>>> only happens for later, assuming kernel does not returns anything
>>> bogus.
>>>
>>> What strace shows in this scenario?
>>

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Pádraig Brady]

Marking this as done in the coreutils bug tracker,
now that this is being tracked in glibc.

thank you!

On 27/04/2022 13:46, Adhemerval Zanella wrote:


On 21/04/2022 14:36, Pádraig Brady wrote:

That suggests the kernel (statx) returns fine,
but glibc's fstatat64 is returning the EOVERFLOW.



It seems to be a glibc missing support indeed.  The coreutils issues indicates
that lchmodat failed somehow:

   if (lchmodat (dst_dirfd, dst_relname, dst_mode | S_IRWXU) != 0)
 {
   error (0, errno, _("setting permissions for %s"),
  quoteaf (dst_name));
   goto un_backup;
 }

And lchmodat is a gnulib wrapper for fchmodat:

CHMODAT_INLINE int
lchmodat (int fd, char const *file, mode_t mode)
{
   return fchmodat (fd, file, mode, AT_SYMLINK_NOFOLLOW);
}

And since Linux fchmodat syscall does not provide a 'flag' argument (to
handle AT_SYMLINK_NOFOLLOW), glibc emulates it through opening a procfs file
descriptor, issuing fstatat to check if it is link (since some kernels and
filesystem it returns in inconsistent results), and then issue chmod.

However, the glibc internal fstat does not use the 64-bit version, which
then results in EOVERFLOW.

I have opened https://sourceware.org/bugzilla/show_bug.cgi?id=29097 and I will
fix it upstream and backport to 2.34 and 2.35.

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Arnaud Panaïotis]

On 21/04/2022 22:59, Pádraig Brady wrote:

On 21/04/2022 18:41, Adhemerval Zanella wrote:



On 21/04/2022 12:25, Pádraig Brady wrote:

On 21/04/2022 15:53, Arnaud Panaïotis wrote:


On 21/04/2022 14:36, Pádraig Brady wrote:

On 19/04/2022 16:01, Arnaud Panaïotis wrote:

On 19/04/2022 16:00, Pádraig Brady wrote:

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now.
Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for
additional
information if needed.

I'd like to have a ticket link to follow the advancement of
this issue
(if possible). I'm available to test a patch if you are able to
provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux
Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils
9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty
folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large
for defined data type

Note: The folder is copied before the error occurs. The copy
works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used
/usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
are defined.

An strace of the cp command would be useful to confirm the
problematic syscall.


That suggests the kernel (statx) returns fine,
but glibc is returning the EOVERFLOW.
That suggests fstatat() rather than fstatat64() is being used
(inferring that by comparing
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c

https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)


So why is fstatat() being used if compiling with
-D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64

You might confirm what statat is being called with:
nm cp | grep U.*statat

The redirect from fstatat() in code to the appropriate 64 bit
interface
should be done with asm redirects and so immune to any undef etc.
that gnulib may be doing.

So in summary please look at how fstatat() is being referenced
on your system (by gnulib).

thanks,
Pádraig


Hi,

I had to add -D option to nm to avoid "no symbols" error.

  U __fstatat64_time64@GLIBC_2.34

All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.


That looks correct.

So perhaps the EOVERFLOW is within your glibc's fstatat64
implementation?
You might get to the issue more quickly by installing debug symbols for
your coreutils and/or glibc and using: gdb -tui -args cp rest of cp
args



Assuming a default config option (without --enable-kernel) the
_fstatat64_time64 should first try statx and then the old fstatat64
if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
only happens for later, assuming kernel does not returns anything
bogus.

What strace shows in this scenario?


strace shows statx() returning non error on this 5.15 kernel

Hi,

Sorry for the delay, had trouble to generate all requirements within
Buildroot (now with gdb+coreutils debugs). I can add glibc debug if needed.

quotearg_buffer_restyled (buffer=buffer@entry=0x41b5a0  "",
 buffersize=, buffersize@entry=256,
arg=arg@entry=0x41c5f0 "folder/test1", argsize=4294967295,

    quoting_style=, flags=,
quote_these_too=, left_quote=0x0, right_quote=0x0)
at lib/quotearg.c:262

(gdb)
quotearg_n_options (n=n@entry=0, arg=arg@entry=0x41c5f0 "folder/test1",
argsize=argsize@entry=4294967295, options=0xb630) at
lib/quotearg.c:905

0x00406b46 in copy_reg (src_sb=,
new_dst=, omitted_permissions=, dst_mode=, x=,
dst_name=,

    src_name=) at src/copy.c:1146

/bin/cp: setting permissions for 'folder/test1': Value too large for defined 
data type

I don't have layout src working but layout asm works.

(breakpoint at fstatat, then step by step).

Let me know if you need more details, I'm available for a shared screen
visio if needed (I'm in 

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Adhemerval Zanella]

On 21/04/2022 12:25, Pádraig Brady wrote:
> On 21/04/2022 15:53, Arnaud Panaïotis wrote:
>>
>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
 On 19/04/2022 16:00, Pádraig Brady wrote:
> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>> Hello,
>>
>> I did not received any feedback from this request right now. Have you
>> made any progress on this subject ?
>>
>> Please let me know the progress for this, or contact me for additional
>> information if needed.
>>
>> I'd like to have a ticket link to follow the advancement of this issue
>> (if possible). I'm available to test a patch if you are able to provide
>> me one.
>>
>> Best regards,
>>
>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>
>>> Hello,
>>>
>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>> working after y2038 issue.
>>>
>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>
>>> The Kernel pass y2038 but I found an issue with cp:
>>> After analysis, the error occurs when trying to move an empty folder
>>> without all user mode rights.
>>>
>>> Here how to reproduce:
>>>
>>> # mkdir -p test/test1 folder
>>> # chmod u-w test/test1
>>> # date -s "2040-04-02"
>>> # cp -a test/* folder/
>>> cp: setting permissions for 'folder/test1' : Value too large for 
>>> defined data type
>>>
>>> Note: The folder is copied before the error occurs. The copy works
>>> fine before y2038.
>>>
>>>
>>> The issue comes from coreutils-9.0/src/cp.c
>>>
>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>
>>> FYI I had a previous issue while calling lstat function from
>>>  which is included in lib/lchmod.c. I used /usr/bin/stat
>>> as a workaround.
>>>
>>>
>>> Keep me in touch if you need more information.
>
> The original mail seems to not have hit the lists, sorry.
>
> The error suggests that the fstatat() done within lchmod()
> is using a 32 bit time_t component of the stat structure.
> Your kernel is new enough to support the 64 bit equivalent,
> but you don't mention glibc. Can you ensure you're using
> at least glibc 2.34, which added support for the 64 bit variants.
>
> coreutils is configured by default to enable use
> of the 64 bit variants where available, and you've confirmed
> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are 
> defined.
>
> An strace of the cp command would be useful to confirm the problematic 
> syscall.
>>>
>>> That suggests the kernel (statx) returns fine,
>>> but glibc is returning the EOVERFLOW.
>>> That suggests fstatat() rather than fstatat64() is being used
>>> (inferring that by comparing
>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>>
>>> So why is fstatat() being used if compiling with
>>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>>
>>> You might confirm what statat is being called with:
>>> nm cp | grep U.*statat
>>>
>>> The redirect from fstatat() in code to the appropriate 64 bit interface
>>> should be done with asm redirects and so immune to any undef etc.
>>> that gnulib may be doing.
>>>
>>> So in summary please look at how fstatat() is being referenced
>>> on your system (by gnulib).
>>>
>>> thanks,
>>> Pádraig
>>
>> Hi,
>>
>> I had to add -D option to nm to avoid "no symbols" error.
>>
>>  U __fstatat64_time64@GLIBC_2.34
>>
>> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.
> 
> That looks correct.
> 
> So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
> You might get to the issue more quickly by installing debug symbols for
> your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args
>

Assuming a default config option (without --enable-kernel) the 
_fstatat64_time64 should first try statx and then the old fstatat64
if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
only happens for later, assuming kernel does not returns anything
bogus.

What strace shows in this scenario?

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Pádraig Brady]

On 21/04/2022 18:41, Adhemerval Zanella wrote:



On 21/04/2022 12:25, Pádraig Brady wrote:

On 21/04/2022 15:53, Arnaud Panaïotis wrote:


On 21/04/2022 14:36, Pádraig Brady wrote:

On 19/04/2022 16:01, Arnaud Panaïotis wrote:

On 19/04/2022 16:00, Pádraig Brady wrote:

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for defined data 
type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.

An strace of the cp command would be useful to confirm the problematic syscall.


That suggests the kernel (statx) returns fine,
but glibc is returning the EOVERFLOW.
That suggests fstatat() rather than fstatat64() is being used
(inferring that by comparing
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)

So why is fstatat() being used if compiling with
-D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64

You might confirm what statat is being called with:
nm cp | grep U.*statat

The redirect from fstatat() in code to the appropriate 64 bit interface
should be done with asm redirects and so immune to any undef etc.
that gnulib may be doing.

So in summary please look at how fstatat() is being referenced
on your system (by gnulib).

thanks,
Pádraig


Hi,

I had to add -D option to nm to avoid "no symbols" error.

  U __fstatat64_time64@GLIBC_2.34

All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.


That looks correct.

So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
You might get to the issue more quickly by installing debug symbols for
your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args



Assuming a default config option (without --enable-kernel) the
_fstatat64_time64 should first try statx and then the old fstatat64
if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
only happens for later, assuming kernel does not returns anything
bogus.

What strace shows in this scenario?


strace shows statx() returning non error on this 5.15 kernel

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Pádraig Brady]

On 21/04/2022 15:53, Arnaud Panaïotis wrote:


On 21/04/2022 14:36, Pádraig Brady wrote:

On 19/04/2022 16:01, Arnaud Panaïotis wrote:

On 19/04/2022 16:00, Pádraig Brady wrote:

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for defined data 
type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.

An strace of the cp command would be useful to confirm the problematic syscall.


That suggests the kernel (statx) returns fine,
but glibc is returning the EOVERFLOW.
That suggests fstatat() rather than fstatat64() is being used
(inferring that by comparing
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)

So why is fstatat() being used if compiling with
-D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64

You might confirm what statat is being called with:
nm cp | grep U.*statat

The redirect from fstatat() in code to the appropriate 64 bit interface
should be done with asm redirects and so immune to any undef etc.
that gnulib may be doing.

So in summary please look at how fstatat() is being referenced
on your system (by gnulib).

thanks,
Pádraig


Hi,

I had to add -D option to nm to avoid "no symbols" error.

     U __fstatat64_time64@GLIBC_2.34

All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.


That looks correct.

So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
You might get to the issue more quickly by installing debug symbols for
your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args

thanks,
Pádraig.

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Arnaud Panaïotis]

On 21/04/2022 14:36, Pádraig Brady wrote:

On 19/04/2022 16:01, Arnaud Panaïotis wrote:

On 19/04/2022 16:00, Pádraig Brady wrote:

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to
provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for
defined data type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are
defined.

An strace of the cp command would be useful to confirm the
problematic syscall.


That suggests the kernel (statx) returns fine,
but glibc is returning the EOVERFLOW.
That suggests fstatat() rather than fstatat64() is being used
(inferring that by comparing
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c

https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)


So why is fstatat() being used if compiling with
-D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64

You might confirm what statat is being called with:
nm cp | grep U.*statat

The redirect from fstatat() in code to the appropriate 64 bit interface
should be done with asm redirects and so immune to any undef etc.
that gnulib may be doing.

So in summary please look at how fstatat() is being referenced
on your system (by gnulib).

thanks,
Pádraig


Hi,

I had to add -D option to nm to avoid "no symbols" error.

    U __fstatat64_time64@GLIBC_2.34

All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.

Thanks,

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaio...@gmx.fr 

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr 

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Pádraig Brady]

On 19/04/2022 16:01, Arnaud Panaïotis wrote:

On 19/04/2022 16:00, Pádraig Brady wrote:

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for defined data 
type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.

An strace of the cp command would be useful to confirm the problematic syscall.


That suggests the kernel (statx) returns fine,
but glibc is returning the EOVERFLOW.
That suggests fstatat() rather than fstatat64() is being used
(inferring that by comparing
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)

So why is fstatat() being used if compiling with
-D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64

You might confirm what statat is being called with:
nm cp | grep U.*statat

The redirect from fstatat() in code to the appropriate 64 bit interface
should be done with asm redirects and so immune to any undef etc.
that gnulib may be doing.

So in summary please look at how fstatat() is being referenced
on your system (by gnulib).

thanks,
Pádraig

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Arnaud Panaïotis]

On 19/04/2022 16:00, Pádraig Brady wrote:

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for
defined data type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are
defined.

An strace of the cp command would be useful to confirm the problematic
syscall.

thanks,
Pádraig


Hello,

Thanks for the quick answer.

My glibc is 2.34-9 (2.34-9-g9acab0bba6a5a57323b1f94bf95b21618a9e5aa4).

Strace attached.

Except for the stat.h, I encountered no issue with Kernel, mysql, my
application...

I indeed suspect stat.h to use a 32 bits time_t somewhere.

Regards,

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaio...@gmx.fr 

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr 

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus
execve("/bin/cp", ["cp", "-a", "test/test1", "folder"], 0xbff8c5c4 /* 14 vars 
*/) = 0
brk(NULL)   = 0x515000
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0xb7fc1000
access("/etc/ld.so.preload", R_OK)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/i686/sse2/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 
-1 ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/i686/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/sse2/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "tls/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT 
(No such file or directory)
openat(AT_FDCWD, "i686/sse2/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "i686/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "sse2/libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "libacl.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No 
such file or directory)
openat(AT_FDCWD, "/opt/apps/current/usr/lib/tls/i686/sse2/libacl.so.1", 
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
statx(AT_FDCWD, "/opt/apps/current/usr/lib/tls/i686/sse2", 
AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, 0xbfabe608) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/apps/current/usr/lib/tls/i686/libacl.so.1", 
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
statx(AT_FDCWD, "/opt/apps/current/usr/lib/tls/i686", 
AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, 0xbfabe608) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/apps/current/usr/lib/tls/sse2/libacl.so.1", 
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
statx(AT_FDCWD, "/opt/apps/current/usr/lib/tls/sse2", 
AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, 0xbfabe608) = -1 
ENOENT (No such file or directory)
openat(AT_FDCWD, "/opt/apps/current/usr/lib/tls/libacl.so.1", 

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Pádraig Brady]

On 19/04/2022 08:47, Arnaud Panaïotis wrote:

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for defined data 
type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.


The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.

An strace of the cp command would be useful to confirm the problematic syscall.

thanks,
Pádraig

bug#55023: Issue with CP empty folder after y2038 on 32-bits Kernel

[Arnaud Panaïotis]

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:


Hello,

I'm working for a client to generate embedded 32-bits Linux Kernel
working after y2038 issue.

I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64

The Kernel pass y2038 but I found an issue with cp:
After analysis, the error occurs when trying to move an empty folder
without all user mode rights.

Here how to reproduce:

# mkdir -p test/test1 folder
# chmod u-w test/test1
# date -s "2040-04-02"
# cp -a test/* folder/
cp: setting permissions for 'folder/test1' : Value too large for defined data 
type

Note: The folder is copied before the error occurs. The copy works
fine before y2038.


The issue comes from coreutils-9.0/src/cp.c

Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)

FYI I had a previous issue while calling lstat function from
 which is included in lib/lchmod.c. I used /usr/bin/stat
as a workaround.


Keep me in touch if you need more information.

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaio...@gmx.fr 

18 place Jean Moulin - 38000 Grenoble - France
APsudo - www.panaiotis.fr 

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaio...@gmx.fr 

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr 

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel 
antivirus Avast.
https://www.avast.com/antivirus