bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[C de-Avillez]

On Tue, Apr 16, 2019 at 1:37 PM Assaf Gordon  wrote:

> Thank you both for testing.
>
> So, to summarize:
> whenever "inv-year" fails - it is a problem with glibc on your
> setup, *not* a problem in coreutils' date(1) program.
>
> If there is a setup where "inv-year" succeeds but date(1) still fails,
> then it is a problem in coreutils.
>
> I'm glad to hear latest Ubuntu 19.04 is working fine
> (though the reason for the earlier failure is still a mystery).
>
> As Paul suggested, trying 'strace' on the failing system
> might reveal more details.

Will run strace again. Meanwhile, I decided to test time zones. My
default TZ is America/Chicago -- in my laptop date fails. I then moved
to TZ=UTC, and it worked:

cerdea@piatam:~/Downloads$ echo $TZ
America/Chicago
cerdea@piatam:~/Downloads$ date
Tue Apr 16 10:04:48 CDT 2019

130 cerdea@piatam:~/Downloads$ date --debug +%-Y -d '- 110 years'
date: parsed relative part: -110 year(s)
date: input timezone: TZ="America/Chicago" environment value
date: using current time as starting value: '10:05:01'
date: using current date as starting value: '(Y-M-D) 2019-04-16'
date: starting date/time: '(Y-M-D) 2019-04-16 10:05:01'
date: warning: when adding relative months/years, it is recommended to
specify the 15th of the months
date: error: adding relative date resulted in an invalid date:
'(Y-M-D) 1909-04-16 10:05:01'
date: invalid date ‘- 110 years’

1 cerdea@piatam:~/Downloads$ export TZ=UTC
cerdea@piatam:~/Downloads$ date --debug +%-Y -d '- 110 years'
date: parsed relative part: -110 year(s)
date: input timezone: TZ="UTC" environment value
date: using current time as starting value: '15:05:21'
date: using current date as starting value: '(Y-M-D) 2019-04-16'
date: starting date/time: '(Y-M-D) 2019-04-16 15:05:21'
date: warning: when adding relative months/years, it is recommended to
specify the 15th of the months
date: after date adjustment (-110 years, +0 months, +0 days),
date: new date/time = '(Y-M-D) 1909-04-16 15:05:21'
date: '(Y-M-D) 1909-04-16 15:05:21' = -1915865679 epoch-seconds
date: timezone: TZ="UTC" environment value
date: final: -1915865679.603138125 (epoch-seconds)
date: final: (Y-M-D) 1909-04-16 15:05:21 (UTC)
date: final: (Y-M-D) 1909-04-16 15:05:21 (UTC+00)
1909

Will keep on trying.
-- 
..hggdh..

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[Assaf Gordon]

Hello,

On 2019-04-15 5:10 p.m., O. Emmerson wrote:

For me it gives:

$ ./inv-year
time() = 1555369320
localtime() = 2019-04-16 00:02:00
   (mday=16 wday=2, isdst=1)
struct tm (after adjustment) = 0009-04-16 00:02:00
   (mday=16 wday=2, isdst=1)
inv-year: mktime() failed: Value too large for defined data type

>

On 2019-04-15 6:50 p.m., C de-Avillez wrote:
[...]

root@u1904:~# gcc -o inv-year inv-year.c
root@u1904:~# ./inv-year
time() = 1555375408
localtime() = 2019-04-16 00:43:28
   (mday=16 wday=2, isdst=0)
struct tm (after adjustment) = 0009-04-16 00:43:28
(mday=16 wday=2, isdst=0)
mktime() after date adjustment = -61874061392

So: a pristine 19.04 runs it. My laptop (which is my work machine,
full of other packages & programs), does not.



Thank you both for testing.

So, to summarize:
whenever "inv-year" fails - it is a problem with glibc on your
setup, *not* a problem in coreutils' date(1) program.

If there is a setup where "inv-year" succeeds but date(1) still fails,
then it is a problem in coreutils.

I'm glad to hear latest Ubuntu 19.04 is working fine
(though the reason for the earlier failure is still a mystery).

As Paul suggested, trying 'strace' on the failing system
might reveal more details.

regards,
 - assaf

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[Paul Eggert]

C de-Avillez wrote:

So: a pristine 19.04 runs it. My laptop (which is my work machine,
full of other packages & programs), does not.


You might try running 'strace' on pristine 19.04 and on your laptop, and see if 
you can spot the difference in system calls. Possibly you have some stuff on 
your laptop that is leftover from an earlier release.

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[C de-Avillez]

It has to be something messing/interacting with coreutils/date on
19.04 (and probably on Tumbleweed).

I created a new container with 19.04, and then:
* apt build-dep coreutils
* apt install rsync wget git
* git clone git://git.sv.gnu.org/coreutils
* cd coreutils
* export FORCE_UNSAFE_CONFIGURE=1 # too lazy to adduser
* ./boostrap
* ./configure
* make

at the end of the build I ran both the provided date and the
just-built one. Both ran correctly:

root@u1904:~/coreutils# date --debug +%-Y -d '- 2010 years'
date: parsed relative part: -2010 year(s)
date: input timezone: system default
date: using current time as starting value: '00:30:39'
date: using current date as starting value: '(Y-M-D) 2019-04-16'
date: starting date/time: '(Y-M-D) 2019-04-16 00:30:39'
date: warning: when adding relative months/years, it is recommended to
specify the 15th of the months
date: after date adjustment (-2010 years, +0 months, +0 days),
date: new date/time = '(Y-M-D) 0009-04-16 00:30:39'
date: '(Y-M-D) 0009-04-16 00:30:39' = -61874062161 epoch-seconds
date: timezone: system default
date: final: -61874062161.081576777 (epoch-seconds)
date: final: (Y-M-D) 0009-04-16 00:30:39 (UTC)
date: final: (Y-M-D) 0009-04-16 00:30:39 (UTC+00)
9

root@u1904:~/coreutils# src/date --debug +%-Y -d '- 2010 years'
date: parsed relative part: -2010 year(s)
date: input timezone: system default
date: using current time as starting value: '00:30:49'
date: using current date as starting value: '(Y-M-D) 2019-04-16'
date: starting date/time: '(Y-M-D) 2019-04-16 00:30:49'
date: warning: when adding relative months/years, it is recommended to
specify the 15th of the months
date: after date adjustment (-2010 years, +0 months, +0 days),
date: new date/time = '(Y-M-D) 0009-04-16 00:30:49'
date: '(Y-M-D) 0009-04-16 00:30:49' = -61874062151 epoch-seconds
date: timezone: system default
date: final: -61874062151.239637280 (epoch-seconds)
date: final: (Y-M-D) 0009-04-16 00:30:49 (UTC)
date: final: (Y-M-D) 0009-04-16 00:30:49 (UTC+00)
9

I then compiled & ran Asaf's inv-year.c:

root@u1904:~# gcc -o inv-year inv-year.c
root@u1904:~# ./inv-year
time() = 1555375408
localtime() = 2019-04-16 00:43:28
  (mday=16 wday=2, isdst=0)
struct tm (after adjustment) = 0009-04-16 00:43:28
   (mday=16 wday=2, isdst=0)
mktime() after date adjustment = -61874061392

So: a pristine 19.04 runs it. My laptop (which is my work machine,
full of other packages & programs), does not.

Oh -- Assaf, yes, I am very well aware 19.04 has not yet been
released. But -- unless we find something critical -- it is basically
what will be  released in a few days. I do not expect a rebuild of the
coreutils package, for example.

Cheers,

..C..

On Mon, Apr 15, 2019 at 6:11 PM O. Emmerson  wrote:
>
> On 15/04/2019 21:55, Assaf Gordon wrote:
> > To see if this is glibc issue, or perhaps an gnulib/mktime_z wrapper
> > issue, can you (and/or others) try the attached C program?
> >
> > It calls time(2)+localtime(3)+mktime(3) to emulate the date adjustment.
> >
> > Because the adjustment is to year 9 (about 1961 years before epoch),
> > the time_t value is negative. perhaps that's the issue? or perhaps
> > combined with a specific timezone it becomes problematic?
>
> For me it gives:
>
> $ ./inv-year
> time() = 1555369320
> localtime() = 2019-04-16 00:02:00
>(mday=16 wday=2, isdst=1)
> struct tm (after adjustment) = 0009-04-16 00:02:00
> (mday=16 wday=2, isdst=1)
> inv-year: mktime() failed: Value too large for defined data type
>
>
> And debug gives:
>
> $ date +%-Y -d "- 112 years" --debug
> date: parsed relative part: -112 year(s)
> date: input timezone: system default
> date: using current time as starting value: '00:04:32'
> date: using current date as starting value: '(Y-M-D) 2019-04-16'
> date: starting date/time: '(Y-M-D) 2019-04-16 00:04:32'
> date: warning: when adding relative months/years, it is recommended to
> specify the 15th of the months
> date: error: adding relative date resulted in an invalid date: '(Y-M-D)
> 1907-04-16 00:04:32'
> date: invalid date ‘- 112 years’
>
>
>
>
>


-- 
..hggdh..

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[O. Emmerson]

On 15/04/2019 21:55, Assaf Gordon wrote:

To see if this is glibc issue, or perhaps an gnulib/mktime_z wrapper
issue, can you (and/or others) try the attached C program?

It calls time(2)+localtime(3)+mktime(3) to emulate the date adjustment.

Because the adjustment is to year 9 (about 1961 years before epoch),
the time_t value is negative. perhaps that's the issue? or perhaps
combined with a specific timezone it becomes problematic?


For me it gives:

$ ./inv-year
time() = 1555369320
localtime() = 2019-04-16 00:02:00
  (mday=16 wday=2, isdst=1)
struct tm (after adjustment) = 0009-04-16 00:02:00
   (mday=16 wday=2, isdst=1)
inv-year: mktime() failed: Value too large for defined data type


And debug gives:

$ date +%-Y -d "- 112 years" --debug
date: parsed relative part: -112 year(s)
date: input timezone: system default
date: using current time as starting value: '00:04:32'
date: using current date as starting value: '(Y-M-D) 2019-04-16'
date: starting date/time: '(Y-M-D) 2019-04-16 00:04:32'
date: warning: when adding relative months/years, it is recommended to
specify the 15th of the months
date: error: adding relative date resulted in an invalid date: '(Y-M-D)
1907-04-16 00:04:32'
date: invalid date ‘- 112 years’

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[C de-Avillez]

cerdea@piatam:~/Downloads$ gcc -o inv-year inv-year.c
cerdea@piatam:~/Downloads$ ./inv-year
time() = 1555368087
localtime() = 2019-04-15 17:41:27
 (mday=15 wday=1, isdst=1)
struct tm (after adjustment) = 0009-04-15 17:41:27
  (mday=15 wday=1, isdst=1)
inv-year: mktime() failed: Value too large for defined data type
1 cerdea@piatam:~/Downloads$

On Mon, Apr 15, 2019 at 3:55 PM Assaf Gordon  wrote:
>
> Thanks Bernhard,
>
> On 2019-04-15 2:14 p.m., Bernhard Voelker wrote:
> > I can easily reproduce here on my regular openSUSE:Tumbleweed from latest 
> > git:
> >
> >$ src/date --debug '+%-Y' -d '- 2010 years'
> []
> >date: error: adding relative date resulted in an invalid date: '(Y-M-D) 
> > 0009-04-15 22:10:37'
>
> This makes it easy to pinpoint (hooray for "--debug" :) ).
>
> This error is given if gnulib's "mktime_z" fails
> to convert the adjusted "struct tm" to "time_t"
> (adjusted because its tm_year was decremented by 2010).
>
> https://opengrok.housegordon.com/source/xref/gnulib/lib/parse-datetime.y#2177
>
> To see if this is glibc issue, or perhaps an gnulib/mktime_z wrapper
> issue, can you (and/or others) try the attached C program?
>
> It calls time(2)+localtime(3)+mktime(3) to emulate the date adjustment.
>
> Because the adjustment is to year 9 (about 1961 years before epoch),
> the time_t value is negative. perhaps that's the issue? or perhaps
> combined with a specific timezone it becomes problematic?
>
> On my system it gives:
> 
>$ gcc -o inv-year inv-year.c
>
>$ ./inv-year
>time() = 1555361050
>localtime() = 2019-04-15 14:44:10
>  (mday=15 wday=1, isdst=1)
>struct tm (after adjustment) = 0009-04-15 14:44:10
>   (mday=15 wday=1, isdst=1)
>mktime() after date adjustment = -61874070118
> 
>
>
> regards,
>   - assaf
>
>
>
>
>
>


-- 
..hggdh..

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[Assaf Gordon]

Thanks Bernhard,

On 2019-04-15 2:14 p.m., Bernhard Voelker wrote:

I can easily reproduce here on my regular openSUSE:Tumbleweed from latest git:

   $ src/date --debug '+%-Y' -d '- 2010 years'

[]

   date: error: adding relative date resulted in an invalid date: '(Y-M-D) 
0009-04-15 22:10:37'


This makes it easy to pinpoint (hooray for "--debug" :) ).

This error is given if gnulib's "mktime_z" fails
to convert the adjusted "struct tm" to "time_t"
(adjusted because its tm_year was decremented by 2010).

https://opengrok.housegordon.com/source/xref/gnulib/lib/parse-datetime.y#2177

To see if this is glibc issue, or perhaps an gnulib/mktime_z wrapper
issue, can you (and/or others) try the attached C program?

It calls time(2)+localtime(3)+mktime(3) to emulate the date adjustment.

Because the adjustment is to year 9 (about 1961 years before epoch),
the time_t value is negative. perhaps that's the issue? or perhaps
combined with a specific timezone it becomes problematic?

On my system it gives:

  $ gcc -o inv-year inv-year.c

  $ ./inv-year
  time() = 1555361050
  localtime() = 2019-04-15 14:44:10
(mday=15 wday=1, isdst=1)
  struct tm (after adjustment) = 0009-04-15 14:44:10
 (mday=15 wday=1, isdst=1)
  mktime() after date adjustment = -61874070118



regards,
 - assaf






/* A test program to help with https://bugs.gnu.org/35289#28

compile with:
gcc -o inv-year inv-year.c

written by Assaf Gordon (assafgor...@gmail.com).
placed under public domain.
*/
#include 
#include 
#include 
#include 

int main()
{
	time_t now = time(NULL);
	if ( now == ((time_t) -1))
		err(1, "time() failed");
	printf("time() = %ld\n", (signed long)now);

	struct tm *a = localtime(&now);
	if (!a)
		err(1, "localtime(now) failed");

	printf("localtime() = %04d-%02d-%02d %02d:%02d:%02d\n"
   "  (mday=%d wday=%d, isdst=%d)\n",
		a->tm_year+1900,a->tm_mon+1,a->tm_mday,
		a->tm_hour, a->tm_min, a->tm_sec,
		a->tm_mday, a->tm_wday, a->tm_isdst);

	struct tm b;
	memcpy (&b, a, sizeof b);

	/**
	*  Test date adjustment by changing 'b' members
	**/
	b.tm_year -= 2010;

	printf("struct tm (after adjustment) = %04d-%02d-%02d %02d:%02d:%02d\n"
	   "   (mday=%d wday=%d, isdst=%d)\n",
		b.tm_year+1900,b.tm_mon+1,b.tm_mday,
		b.tm_hour, b.tm_min, b.tm_sec,
		b.tm_mday, b.tm_wday, b.tm_isdst);

	time_t notnow = mktime (&b);
	if ( notnow == ((time_t) -1) )
		err(1, "mktime() failed");

	printf("mktime() after date adjustment = %ld\n", (signed long)notnow);

	return 0;
}

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[Bernhard Voelker]

On 4/15/19 9:41 PM, Assaf Gordon wrote:
> A good starting point is adding the "--debug" option to date(1)
> and examining its output.

Hi Assaf,

I can easily reproduce here on my regular openSUSE:Tumbleweed from latest git:

  $ src/date --debug '+%-Y' -d '- 2010 years'
  date: parsed relative part: -2010 year(s)
  date: input timezone: system default
  date: using current time as starting value: '22:10:37'
  date: using current date as starting value: '(Y-M-D) 2019-04-15'
  date: starting date/time: '(Y-M-D) 2019-04-15 22:10:37'
  date: error: adding relative date resulted in an invalid date: '(Y-M-D) 
0009-04-15 22:10:37'
  src/date: invalid date '- 2010 years'

Have a nice day,
Berny

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[Assaf Gordon]

Hello,

On 2019-04-15 11:55 a.m., C de-Avillez wrote:

19.04:


It is worth noting that Ubuntu 19.04 has not been officially released
yet, so you are testing on a development branch (or a release-candidate,
or a special built infrastructure as hinted by your path).


cerdea@piatam:/data/buildd/coreutils$ date +%-Y -d '- 2010 years'
date: invalid date ‘- 2010 years’
1 cerdea@piatam:/data/buildd/coreutils$ date --version
date (GNU coreutils) 8.30


[...]


On Mon, Apr 15, 2019 at 12:16 PM O. Emmerson  wrote:


$ file /bin/date
/bin/date: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for
GNU/Linux 3.2.0, BuildID[sha1]=26fa7f6c43c354d8c5647ebf946255a2b8e3c53d,
stripped


I just downloaded a recent daily snapshot of
ubuntu desktop live CD for amd64 from 
http://cdimage.ubuntu.com/daily-live/current/ .

The file "disco-desktop-amd64.iso" dated 2019-04-13 22:28 size 1.9GB,
with the following checksum:

  $ sha1sum disco-desktop-amd64.iso
  b89fb143b51e17482a3882abe2f5f4e3b69942fe  disco-desktop-amd64.iso

Booting with QEMU as live-cd, I tested the same command and got "9" as
the (correct) result. So this can't be easily reproduced.

An interesting benefit of reproducible builds is that I see on the
live-cd image the sha1 checksum of "/bin/date" is the same as you listed
above. This hints to me the problem is somewhere else in your setup.

As this is not an official release, we really can't support it.
You'll have to dig further and see what is the issue.

A good starting point is adding the "--debug" option to date(1)
and examining its output.

regards,
 - assaf

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[C de-Avillez]

I have ran a few tests on Ubuntu 16.04, 18.04, 18.10, and 19.04 (all X86_64):

16.04:

root@u1604:~# date +%-Y -d '- 2010 years'
9
root@u1604:~# date --version
date (GNU coreutils) 8.25
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.

18.04:

root@bionic:~# date +%-Y -d '- 2010 years'
9
root@bionic:~# date --version
date (GNU coreutils) 8.28
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.

18.10:

root@u1810:~# date +%-Y -d '- 2010 years'
9
root@u1810:~# date --version
date (GNU coreutils) 8.28
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.

19.04:

cerdea@piatam:/data/buildd/coreutils$ date +%-Y -d '- 2010 years'
date: invalid date ‘- 2010 years’
1 cerdea@piatam:/data/buildd/coreutils$ date --version
date (GNU coreutils) 8.30
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.

Interestingly, it is also failing on git head:

130 cerdea@piatam:/data/buildd/coreutils$ src/date +%-Y -d '- 2010 years'
src/date: invalid date ‘- 2010 years’
1 cerdea@piatam:/data/buildd/coreutils$ src/date --version
date (GNU coreutils) 8.31.12-6d78a
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later .
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by David MacKenzie.

On Mon, Apr 15, 2019 at 12:16 PM O. Emmerson  wrote:
>
> $ file /bin/date
> /bin/date: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
> dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for
> GNU/Linux 3.2.0, BuildID[sha1]=26fa7f6c43c354d8c5647ebf946255a2b8e3c53d,
> stripped
>
>
>
>


-- 
..hggdh..

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[O. Emmerson]

$ file /bin/date
/bin/date: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV),
dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for
GNU/Linux 3.2.0, BuildID[sha1]=26fa7f6c43c354d8c5647ebf946255a2b8e3c53d,
stripped

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[O. Emmerson]

I have downloaded and compiled 8.31 from source to see if it makes any
difference and have gotten the same error.

bug#35289: closed (Re: bug#35289: date+%-Y -d "- N years" errors when N > 111)

[O. Emmerson]

On 15/04/2019 17:02, GNU bug Tracking System wrote:

It works for me with coreutils 8.31 on RHEL 7 x86-64:

$ date +%-Y -d "- 2010 years"
9

Most likely you are running on a 32-bit machine, and dates in the year 9
cannot be represented in a 32-bit timestamp. So a simple fix would be
for you to switch to a 64-bit machine.


I am running a 64bit version of Ubuntu.
Anyway, as I said, I have been running this command successfully for
years until now.

Here is the full system info from reportbug copied from by original bug
on Ubuntu's tracker:

ProblemType: Bug
DistroRelease: Ubuntu 19.04
Package: coreutils 8.30-1ubuntu1
ProcVersionSignature: Ubuntu 5.0.0-11.12-generic 5.0.6
Uname: Linux 5.0.0-11-generic x86_64
ApportVersion: 2.20.10-0ubuntu27
Architecture: amd64
Date: Sun Apr 14 10:59:36 2019
InstallationDate: Installed on 2017-10-08 (553 days ago)
InstallationMedia: Ubuntu 16.04.2 LTS "Xenial Xerus" - Release amd64
(20170215.2)
SourcePackage: coreutils
UpgradeStatus: Upgraded to disco on 2019-04-13 (1 days ago)