Bug#1059363: gbp-import-dsc crashes when changelog contains leap seconds ("Sun, 1 Jan 2006, 00:59:60 +0100")

2023-12-30 Thread Gioele Barabucci 

On 30/12/23 11:27, Guido Günther wrote:

  from gbp.git.modifier import GitModifier   # noqa: F401
@@ -41,7 +42,11 @@ def rfc822_date_to_git(rfc822_date, fuzzy=False):
  >>> rfc822_date_to_git('So, 26 Feb 1998 8:50:00 +0100', fuzzy=True)
  '888479400 +0100'
  """
+rfc822_date_orig = rfc822_date
+rfc822_date = rfc822_date.replace(":60 ", ":59 ")
  d = dateutil.parser.parse(rfc822_date, fuzzy=fuzzy)
+if rfc822_date != rfc822_date_orig: # Handle leap seconds.
+d += datetime.timedelta(seconds=1)


That works. Could you add a line to the doctest in that function too so
we check for and don't break it on accident again. Otherwise I can do it
when applying the patch.


Hi Guido, I'll be happy to leave the doctest to you. :)

Thanks for accepting the patch.

PS: Beware: the indentation may be wrong.

Regards,

--
Gioele Barasbucci.

Bug#1059363: gbp-import-dsc crashes when changelog contains leap seconds ("Sun, 1 Jan 2006, 00:59:60 +0100")

2023-12-30 Thread Guido Günther 
Hi,
On Sat, Dec 23, 2023 at 05:09:48PM +0100, Gioele Barabucci wrote:
> Control: tags -1 patch
> 
> Hi Guido,
> 
> On 23/12/23 16:57, Guido Günther wrote:
> > > the gbp-import-dsc script crashes while parsing changelogs that include 
> > > leap
> > > seconds.
> > > 
> > > dateutil.parser._parser.ParserError: second must be in 0..59: Sun,  1 Jan
> > > 2006 00:59:60 +0100
> > 
> > This looks like a bug in dateutil, can you reassign there?
> 
> Not really:
> 
> > > As of 2023-12 there is an open issue asking for support for leap seconds 
> > > in
> > > dateutils [3].

Ah...I missed that one. We should work around it in gbp then.

>  from gbp.git.modifier import GitModifier   # noqa: F401
> @@ -41,7 +42,11 @@ def rfc822_date_to_git(rfc822_date, fuzzy=False):
>  >>> rfc822_date_to_git('So, 26 Feb 1998 8:50:00 +0100', fuzzy=True)
>  '888479400 +0100'
>  """
> +rfc822_date_orig = rfc822_date
> +rfc822_date = rfc822_date.replace(":60 ", ":59 ")
>  d = dateutil.parser.parse(rfc822_date, fuzzy=fuzzy)
> +if rfc822_date != rfc822_date_orig: # Handle leap seconds.
> +d += datetime.timedelta(seconds=1)

That works. Could you add a line to the doctest in that function too so
we check for and don't break it on accident again. Otherwise I can do it
when applying the patch.

Cheers,
 -- Guido

>  seconds = calendar.timegm(d.utctimetuple())
>  tz = d.strftime("%z")
>  return '%d %s' % (seconds, tz)
> -- 
> 2.43.0
>

Bug#1059363: gbp-import-dsc crashes when changelog contains leap seconds ("Sun, 1 Jan 2006, 00:59:60 +0100")

2023-12-23 Thread Gioele Barabucci 

Control: tags -1 patch

Hi Guido,

On 23/12/23 16:57, Guido Günther wrote:

the gbp-import-dsc script crashes while parsing changelogs that include leap
seconds.

dateutil.parser._parser.ParserError: second must be in 0..59: Sun,  1 Jan
2006 00:59:60 +0100


This looks like a bug in dateutil, can you reassign there?


Not really:


As of 2023-12 there is an open issue asking for support for leap seconds in
dateutils [3].


That issue has been open for almost three years. The dateutils 
developers say that Python's datetime does not handle leap seconds, so 
they are not going to mess around with it.


In order to deal with Debian policy's request to support :60, 
application-level workarounds are needed.


Please find attached a small patch that fix this issue.

Regards,

--
Gioele BarabucciFrom 5dd36d1a7ed44f7cd10eb7afeb58e6c747558e91 Mon Sep 17 00:00:00 2001
From: Gioele Barabucci 
Date: Sat, 23 Dec 2023 17:03:39 +0100
Subject: [PATCH] git: rfc822_date_to_git: Handle leap seconds

Closes: #1059363
---
 gbp/git/__init__.py | 5 +
 1 file changed, 5 insertions(+)

diff --git a/gbp/git/__init__.py b/gbp/git/__init__.py
index 3d0bb0b9..773ba822 100644
--- a/gbp/git/__init__.py
+++ b/gbp/git/__init__.py
@@ -17,6 +17,7 @@
 """Accessing Git from python"""
 
 import calendar
+import datetime
 import dateutil.parser
 
 from gbp.git.modifier import GitModifier   # noqa: F401
@@ -41,7 +42,11 @@ def rfc822_date_to_git(rfc822_date, fuzzy=False):
 >>> rfc822_date_to_git('So, 26 Feb 1998 8:50:00 +0100', fuzzy=True)
 '888479400 +0100'
 """
+rfc822_date_orig = rfc822_date
+rfc822_date = rfc822_date.replace(":60 ", ":59 ")
 d = dateutil.parser.parse(rfc822_date, fuzzy=fuzzy)
+if rfc822_date != rfc822_date_orig: # Handle leap seconds.
+d += datetime.timedelta(seconds=1)
 seconds = calendar.timegm(d.utctimetuple())
 tz = d.strftime("%z")
 return '%d %s' % (seconds, tz)
-- 
2.43.0

Bug#1059363: gbp-import-dsc crashes when changelog contains leap seconds ("Sun, 1 Jan 2006, 00:59:60 +0100")

2023-12-23 Thread Guido Günther 
Hi,
On Sat, Dec 23, 2023 at 04:48:51PM +0100, Gioele Barabucci wrote:
> Package: git-buildpackage
> Version: 0.9.33
> 
> Dear git-buildpackage maintainer,
> 
> the gbp-import-dsc script crashes while parsing changelogs that include leap
> seconds.
> 
> For example, while parsing the changelog of unicode/5.0, gbp-import-dsc
> crashes with
> 
> ```
>   File "/usr/lib/python3/dist-packages/gbp/scripts/import_dsc.py", line 116,
> in get_author_from_changelog
> date = rfc822_date_to_git(dch.date, fuzzy=True)
>
>   File "/usr/lib/python3/dist-packages/gbp/git/__init__.py", line 44, in
> rfc822_date_to_git
> d = dateutil.parser.parse(rfc822_date, fuzzy=fuzzy)
> ^^^
>   File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", line
> 1368, in parse
> return DEFAULTPARSER.parse(timestr, **kwargs)
>^^
>   File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", line
> 651, in parse
> six.raise_from(ParserError(str(e) + ": %s", timestr), e)
>   File "", line 3, in raise_from
> dateutil.parser._parser.ParserError: second must be in 0..59: Sun,  1 Jan
> 2006 00:59:60 +0100
> ```

This looks like a bug in dateutil, can you reassign there?
Cheers,
 -- Guido

> 
> Leap seconds (:60) are explicitly permitted by Debian Policy [1] and RFC
> 5322 [2] and produced by `date -R`.
> 
> As of 2023-12 there is an open issue asking for support for leap seconds in
> dateutils [3].
> 
> A possible solution to this issue, suggested by olasd, would to parse the
> date using the stdlib method `email.utils.parsedate_tz` and handle the leap
> seconds manually:
> 
> ```
> import email, datetime
> 
> add_leap_second = False
> dateinfo = list(email.utils.parsedate_tz(rfc822_date))
> if dateinfo[5] == 60:
> dateinfo[5] = 59
> add_leap_second = True
> 
> date = datetime.datetime(*dateinfo[:6])
> 
> if add_leap_second:
> date += datetime.timedelta(seconds=1)
> ```
> 
> Alternatively:
> 
> ```
> dhc_date = dch.date.replace(":60 ", ":59 ")
> date = rfc822_date_to_git(dch.date, fuzzy=True)
> if dch_date != dch.date:
> date += datetime.timedelta(seconds=1)
> ```
> 
> Regards,
> 
> [1] «ss is the two-digit seconds (00-60)»
> https://www.debian.org/doc/debian-policy/ch-source.html#s-dpkgchangelog
> 
> [2] «the time-of-day MUST be in the range 00:00:00 through 23:59:60 (the
> number of seconds allowing for a leap second; see [RFC1305])»
> https://datatracker.ietf.org/doc/html/rfc5322#section-3.3
> 
> [3] https://github.com/dateutil/dateutil/issues/1018
> 
> -- 
> Gioele Barabucci
>

Bug#1059363: gbp-import-dsc crashes when changelog contains leap seconds ("Sun, 1 Jan 2006, 00:59:60 +0100")

2023-12-23 Thread Gioele Barabucci 

Package: git-buildpackage
Version: 0.9.33

Dear git-buildpackage maintainer,

the gbp-import-dsc script crashes while parsing changelogs that include 
leap seconds.


For example, while parsing the changelog of unicode/5.0, gbp-import-dsc 
crashes with


```
  File "/usr/lib/python3/dist-packages/gbp/scripts/import_dsc.py", line 
116, in get_author_from_changelog

date = rfc822_date_to_git(dch.date, fuzzy=True)
   
  File "/usr/lib/python3/dist-packages/gbp/git/__init__.py", line 44, 
in rfc822_date_to_git

d = dateutil.parser.parse(rfc822_date, fuzzy=fuzzy)
^^^
  File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", 
line 1368, in parse

return DEFAULTPARSER.parse(timestr, **kwargs)
   ^^
  File "/usr/lib/python3/dist-packages/dateutil/parser/_parser.py", 
line 651, in parse

six.raise_from(ParserError(str(e) + ": %s", timestr), e)
  File "", line 3, in raise_from
dateutil.parser._parser.ParserError: second must be in 0..59: Sun,  1 
Jan 2006 00:59:60 +0100

```

Leap seconds (:60) are explicitly permitted by Debian Policy [1] and RFC 
5322 [2] and produced by `date -R`.


As of 2023-12 there is an open issue asking for support for leap seconds 
in dateutils [3].


A possible solution to this issue, suggested by olasd, would to parse 
the date using the stdlib method `email.utils.parsedate_tz` and handle 
the leap seconds manually:


```
import email, datetime

add_leap_second = False
dateinfo = list(email.utils.parsedate_tz(rfc822_date))
if dateinfo[5] == 60:
dateinfo[5] = 59
add_leap_second = True

date = datetime.datetime(*dateinfo[:6])

if add_leap_second:
date += datetime.timedelta(seconds=1)
```

Alternatively:

```
dhc_date = dch.date.replace(":60 ", ":59 ")
date = rfc822_date_to_git(dch.date, fuzzy=True)
if dch_date != dch.date:
date += datetime.timedelta(seconds=1)
```

Regards,

[1] «ss is the two-digit seconds (00-60)» 
https://www.debian.org/doc/debian-policy/ch-source.html#s-dpkgchangelog


[2] «the time-of-day MUST be in the range 00:00:00 through 23:59:60 (the 
number of seconds allowing for a leap second; see [RFC1305])» 
https://datatracker.ietf.org/doc/html/rfc5322#section-3.3


[3] https://github.com/dateutil/dateutil/issues/1018

--
Gioele Barabucci