[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Irit Katriel]

Change by Irit Katriel :


--
keywords:  -patch
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Paul Ganssle]

Paul Ganssle  added the comment:

> Should we fix utcfromtimestamp() internally to avoid the OverflowError, 
> rather than only fixing the http.cookiejar module?

I'm not a big fan of utcfromtimestamp (as you can see here: 
https://blog.ganssle.io/articles/2019/11/utcnow.html ), but it seems we do have 
the same issue in `datetime.datetime.fromtimestamp`, so I think maybe we should 
spawn a new issue to at least look into the possibility?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor  added the comment:

@Paul Ganssle: it's a little sad that I have work around 
datetime.datetime.utcfromtimestamp() function to avoid an OverflowError 
exception: msg361972.

Should we fix utcfromtimestamp() internally to avoid the OverflowError, rather 
than only fixing the http.cookiejar module?

--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Neil MacLeod]

Neil MacLeod  added the comment:

Just an FYI, this is also broken on 32-bit with Python2.7.16, so possibly it 
was never fixed originally (rather than being a regression).

LibreELEC (Milhouse): devel-20191012235627-#1012-ge416c8b (RPi2.arm)
rpi22:~ # python -c "import sys, cookielib; print sys.version; print 
cookielib.time2isoz(2322923767)"
2.7.16 (default, Sep 29 2019, 04:11:31)
[GCC 9.2.0]
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.7/cookielib.py", line 99, in time2isoz
ValueError: timestamp out of range for platform time_t

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Neil MacLeod]

Neil MacLeod  added the comment:

Hi Victor

I can confirm the patch is working on both 32-bit and 64-bit systems running 
Python3.7.6, with both platforms returning the same result after patching - 
many thanks!

 UNPATCHED, 32-bit (RPi3+)
LibreELEC (Milhouse.testing): devel-20200213234919-#0213f-g70b69eb (RPi2.arm)
rpi22:~ # python -c "import sys, http.cookiejar; print(sys.version); 
print(http.cookiejar.time2isoz(2322923767))"
3.7.6 (default, Feb 14 2020, 00:35:22)
[GCC 9.2.0]
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.7/cookiejar.py", line 101, in time2isoz
OverflowError: timestamp out of range for platform time_t

 PATCHED, 32-bit (RPi3+)
LibreELEC (Milhouse.testing): devel-20200214012134-#0213g-g70b69eb (RPi2.arm)
rpi22:~ # python -c "import sys, http.cookiejar; print(sys.version); 
print(http.cookiejar.time2isoz(2322923767))"
3.7.6 (default, Feb 14 2020, 01:22:50)
[GCC 9.2.0]
2043-08-11 16:36:07Z

 UNPATCHED, 64-bit (x86_64)
LibreELEC (Milhouse.next): devel-20200213053123-#0212x-g67aedc9 (Generic.x86_64)
NUC:~ # python -c "import sys, http.cookiejar; print(sys.version); 
print(http.cookiejar.time2isoz(2322923767))"
3.7.6 (default, Feb 12 2020, 20:54:03)
[GCC 9.2.0]
2043-08-11 16:36:07Z

 PATCHED, 64-bit (x86_64)
LibreELEC (Milhouse.next): devel-20200214005606-#0214-g70b69eb (Generic.x86_64)
NUC:~ # python -c "import sys, http.cookiejar; print(sys.version); 
print(http.cookiejar.time2isoz(2322923767))"
3.7.6 (default, Feb 14 2020, 01:00:37)
[GCC 9.2.0]
2043-08-11 16:36:07Z

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor  added the comment:

Oh, maybe the bug wasn't properly fixed?

Can you please try the patch above?

diff --git a/Lib/http/cookiejar.py b/Lib/http/cookiejar.py
index 47ed5c3d64..55915cf18a 100644
--- a/Lib/http/cookiejar.py
+++ b/Lib/http/cookiejar.py
@@ -99,7 +99,7 @@ def time2isoz(t=None):
 if t is None:
 dt = datetime.datetime.utcnow()
 else:
-dt = datetime.datetime.utcfromtimestamp(t)
+dt = datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=t)
 return "%04d-%02d-%02d %02d:%02d:%02dZ" % (
 dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second)
 

For example, copy http/ subdirectory in the current directory, and then patch 
manually http/cookiejar.py file.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[nmacleod]

nmacleod  added the comment:

Apologies for the necro on this issue, but should this now be fixed in 
Python3.7? As it appears to still be an issue.

Testing on a Raspberry Pi with LibreELEC (32-bit OS):

rpi512:~ # python -c "import sys, http.cookiejar; print(sys.version); 
print(http.cookiejar.time2isoz(2322923767))"
3.7.6 (default, Feb 12 2020, 17:36:39)
[GCC 9.2.0]
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.7/cookiejar.py", line 101, in time2isoz
OverflowError: timestamp out of range for platform time_t


This is on a distribution built with latest glibc-2.31.

--
nosy: +nmacleod

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Alex Quinn]

Alex Quinn added the comment:

This bug still exists in Python 2.7.3 32-bit on Linux.

I wonder if this might be because the patch (posted 2011-02-18) used 
utcfromtimestamp().  datetime.datetime.utcfromtimestamp(2**32) will fail on 
32-bit systems.

The bug does NOT exist in Python 2.7.3 32-bit on Windows (64-bit OS).

==
32-BIT PYTHON ON 32-BIT LINUX
--
$ python -c import sys, cookielib; print sys.version; print 
cookielib.time2isoz(2322923767)
2.7.3 (default, Apr 10 2013, 05:46:21) 
[GCC 4.6.3]
Traceback (most recent call last):
  File string, line 1, in module
  File /usr/lib/python2.7/cookielib.py, line 99, in time2isoz
year, mon, mday, hour, min, sec = time.gmtime(t)[:6]
ValueError: timestamp out of range for platform time_t
==


==
64-BIT PYTHON ON 64-BIT LINUX
--
$ python -c import sys, cookielib; print sys.version; print 
cookielib.time2isoz(2322923767)
2.7.3 (default, Aug  3 2012, 17:21:07)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
2043-08-11 16:36:07Z
==


==
32-BIT PYTHON ON 64-BIT WINDOWS
--
C:\python -c import sys, cookielib; print sys.version; print 
cookielib.time2isoz(2322923767)
2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
2043-08-11 16:36:07Z
==

--
nosy: +Alex Quinn
Added file: http://bugs.python.org/file31370/monkey_patch_cookielib_time2isoz.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Alex Quinn]

Alex Quinn added the comment:

For those who are affected by this bug, here's a snippet to monkey-patch 
cookielib on any Python 2.4 to 2.7.

A more complete version of this was attached to my message a moment ago.

==
import cookielib
try:
cookielib.time2isoz(2**32)
except ValueError:
from datetime import datetime, timedelta
def time2isoz(t=None):
if t is None:
dt = datetime.now()
else:
dt = datetime.utcfromtimestamp(0) + timedelta(seconds=int(t))
return %04d-%02d-%02d %02d:%02d:%02dZ%\
(dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second)
cookielib.time2isoz = time2isoz
==

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Roundup Robot]

Roundup Robot devnull@devnull added the comment:

New changeset b15f60f9e256 by Victor Stinner in branch '3.1':
Issue #5537: Fix time2isoz() and time2netscape() functions of httplib.cookiejar
http://hg.python.org/cpython/rev/b15f60f9e256

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

 While it is unlikely that a purely numeric format such as %Y-%m-%d
 %H:%M:%S will be locale dependent, it is possible that some pre-C99
 systems would format dates using exotic digits is some locales.

Ok, I rewrote my patch to avoid strftime(). It should now be fixed.

FYI datetime.fromtimestamp() converts the timestamp to a double, which has a 
precision of 53 bits (no precision loss for year  285,422,890 and so it's 
enough for year 2038).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

Changes by STINNER Victor victor.stin...@haypocalc.com:


--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

Oh, my patch is incomplete: time2netscape() has the same issue.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

Victor,

I don't see your patch.  Did you remove it?

--
type:  - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

No, I forgot to upload it...

--
keywords: +patch
Added file: http://bugs.python.org/file20785/cookiejar_datetime.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

While it is unlikely that a purely numeric format such as %Y-%m-%d
%H:%M:%S will be locale dependent, it is possible that some pre-C99
systems would format dates using exotic digits is some locales.  Given
that format is so simple, I would just use explicit formatting of dt
components instead of datetime.strftime.  Doing so will also eliminate
a call to system strftime which is known to be quirky on popular
platforms even in C locale.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

If datetime.strftime() is not reliable, we should maybe fix it instead of using 
a workaround?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Alexander Belopolsky]

Alexander Belopolsky belopol...@users.sourceforge.net added the comment:

On Fri, Feb 18, 2011 at 11:04 AM, STINNER Victor rep...@bugs.python.org wrote:
..
 If datetime.strftime() is not reliable, we should maybe fix it instead of 
 using a workaround?

The real fix would be to rewrite strftime so that it does not call
system strftime.  See issue3173.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

 Shouldn't module time be changed to use a cross-platform implementation
 that uses a 64 bit time_t-like type? Apparently Perl 6 has made the
 equivalent change.

The error occurs on time.gmtime(t): even if we use 64 bits time_t type, we have 
to downcast it to system time_t later, because we would like to call gmtime() 
function at the end.

To workaround gmtime() limitation: we can simply use datetime instead. Attached 
patch replaces gmtime() by datetime.utcfromtimestamp(), and use its .strftime() 
method which has no such limitation.

--
nosy: +belopolsky, haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[John J Lee]

John J Lee jj...@users.sourceforge.net added the comment:

Shouldn't module time be changed to use a cross-platform implementation that 
uses a 64 bit time_t-like type?  Apparently Perl 6 has made the equivalent 
change.

Admittedly there seems to be no sign of that actually happening.

--
nosy: +jjlee

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5537] LWPCookieJar cannot handle cookies with expirations of 2038 or greater on 32-bit platforms

[Chris Hollenbeck]

New submission from Chris Hollenbeck chris.hollenb...@gmail.com:

The LWPCookieJar can be saved on 64-bit Ubuntu, but not on 32-bit Ubuntu
when the expiration year is greater than 2038.  This has not been tested
on any other Intel-compatible Linux platform, though it appears related
to the Year 2038 bug.  The MozillaCookieJar does not have a problem
saving on either architecture.

A sample crash is shown below:

  File /home/user/xblstatus/LiveConnect.py, line 189, in connect
self.cookiejar.save(self.cookieFile)
  File /usr/lib/python2.5/_LWPCookieJar.py, line 89, in save
f.write(self.as_lwp_str(ignore_discard, ignore_expires))
  File /usr/lib/python2.5/_LWPCookieJar.py, line 75, in as_lwp_str
r.append(Set-Cookie3: %s % lwp_cookie_str(cookie))
  File /usr/lib/python2.5/_LWPCookieJar.py, line 35, in lwp_cookie_str
time2isoz(float(cookie.expires
  File /usr/lib/python2.5/cookielib.py, line 98, in time2isoz
year, mon, mday, hour, min, sec = time.gmtime(t)[:6]
ValueError: timestamp out of range for platform time_t

---

The cookie jar and urllib2 integration was done with:

self.cookiejar = cookielib.LWPCookieJar()

self.opener =
urllib2.build_opener(urllib2.HTTPCookieProcessor(self.cookiejar))

urllib2.install_opener(self.opener)

---

The code used to save the cookie after accessing the web page was:

self.cookiejar.save(self.cookieFile)

The cookieFile variable is simply the default location of the cookie
file for saving in the program.

--
components: Library (Lib)
messages: 83979
nosy: hollec
severity: normal
status: open
title: LWPCookieJar cannot handle cookies with expirations of 2038 or greater 
on 32-bit platforms
versions: Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5537
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com