[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

The test doesn't fail anymore on buildbots. I ran manually test_strptime with a 
memory limit of 1 GB on Python 3.5 and 3.6: the test pass.

Can we close the issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Steve Dower]

Steve Dower added the comment:

Done.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Roundup Robot]

Roundup Robot added the comment:

New changeset bd7aed300a1b by Steve Dower in branch '3.5':
Issue #25029: MemoryError in test_strptime
https://hg.python.org/cpython/rev/bd7aed300a1b

New changeset 706b9eaba734 by Steve Dower in branch '3.5':
Merge fix for #25029
https://hg.python.org/cpython/rev/706b9eaba734

New changeset cceaccb6ec5a by Steve Dower in branch '3.5':
Adds Mics/NEWS entry for issue #25029.
https://hg.python.org/cpython/rev/cceaccb6ec5a

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

All builds on edelsohn-fedora-ppc64 are red starting from build 55 for 3.x and 
build 41 for 3.5. 3.4 is green.
http://buildbot.python.org/all/builders/PPC64%20Fedora%203.x/builds/55/
http://buildbot.python.org/all/builders/PPC64%20Fedora%203.5/builds/41/

==
ERROR: test_TimeRE_recreation (test.test_strptime.CacheTests)
--
Traceback (most recent call last):
  File 
"/home/shager/cpython-buildarea/3.5.edelsohn-fedora-ppc64/build/Lib/test/test_strptime.py",
 line 565, in test_TimeRE_recreation
_strptime._strptime_time('10', '%d')
  File 
"/home/shager/cpython-buildarea/3.5.edelsohn-fedora-ppc64/build/Lib/_strptime.py",
 line 494, in _strptime_time
tt = _strptime(data_string, format)[0]
  File 
"/home/shager/cpython-buildarea/3.5.edelsohn-fedora-ppc64/build/Lib/_strptime.py",
 line 312, in _strptime
_TimeRE_cache = TimeRE()
  File 
"/home/shager/cpython-buildarea/3.5.edelsohn-fedora-ppc64/build/Lib/_strptime.py",
 line 190, in __init__
self.locale_time = LocaleTime()
  File 
"/home/shager/cpython-buildarea/3.5.edelsohn-fedora-ppc64/build/Lib/_strptime.py",
 line 75, in __init__
self.__calc_am_pm()
  File 
"/home/shager/cpython-buildarea/3.5.edelsohn-fedora-ppc64/build/Lib/_strptime.py",
 line 114, in __calc_am_pm
am_pm.append(time.strftime("%p", time_tuple).lower())
MemoryError

--

--
components: Extension Modules
messages: 250177
nosy: David.Edelsohn, larry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: MemoryError in test_strptime
type: crash
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Larry Hastings]

Larry Hastings added the comment:

Any clue here?  Is this unaligned access?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[David Edelsohn]

David Edelsohn added the comment:

PPC64 is not a strict alignment system.  The system is running a non-recent 
release of Fedora, so it could be a bad interaction with libc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

It's a regression introduced by c31dad22c80d (Issue #24917).

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

Script to reproduce the issue:
---
import time
import locale
import pprint

time_tuple = time.struct_time((1999,3,17,1,44,55,2,76,0))

p1 = time.strftime("%p", time_tuple)
print("current LC_TIME", repr(p1))

locale.setlocale(locale.LC_TIME, ('de_DE', 'UTF8'))
p2 = time.strftime("%p", time_tuple)
print("de_DE (UTF8)", repr(p2))
---

Output:
---
$ python3.4 bug.py
current LC_TIME 'AM'
de_DE (UTF8) ''
---

The problem is that strftime()/wcsftime() *can* return 0, it's not an error. 
Whereas c31dad22c80d considers that if buflen is 0 but fmtlen is smaller than 
5, we must retry with a larger buffer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

Again, this issue remembers me the idea of rewriting strftime() in Python. We 
already have _strptime.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

Oh wait, the VmPeak is *much* high when running test_strptime:

* test_os VmPeak: 380 992 kB
* test_strptime VmPeak: 8 608 972 kB (8 GB!?)

1 GB should be enough for everybody:

$ bash -c 'ulimit -v 100; ./python -m test -v test_strptime'
..
==
ERROR: test_TimeRE_recreation (test.test_strptime.CacheTests)
--
Traceback (most recent call last):
  ...
  File "/home/haypo/prog/python/default/Lib/_strptime.py", line 75, in __init__
self.__calc_am_pm()
  File "/home/haypo/prog/python/default/Lib/_strptime.py", line 114, in 
__calc_am_pm
am_pm.append(time.strftime("%p", time_tuple).lower())
MemoryError

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

I ran test_strptime & test_os with tracemalloc: test_os memory peak is higher 
than test_strptime memory peak (I ran the two tests independently). So the bug 
cannot be reproduce on my Fedora 22 (x86_64).

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Steve Dower]

Steve Dower added the comment:

Thanks. I'll submit a PR for 3.5.0 later tonight - can't seem to clone Larry's 
repo successfully at work for some reason.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[eryksun]

eryksun added the comment:

Steve, it seems to me that for MSVC the EINVAL test should come first:
   
_Py_BEGIN_SUPPRESS_IPH
olderr = errno;
errno = 0;
buflen = format_time(outbuf, i, fmt, );
err = errno;
errno = olderr;
_Py_END_SUPPRESS_IPH

if (buflen == 0 && err == EINVAL) {
PyErr_SetString(PyExc_ValueError, "Invalid format string");
break;
}

Then the old test could be restored, i.e. (buflen > 0 || i >= 256 * fmtlen).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Steve Dower]

Steve Dower added the comment:

@eryksun - that's exactly what I've just done :)

Unfortunately, I don't have a _locale module, so I can't do Victor's test. 
Attached my patch in case Victor is around to test it.

--
keywords: +patch
Added file: http://bugs.python.org/file40407/25029_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[STINNER Victor]

STINNER Victor added the comment:

My test runs the following command on Linux (Fedora 22):

make && bash -c 'ulimit -v 100; ./python -u -m test -v test_strptime' 

* current default branch (rev 3c0c153d6b02): MemoryError
* with 25029_1.patch: the test pass

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Steve Dower]

Steve Dower added the comment:

Sorry Larry.

I'll fix it.

--
assignee:  -> steve.dower
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Steve Dower]

Steve Dower added the comment:

PR at 
https://bitbucket.org/larry/cpython350/pull-requests/21/issue-25029-memoryerror-in-test_strptime/diff

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Steve Dower]

Steve Dower added the comment:

Probably should, since the fix that caused it was in for rc3. There's no 
section for 3.5.0 final yet though (that's my excuse, anyway :) )

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Larry Hastings]

Larry Hastings added the comment:

Does this need a Misc/NEWS entry?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25029] MemoryError in test_strptime

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I would blame this change: 
.  The rest should not 
have any effect on Linux.

--
nosy: +belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com