[issue15328] datetime.strptime slow
Changes by Éric Araujo mer...@netwok.org: -- components: +Extension Modules -None versions: +Python 3.4 -Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15328 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15328] datetime.strptime slow
Changes by Tshepang Lekhonkhobe tshep...@gmail.com: -- nosy: +tshepang status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15328 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15328] datetime.strptime slow
R. David Murray rdmur...@bitdance.com added the comment: If someone wants to propose a patch we can reopen the issue. -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15328 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15328] datetime.strptime slow
Changes by Brett Cannon br...@python.org: -- status: open - pending ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15328 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15328] datetime.strptime slow
New submission from Lars Nordin lars.nor...@gmail.com:
The datetime.strptime works well enough for me it is just slow.
I recently added a comparison to a log parsing script to skip log lines earlier
than a set date. After doing so my script ran much slower.
I am processing 4,784,212 log lines in 1,746 files.
Using Linux time, the measured run time is:
real5m12.884s
user4m54.330s
sys 0m2.344s
Altering the script to cache the datetime object if the date string is the
same, reduces the run time to:
real1m3.816s
user0m49.635s
sys 0m1.696s
# --- code snippet ---
# start_dt calculated at script start
...
day_dt = datetime.datetime.strptime(day_str, %Y-%m-%d)
if day_dt start_dt:
...
$ python
import platform
print 'Version :', platform.python_version()
print 'Version tuple:', platform.python_version_tuple()
print 'Compiler :', platform.python_compiler()
print 'Build:', platform.python_build()
Version : 2.7.2+
Version tuple: ('2', '7', '2+')
Compiler : GCC 4.6.1
Build: ('default', 'Oct 4 2011 20:03:08')
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 11.10
Release:11.10
Codename: oneiric
--
components: None
messages: 165256
nosy: Lars.Nordin
priority: normal
severity: normal
status: open
title: datetime.strptime slow
type: performance
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue15328
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15328] datetime.strptime slow
Lars Nordin lars.nor...@gmail.com added the comment: Running the script without any timestamp comparison (and parsing more log lines), gives these performance numbers: log lines: 7,173,101 time output: real1m9.892s user0m53.563s sys 0m1.592s -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15328 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15328] datetime.strptime slow
R. David Murray rdmur...@bitdance.com added the comment: Thanks for the report. However, do you have a patch to propose? Otherwise I'm not sure there is a reason to keep this issue open...one can always say various things are slow; that by itself is not a bug. Performance enhancement patches are welcome, though. If you are proposing adding an LRU cache, I think it may be that that should be left up to the application, as you did in your case. I'm not convinced there would be enough general benefit to make it worth adding to the stdlib, since the characteristics of date parsing workloads probably vary widely. -- nosy: +belopolsky, r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15328 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
