poostenr added the comment:
My initial observations with my Python script using:
s = "{0},".format(columnvalue) # fast
Processed ~360MB of data from 2:16PM - 2:51PM (35 minutes, ~10MB/min)
One particular file 6.5MB took ~1 minute.
When I changed this line of code to:
s = "'{0}',".format(columnvalue) # ~30x slower (1 min. vs 30 min.)
Same particular file of 6.5MB took ~30 minutes (228KB/min).
My Python environment is:
C:\Data>python -V
Python 3.5.1 :: Anaconda 2.4.1 (32-bit)
I did some more testing with a very simplified piece of code, but is not
conclusive. But there is a significant jump when I introduce the single quotes
where I see a jump from 0m2.410s to 0m3.875s.
$ python -V
Python 3.5.1
//
// s='test'
// for x in range(10000000):
// y = "{0}".format(s)
$ time python test.py
real 0m2.410s
user 0m2.356s
sys 0m0.048s
// s='test'
// for x in range(10000000):
// y = "'%s'" % (s)
$ time python test2.py
real 0m2.510s
user 0m2.453s
sys 0m0.051s
// s='test'
// for x in range(10000000):
// y = "'{0}'".format(s)
$ time python test3.py
real 0m3.875s
user 0m3.819s
sys 0m0.048s
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26118>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
