Re: urllib2 performance on windows, usb connection
MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Regards, Martin Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... -- http://mail.python.org/mailman/listinfo/python-list
Re: Trouble sorting a list of objects by attributes
On Feb 6, 2:34 pm, Robocop btha...@physics.ucsd.edu wrote:
On Feb 6, 2:20 pm, Robocop btha...@physics.ucsd.edu wrote:
On Feb 6, 2:17 pm, Robocop btha...@physics.ucsd.edu wrote:
On Feb 6, 1:03 pm, bearophileh...@lycos.com wrote:
Robocop:
then within each department block of the list, have it organized by
projects.
I don't know what does it means.
timesheets.sort(key=operator.attrgetter('string'))
Try something like:
timesheets.sort(key=attrgetter(department, engagement, date,
stare_hour))
My brain might explode if i continue.
Relax.
Bye,
bearophile
UH OH GUYS!
line 110, in sorter
timesheets.sort(key=attrgetter(department, engagement,
date,start))
TypeError: attrgetter expected 1 arguments, got 4
I think there may have been a misunderstanding. I was already using
attrgetter, my problem is that it doesn't appear to be sorting by the
argument i give it. How does sort work with strings? How about with
datetime.time or datetime.date?
So far i can get it sorting strictly by the datetime objects, but i
need all of this sorting done within the constraints imposed by doing
sorts via department and engagements.
Any ideas?
I'm stuck with python 2.4 right now:(
I've got a python 2.4 fix though!
timesheets.sort(key= lambda i:(i.department,i.engagement,i.start))
Thanks for the help and time!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trouble sorting a list of objects by attributes
I think there may have been a misunderstanding. I was already using
attrgetter, my problem is that it doesn't appear to be sorting by the
argument i give it. How does sort work with strings? How about with
datetime.time or datetime.date?
You were using the attrgetter, but it looks like you weren't doing it correctly:
timesheets.sort(key=operator.attrgetter('string'))
You don't specify a type, be it string or date or anything: you
specify the /name/ of the attribute to get. It'll handle sorting by
any data type that has an order to it without you having to worry
about it at all. Strings, ints, dates, times.
It should be:
timesheets.sort(key=operator.attrgetter(department))
If you want to sort by the value of the department attribute, which
happens to be a string.
If you want to sort by the date attribute, which happens to be a
datetime.date, you do:
timesheets.sort(key=operator.attrgetter(date))
Where date is not a datatype a la datetime.date, but the name of an
attribute on your TimeSheet instances.
--S
--
http://mail.python.org/mailman/listinfo/python-list
Module unloading resulting from SystemExit exception
I am trying to understand how the SystemExit exception influences
module unloading. The situation that I am facing is that a Python
application behaves just fine upon exiting when executed using the
standard Python interpreter, but when run as a frozen executable,
modules are unloaded unexpectedly as a result of sys.exit() being
called. This wouldn't be an issue at all except that callbacks
registered with the atexit module have to re-import all modules that
they want to use, and this can be rather unwieldy.
I have narrowed it down to the SystemExit exception by determining
that the problem does not occur if my __main__ module catches
SystemExit and allows the script to terminate by reaching the end of
__main__. If the exception is not caught or is re-raised, then the
problem occurs. Since SystemExit influences the shutdown of threads,
it seems that I need to let it be raised to keep the application from
deadlocking on exit.
I am working with Python 2.6.1, and I saw this behavior with 2.5 as
well. It happens on Windows, Mac OS X, and Linux. The code that I am
using for the frozen executable is generated by sib.py from VendorID
1.0.0. The fact that VendorID is being used is inconsequential. I have
verified that its use in the executable is not causing this behavior
by removing all references to it. I have reduced this to the minimum
amount of code required to reproduce the problem and attached the
Python script an the C code generated by sib.py for the application
main() function.
My guess is that the code that sib.py generates is failing to do
something that Py_Main() would normally do. To that end, I have tried
modifying the main() function in a variety of ways to get it as close
to Py_Main() as I can, but it still exhibits the undesirable behavior
when sys.exit() is called. That pretty much leaves me thinking that
PyImport_ExecCodeModule() or PyImport_ImportFrozenModule() (I have
also looked at Py_FrozenMain() for insight) is the culprit.
Can anyone point me in the right direction for understanding how the
SystemExit exception results in this behavior? My hope is that there
is a change (simple or not) that can be made to the C code for the
frozen executable to resolve the problem.
Thanks in advance.
-Patrick
--
Patrick L. Hartling
http://www.137.org/patrick/
/*==*/
/* Main startup code for test */
/* This is generated code; Do not modify it!*/
/*--*/
#include stdarg.h
#include Python.h
#include marshal.h
/* #include vendorid.h */
extern unsigned char M___main__[];
static struct _frozen _PyImport_FrozenModules[] =
{
{0, 0, 0} /* sentinel */
};
int main(int argc, char **argv)
{
PyObject *v, *co;
PyImport_FrozenModules = _PyImport_FrozenModules;
Py_SetProgramName((char *)test);
/*vendorid_init(); */
Py_Initialize();
PySys_SetArgv(argc, argv);
co = PyMarshal_ReadObjectFromString((char *)M___main__, 655);
v = PyImport_ExecCodeModule((char *)__main__, co);
if(v == NULL)
{
PyErr_Print();
return -1;
}
Py_DECREF(v);
Py_Finalize();
return 0;
}
/*==*/
/* EOF */
/*--*/
import atexit
import os
import sys
def onExit():
print os.listdir(.)
def main():
atexit.register(onExit)
sys.exit()
if __name__ == __main__:
try:
main()
except SystemExit:
#pass
raise
--
http://mail.python.org/mailman/listinfo/python-list
Re: Ordered dict by default
Cameron Simpson: increases the unrealised assumptions about mappings in general which a newbie may acquire, causing them pain/complaint later with other mappings This is wrong in several different ways. I would much rather keep dictionaries as performant as possible, as a bare mapping, and add an odict for when order matters. In Python 3 strings are all unicode, integral numbers are all multiprecision, chars in Python 2.x+ are strings, lists are arrays that can grow dynamically, and so on because the Purpose of Python isn't to be as fast as possible, but to be first of all flexible, safe, easy, not but-prone, even if other solution or older versions were faster. Ruby shares almost same purposes. I presume Ruby wants to become a bit higher level than Python, because it now has a more flexible built-in. But even in a language designed to run way faster than Python, like D, I think the right thing for built-ins is to be as flexibleeasy as possible, so they are good enough in as many situations as possible, where performance isn't the most important thing, and to put the more specialized and faster versions into external libs. Making the built-ins be as optimized as possible (but limited too) looks like premature optimization to me, and in a language like Python premature optimization looks even more silly than usual. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Ordered dict by default
bearophile: In Python 3 strings are all unicode, integral numbers are all multiprecision, chars in Python 2.x+ are strings, lists are arrays that can grow dynamically, and so on because the Purpose of Python isn't to be as fast as possible, but to be first of all flexible, safe, easy, not but-prone, even if other solution or older versions were faster. I have missed another example: It may be possible to create a sorting routine that's not stable but is faster than the current stable timsort (for example in C++ STL you have both sorting routines, and the unstable one is a variant of introsort that is faster than the stable version). I think Python will keep the stable one, because even if (in theory. In practice it may be quite difficult to write such unstable sorter faster than timsort) it can be slower, it's safer and more useful than an unstable sort. Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
dq wrote: MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... How long does it take to transfer 4KB? If it can transfer 1MB/s then I'd say that 4KB is too small. Generally speaking, the higher the data rate, the larger the blocks you should be transferring at a time, IMHO. You could write a script to test the transfer speed with different block sizes. -- http://mail.python.org/mailman/listinfo/python-list
Re: Ordered dict by default
bearophileh...@lycos.com wrote: Cameron Simpson: increases the unrealised assumptions about mappings in general which a newbie may acquire, causing them pain/complaint later with other mappings This is wrong in several different ways. I would much rather keep dictionaries as performant as possible, as a bare mapping, and add an odict for when order matters. In Python 3 strings are all unicode, integral numbers are all multiprecision, chars in Python 2.x+ are strings, lists are arrays that can grow dynamically, and so on because the Purpose of Python isn't to be as fast as possible, but to be first of all flexible, safe, easy, not but-prone, even if other solution or older versions were faster. Ruby shares almost same purposes. I presume Ruby wants to become a bit higher level than Python, because it now has a more flexible built-in. But even in a language designed to run way faster than Python, like D, I think the right thing for built-ins is to be as flexibleeasy as possible, so they are good enough in as many situations as possible, where performance isn't the most important thing, and to put the more specialized and faster versions into external libs. Making the built-ins be as optimized as possible (but limited too) looks like premature optimization to me, and in a language like Python premature optimization looks even more silly than usual. You'll be wanting ordered sets next! :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
dq wrote: MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Regards, Martin Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... After some tweaking of the block size, I managed to get the DL speed up to about 900 Mb/s. It's still not quite Ubuntu, but it's a good order of magnitude better. The new DL code is pretty much this: blocksize = 2 ** 16# plus or minus a power of 2 source = urllib2.urlopen( 'url://string' ) target = open( pathname, 'wb') fullsize = float( source.info()['Content-Length'] ) DLd = 0 while DLd fullsize: DLd = DLd + blocksize # optional: write some DL progress info # somewhere, e.g. stdout target.close() source.close() -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
MRAB wrote: dq wrote: MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... How long does it take to transfer 4KB? If it can transfer 1MB/s then I'd say that 4KB is too small. Generally speaking, the higher the data rate, the larger the blocks you should be transferring at a time, IMHO. You could write a script to test the transfer speed with different block sizes. Thanks MRAB, 32 or 64 KB seems to be quickest, but I'll do a more scientific test soon and see what turns up. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flattening lists
On Feb 6, 3:23 pm, Rhamphoryncus rha...@gmail.com wrote: On Feb 5, 1:16 pm, Michele Simionato michele.simion...@gmail.com wrote: On Feb 5, 7:24 pm, a...@pythoncraft.com (Aahz) wrote: In article a22c77c4-a812-4e42-8972-6f3eedf72...@l33g2000pri.googlegroups.com, Michele Simionato michele.simion...@gmail.com wrote: Looks fine to me. In some situations you may also use hasattr(el, '__iter__') instead of isinstance(el, list) (it depends if you want to flatten generic iterables or only lists). Of course, once you do that, you need to special-case strings... Strings are iterable but have no __iter__ method, which is fine in this context, since I would say 99.9% of times one wants to treat them as atomic objects, so no need to special case. Don't worry, that little oddity was fixed for you: Python 3.0+ (unknown, Dec 8 2008, 14:26:15) [GCC 4.3.2] on linux2 Type help, copyright, credits or license for more information. str.__iter__ slot wrapper '__iter__' of 'str' objects bytes.__iter__ slot wrapper '__iter__' of 'bytes' objects bytearray.__iter__ slot wrapper '__iter__' of 'bytearray' objects I'm in the why do you need more than 1 depth? camp. Dispatching based on your own type should be given an extra look. Dispatching based passed in types should be given three extra looks. I didn't realize itertools.chain(*iterable) worked. I guess that needs to be pushed as the canonical form. What about this (from the Recipes section of the itertools manual)? def flatten(listOfLists): return list(chain.from_iterable(listOfLists)) -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
dq wrote: dq wrote: MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Regards, Martin Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... After some tweaking of the block size, I managed to get the DL speed up to about 900 Mb/s. It's still not quite Ubuntu, but it's a good order of magnitude better. The new DL code is pretty much this: blocksize = 2 ** 16# plus or minus a power of 2 source = urllib2.urlopen( 'url://string' ) target = open( pathname, 'wb') fullsize = float( source.info()['Content-Length'] ) DLd = 0 while DLd fullsize: DLd = DLd + blocksize # optional: write some DL progress info # somewhere, e.g. stdout target.close() source.close() I'd like to suggest that the block size you add to 'DLd' be the actual size of the returned block, just in case the read() doesn't return all you asked for (it might not be guaranteed, and the chances are that the final block will be shorter, unless 'fullsize' happens to be a multiple of 'blocksize'). If less is returned by read() then the while-loop might finish before all the data has been downloaded, and if you just add 'blocksize' each time it might end up 'fullsize', ie apparently 100% downloaded! -- http://mail.python.org/mailman/listinfo/python-list
Why doesn't this RE match?
I'm confused by this behaviour:
import re
regex = re.compile('foo')
match = regex.match('whatfooever')
In my experience with regular expressions, regex should have found a
match. However, in this case regex.match() returns None. Why is that?
What am I missing?
Thank you...
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't this RE match?
2009/2/7 Just Another Victim of the Ambient Morality ihates...@hotmail.com:
I'm confused by this behaviour:
import re
regex = re.compile('foo')
match = regex.match('whatfooever')
In my experience with regular expressions, regex should have found a
match. However, in this case regex.match() returns None. Why is that?
What am I missing?
Thank you...
--
http://mail.python.org/mailman/listinfo/python-list
Try re.search() instead of match(), if nod only the beginning should
be checked for a match:
see
http://docs.python.org/library/re.html#matching-vs-searching
vbr
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't this RE match?
Just Another Victim of the Ambient Morality wrote:
I'm confused by this behaviour:
import re
regex = re.compile('foo')
match = regex.match('whatfooever')
In my experience with regular expressions, regex should have found a
match. However, in this case regex.match() returns None. Why is that?
What am I missing?
Thank you...
match() is anchored at (ie matches only at) the start of the string. You
need search(). It's in the docs! :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't this RE match?
In my experience with regular expressions, regex should have found a match. However, in this case regex.match() returns None. Why is that? What am I missing? You want regex.search(). match specifically looks for the pattern from the start of the screen, search anywhere. --S -- http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't this RE match?
On 2009-02-06 18:23, Just Another Victim of the Ambient Morality wrote:
I'm confused by this behaviour:
import re
regex = re.compile('foo')
match = regex.match('whatfooever')
In my experience with regular expressions, regex should have found a
match. However, in this case regex.match() returns None. Why is that?
What am I missing?
http://docs.python.org/library/re#re.RegexObject.match
Basically, regex.match() tries to apply the regexp to the beginning of the
string. Use regex.search() to search for a match later in the string.
--
Robert Kern
I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth.
-- Umberto Eco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Trouble sorting a list of objects by attributes
On Feb 6, 2:41 pm, Stephen Hansen apt.shan...@gmail.com wrote:
I think there may have been a misunderstanding. I was already using
attrgetter, my problem is that it doesn't appear to be sorting by the
argument i give it. How does sort work with strings? How about with
datetime.time or datetime.date?
You were using the attrgetter, but it looks like you weren't doing it
correctly:
timesheets.sort(key=operator.attrgetter('string'))
You don't specify a type, be it string or date or anything: you
specify the /name/ of the attribute to get. It'll handle sorting by
any data type that has an order to it without you having to worry
about it at all. Strings, ints, dates, times.
It should be:
timesheets.sort(key=operator.attrgetter(department))
If you want to sort by the value of the department attribute, which
happens to be a string.
If you want to sort by the date attribute, which happens to be a
datetime.date, you do:
timesheets.sort(key=operator.attrgetter(date))
Where date is not a datatype a la datetime.date, but the name of an
attribute on your TimeSheet instances.
--S
Yeah this i totally understand. In my code i was using attrgetter
correctly, in the above answer i used string to represent the datatype
of the attribute, not what i actually put in there. More my laziness
than anything else. The problem boiled down attrgetter taking
multiple arguments in 2.5, but only one in 2.4. Thanks for the input
though.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why doesn't this RE match?
On Feb 7, 11:23 am, Just Another Victim of the Ambient Morality
ihates...@hotmail.com wrote:
I'm confused by this behaviour:
import re
regex = re.compile('foo')
match = regex.match('whatfooever')
In my experience with regular expressions, regex should have found a
match. However, in this case regex.match() returns None. Why is that?
Because that is exactly what it is documented to do.
What am I missing?
Inter alia:
(1) The whole section of the fantastic manual devoted to this topic:
http://www.python.org/doc/2.6/library/re.html#matching-vs-searching
(2) a section in the fabulous HOWTO:
http://www.python.org/doc/2.6/howto/regex.html#performing-matches
(3) the re section in the phantasmagorical help sub-system (very handy
if your internet connection goes on the fritz):
Python 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit
(Intel)] on win32
Type help, copyright, credits or license for more information.
import re
help(re)
Help on module re:
[snip]
This module exports the following functions:
matchMatch a regular expression pattern to the beginning
of a string.
search Search a string for the presence of a pattern.
[snip]
--
http://mail.python.org/mailman/listinfo/python-list
Possible to access MS Access 2007 password-protected database from Python?
Googling has shown me various ways of connecting to a non-password- protected Access database, but I was wondering if someone could point to code illustrating how to use an Access db that's password- protected. I haven't been able to find anything on this. Thanks, Ken -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
MRAB wrote: dq wrote: dq wrote: MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Regards, Martin Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... After some tweaking of the block size, I managed to get the DL speed up to about 900 Mb/s. It's still not quite Ubuntu, but it's a good order of magnitude better. The new DL code is pretty much this: blocksize = 2 ** 16# plus or minus a power of 2 source = urllib2.urlopen( 'url://string' ) target = open( pathname, 'wb') fullsize = float( source.info()['Content-Length'] ) DLd = 0 while DLd fullsize: DLd = DLd + blocksize # optional: write some DL progress info # somewhere, e.g. stdout target.close() source.close() I'd like to suggest that the block size you add to 'DLd' be the actual size of the returned block, just in case the read() doesn't return all you asked for (it might not be guaranteed, and the chances are that the final block will be shorter, unless 'fullsize' happens to be a multiple of 'blocksize'). If less is returned by read() then the while-loop might finish before all the data has been downloaded, and if you just add 'blocksize' each time it might end up 'fullsize', ie apparently 100% downloaded! Interesting. I'll if to see if any of the downloaded files end prematurely :) btw, I forgot the most important line of the code! blocksize = 2 ** 16# plus or minus a power of 2 source = urllib2.urlopen( 'url://string' ) target = open( pathname, 'wb') fullsize = float( source.info()['Content-Length'] ) DLd = 0 while DLd fullsize: # +++ target.write( source.read( blocksize ) ) # +++ # +++ DLd = DLd + blocksize # optional: write some DL progress info # somewhere, e.g. stdout target.close() source.close() Using that, I'm not quite sure where I can grab onto the value of how much was actually read from the block. I suppose I could use an intermediate variable, read the data into it, measure the size, and then write it to the file stream, but I'm not sure it would be worth the overhead. Or is there some other magic I should know about? If I start to get that problem, at least I'll know where to
Running all unit tests
I'm working on my first substantial Python project, and I'm following a fully test-first approach. I'd like to know how Pythonistas typically go about running all of their tests to ensure that my application stays green. In Ruby, I would have a Rake task so that I could say rake test and all tests would be executed. In C or C++ I would have a make target so I could run all my tests with make test. In Java it would be an Ant task and ant test. And so forth and so on. What's the recommended approach for Python programs? I'm sure I could write a shell script (or a Python script even) that scans my test directory for test cases and runs them, but I'm wondering if there's something already built in that could do this for me. -- Jason Voegele Only fools are quoted. -- Anonymous -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
dq wrote: MRAB wrote: dq wrote: dq wrote: MRAB wrote: dq wrote: Martin v. Löwis wrote: So does anyone know what the deal is with this? Why is the same code so much slower on Windows? Hope someone can tell me before a holy war erupts :-) Only the holy war can give an answer here. It certainly has *nothing* to do with Python; Python calls the operating system functions to read from the network and write to the disk almost directly. So it must be the operating system itself that slows it down. To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) If the network read speed is fine, but writing slows down, I ask the same questions. Perhaps you have some virus scanner installed that filters all write operations? Maybe Windows sucks? Regards, Martin Thanks for the ideas, Martin. I ran a couple of experiments to find the culprit, by downloading the same 20 MB file from the same fast server. I compared: 1. DL to HD vs USB iPod. 2. AV on-access protection on vs. off 3. source. read() only vs. file.write( source.read() ) The culprit is definitely the write speed on the iPod. That is, everything runs plenty fast (~1 MB/s down) as long as I'm not writing directly to the iPod. This is kind of odd, because if I copy the file over from the HD to the iPod using windows (drag-n-drop), it takes about a second or two, so about 10 MB/s. So the problem is definitely partially Windows, but it also seems that Python's file.write() function is not without blame. It's the combination of Windows, iPod and Python's data stream that is slowing me down. I'm not really sure what I can do about this. I'll experiment a little more and see if there's any way around this bottleneck. If anyone has run into a problem like this, I'd love to hear about it... You could try copying the file to the iPod using the command line, or copying data from disk to iPod in, say, C, anything but Python. This would allow you to identify whether Python itself has anything to do with it. Well, I think I've partially identified the problem. target.write( source.read() ) runs perfectly fast, copies 20 megs in about a second, from HD to iPod. However, if I run the same code in a while loop, using a certain block size, say target.write( source.read(4096) ), it takes forever (or at least I'm still timing it while I write this post). The mismatch seems to be between urllib2's block size and the write speed of the iPod, I might try to tweak this a little in the code and see if it has any effect. Oh, there we go: 20 megs in 135.8 seconds. Yeah... I might want to try to improve that... After some tweaking of the block size, I managed to get the DL speed up to about 900 Mb/s. It's still not quite Ubuntu, but it's a good order of magnitude better. The new DL code is pretty much this: blocksize = 2 ** 16# plus or minus a power of 2 source = urllib2.urlopen( 'url://string' ) target = open( pathname, 'wb') fullsize = float( source.info()['Content-Length'] ) DLd = 0 while DLd fullsize: DLd = DLd + blocksize # optional: write some DL progress info # somewhere, e.g. stdout target.close() source.close() I'd like to suggest that the block size you add to 'DLd' be the actual size of the returned block, just in case the read() doesn't return all you asked for (it might not be guaranteed, and the chances are that the final block will be shorter, unless 'fullsize' happens to be a multiple of 'blocksize'). If less is returned by read() then the while-loop might finish before all the data has been downloaded, and if you just add 'blocksize' each time it might end up 'fullsize', ie apparently 100% downloaded! Interesting. I'll if to see if any of the downloaded files end prematurely :) btw, I forgot the most important line of the code! blocksize = 2 ** 16# plus or minus a power of 2 source = urllib2.urlopen( 'url://string' ) target = open( pathname, 'wb') fullsize = float( source.info()['Content-Length'] ) DLd = 0 while DLd fullsize: # +++ target.write( source.read( blocksize ) ) # +++ # +++ DLd = DLd + blocksize # optional: write some DL progress info # somewhere, e.g. stdout target.close() source.close() Using that, I'm not quite sure where I can grab onto the value of how much was actually read from the block. I suppose I could use an intermediate variable, read the data into it, measure the size, and then write it to the file stream, but I'm not sure it would be worth the overhead. Or is there some other magic I should know about? If I start to get that problem, at least I'll know
sys.exc_info() different between python 2.5.4 and 2.6.1?
Hi all,
I just noticed a changed behaviour of sys.exc_info() between python
2.5.4 and 2.6.1 and wanted to ask, wheter it was intended, and how to
deal with the new state.
Some code triggers an error while opening a text file (windows 1250 -
with non-ascii characters) wrongly as utf-8,
this gets catched by a bare except clause.
Further the mentioned versions of python differ in handling sys.exc_info()
The following lines:
print sys.exc_info()[1]
print repr(sys.exc_info()[1])
print str(sys.exc_info()[1])
print unicode(sys.exc_info()[1])
result in python 2.5 in:
'utf8' codec can't decode byte 0x9a in position 2: unexpected code byte
UnicodeDecodeError('utf8', 'ab\x9acd', 2, 3, 'unexpected code byte')
'utf8' codec can't decode byte 0x9a in position 2: unexpected code byte
'utf8' codec can't decode byte 0x9a in position 2: unexpected code byte
in python 2.6 it is:
'utf8' codec can't decode byte 0x9a in position 2: unexpected code byte
UnicodeDecodeError('utf8', 'ab\x9acd', 2, 3, 'unexpected code byte')
'utf8' codec can't decode byte 0x9a in position 2: unexpected code byte
('utf8', 'ab\x9acd', 2, 3, 'unexpected code byte')
Which is kind of confusing as my the excepthook function uses the
unicode() form of sys.exc_info()[1]
(The second part ab\x9acd is the whole content of the file being
read - which is normally quite a bit longer than this sample...)
Is this the expected behaviour? If so, what might be the reason for
differently outputting str() and unicode()?
How can this be dealt with, are the standard error values somehow
expected to be ascii, so that str() can be used reasonably?
Any hints are much appreciated,
Vlasta
--
http://mail.python.org/mailman/listinfo/python-list
Re: Using while loop and if statement to tell if a binary has an odd or even number of 1's.
Duncan Booth duncan.bo...@invalid.invalid writes: Mark Dickinson dicki...@gmail.com wrote: [snip] while n: count += 1 n = n-1 return count is_even = count_set_bits(the_int) % 2 == 0 ...but anyone submitting this as a homework solution had better be prepared to explain why it works. I remember a programming exercise when I was an undergraduate and anyone who *didn't* use that trick got marked down for writing inefficient code. Curious. I don't see why def parity(x): b = 2 l = 1 while True: b = l if x b: break l = 1 while l: b = l x ^= x l x = b - 1 l = 1 return x 1 is any less efficient. Indeed, it seems more so to me. The number of top-level loop iterations is bounded by the logarithm of the total number of bits in the input rather than the Hamming weight. In terms of single-precision operations (if we're dealing with bignums) the analysis is more complicated; but the number of single-precision operations in one of my loops is a linear function of l (assuming that the comparison is done sensibly), and l increases and decreases geometrically, so I have performance which is O(log x). Assuming no special Hamming-weight distribution on the input, the `efficient' version takes O(log^2 x) single-precision operations. Given an /a-priori/ upper bound on the length of the input, e.g., it fits in a machine word, the above technique is still probably faster. That is, assuming arithmetic and bitwise operations on integers take constant time, my version runs in O(log log x) time whereas the `efficient' version takes O(log x) time. (My function returns the complement of the value requested. Fixing it is obviously trivial.) -- [mdw] -- http://mail.python.org/mailman/listinfo/python-list
Re: Running all unit tests
On Feb 6, 9:11 pm, Jason Voegele ja...@jvoegele.com wrote: I'm working on my first substantial Python project, and I'm following a fully test-first approach. I'd like to know how Pythonistas typically go about running all of their tests to ensure that my application stays green. In Ruby, I would have a Rake task so that I could say rake test and all tests would be executed. In C or C++ I would have a make target so I could run all my tests with make test. In Java it would be an Ant task and ant test. And so forth and so on. What's the recommended approach for Python programs? I'm sure I could write a shell script (or a Python script even) that scans my test directory for test cases and runs them, but I'm wondering if there's something already built in that could do this for me. -- Jason Voegele Only fools are quoted. -- Anonymous I don't know about the recommended approach, but I've done something like you suggest in a library I authored. Any files named test*.py are found and added to the unittest test suite. See http://code.google.com/p/pydicom/source/browse/trunk/source/dicom/test/run_tests.py. HTH Darcy -- http://mail.python.org/mailman/listinfo/python-list
Re: Running all unit tests
On Fri, 06 Feb 2009 21:11:15 -0500 Jason Voegele ja...@jvoegele.com wrote: I'm working on my first substantial Python project, and I'm following a fully test-first approach. I'd like to know how Pythonistas typically go about running all of their tests to ensure that my application stays green. I check in my unit tests to CVS along with the code and name the unit test files with the form TEST_xxx.py to identify them. I then put the following script into my crontab so that my tests are run every day. I make the TEST_xxx.py executible so that I can always run a specific test at any time if I am working on specific code. Note that the script only sends email when something goes red so I don't have to wade through green reports every day. The errors stand out that way. #! /bin/sh # $Id: run_unit_tests,v 1.4 2008/06/23 01:07:24 darcy Exp $ # This script runs all of the unit tests if [ $# -lt 2 ] then echo Usage: $0 base directory [email address] exit 1 fi cd $1 trap rm -f /tmp/run_tests.$$ 0 HOST=`hostname` find -L . -type f -name 'TEST_*.py' | while read TEST do $TEST /tmp/run_tests.$$ 21 || mail -s TEST failure on $HOST: $TEST $2 /tmp/run_tests.$$ done -- D'Arcy J.M. Cain da...@druid.net | Democracy is three wolves http://www.druid.net/darcy/| and a sheep voting on +1 416 425 1212 (DoD#0082)(eNTP) | what's for dinner. -- http://mail.python.org/mailman/listinfo/python-list
Lists implemented as integer-hashed Dictionaries?
Somebody much more intelligent than I said today that someone told him that Python lists are just dictionaries with lists hashed by integers. Since he said that someone else told him this, I piped up and said that I thought that wasn't true. I looked at the source code for lists in python, and I did not expressly remember seeing dictionaries. Unfortunately I am not somewhere where I can easily look at the code right now (I'm logged into Windows!), and I might not realize exactly what I'm looking at anyways, but I'd like to extend an apology to the guy soon if I was wrong. So, can anyone tell me if lists are really just dictionaries? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists implemented as integer-hashed Dictionaries?
Correction, the first sentence should read, lists are just dictionaries keyed with integers. On Fri, Feb 6, 2009 at 10:18 PM, er eroberer...@gmail.com wrote: Somebody much more intelligent than I said today that someone told him that Python lists are just dictionaries with lists hashed by integers. Since he said that someone else told him this, I piped up and said that I thought that wasn't true. I looked at the source code for lists in python, and I did not expressly remember seeing dictionaries. Unfortunately I am not somewhere where I can easily look at the code right now (I'm logged into Windows!), and I might not realize exactly what I'm looking at anyways, but I'd like to extend an apology to the guy soon if I was wrong. So, can anyone tell me if lists are really just dictionaries? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists implemented as integer-hashed Dictionaries?
On Fri, Feb 6, 2009 at 7:18 PM, er eroberer...@gmail.com wrote: Somebody much more intelligent than I said today that someone told him that Python lists are just dictionaries with lists hashed by integers. Since he said that someone else told him this, I piped up and said that I thought that wasn't true. I looked at the source code for lists in python, and I did not expressly remember seeing dictionaries. Unfortunately I am not somewhere where I can easily look at the code right now (I'm logged into Windows!), and I might not realize exactly what I'm looking at anyways, but I'd like to extend an apology to the guy soon if I was wrong. So, can anyone tell me if lists are really just dictionaries? Thanks! They most certainly are not. This is Python, not Lua or PHP (where for reasons I cannot fathom, they've seen fit to conflate dictionaries and lists; although it does make slightly more sense in Lua's case) Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Running all unit tests
Jason Voegele ja...@jvoegele.com writes: What's the recommended approach for Python programs? I'm sure I could write a shell script (or a Python script even) that scans my test directory for test cases and runs them, but I'm wondering if there's something already built in that could do this for me. The lack of a built-in “collect and run all the tests in this working tree” in the Python unit test system is a known problem; discussions are ongoing what to do about it. Meanwhile, the third-party ‘nose’ system URL:http://somethingaboutorange.com/mrl/projects/nose/ provides this and much more, while remaining compatible with both testing systems in the standard library. I generally set up a ‘test’ target in my Makefile, such that it will use ‘nosetests’ to collect and run all the tests; then I just run ‘make test’ in a loop that is triggered by any filesystem change in my project working tree. -- \“I have a microwave fireplace in my house. The other night I | `\ laid down in front of the fire for the evening in two minutes.” | _o__) —Steven Wright | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists implemented as integer-hashed Dictionaries?
Thanks Chris. Lua tables are one of my favorite linguistic traits, which was actually part of the discussion that brought up this nugget. Nevertheless, any details you care to provide about the details. I'm going to dive into the source code in more depth tomorrow, just so I can get a better understanding anyway, but I'd love to hear some details, or see any links, if you have them. On Fri, Feb 6, 2009 at 10:25 PM, Chris Rebert c...@rebertia.com wrote: On Fri, Feb 6, 2009 at 7:18 PM, er eroberer...@gmail.com wrote: Somebody much more intelligent than I said today that someone told him that Python lists are just dictionaries with lists hashed by integers. Since he said that someone else told him this, I piped up and said that I thought that wasn't true. I looked at the source code for lists in python, and I did not expressly remember seeing dictionaries. Unfortunately I am not somewhere where I can easily look at the code right now (I'm logged into Windows!), and I might not realize exactly what I'm looking at anyways, but I'd like to extend an apology to the guy soon if I was wrong. So, can anyone tell me if lists are really just dictionaries? Thanks! They most certainly are not. This is Python, not Lua or PHP (where for reasons I cannot fathom, they've seen fit to conflate dictionaries and lists; although it does make slightly more sense in Lua's case) Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Portable way to refer to the null device?
I need to run a command using subprocess.Popen() and have stdin
connected to the null device. On unix, I would do:
self.process = subprocess.Popen(argv,
env=new_env,
stdout=open(outfile, 'w'),
stderr=open(errfile, 'w'),
stdin=open('/dev/null')
)
but that's not portable to windows. Does Python have a portable way
to get a file object connected to the null device, regardless of what
operating system you're running on?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Lists implemented as integer-hashed Dictionaries?
On Fri, Feb 6, 2009 at 10:25 PM, Chris Rebert c...@rebertia.com wrote: On Fri, Feb 6, 2009 at 7:18 PM, er eroberer...@gmail.com wrote: Somebody much more intelligent than I said today that someone told him that Python lists are just dictionaries with lists hashed by integers. Since he said that someone else told him this, I piped up and said that I thought that wasn't true. I looked at the source code for lists in python, and I did not expressly remember seeing dictionaries. Unfortunately I am not somewhere where I can easily look at the code right now (I'm logged into Windows!), and I might not realize exactly what I'm looking at anyways, but I'd like to extend an apology to the guy soon if I was wrong. So, can anyone tell me if lists are really just dictionaries? Thanks! They most certainly are not. This is Python, not Lua or PHP (where for reasons I cannot fathom, they've seen fit to conflate dictionaries and lists; although it does make slightly more sense in Lua's case) On Fri, Feb 6, 2009 at 7:32 PM, er eroberer...@gmail.com wrote: Thanks Chris. Lua tables are one of my favorite linguistic traits, which was actually part of the discussion that brought up this nugget. Nevertheless, any details you care to provide about the details. I'm going to dive into the source code in more depth tomorrow, just so I can get a better understanding anyway, but I'd love to hear some details, or see any links, if you have them. Unfortunately, I don't know the details as I'm not a CPython dev. I only indirectly know that arrays are used from having read so previously in writings talking about the implementation. Also, Python generally tries to make its basic datatypes speedy, and using dictionaries to implement lists would be completely counter to that goal, no matter how excellent the dictionary implementation is. In all likelihood, someone who /is/ familiar with the implementation will probably chime in with the actual details. Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Portable way to refer to the null device?
Roy I need to run a command using subprocess.Popen() and have stdin Roy connected to the null device. os.path.devnull should do what you want: os.path.devnull '/dev/null' import ntpath ntpath.devnull 'nul' -- Skip Montanaro - s...@pobox.com - http://www.smontanaro.net/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Portable way to refer to the null device?
Roy Smith r...@panix.com writes:
I need to run a command using subprocess.Popen() and have stdin
connected to the null device. On unix, I would do:
self.process = subprocess.Popen(argv,
env=new_env,
stdout=open(outfile, 'w'),
stderr=open(errfile, 'w'),
stdin=open('/dev/null')
)
Yikes, that's a nasty indentation you've got going on there. I'd be
writing the above as:
self.process = subprocess.Popen(
argv,
env=new_env,
stdout=open(outfile, 'w'),
stderr=open(errfile, 'w'),
stdin=open('/dev/null'),
)
but that's not portable to windows. Does Python have a portable way
to get a file object connected to the null device, regardless of what
operating system you're running on?
Almost: the ‘os.devnull’ attribute is documented (in the documentation
for the ‘os’ module) as “os.devnull is the file path of the null
device ('/dev/null', etc.)”.
So the above becomes:
self.process = subprocess.Popen(
argv,
env=new_env,
stdout=open(outfile, 'w'),
stderr=open(errfile, 'w'),
stdin=open(os.devnull),
)
--
\ “Pinky, are you pondering what I'm pondering?” “I think so, |
`\ Brain, but how will we get a pair of Abe Vigoda's pants?” |
_o__) —_Pinky and The Brain_ |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is c.l.py becoming less friendly?
Probably that [c.l.]python is becoming more popular and, like most things as they become popular, it loses its purity... much like the Internet in the early 1990s. -- http://mail.python.org/mailman/listinfo/python-list
Re: Where to host a (Python) project?
In article 6dcb8ce5-c93e-458c-9047-e5db60f27...@v18g2000pro.googlegroups.com, andrew cooke and...@acooke.org wrote: On Feb 1, 8:45=A0pm, a...@pythoncraft.com (Aahz) wrote: [...]=A0I for one won't participate in any list hosted on Google because of the need for a Google login. hi, just fyi, i investigated this and you can join any publicly readable group by sending an email to the -subscribe address. you do not need a google login for this and, as far as i can tell, it then operates for you like a normal mailing list. for example, to subscribe to the group foo, you would send an email to foo-subscr...@googlegroups.com. to unsubscribe, use foo- unsubscr...@googlegroups.com. this isn't exactly well-publicised, but i tested it and it does work. The same thing is theoretically true for Yahoo groups, but I've heard from people over the years about various difficulties fixing problems with list subscriptions in the absence of a real Yahoo login and I'm not particularly interested in finding out that the same thing ends up being true for Google lists. -- Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. -- http://mail.python.org/mailman/listinfo/python-list
GOZERBOT 0.9 RELEASED
Finally gozerbot 0.9 has been released. This is a huge step forward to version 1.0 and contains a number of changes: * use json as the format to save data in instead of pickles * let config files also use json, this makes them more readable and human editable * remove popen usage from the bot core * remove execfile() calls from the bot core * rewrite the gozerbot package into several subpackages * use sqlaclhemy to provide database backend (sqlite3 is default) * require python2.5 * move most of the plugins into their own package * restructure the gozerdata layout so its more readable All these changes makes upgrading from older versions of gozerbot necessary so a gozerbot-upgrade program is provided (upgrading from 0.7 is not supported yet, will follow soon). See http://gozerbot.org on how to install gozerbot 0.9 About GOZERBOT: 0.9 Requirements * a shell * python 2.5 or higher * gnupg * simplejson * sqlalchemy * xmpppy Why gozerbot? * provide both IRC and Jabber support * user management by userhost .. bot will not respond if it doesn't know you (see USER) * fleet .. use more than one bot in a program (list of bots) (see FLEET) * use the bot through dcc chat * fetch rss feeds (see RSS) * remember items * relaying between bots (see RELAY) * program your own plugins (see PROGRAMPLUGIN) * query other bots with json REST (see CLOUD) * serve as a udp - irc or jabber notification bot (see UDP * sqlalchemy support Bart -- http://mail.python.org/mailman/listinfo/python-list
What is difference between ADO and RDO
Explain ADO and RDO -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 performance on windows, usb connection
On 2009-02-06, Martin v. L?wis mar...@v.loewis.de wrote: To investigate further, you might drop the write operating, and measure only source.read(). If that is slower, then, for some reason, the network speed is bad on Windows. Maybe you have the network interfaces misconfigured? Maybe you are using wireless on Windows, but cable on Linux? Maybe you have some network filtering software running on Windows? Maybe it's just that Windows sucks?-) MAYBE? Good one. A friend of mine signed up for 6MB/s DSL a while back. She set up the DSL modem/WAP/firewall according to QWest's instructions and it basically worked, but her download speeds were much slower than they should have been. It turns out that the Windows drivers for the Intel WiFi chipset used in many laptops completely and utterly sucked. Her laptop and my laptop used the same WiFi chipset and neither could sustain more than about 1MB/s speeds when running Windows. I know my Windows setup was running no firewall or virus scanner software. Connecting to the DSL modem via Ethernet cable brought the download speeds up to 6MB/s. When Linux was running on either laptop, the WiFi link ran at full speed, and DSL was the bottleneck as it should have been. Windows support was just plain broken for one of the most popular WiFi chipsets (the Intel Pro 2200BG). -- Grant Edwards grante Yow! I'm wearing PAMPERS!! at visi.com -- http://mail.python.org/mailman/listinfo/python-list
WebError documentation?
Is there any? Where is it? Extensive Googling has proven fruitless. Thanks, rg -- http://mail.python.org/mailman/listinfo/python-list
simple web app, where to start
I have a simple script that takes a few input values and returns a csv file and a few stats. If I wanted to host this on the web how would I. I have no idea where to begin. If someone could point me in the right direction like maybe a tutorial, tools I will need, functions. I would appreciate it.I know a little html but am not sure how to integrate python or know what servers will handle it . Thanks Vincent Davis -- http://mail.python.org/mailman/listinfo/python-list
Re: Returning a variable number of things...
Chris Rebert wrote: On Fri, Feb 6, 2009 at 10:50 AM, r0g aioe@technicalbloke.com wrote: Hi There, I have a function that uses *args to accept a variable number of parameters and I would like it to return a variable number of objects. I could return a list but I would like to take advantage of tuple unpacking with the return values e.g. Despite its name, tuple unpacking works with lists too. Cheers, Chris Ah so it does! Thanks Chris! I'm losing count of all the reasons I have to love this language :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: WebError documentation?
On Thu, Feb 5, 2009 at 12:52 PM, Ron Garret rnospa...@flownet.com wrote: Is there any? Where is it? Extensive Googling has proven fruitless. It's not a standard Python exception. A third-party library you're using must be raising it. Check the exception traceback. Cheers, Chris -- Follow the path of the Iguana... http://rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Flattening lists
On Feb 6, 10:23 pm, Rhamphoryncus rha...@gmail.com wrote: On Feb 5, 1:16 pm, Michele Simionato michele.simion...@gmail.com wrote: On Feb 5, 7:24 pm, a...@pythoncraft.com (Aahz) wrote: In article a22c77c4-a812-4e42-8972-6f3eedf72...@l33g2000pri.googlegroups.com, Michele Simionato michele.simion...@gmail.com wrote: Looks fine to me. In some situations you may also use hasattr(el, '__iter__') instead of isinstance(el, list) (it depends if you want to flatten generic iterables or only lists). Of course, once you do that, you need to special-case strings... Strings are iterable but have no __iter__ method, which is fine in this context, since I would say 99.9% of times one wants to treat them as atomic objects, so no need to special case. Don't worry, that little oddity was fixed for you: Acc! I have a few places in my code with checks of the kind ``hasattr(x, '__iter__')`` and I guess those spots will be tricky when converting to Python 3. I guess 2to3 cannot help either :-( -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists implemented as integer-hashed Dictionaries?
On Fri, Feb 6, 2009 at 7:32 PM, er eroberer...@gmail.com wrote: Thanks Chris. Lua tables are one of my favorite linguistic traits, which was actually part of the discussion that brought up this nugget. Nevertheless, any details you care to provide about the details. I'm going to dive into the source code in more depth tomorrow, just so I can get a better understanding anyway, but I'd love to hear some details, or see any links, if you have them. Yeah, as Chris said, Python lists are not dictionaries at all. They're PyObjects that contain an array of pointers to other PyObjects. Basic Python types try to be efficient and reliable. They're not always the perfect choice, but they strive to be good enough for most situations in terms of both functionality and speed... and while they have put a lot of effort into tuning Python's hash implementation, an array has to beat it hands down when you're implementing an ordered sequence. Now, I believe Python sets *are* for all intents and purposes dictionaries, but I think that's just because its the easiest and most efficient way to implement their uniqueness properties; they took the very-well-tuned dictionary implementation and cut out the stuff not needed by sets and did some tweaks here or there. I /believe/. --S -- http://mail.python.org/mailman/listinfo/python-list
Re: Running all unit tests
On Fri, 06 Feb 2009 21:11:15 -0500, Jason Voegele ja...@jvoegele.com wrote: I'm working on my first substantial Python project, and I'm following a fully test-first approach. I'd like to know how Pythonistas typically go about running all of their tests to ensure that my application stays green. In Ruby, I would have a Rake task so that I could say rake test and all tests would be executed. In C or C++ I would have a make target so I could run all my tests with make test. In Java it would be an Ant task and ant test. And so forth and so on. What's the recommended approach for Python programs? I'm sure I could write a shell script (or a Python script even) that scans my test directory for test cases and runs them, but I'm wondering if there's something already built in that could do this for me. There are a bunch of tools for this. I use trial (part of Twisted), which will collect your tests, run them, and report the results (and has helpers for debugging, profiling, and some other stuff) and buildbot. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Flattening lists
Quoth Mensanator mensana...@aol.com: On Feb 6, 3:23=A0pm, Rhamphoryncus rha...@gmail.com wrote: On Feb 5, 1:16=A0pm, Michele Simionato michele.simion...@gmail.com wrote: On Feb 5, 7:24=A0pm, a...@pythoncraft.com (Aahz) wrote: In article a22c77c4-a812-4e42-8972-6f3eedf72...@l33g2000pri.googlegroups.com, Michele Simionato =A0michele.simion...@gmail.com wrote: Looks fine to me. In some situations you may also use hasattr(el, '__iter__') instead of isinstance(el, list) (it depends if you want to flatten generic iterables or only lists). Of course, once you do that, you need to special-case strings... Strings are iterable but have no __iter__ method, which is fine in this context, since I would say 99.9% of times one wants to treat them as atomic objects, so no need to special case. Don't worry, that little oddity was fixed for you: Python 3.0+ (unknown, Dec =A08 2008, 14:26:15) [GCC 4.3.2] on linux2 Type help, copyright, credits or license for more information. str.__iter__ slot wrapper '__iter__' of 'str' objects bytes.__iter__ slot wrapper '__iter__' of 'bytes' objects bytearray.__iter__ slot wrapper '__iter__' of 'bytearray' objects I'm in the why do you need more than 1 depth? camp. Dispatching based on your own type should be given an extra look. Dispatching based passed in types should be given three extra looks. I didn't realize itertools.chain(*iterable) worked. I guess that needs to be pushed as the canonical form. What about this (from the Recipes section of the itertools manual)? def flatten(listOfLists): return list(chain.from_iterable(listOfLists)) Python 2.6.1 (r261:67515, Jan 7 2009, 17:09:13) [GCC 4.3.2] on linux2 Type help, copyright, credits or license for more information. from itertools import chain list(chain.from_iterable([1, 2, [3, 4]])) Traceback (most recent call last): File stdin, line 1, in module TypeError: 'int' object is not iterable list(chain(*[1, 2, [3, 4]])) Traceback (most recent call last): File stdin, line 1, in module TypeError: 'int' object is not iterable list(chain.from_iterable(['abcd', 'efg', [3, 4]])) ['a', 'b', 'c', 'd', 'e', 'f', 'g', 3, 4] --RDM -- http://mail.python.org/mailman/listinfo/python-list
Re: simple web app, where to start
have a loo at the django framework http://www.djangoproject.com/ On Fri, Feb 6, 2009 at 8:16 PM, Vincent Davis vinc...@vincentdavis.net wrote: I have a simple script that takes a few input values and returns a csv file and a few stats. If I wanted to host this on the web how would I. I have no idea where to begin. If someone could point me in the right direction like maybe a tutorial, tools I will need, functions. I would appreciate it. I know a little html but am not sure how to integrate python or know what servers will handle it . Thanks Vincent Davis -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Python3.0 has more duplication in source code than Python2.5
I used a CPD (copy/paste detector) in PMD to analyze the code duplication in Python source code. I found that Python3.0 contains more duplicated code than the previous versions. The CPD tool is far from perfect, but I still feel the analysis makes some sense. |Source Code | NLOC | Dup60 | Dup30 | Rate60| Rate 30 | Python1.5(Core) 19418 10723023 6% 16% Python2.5(Core) 35797 16566441 5% 18% Python3.0(Core) 40737 34609076 8% 22% Apache(server) 18693 11142553 6% 14% NLOC: The net lines of code Dup60: Lines of code that has 60 continuous tokens duplicated to other code (counted twice or more) Dup30: 30 tokens duplicated Rate60: Dup60/NLOC Rate30: Dup30/NLOC We can see that the common duplicated rate is tended to be stable. But Python3.0 is slightly bigger than that. Consider the small increase in NLOC, the duplication rate of Python3.0 might be too big. Does that say something about the code quality of Python3.0? -- http://mail.python.org/mailman/listinfo/python-list
Re: Lists implemented as integer-hashed Dictionaries?
er wrote: Somebody much more intelligent than I said today that someone told him that Python lists are just dictionaries with lists hashed by integers. Abstractly, which is to say, behaviorally, a Python list is a sequence class as defined under Built-in Types in the Library manual. Dictionaries are a mapping class. The two categories have different methods. So at this level, the claim is nonsensical, wrong by definition. A list *must* have it n entries indexed from 0 to n-1 and dicts do not and could not enforce such an invariant. Concretely, an implementation could do as claimed under the covers, but CPython and I suspect all the other implementations actually use extensible arrays. People *do* use dicts for sparse arrays, but then they are *not* sequences. Good for you for asking here. Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Is c.l.py becoming less friendly?
Albert Hopkins wrote: Probably that [c.l.]python is becoming more popular and, like most things as they become popular, it loses its purity... much like the Internet in the early 1990s. Several years ago when I proposed the addition of list.pop(), a couple of people accused me of trying to ruin Python (by spoiling its 'purity', I guess). There were some other unfriendly things said a few years later, by and to various people, in the discussion of integer division. So I think python-list has become more friendly since. Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3.0 has more duplication in source code than Python2.5
Does that say something about the code quality of Python3.0? Not necessarily. IIUC, copying a single file with 2000 lines completely could already account for that increase. It would be interesting to see what specific files have gained large numbers of additional files, compared to 2.5. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list
[issue3031] distutils package_dir/package_data failure
Tarek Ziadé ziade.ta...@gmail.com added the comment: works, and even tested in test_build_py.test_empty_package_dir -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3031 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3987] removed types from distutils.core [patch]
Tarek Ziadé ziade.ta...@gmail.com added the comment: done in r69356 -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3987 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3386] [PATCH] distutils.sysconfig.get_python_lib prefix argument broken
Tarek Ziadé ziade.ta...@gmail.com added the comment: The patch looks fine to me. I'll send a mail to Andrew to ask him for a demonstration, and wait a week to commit it. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3386 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3986] removed string and type usage from distutils.cmd [patch]
Tarek Ziadé ziade.ta...@gmail.com added the comment: worked started in r3986 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3986 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5164] backport distutils 3.x changes into 2.7 when appliabl
New submission from Tarek Ziadé ziade.ta...@gmail.com: It's annoying to get conflicts on changes in distutils in the trunk, when forward-porting to 3.x, because other changes where made there and only there. Things like PEP8-ification and modern syntax changes needs to be backported to 2.7 when appliable to reduce the gap. -- assignee: tarek components: Distutils messages: 81253 nosy: tarek priority: high severity: normal status: open title: backport distutils 3.x changes into 2.7 when appliabl type: behavior versions: Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5164 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2445] Use The CygwinCCompiler Under Cygwin
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.7, Python 3.1 -Python 2.6, Python 3.0 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2445 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2279] distutils sdist add_defaults does not add data_files
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.7, Python 3.1 -Python 2.4, Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2279 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2624] swig support in distutils should use the build and temp dirs
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.7, Python 3.1 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2624 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2943] Distutils should generate a better error message when the SDK is not installed
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek type: - behavior versions: +Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2943 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2945] bdist_rpm does not list dist files (should effect upload)
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.7, Python 3.1 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2945 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3621] it would be nice if installers made by bdist_wininst stored an EstimatedSize property in the Windows registry
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.7, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3621 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3902] distutils does not correctly create packages for compiled extensions
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.6, Python 2.7, Python 3.0, Python 3.1 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue3902 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4032] distutils cannot recognize .dll.a as library on cygwin
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4032 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4359] at runtime, distutils uses buildtime files
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: - tarek nosy: +tarek versions: +Python 2.6, Python 2.7, Python 3.0, Python 3.1 -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4359 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5134] Compiler warnings in sqlite module
Ulrich Eckhardt eckha...@satorlaser.com added the comment: Technically, both changes (or neither of them) generate the same output binaries, so I Don't Care(tm). My approach for disabling the warnings in the code has (to me) two advantages: 1. You immediately see that warnings are disabled. I would otherwise never expect that to happen in any serious project, because warnings are valuable. 2. You only have to do it once for the single source file, not in every VC project file. Happy weekend! ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5134 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5160] Intermittant segmentation fault with ctrl-c (threads and queues)
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment:
Here is stack trace.
PyEval_EvalFrameEx(_frame * 0x00a62060, int 83) line 2841 + 6 bytes
fast_function(_object * 0x, _object * * * 0x00fbfa98, int 1, int
1, int 9870576) line 3946
call_function(_object * * * 0x00fbfa98, int 0) line 3880 + 16 bytes
PyEval_EvalFrameEx(_frame * 0x00a60ac0, int 131) line 2515
fast_function(_object * 0x, _object * * * 0x00fbfbe0, int 1, int
1, int 9870576) line 3946
call_function(_object * * * 0x00fbfbe0, int 0) line 3880 + 16 bytes
PyEval_EvalFrameEx(_frame * 0x00a60918, int 131) line 2515
PyEval_EvalCodeEx(PyCodeObject * 0x00a142c8, _object * 0x00a60918,
_object * 0x0001, _object * * 0x00a194ac, int 1, _object * *
0x, int 0, _object * * 0x, int 0, _object * 0x)
line 3104 + 11 bytes
function_call(_object * 0x00a35350, _object * 0x00a19498, _object *
0x) line 529 + 64 bytes
PyObject_Call(_object * 0x00a35350, _object * 0x00a19498, _object *
0x) line 2506 + 15 bytes
instancemethod_call(_object * 0x00a35350, _object * 0x00a19498, _object
* 0x) line 2579 + 17 bytes
PyObject_Call(_object * 0x009fdef8, _object * 0x008c1038, _object *
0x) line 2506 + 15 bytes
PyEval_CallObjectWithKeywords(_object * 0x009fdef8, _object *
0x008c1038, _object * 0x) line 3729
t_bootstrap(void * 0x008c7608) line 426 + 26 bytes
bootstrap(void * 0x00234d08) line 122 + 7 bytes
_threadstartex(void * 0x009aeaf0) line 227 + 13 bytes
KERNEL32! 77e5b3bc()
//
if (tstate-frame-f_exc_type != NULL) /* crash: tstate-frame is NULL
*/
reset_exc_info(tstate);
else {
assert(tstate-frame-f_exc_value == NULL);
assert(tstate-frame-f_exc_traceback == NULL);
}
--
nosy: +ocean-city
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5160
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5157] os.stat('foo') succeds if 'foo.exe' exists on cygwin
Antoine Calando acala...@free.fr added the comment: Hi Martin, Actually, I just investigated the problem in the libs and did not check the python exe source code. I guess you are right, this looks more like an issue from cygwin. I was a bit irritated by hours of debugging when entering the bug, and also tired of thinking it seems :) Regards, Antoine - Martin v. Löwis rep...@bugs.python.org a écrit : Martin v. Löwis mar...@v.loewis.de added the comment: Why do you think this is a bug in Python? It sounds like a bug in Cygwin to me? Python delegates to the C library as-is, with not attempt to second-guessing the C library. So if the C library says file exists, then this is also what Python must tell you. Closing as third-party bug. -- nosy: +loewis resolution: - invalid status: open - closed versions: +3rd party -Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5157 ___ ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5157 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5165] os.rename and other raise WindowsError
New submission from Kristján Valur Jónsson krist...@ccpgames.com: in lib/test/test_os.py, there is a test class, Win32ErrorTests, that tests that certain functions return a WindowsError on failure. And indeed they do that, but that is in contradiction with the documentation. From the 2.6 docs: Note All functions in this module raise OSError in the case of invalid or inaccessible file names and paths, or other arguments that have the correct type, but are not accepted by the operating system. -- components: Interpreter Core messages: 81256 nosy: krisvale severity: normal status: open title: os.rename and other raise WindowsError type: behavior versions: Python 2.5, Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5165 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1835] Update version number in __init__.py
Marc-Andre Lemburg m...@egenix.com added the comment: On 2009-02-05 14:23, Thomas Heller wrote: Thomas Heller thel...@ctypes.org added the comment: The distutils version number is now updated automatically by the Python release process, so the comment in that file can be removed. How does this mechanism work? I'm wondering if I should use a similar mechanism for the ctypes version number... This is done by Barry's release.py script: http://svn.python.org/view/sandbox/trunk/release/release.py in bump(). ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1835 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5167] distutils/test_customize_compiler fails on windows
New submission from Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp: test_customize_compiler fails on windows with following error message. ERROR: test_customize_compiler (distutils.tests.test_sysconfig.SysconfigTestCase ) -- Traceback (most recent call last): File e:\python-dev\trunk\lib\distutils\tests\test_sysconfig.py, line 54, in test_customize_compiler sysconfig.customize_compiler(comp) File e:\python-dev\trunk\lib\distutils\sysconfig.py, line 181, in customize_ compiler cpp = cc + -E # not always TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' I hope attached patch will fix this. -- components: Distutils, Tests files: fix_distutils_test.patch keywords: patch messages: 81261 nosy: ocean-city, tarek severity: normal stage: patch review status: open title: distutils/test_customize_compiler fails on windows versions: Python 2.7 Added file: http://bugs.python.org/file12951/fix_distutils_test.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5167 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4804] Python on Windows disables all C runtime library assertions
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment: Sorry, I don't have VS2005. By the way, _PyVerify_fd seems to be required to VC6 too. Because fdopen(fd = _NHANDLE_) crashes on debug build and fdopen(bad fd _NHANDLE_) won't set errno to EBADF. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5168] shutil.copystat does not copy the hidden flag on windows
New submission from Horváth István Róbert thr...@gmail.com: Hi! While copying hidden files on windows, the hidden flag is lost (surprisingly the read-only flag is copied). Is this a bug or a feature? The documentation only says that flags are also copied by shutil.copystat, but no details. See the attached file for an example. Cheers, Robert -- components: Windows files: copyHiddenFile.zip messages: 81263 nosy: throbi severity: normal status: open title: shutil.copystat does not copy the hidden flag on windows type: behavior versions: Python 2.5 Added file: http://bugs.python.org/file12952/copyHiddenFile.zip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5168 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5165] os.rename and other raise WindowsError
Matthew Barnett pyt...@mrabarnett.plus.com added the comment: WindowsError is a subclass of OSError, so it's not entirely contradictory, just a little misleading... :-) -- nosy: +mrabarnett ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5165 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5165] os.rename and other raise WindowsError
Kristján Valur Jónsson krist...@ccpgames.com added the comment: ah, well, silly me. then I'll just close this as Invalid. -- resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5165 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue2527] Pass a namespace to timeit
Georg Brandl ge...@python.org added the comment: I'm sorry, this should have been another issue. Reassigning to you. -- assignee: pitrou - rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue2527 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5066] IDLE documentation for Unix obsolete/incorrect
Georg Brandl ge...@python.org added the comment: I don't know any. But since rst is so lightweight, it is usually not too much of a pain to just copy the text from the browser and reintroduce formatting because you're likely to have to go over and edit the whole content anyway. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5066 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5158] Document distutils 'depends' option for extensions
Changes by Tarek Ziadé ziade.ta...@gmail.com: -- assignee: georg.brandl - tarek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5158 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5169] Default hash not equal to id on AMD Sempron
New submission from Chema Cortés dev.xt...@gmail.com: Sometimes, the default hash for user-defined object is not equal to the id of the object: In [1]: class A: ...: pass In [2]: a=A() In [3]: id(a),hash(a) Out[3]: (3082955212L, -1212012084) The test box has an AMD Sempron, a 64bit CPU archictecture emulating a 32bit one. This following relation can be deduced: hash(a)=id(a)-2**32 -- components: Interpreter Core messages: 81269 nosy: chemacortes, jcea severity: normal status: open title: Default hash not equal to id on AMD Sempron type: behavior versions: Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5169 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5169] Default hash not equal to id on AMD Sempron
Antoine Pitrou pit...@free.fr added the comment: I wouldn't qualify this as a bug. hash() doesn't need to be equal to the id() even in the default case. Actually, it may be better for hash() to be equal to id()/4 or id()/8, depending on the standard alignment of the memory allocator. -- nosy: +pitrou, rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5169 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4804] Python on Windows disables all C runtime library assertions
Kristján Valur Jónsson krist...@ccpgames.com added the comment: I see. I had thought your code was for VS2005 (VC8) I have changed the patch to work with VS2005 as well. I don't think we need to worry about VS2003 so much, as I don't think it is an officially supported compiler for the current python versions. Also, the problem with close() et al only started when we ported the code to Visual Studio 2005, and this is the version of VisualStudio that starts to make assertions about its file descriptors. Anyway, I upload an updated version of the patch, which copes with VS2005 as well as VS2008 Added file: http://bugs.python.org/file12953/__pioinfo.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5122] test_tcl and test_ttk_guionly don't like each other
Guilherme Polo ggp...@gmail.com added the comment: Eric, I noticed you said Fedora 6, so I checked which tcl/tk it includes and apparently it is 8.4.13, is that correct ? I tried using tcl/tk 8.4.13 and I managed to hang it too :) Try running test_tcl followed by test_ttk_guionly with regrtest. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5122 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5166] ElementTree and minidom don't prevent creation of not well-formed XML
New submission from Denis S. Otkidach denis.otkid...@gmail.com:
ElementTree and minidom allow creation of not well-formed XML, that
can't be parsed:
from xml.etree import ElementTree
element = ElementTree.Element('element')
element.text = u'\0'
xml = ElementTree.tostring(element, encoding='utf-8')
ElementTree.fromstring(xml)
[...]
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1,
column 9
from xml.dom import minidom
doc = minidom.getDOMImplementation().createDocument(None, None, None)
element = doc.createElement('element')
element.appendChild(doc.createTextNode(u'\0'))
DOM Text node
doc.appendChild(element)
DOM Element: element at 0xb7ca688c
xml = doc.toxml(encoding='utf-8')
minidom.parseString(xml)
[...]
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, colum
I believe they should raise some exception when there are characters
not allowed in XML (http://www.w3.org/TR/REC-xml/#NT-Char) are used in
attribute values, text nodes and CDATA sections.
--
components: Library (Lib)
messages: 81259
nosy: ods
severity: normal
status: open
title: ElementTree and minidom don't prevent creation of not well-formed XML
type: behavior
versions: Python 2.5, Python 2.6, Python 3.0
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue5166
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5122] test_tcl and test_ttk_guionly don't like each other
Eric Smith e...@trueblade.com added the comment: Yes, I have these installed: tcl-8.4.13-3.fc6 tcl-devel-8.4.13-3.fc6 tk-8.4.13-3.fc6 tk-devel-8.4.13-3.fc6 When I run ./python Lib/test/regrtest.py test_tcl test_ttk_guionly, it hangs. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5122 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
New submission from shamilbi shami...@gmail.com: if i configure logging into a file with encoding = 'cp1251' and do logger.debug(u'...') then i get crash with UnicodeError i suggest reimplementing method FileHandler.emit(): ... if isinstance(msg, unicode): stream.write(f % msg)# it works! ... -- components: Library (Lib) messages: 81274 nosy: shamilbi severity: normal status: open title: logging to file + encoding type: crash versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4804] Python on Windows disables all C runtime library assertions
Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp added the comment: I agree. Please focus on _MSC_VER = 1400. I'll post new issue about VC6 after this issue will be solved. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5170] logging to file + encoding
Changes by Benjamin Peterson benja...@python.org: -- assignee: - vsajip nosy: +vsajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5170 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5122] test_tcl and test_ttk_guionly don't like each other
Guilherme Polo ggp...@gmail.com added the comment: When I run ./python Lib/test/regrtest.py test_tcl test_ttk_guionly, it hangs. I have isolated it now: Traceback (most recent call last): File stdin, line 1, in module File /home/gpolo/python-dev/python-trunk/Lib/lib-tk/Tkinter.py, line 1649, in loadtk self.tk.loadtk() _tkinter.TclError: no display name and no $DISPLAY environment variable Tkinter.Tk() hang! Very nice, much more solvable now after getting to know the tcl/tk version. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5122 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5169] Default hash not equal to id on AMD Sempron
Mark Dickinson dicki...@gmail.com added the comment: It looks like this is a platform with sizeof(long) == 4 and sizeof(void *) == 8. Is that right? As Antoine says, I can't see any problem here. Why do you think that hash(a) should be equal to id(a) in this case? Antoine, in what way would id()/4 be better than id()? -- nosy: +marketdickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5169 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4010] configure options don't trickle down to distutils
Changes by Akira Kitada akit...@gmail.com: Removed file: http://bugs.python.org/file12868/issue4010.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4010 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4010] configure options don't trickle down to distutils
Akira Kitada akit...@gmail.com added the comment: s/get_config_vars/get_config_var/ -- versions: +Python 3.1 Added file: http://bugs.python.org/file12954/issue4010.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4010 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4804] Python on Windows disables all C runtime library assertions
Kristján Valur Jónsson krist...@ccpgames.com added the comment: I've taken the patch from Hirokazu and enhanced it: 1) it needed work to function with Visual Studio 2008 2) It now exposes a function so that _fileio.c can make use of it too. 3) Turned off the CRT manipulation in exceptions.c 4) Fixed minor problems, e.g. with dup2 5) Added comments A VS2008 compilation runs the testsuite just fine. Maybe Hirozaku can test this with his VS2005? Added file: http://bugs.python.org/file12950/__pioinfo.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4804 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5167] distutils/test_customize_compiler fails on windows
Tarek Ziadé ziade.ta...@gmail.com added the comment: fixed in r69366 thanks for the patch -- assignee: - tarek status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5167 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5122] test_tcl and test_ttk_guionly don't like each other
Guilherme Polo ggp...@gmail.com added the comment: After I have isolated it now:, there should be this interactive session (but gmail ate it apparently): import os del os.environ['DISPLAY'] import Tkinter t = Tkinter.Tcl() t.loadtk() Traceback (most recent call last): File stdin, line 1, in module File /home/gpolo/python-dev/python-trunk/Lib/lib-tk/Tkinter.py, line 1649, in loadtk self.tk.loadtk() _tkinter.TclError: no display name and no $DISPLAY environment variable Tkinter.Tk() hang! ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5122 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5122] test_tcl and test_ttk_guionly don't like each other
Guilherme Polo ggp...@gmail.com added the comment: Ok, I've found the cause of the problem and the patch attached should solve it but shouldn't be applied! This was just a quick fix I did to confirm what I expected, I will add something more correct later. _tkinter is aware of this deadlock that we are getting (see Tkapp_TkInit), but it is not collaborating to solve it. The problem is avoided if we always use loadtk in Tkinter.py instead of using create to load tk. -- keywords: +patch Added file: http://bugs.python.org/file12955/dont_load_tk_in_tkintercreate.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5122 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5171] itertools.product docstring missing 'repeat' argument
New submission from Mark Dickinson dicki...@gmail.com: The docstring for itertools.product seems to be missing any mention of the repeat keyword argument, in both the trunk and py3k, and the maintenance branches. (The itertools.rst docs are fine, though.) -- assignee: rhettinger components: Library (Lib) messages: 81281 nosy: marketdickinson, rhettinger severity: normal status: open title: itertools.product docstring missing 'repeat' argument versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5171 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5169] Default hash not equal to id on AMD Sempron
Antoine Pitrou pit...@free.fr added the comment: Because with hash() == id() == address of the PyObject, the hash is always a multiple of 4 or 8 (I think it's 8), so (hash() % dict_or_set_table_size) is non-uniformly distributed in most cases. ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5169 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
