Re: [Python-Dev] LZW support in tarfile ?
Michael Foord fuzzy...@voidspace.org.uk wrote: Antoine Pitrou wrote: Tarek Ziadé ziade.tarek at gmail.com writes: But I was wondering if we should we add a LZW support in tarinfo, besides gzip and bzip2 ? Although this compression standard doesn't seem very used these days, It would be more useful to add LZMA / xz support. I don't think compress is used anymore, except perhaps on old legacy systems. On my Linux system, I have lots of .gz, .bz2 and .lzma files, but absolutely no .Z file. I've seen the occasional .Z file in recent years, but never that I recall for a Python package. On my unix filesystem (which has files stretching back over 20 years) I find only two .Z files, one dated 1989 and one 2002. I think you can safely say that compress is gone! The worst you are doing by removing compress support is getting the user of some ancient platform to download one of the binaries here first. http://www.gzip.org/#exe As plugging in external compression tools is less likely to work cross-platform wouldn't it be both easier and better to deprecate (and not replace) the compress support. Agreed. -- Nick Craig-Wood n...@craig-wood.com -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] subprocess crossplatformness and async communication
On Sat, Jan 24, 2009 at 07:58:40AM -0800, Guido van Rossum wrote: I'm confused. The subprocess already allows reading/writing its stdin/stdout/stderr, and AFAIK it's a platform-neutral API. I'm sure there's something missing, but your post doesn't make it clear what exactly, and the recipe you reference is too large to digest easily. Can you explain what it is that the current subprocess does't have beyond saying async communication (which could mean many things to many people)? The main problem with subprocess is that it doesn't work if you want to have a conversation / interact with your child process. subprocess works very well indeed for this case :- run child send stuff to stdin child reads stdin and writes stdout child exits read stuff from stdout But for the conversational case (eg using it to do a remote login) it doesn't work at all :- run child send stuff to stdin child reads stdin and writes stdout read stuff from stdout send stuff to stdin child reads stdin and writes stdout read stuff from stdout send stuff to stdin child reads stdin and writes stdout read stuff from stdout child exits In subprocess read stuff from stdout means read stdout until the other end closes it, not read what is available and return it, so it blocks on reading the first reply and never returns. Hence Anatoly's request for async communication and the existence of that recipe. http://code.activestate.com/recipes/440554/ I've spend quite a lot of time explaning this to people on comp.lang.python. I usually suggest either the recipe as suggested by Anatoly or if on unix the pexpect module. There are other solutions I know of, eg in twisted and wxPython. I heard rumours of a pexpect port to Windows but I don't know how far that has progressed. A cross platform async subprocess would indeed be a boon! -- Nick Craig-Wood n...@craig-wood.com -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] py3k: TypeError: object.__init__() takes no parameters
I've noticed with latest python 3.1 checkout (68631) if I have this
object hierarchy with a default __init__ in the superclass to be used
by the subclasses which don't necessarily need an __init__ it blows up
with a TypeError.
class Field(object):
def __init__(self, data):
Default init for the subclasses
print(init class=%r, self=%r % (self.__class__.__name__, self))
super(Field, self).__init__(data)
self.data = self.orig = data
class IntegerField(Field):
def __init__(self, data):
Overridden init
super(IntegerField, self).__init__(data)
self.data = int(data)
class StringField(Field):
pass
f1 = StringField('abc')
f2 = IntegerField('10')
print(f1=%r % f1.data)
print(f2=%r % f2.data)
print(type(f1))
print(type(f2))
It blows up with
init class='StringField', self=__main__.StringField object at 0xb7d47b4c
Traceback (most recent call last):
File subclass-super-problem-py3k.py, line 17, in module
f1 = StringField('abc')
File subclass-super-problem-py3k.py, line 5, in __init__
super(Field, self).__init__(data)
TypeError: object.__init__() takes no parameters
The exact same code runs under py 2.5 just fine.
I can't think of anything to write in Field.__init__ to tell whether
super is about to run __init__ on object.
The problem can be fixed (inelegantly IMHO) like this
class BaseField(object):
def __init__(self, data):
Default init for the subclasses
self.data = self.orig = data
class Field(BaseField):
def __init__(self, data):
Another Default init for the subclasses
super(Field, self).__init__(data)
class IntegerField(Field):
def __init__(self, data):
Overridden init
super(IntegerField, self).__init__(data)
self.data = int(data)
class StringField(Field):
pass
f1 = StringField('abc')
f2 = IntegerField('10')
print(f1=%r % f1.data)
print(f2=%r % f2.data)
print(type(f1))
print(type(f2))
Is this a bug or a feature? Is there a better work-around?
--
Nick Craig-Wood n...@craig-wood.com -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py3k: TypeError: object.__init__() takes no parameters
Terry Reedy tjre...@udel.edu wrote: Nick Craig-Wood wrote: I've noticed with latest python 3.1 checkout (68631) if I have this object hierarchy with a default __init__ in the superclass to be used by the subclasses which don't necessarily need an __init__ it blows up with a TypeError. class Field(object): object is default baseclass, hence not needed Agreed, but I wanted the code to run with py 3 also! def __init__(self, data): Default init for the subclasses print(init class=%r, self=%r % (self.__class__.__name__, self)) super(Field, self).__init__(data) This line is the problem: remove it and I believe all is fine. Since object()s are immutable, its init cannot do anything as far as I know. Deleting this is effectively what you did below. Yes you are absolutely right - that super is never needed. I don't know what I was thinking of! Without that the problem disappears. [snip] Perhaps 2.5's object.__init__ just swallowed all args, thus hiding bogus calls. Yes it did which is the fundamental difference in behaviour between py2 and py3 as far as I can see. [snip] Eliminate bad call. Check! (Bashes head against wall!) Thanks Nick -- Nick Craig-Wood n...@craig-wood.com -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Optionally using GMP to implement long if available
Tim Peters [EMAIL PROTECTED] wrote:
And for 32x32 - 64, can't this simply be replaced by (uint64_t)i * j,
where uint64_t is as in C99? I'd hope that most compilers would
manage to turn this into the appropriate 32x32-bit hardware multiply.
1. That's C99, not C89, and therefore less portable.
2. On platforms that support it, this is at least 64x64-64 multiplication,
potentially much more expensive than the 32x32-64 (or 31x31-62?)
flavor you /intend/ to move to.
3. There's no way to know exactly what compilers will do with this short of
staring at generated code.
I've relied in the past for the compiler generating a 32*32-64 bit
multiply for this code
#include stdint.h
uint64_t mul(uint32_t a, uint32_t b)
{
return a*b;
}
Looking at the assembler it produces (x86)
mul:
pushl %ebp
xorl%edx, %edx
movl%esp, %ebp
movl12(%ebp), %eax
imull 8(%ebp), %eax
popl%ebp
ret
Which I'm pretty sure is a 32x32-64 bit mul (though my x86 assembler
foo is weak).
I think a compiler would have to be pretty stupid not to take this
optimisation... But then there are some pretty stupid compilers out
there!
--
Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Lazy module imports and post import hook
Christian Heimes [EMAIL PROTECTED] wrote: I've attached the first public draft of my first PEP. Some brief thoughts from me on the PEP... Post import hooks sound great and a good idea. Lazy importing sounds like a recipe for hard to find bugs and rather too magic for my taste. Perhaps you should separate these two things into two PEPs and implementations? -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] csv changed from python 2.4 to 2.5
Christian K [EMAIL PROTECTED] wrote:
I could not find documentation of the following change in python2.5. What is
the
reason for that?
Python 2.4.4 (#2, Apr 12 2007, 21:03:11)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type help, copyright, credits or license for more information.
import csv
d=csv.get_dialect('excel')
d.delimiter = ','
[EMAIL PROTECTED]:/media/hda6/home/ck/prog/peak-o-mat/trunk$ python2.5
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type help, copyright, credits or license for more information.
import csv
d=csv.get_dialect('excel')
d.delimiter = ','
Traceback (most recent call last):
File stdin, line 1, in module
TypeError: readonly attribute
Looks like this is the reason - the get_dialect call (which is
implemented in C) is now returning a read only Dialect object rather
than an instance of the original class :-
2.5
import csv
d = csv.get_dialect('excel')
d.__class__
type '_csv.Dialect'
2.4
import csv
d = csv.get_dialect('excel')
d.__class__
class csv.excel at 0xb7d1f74c
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type help, copyright, credits or license for more information.
import csv
d = csv.excel
d.delimiter = ','
Don't you want to do this anyway?
import csv
class my_dialect(csv.excel):
delimeter = ,
--
Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docs, reloaded
On Thu, May 24, 2007 at 12:43:18PM -0500, Ron Adam wrote: And for console text output, is the unmodified reST suitable, or would it be desired to modify it in some way? Currently pydoc output looks like a man page under Unix. if it could look like that then that would be great. Otherwise raw reST is fine! Should a subset of the main documents be included with pydoc to avoid the documents not available messages if they are not installed? Or should the topics retrieval code be moved from pydoc to the main document tools so it's installed with the documents. Then that can be maintianed with the documents instead of being maintained with pydoc. Then pydoc will just looks for it, instead of looking for the html pages. I think the latter proposal sounds like the correct one. In debian for instance, the python docs are a seperate package, and it would seem reasonable that you'd have to have that package installed to get the long docs. I think that if reST was an acceptable form for the documentation, and it could be auto included in the main docs from docstrings then you would find more modules completely documented in __doc__. That would be fine for third party modules if they want to do that or if there is not much difference between the two. If you look at the documentation for subprocess for instance, you'll see that the docstring is pretty much the same as the library reference documentation which seems like needless duplication and opportunity for code/doc skew. Maybe one is auto generated from the other - I don't know! Actually if it gave both sets of docs quick, then long, one after the other that would suit me fine. That may work well for the full documentation, but the quick reference wouldn't be a short quick reference any more. Well you could stop after reading the short bit! I'm attempting to have a pydoc api call that gets a single item or sub-item and format it to a desired output so it can be included in other content. That's makes it possible for the full docs (not necessarily pythons) to embed pydoc output in it if it's desirable. This will need pydoc formatters for the target document type. I hope to include a reST output formatter for pydoc. The help() function is imported from pydoc by site.py when you start python. It may not be difficult to have it as a function that first tries pydoc to get a request, and if the original request is returned unchanged, tries to get information from the full documentation. There could be a way to select one or the other, (or both). But this feature doesn't need to be built into pydoc, or the full documentation. They just need to be able to work together so things like this are possible in an easy to write 4 or 5 line function. (give or take a few lines) So it looks like most of these issues are more a matter of how to organize the interfaces. It turns out that what I've done with pydoc, and what Georg is doing with the main documentation should work together quite nicely. Sounds good! Nick -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docs, reloaded
On Wed, May 23, 2007 at 05:39:38AM -0500, [EMAIL PROTECTED] wrote: Nick If you type pydoc re at the moment then it says in it Nick MODULE DOCS Nick http://www.python.org/doc/current/lib/module-re.html Nick which is pretty much useless to me when ssh-ed in to a linux box Nick half way around the world... I get quite a bit of information about re (I've never known /F to be a documentation slouch). Yes it is certainly better than no docs. It doesn't for instance have any regexp info, and I can never remember all the special non matching brackets (eg (?:...) so I have to read for the full docs for that. Only one bit of that information is a reference to the page in the library reference manual. And if I happen to be ssh'd into a machine halfway round the world through a Gnome terminal I can right mouse over that URL and pop the page up in my default local browser. If you set the PYTHONDOCS environment variable you can point it to a local (or at least different) copy of the libref manual. I take your point. However the unix tradition is that everything is in the man pages. man pages have expanded over the years to include info pages and you *can* read the full python docs via info, it just isn't quite as convenient as pydoc. I think perl had the right idea with perldoc. You can read all the perl documentation whether it is in module documentation (like docstrings) or general documentation (like the latex docs under discussion). I'd like to see pydoc be a viewer for all the python documentation, not just a subset of it. A flag could be added to pydoc to show that content instead, however being html it probably would be difficult to read unless pumped through lynx -dump or something similar. I'm assuming that we do reST all the python documentation which would make it easier. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docs, reloaded
On Wed, May 23, 2007 at 12:46:50PM -0500, Ron Adam wrote:
Nick Craig-Wood wrote:
So I'll be able to read the main docs for a module in a terminal
without reaching for the web browser (or info)? That would be great!
How would pydoc decide which bit of docs it is going to show?
Pydoc currently gets topic info for some items by scraping the text from
document 'local' web pages. This is kind of messy for a couple of reasons.
- The documents may not be installed locally.
- It can be problematic locating the docs even if they are installed.
- They are not formatted well after they are retrieved.
I think this is an area for improvement.
And it would be improved by converting the docs to reST I imagine.
This feature is also limited to a small list where the word being searched
for is a keyword, or a very common topic reference, *and* they are not
likely to clash with other module, class, or function names.
The introspection help parts of pydoc are completely separate from topic
help parts. So replacing this part can be done without much trouble. What
the best behavior is and how it should work would need to be discussed.
Keep in mind doc strings are meant to be more of a quick reference to an
item, and Pydoc is the interface for that.
I think that if reST was an acceptable form for the documentation, and
it could be auto included in the main docs from docstrings then you
would find more modules completely documented in __doc__.
If I type pydoc re is it going to give me the rather sparse __doc__
from the re module or the nice reST docs? Or maybe both, one after
the other? Or will I have to use a flag to dis-ambiguate?
If retrieval from the full docs is desired, then it will probably need to
be disambiguated in some way or be a separate interface.
help('re')# Quick reference on 're'.
helpdocs('re')# Full documentation for 're'.
Actually if it gave both sets of docs quick, then long, one after the
other that would suit me fine.
--
Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docs, reloaded
Georg Brandl [EMAIL PROTECTED] wrote: Nick Craig-Wood schrieb: Being a seasoned unix user, I tend to reach for pydoc as my first stab at finding some documentation rather than (after excavating the mouse from under a pile of paper) use a web browser. If you've ever used pydoc you'll know it reads docstrings and for some modules they are great and for others they are sorely lacking. If pydoc could show all this documentation as well I'd be very happy! Maybe your quick dispatch feature could be added to pydoc too? It is my intention to work together with Ron Adam to make the pydoc - documentation integration as seamless as possible. So I'll be able to read the main docs for a module in a terminal without reaching for the web browser (or info)? That would be great! How would pydoc decide which bit of docs it is going to show? If I type pydoc re is it going to give me the rather sparse __doc__ from the re module or the nice reST docs? Or maybe both, one after the other? Or will I have to use a flag to dis-ambiguate? If you type pydoc re at the moment then it says in it MODULE DOCS http://www.python.org/doc/current/lib/module-re.html which is pretty much useless to me when ssh-ed in to a linux box half way around the world... It is missing conversion of ``comment'' at the moment as I'm sure you know... Sorry, what did you mean? ``comment'' produces smart quotes in latex if I remember correctly. You probably want to convert it somehow because it looks a bit odd on the web page as it stands. I'm not sure what the reST replacement might be, but converting it just to comment would probably be OK. Likewise with `comment' to 'comment'. For an example see the first paragraph here: http://pydoc.gbrandl.de/reference/index.html -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docs, reloaded
Georg Brandl [EMAIL PROTECTED] wrote: over the last few weeks I've hacked on a new approach to Python's documentation. As Python already has an excellent documentation framework, the docutils, with a readable yet extendable markup format, reST, I thought that it should be possible to use those instead of the current LaTeX-latex2html toolchain. Good idea! Latex is a barrier for contribution to the docs. I imagine most people would be much better at contributing to the docs in reST. (Me included: I learnt latex at university a couple of decades ago and have now forgotten it completely!) - a quick-dispatch function: e.g., docs.python.org/q?os.path.split would redirect you to the matching location. Being a seasoned unix user, I tend to reach for pydoc as my first stab at finding some documentation rather than (after excavating the mouse from under a pile of paper) use a web browser. If you've ever used pydoc you'll know it reads docstrings and for some modules they are great and for others they are sorely lacking. If pydoc could show all this documentation as well I'd be very happy! Maybe your quick dispatch feature could be added to pydoc too? Concluding, a small caveat: The conversion/build tools are, of course, not complete yet. There are a number of XXX comments in the text, most of them indicate that the converter could not handle a situation -- that would have to be corrected once after conversion is done. It is missing conversion of ``comment'' at the moment as I'm sure you know... You will need to make your conversion perfect before you convince the people who wrote most of that documentation I suspect! -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On Wed, Oct 04, 2006 at 12:42:04AM -0400, Tim Peters wrote:
[EMAIL PROTECTED]
If C90 doesn't distinguish -0.0 and +0.0, how can Python?
With liberal applications of piss vinegar ;-)
Can you give a simple example where the difference between the two is
apparent
to the Python programmer?
Perhaps surprsingly, many (well, comparatively many, compared to none
) people have noticed that the platform atan2 cares a lot:
from math import atan2 as a
z = 0.0 # postive zero
m = -z # minus zero
a(z, z) # the result here is actually +0.0
0.0
a(z, m)
3.1415926535897931
a(m, z)# the result here is actually -0.0
0.0
This actually returns -0.0 under linux...
a(m, m)
-3.1415926535897931
It work like that even on Windows, and these are the results C99's
754-happy appendix mandates for atan2 applied to signed zeroes. I've
even seen a /complaint/ on c.l.py that atan2 doesn't do the same when
z = 0.0
is replaced by
z = 0
That is, at least one person thought it was a bug that integer
zeroes didn't deliver the same behaviors.
Do people actually rely on this? I know I don't, but given that more
than just 2 people have remarked on it seeming to like it, I expect
that changing this would break /some/ code out there.
Probably!
It surely isn't a big problem though is it?
instead of writing
if (result == 0.0)
returned cached_float_0;
we just write something like
if (memcmp((result, static_zero, sizeof(double)) == 0))
returned cached_float_0;
Eg the below prints (gcc/linux)
The memcmp() way
1: 0 == 0.0
2: -0 != 0.0
The == way
3: 0 == 0.0
4: -0 == 0.0
#include stdio.h
#include string.h
int main(void)
{
static double zero_value = 0.0;
double result;
printf(The memcmp() way\n);
result = 0.0;
if (memcmp(result, zero_value, sizeof(double)) == 0)
printf(1: %g == 0.0\n, result);
else
printf(1: %g != 0.0\n, result);
result = -0.0;
if (memcmp(result, zero_value, sizeof(double)) == 0)
printf(2: %g == 0.0\n, result);
else
printf(2: %g != 0.0\n, result);
printf(The == way\n);
result = 0.0;
if (result == 0.0)
printf(3: %g == 0.0\n, result);
else
printf(3: %g != 0.0\n, result);
result = -0.0;
if (result == 0.0)
printf(4: %g == 0.0\n, result);
else
printf(4: %g != 0.0\n, result);
return 0;
}
--
Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On Tue, Oct 03, 2006 at 09:47:03AM +1000, Delaney, Timothy (Tim) wrote: This doesn't actually give us a very useful indication of potential memory savings. What I think would be more useful is tracking the maximum simultaneous count of each value i.e. what the maximum refcount would have been if they were shared. It isn't just memory savings we are playing for. Even if 0.0 is allocated and de-allocated 10,000 times in a row, there would be no memory savings by caching its value. However there would be a) less allocator overhead - allocation objects is relatively expensive b) better caching of the value c) less cache thrashing I think you'll find that even in the no memory saving case a few cycles spent on comparison with 0.0 (or maybe a few other values) will speed up programs. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On Mon, Oct 02, 2006 at 07:53:34PM -0500, [EMAIL PROTECTED] wrote: Terry Kristján V. Jónsson [EMAIL PROTECTED] wrote: Anyway, Skip noted that 50% of all floats are whole numbers between -10 and 10 inclusive, Terry Please, no. He said something like this about Terry *non-floating-point applications* (evidence unspecified, that I Terry remember). But such applications, by definition, usually don't Terry have enough floats for caching (or conversion time) to matter too Terry much. Correct. The non-floating-point application I chose was the one that was most immediately available, make test. Note that I have no proof that regrtest.py isn't terribly floating point intensive. I just sort of guessed that it was. For my application caching 0.0 is by far the most important. 0.0 has ~200,000 references - the next highest reference count is only about ~200. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On Sun, Oct 01, 2006 at 02:01:51PM -0400, Jean-Paul Calderone wrote: Each line in an interactive session is compiled separately, like modules are compiled separately. With the current implementation, literals in a single compilation unit have a chance to be cached like this. Literals in different compilation units, even for the same value, don't. That makes sense - thanks for the explanation! -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On Fri, Sep 29, 2006 at 12:03:03PM -0700, Guido van Rossum wrote: I see some confusion in this thread. If a *LITERAL* 0.0 (or any other float literal) is used, you only get one object, no matter how many times it is used. For some reason that doesn't happen in the interpreter which has been confusing the issue slightly... $ python2.5 Python 2.5c1 (r25c1:51305, Aug 19 2006, 18:23:29) [GCC 4.1.2 20060814 (prerelease) (Debian 4.1.1-11)] on linux2 Type help, copyright, credits or license for more information. a=0.0 b=0.0 id(a), id(b) (134737756, 134737772) $ python2.5 -c 'a=0.0; b=0.0; print id(a),id(b)' 134737796 134737796 But if the result of a *COMPUTATION* returns 0.0, you get a new object for each such result. If you have 70 MB worth of zeros, that's clearly computation results, not literals. In my application I'm receiving all the zeros from a server over TCP as ASCII and these are being float()ed in python. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Caching float(0.0)
On Sat, Sep 30, 2006 at 03:21:50PM -0700, Bob Ippolito wrote:
On 9/30/06, Terry Reedy [EMAIL PROTECTED] wrote:
Nick Coghlan [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED]
I suspect the problem would typically stem from floating point
values that are read in from a human-readable file rather than
being the result of a 'calculation' as such:
Over a TCP socket in ASCII format for my application
For such situations, one could create a translation dict for both common
float values and for non-numeric missing value indicators. For instance,
flotran = {'*': None, '1.0':1.0, '2.0':2.0, '4.0':4.0}
The details, of course, depend on the specific case.
But of course you have to know that common float values are never
cached and that it may cause you problems. Some users may expect them
to be because common strings and integers are cached.
I have to say I was surprised to find out how many copies of 0.0 there
were in my code and I guess I was subconsciously expecting the
immutable 0.0s to be cached even though I know consciously I've never
seen anything but int and str mentioned in the docs.
--
Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Caching float(0.0)
I just discovered that in a program of mine it was wasting 7MB out of 200MB by storing multiple copies of 0.0. I found this a bit suprising since I'm used to small ints and strings being cached. I added the apparently nonsensical lines +if age == 0.0: +age = 0.0 # return a common object for the common case and got 7MB of memory back! Eg :- Python 2.5c1 (r25c1:51305, Aug 19 2006, 18:23:29) [GCC 4.1.2 20060814 (prerelease) (Debian 4.1.1-11)] on linux2 Type help, copyright, credits or license for more information. a=0.0 print id(a), id(0.0) 134738828 134738844 Is there any reason why float() shouldn't cache the value of 0.0 since it is by far and away the most common value? A full cache of floats probably doesn't make much sense though since there are so many 'more' of them than integers and defining small isn't obvious. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Time-out in URL Open
Alex Martelli [EMAIL PROTECTED] wrote: What about doing it with a per-thread-timeout in TLS (overriding the global one if a thread does have it set in its TLS)? Not as clean, but perhaps far easier to implement than patching dozens of modules/functions/classes to provide timeout= options everywhere... Yes please! I wrote a sketch of a module which did this on c.l.py recently http://groups.google.com/group/comp.lang.python/browse_thread/thread/d897c00b67cadca5/fd2ceb4e014de7ce?lnk=stq=TimeoutErrorrnum=2hl=en#fd2ceb4e014de7ce It would be much better if it had help from the core though. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Arena-freeing obmalloc ready for testing
On Thu, Mar 02, 2006 at 01:43:00AM -0600, Tim Peters wrote: I'm optimistic, because the new test compares a quantity already being tested by the macro, a second time against 0, and it's hard to get cheaper than that. However, the new branch isn't predictable, so who knows? When compiling with gcc at least you could give the compiler a hint, eg http://kerneltrap.org/node/4705 For example, in a release build on WinXP, VM size is about 48MB at the full prompt, and drops to 3MB at the empty prompt. In the trunk (without this patch), VM size falls relatively little from what it is at the full prompt Excellent work! -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str with base
On Mon, Jan 16, 2006 at 11:13:27PM -0500, Raymond Hettinger wrote: My reason is that I've rolled-my-own more times than I can count but infrequently enough to where it was easier to re-write than to search for the previous use. Me too! The assymetry is annoying. Its easy to consume base 2..36 integers, but its hard to generate them. However str() seems far too important to monkey with to me. I like a method on int that would be great. That keeps all the base conversions in int (either in __init__ or in as_yet_unnamed_method()). Another suggestion would be to give hex() and oct() another parameter, base, so you'd do hex(123123123, 2). Perhaps a little counter-intuitive, but if you were looking for base conversion functions you'd find hex() pretty quickly and the documentation would mention the other parameter. -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Include ctypes into core Python?
On Wed, Jan 11, 2006 at 07:59:03AM +0100, Thomas Heller wrote: Another possibility would be to emit a warning when the module (dl or ctypes, if included) is imported. warnings.warn(Incorrect usage of this module may crash Python, RuntimeWarning, stacklevel=2) Arrgggh! No!! Warnings are one of the things I really hate about perl and I'd hate to see them infecting python any more than they already have. I want my programming language to tell me, the programmer, about stuff not go blurting to the user behind my back. Death to warnings ;-) (Yes I know about the warnings module!) -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal returning NotImplemented (or not)
On Tue, Mar 01, 2005 at 05:55:50PM -0500, Neil Schemenauer wrote: On Tue, Mar 01, 2005 at 11:45:43PM +1000, Nick Coghlan wrote: Interesting. In that case, my other suggestion was to have raising NotImplementedError from a special method be the equivalent of returning NotImplemented (which makes life much easier for a class like Decimal which has an internal method for doing the type conversion). NotImplementedError has nothing to do with the NotImplemented singleton. It's unfortunate about the naming. IMO, Decimal should be returning NotImplemented instead of raising TypeError. That seems very un-pythonic to me - return an error? If you look at the patched Decimal module you'll see all sorts of contortions necessary to accomodate it, wheras if it could just raise NotImplementedError or NotImplementedTypeError it would make the code a lot cleaner. I have to say when I read the python language docs, I assumed there was a mistake in them and they meant to say raise NotImplementedError instead of return NotImplemented. Why is it like that? And could it be changed (Nick Coghlan's proposal seems good to me)? -- Nick Craig-Wood [EMAIL PROTECTED] -- http://www.craig-wood.com/nick ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
