Re: [Python-Dev] Any reason that any()/all() do not take a predicateargument?
Bill Janssen wrote: Yeah, but you can't do more complicated expressions that way, like any(lambda x: x[3] == thiskey) Not /quite/ sure what this is intended to mean, but most likely, you meant any(x[3]==thiskey for x in seq) I think it makes a lot of sense for any and all to take optional predicate function arguments. I don't believe that adds expressiveness: you can always formulate this with a generator expression - apparently, those are of the read-only nature, i.e. difficult to formulate (assuming you have no difficulties to read above term). I suppose (len([x for x in SEQ if PRED(x)]) 0) will suffice for now. Obvious enough, Martin? It's simpler written as any(PRED(x) for x in SEQ) or any(True for x in SEQ if PRED(x)) if you want Using any() has the advantage over len() that the any() code stops at the first value that becomes true, whereas the len code ill compute them all. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_Finalize does not release all memory, not even closely
Tim Peters wrote: Because new-style classes create cycles that Py_Finalize() doesn't clean up, it may make analysis easier to stick a PyGC_Collect() call (or two! repeat until it returns 0) inside the loop now. I'm shy to do this: the comment in Py_Finalize suggests that things will break if there is a late garbage collection. There is no way at present, short of editing the source for Py_Finalize and recompiling. Presumably this is something that should be addressed in the module initialization/finalization PEP, right? Indeed. This totals to 360, which is for some reason higher than the numbers I get when counting the objects on the global list of objects. How much higher? Well, I counted an increase of 156 objects on the all objects list, and an increase of 360 according to the COUNT_ALLOCS numbers. The first number was without COUNT_ALLOCS being defined, though. Anyway, thanks for your comments. I'll try to look at this from time to time, maybe I can resolve some of the leaks. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Preserving the blamelist
Tim Peters wrote: Since we're spread across time zones, I think 24 hours is a good minimum. Ok, done. If something is set to 12 hours now, doesn't look like it's working: when I wrote my msg, it showed (as I said) about 5 hours of history. Right now it shows only about 3 hrs, from Sat 15 Apr 2006 21:47:13 GMT to now (about 00:50:00 GMT Sunday). This is under Firefox on Windows, so nobody can blame it on an IE bug :-) There is another limit on the height of the waterfall: at most 200 timestamps. I have now doubled that as well. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] need info for externally maintained modules PEP
Brett Cannon wrote: Basically, from all the replies I have gotten has said that package that were/are externally maintained either considers Python HEAD as the current version or watches checkins and the bug tracker and thus the PEP is really not needed. So unless some package steps forward and says that they prefer external reporting of bugs and patches, I will consider this PEP idea dead and just modify HEAD without worrying about it. Not sure whether Fredrik Lundh has responded, but I believe he once said that he would prefer if ElementTree isn't directly modified, but that instead patches are filed on the SF tracker and assigned to him. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_Finalize does not release all memory, not even closely
Tim Peters wrote: Putting a collection call inside an initialize/finalize loop isn't doing it late, it's doing it early. If we can't collect cyclic trash after Py_Initialize(), that would be a showstopper for apps embedding Python in a loop! There's either nothing to fear here, or Python has a very bad bug. Right. I did that, and it collects 308 objects after the first call in the second round of Py_Initialize/Py_Finalize, and then no additional objects. However, I don't think that helps much: Py_Finalize will call PyGC_Collect(), anyway, and before any counts are made. Are you thinking of this comment?: Yes; I was assuming you suggested to enable that block of code. I wrote that, and think it's pretty clear: after PyImport_Cleanup(), so little of the interpreter still exists that _any_ problem while running Python code has a way of turning into a fatal problem. Right. I still haven't tried it, but it might be that, after a plain Py_Initialize/Py_Finalize sequence, no such problems will occur, and that it would be safe to call it in this specific case. Could you check in the code you're using? I had to modify code in ways that shouldn't be checked in, e.g. by putting API calls into _Py_PrintReferenceAddresses, even though the comment says it does't call any API. When I get to clean this up, I'll check it in. With some debugging, I now found a leak that contributes to quite some of these garbage objects: Each round of Py_Initialize/Py_Finalize will leave a CodecInfo type behind. I think it comes from this block of code /* Note that as of Python 2.2, heap-allocated type objects * can go away, but this code requires that they stay alive * until program exit. That's why we're careful with * refcounts here. type_list gets a new reference to tp, * while ownership of the reference type_list used to hold * (if any) was transferred to tp-tp_next in the line above. * tp is thus effectively immortal after this. */ Py_INCREF(tp); so that this leak would only exist if COUNT_ALLOCS is defined. I would guess that even more of the leaking type objects (16 per round) can be attributed to this. This completely obstructs measurements, and could well explain why the number of leaked objects is so much higher when COUNT_ALLOCS is defined. OTOH, I can see why this code requires that they stay alive. Any ideas on how to solve this dilemma? Perhaps the type_list could be a list of weak references, so that the types do have a chance to go away when the last instance disappears? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Neal Norwitz wrote: If the patch won't fix the problem, is there something else we can do to ensure the python DLL is no longer used regardless of whether the previous test passed or not? Rebooting the machine will help, and might be the only cure. It's Windows, after all :-( Of course, we shouldn't do that, and even if it was ok to reboot remotely, the buildbot likely wouldn't come back automatically. If we can get the process handle, can we can subprocess.TerminateProcess()? You get the process handle either from CreateProcess (which buildbot did, so we can't get the handle), or from OpenProcess. For OpenProcess, we need a process id. One way to get that is through Process32First/Process32Next. These would provide the executable path, so it should be easy to find out which one is a python_d.exe binary. None of these functions is exposed through subprocess, so this is no option. In addition, I believe that buildbot *tries* to use TerminateProcess. The code is twisted, though, so it is hard to tell what actually happens. Of course, it would be possible to do this all in VisualBasic, so we could check in a vbscript file, similar to the one in http://support.microsoft.com/kb/q187913/ OTOH, we could just as well check in an executable that does all that, e.g. like the one in http://msdn.microsoft.com/library/default.asp?url=/library/en-us/perfmon/base/enumerating_all_modules_for_a_process.asp Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Tim Peters wrote: 2. The buildbot code tries to kill the process itself. It appears (to judge from the buildbot messges) that this never works on Windows. 3. For reasons that are still unknown, python_d.exe keeps running, and forever. It's actually not too surprising that python_d.exe keeps running. The buildbot has a process handle for the cmd.exe process that runs test.bat. python_d.exe is only a child process of process. So killing cmd.exe wouldn't help, even if it worked. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
David Abrahams wrote: I just wanted to write to encourage some Python developers to look at (and accept!) Christoph's patch. This is really crucial for smooth interoperability between C++ and Python. I did, and accepted the patch. If there is anything left to be done, please submit another patch. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Py_BEGIN_ALLOW_THREADS around readdir()?
Currently, the readdir() call releases the GIL. I believe this is not thread-safe, because readdir() does not need to be re-entrant; we should use readdir_r where available to get a thread-safe version. Comments? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_BEGIN_ALLOW_THREADS around readdir()?
Ronald Oussoren wrote: AFAIK readdir is only unsafe when multiple threads use the same DIR* at the same time. The spec[1] seems to agree with me. [1] : http://www.opengroup.org/onlinepubs/009695399/functions/readdir.html What specific sentence makes you think so? I see The readdir() interface need not be reentrant. which seems to allow for an implementation that returns a static struct dirent. Of course, the most natural implementation associates the storage for the result with the DIR*, so it's probably not a real problem... Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
OTOH, we could just as well check in an executable that does all that, e.g. like the one in I did something like this: I checked the source of a kill_python.exe application which looks at all running processes and tries to kill python_d.exe. After several rounds of experimentation, this now was able to unstick Trent's build slave. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ python-Patches-790710 ] breakpoint command lists in pdb
Grégoire Dooms wrote: What should I do to get it reviewed further ? (perhaps just this : posting to python-dev :-) It didn't help that much, except for keeping your mail in my inbox. In any case, I went back to it and checked it in. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] FishEye on Python CVS Repository
Peter Moore wrote: I'm responsible for setting up free FishEye hosting for community projects. As a long time python user I of course added Python up front. You can see it here: http://fisheye.cenqua.com/viewrep/python/ Can you please move that to the subversion repository (http://svn.python.org/projects/python), or, failing that, remove that entry? The CVS repository is no longer used. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] problem installing current cvs - TabError
Anthony Baxter wrote: There's a scripts Tools/scripts/reindent.py - put it somewhere on your PATH and run it before checkin, like reindent.py -r Lib. It means Tim or I don't have to run it for you wink As I kept forgetting what the name, location, and command line options of that script are, I now added a reindent makefile target. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_BEGIN_ALLOW_THREADS around readdir()?
Ronald Oussoren wrote: A couple of lines down it says: The pointer returned by readdir() points to data which may be overwritten by another call to readdir() on the same directory stream. This data is not overwritten by another call to readdir() on a different directory stream. This explicitly says that implementations cannot use a static dirent structure. Ah, right. I read over this several times, and still managed to miss that point. Thanks. Of course, the most natural implementation associates the storage for the result with the DIR*, so it's probably not a real problem... If this were a problem on some platform I'd expect it to be so ancient that it doesn't offer readdir_r either. Sure - I would have just removed Py_BEGIN_ALLOW_THREADS on systems which don't have readdir_r. But this is now unnecessary. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Tim Peters wrote: No, what's surprising is that it keeps running _forever_. This isn't Unix, and, e.g., a defunct child process doesn't sit around waiting for its parent to reap it. Why doesn't the leftover python_d.exe complete running the test suite, and then go away all by itself? It doesn't, no matter how long you wait. That's the mystery to me. True. But I find that not too surprising: something deadlocks. A perfect deadlock aims to hold until the heat death of the universe; most of them only hold until reboot, or even just process termination. Now, as to *why* it deadlocks: that's indeed a mystery. But hey: it's Windows, so processes just do get stuck. It took them years to make sure they system continues running in such a case. It suppose it's possible that killing cmd.exe actually did work, but the buildbot code misreports the outcome, and python_d.exe runs forever because it's blocked waiting on some resource (console I/O handle?) it inherited from its (no longer there) parent process. It can't be that simple. Python's stdout should indeed be inherited from cmd.exe, but that, in turn, should have obtained it from buildbot. So even though cmd.exe closes its handle, Python's handle should still be fine. If buildbot then closes the other end of the pipe, Python should get ERROR_BROKEN_PIPE. The only deadlock I can see here is when buildbot does *not* close the pipe, but stops reading from it. In that case, Python's WriteFile would block. If that happens, it would be useful to attach with a debugger to find out where Python got stuck. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] How to make _sre.c compile w/ C++?
[EMAIL PROTECTED] wrote:
if (b == 1) {
-literal = sre_literal_template(ptr, n);
+ literal = sre_literal_template((SRE_CHAR *)ptr, n);
} else {
#if defined(HAVE_UNICODE)
-literal = sre_uliteral_template(ptr, n);
+ literal = sre_uliteral_template((Py_UNICODE *)ptr, n);
#endif
../Modules/_sre.c: In function 'PyObject* pattern_subx(PatternObject*,
PyObject*, PyObject*, int, int)':
../Modules/_sre.c:2287: error: cannot convert 'Py_UNICODE*' to 'unsigned
char*' for argument '1' to 'int sre_literal_template(unsigned char*, int)'
During the 16-bit pass, SRE_CHAR expands to Py_UNICODE, so the call to
sre_literal_template is incorrect. Any ideas how to fix things?
sre_literal_template doesn't take SRE_CHAR*, but unsigned char*. So just
cast to that.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45505 - python/trunk/Modules/posixmodule.c
[EMAIL PROTECTED] wrote: Martin Also, I suggest to use None as the return value for no value Martin available; it might be that the configured value is an empty Martin string (in which case confstr returns 1). I'll work on all of this. Are you sure you want the API to change? Wrt. to the no configured value case? If everybody can agree it is the conceptually right thing to do (*), then sure; documentation should get updated, of course (if there is any). This was so broken already that I'm not worried about breaking some user's code: all users apparently only ever used the successful cases. OTOH, if people debate whether this actually is the right thing to do, it should not change. Regards, Martin (*) I believe it is conceptually right, because it allows to distinguish two cases which are currently indistinguishable in Python but distinguishable in C, namely: a) there is no configured value (confstr returns 0 an does not change errno), and b) the configured value is an empty string (confstr returns 1). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ( r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Fredrik Lundh wrote: it's still listed under possible additions in the release PEP, and there don't seem to be a PEP or any other easily located document that explains exactly how it works, what's required from library developers, how it affects existing toolchains, etc. is this really ready for inclusion ? does anyone but Phillip understand how it works ? I don't understand it. My concern is that it appears to involve a lot of magic. This magic might do the right thing in many cases, and it might indeed help the user that the magic is present, yet I still do believe that magic is inherently evil: explicit is better than implicit. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Gerhard Häring wrote: Speaking of which, what about SVN commit privileges for me? Send me your key, and I'll add you. I assume 'gerhard.haering' would work as the commit name? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Greg Ewing wrote: I started refactoring some of the ugliness out of the internals of distutils last year, but was completely stymied by the demand that no existing setup.py scripts be broken. Instead of trying to fix distutils, maybe it would be better to start afresh with a new package and a new name, then there wouldn't be any backwards-compatibility issues to hold it back. It is *precisely* my concern that this happens. For whatever reason, writing packaging-and-deployment software is totally unsexy. This is why setuptools is a one-man show, and this is why the original distutils authors ran away after they convinced everybody that distutils should be part of Python. If distutils is now abandoned and replaced with something else, the same story will happen again: the developers will run away, the package gets abandoned, and, after a few years of sadness, a new, smart developer will come along and provide a super replacement. And that will repeat in cycles of roughly 10 years. We have to stop this. If distutils has flaws, fix them. Never ever even think about rewriting software: http://www.joelonsoftware.com/articles/fog69.html Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Anthony Baxter wrote: It is *precisely* my concern that this happens. For whatever reason, writing packaging-and-deployment software is totally unsexy. This is why setuptools is a one-man show, and this is why the original distutils authors ran away after they convinced everybody that distutils should be part of Python. If distutils is now abandoned and replaced with something else, the same story will happen again: the developers will run away, Well, I've seen no indication that this is Phillip's plan. If it is, could he tell us now? wink I don't know about Phillip's plans, but I do see many indications that people stop using distutils, and use setuptools instead. People change their setup.py files to setuptools.setup instead of distutils.setup, since the former provides the same things to them, just better. So effectively, distutils disappears except as an implementation detail of setuptools. There is much talk about backwards compatibility: these package are then, of course, not backwards compatible with earlier Python versions anymore. No problem: just install setuptools on these earlier versions. So distutils isn't just abandoned for the future versions, but also for past versions. I started looking at this. The number of complaints I got when I started on this that it would break the existing distutils based installers totally discouraged me. I believe this is a myth. Some changes might cause breakage, yes, but others wouldn't. For example, introducing additional parameters to setup is just fine: existing packages won't break; new packages using these parameters won't build on older Python releases, of course. In addition, the existing distutils codebase is ... not good. Then it shouldn't have become part of Python in the first place. Can you please elaborate what specific aspects of distutils you dislike? It is flatly not possible to fix distutils and preserve backwards compatibility. Why? Sometimes you _have_ to rewrite. I point to urllib-urllib2, asyncore-twisted, rfc822/mimelib/c-email. If I had the time, I would question each of these. Yes, it is easier for the author of the new package to build in the green, but it is (nearly) never necessary, and almost always bad for the project. This approach means that people's existing code continues to work, there's a separate installer of the new code that is available for older versions of Python, plus we have the newer features. Yes, and everybody has to rewrite their code, because the old modules don't see fixes, and get obsoleted and eventually removed. Users get the impression that Python breaks their APIs for no good reason every now and then. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Anthony Baxter wrote: And I would reply that sometimes a rewrite is necessary. I doubt firefox would be at the state it is today if it was still based on the ancient netscape codebase. It's off-topic here certainly: but firefox is actually not a complete rewrite; it still has tons of ancient netscape codebase in it. The could not have completed it otherwise. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Anthony Baxter wrote: I'm not sure how people would prefer this be handled. I don't think we need to have a PEP for it - I don't see PEPs for ctypes, elementtree, pysqlite or cProfile, either. I see a significant procedural difference between what happened for ctypes, elementtree, and pysqlite, as opposed to setuptools. For all these packages, there was 1. a desire of users to include it 2. an indication from the package maintainer that it's ok to include the package, and that he is willing to maintain it 3. some discussion on python-dev, which resulted only in support and no objection 4. some (other) committer who approved incorporation of the library. In essence, that committer is a second for the package inclusion. setuptools has 1 and 2, but fails on 3 and 4 so far. There is discussion now after the fact, but it results in objection. I'd like to point out the importance of 4: Fredrik Lundh originally asked who approved that change, which really meant who can I blame for it. I feel that I approved inclusion of ctypes and elementtree: I talked with the developers on how precisely it should happen, and I checked then that everything that I thought should happened indeed happened. And I did the majority of the communication on python-dev. So the package authors can get all the praise, and I happily take all the blame. The same didn't happen for setuptools: there is no second, so Phillip Eby takes all the praise *and* the blame. It's still a one-man show. Now, I know that Neal Norwitz had asked him what the status is and when it will happen, but he apparently did not want to *approve* inclusion of that package. Likewise, Guido van Rossum (apparently) did not want to approve it, either (he just would not object). If you (Anthony) want to act as a second for setuptools, I would feel much happier - because I can then blame you for whatever problems that decision causes five years from now. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in 2.5.
Anthony Baxter wrote: And now let's look at some of the stuff that setuptools gives us: - We have a CPAN-type system I think it is unfair (to Richard Jones) to attribute this to setuptools. We already have a CPAN-type system: the Cheeseshop. What setuptools adds is roughly the CPAN shell (ie. CPAN.pm or however it is implemented). Actually, I think it is ez_setup that provides (something like) the CPAN shell. It is my understanding that setuptools itself has nothing to do with a CPAN-like system, just as Makemakefile.pl has nothing to do with CPAN. - Multiple installs of different versions of the same package, including per-user installs. I never had the need, but I trust others do. - The develop mode This makes life that bit less painful all-round. This could be added to distutils with no problems, right? - The plugin/extension support Extending distutils currently is a total pain in the arse. If that's a desirable feature to have, it *should* be added to distutils; IOW, it should be available if you do from distutils import setup, not just when you do from setuptools import setup. I'm a little suprised by the amount of fear and loathing this has generated. To me, there are such obvious benefits that I don't see why people are so vehemently against setuptools. I haven't seen any arguments that have convinced me that this isn't the right thing to do. Yes, there's still work to be done - but hell, we've only released the first alpha so far. I'm not looking at it from a end-user's point of view. I trust that setuptools provides features that end-users want, and in a convenient way. My fear is in the maintainer's side: how many new bug reports will this add? How much code do I have to digest to even make the slightest change? That is says from itself that it is version 0.7a1dev-r45536 doesn't help to reduce that fear. I'm also suprised by how much some people seem to think that the current state of distutils functionality is acceptable or desirable. It's not - it's a mess. I would like to require that this is solved by contributing to distutils, instead of replacing it. I know this is an unrealistic request to make - in particular because there is only a single developer world-wide which actively develops something like that. Requiring Phillip to rewrite distutils instead is certainly unfair - but I'm still unhappy with the path events take. Finally, I'd like to point out that I think some of the hostility towards Phillip's work has been excessive. I'd like to make clear that my hostility is only towards his work; never towards Phillip Eby himself. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Greg Ewing wrote: I'm not sure whether distutils is really that badly broken. But an earlier poster seemed to be saying that he had great difficulty finding anything that could be changed without breaking something that someone relied on. It's hard to fix something if you can't change it at all. That's not been my experience with modifying distutils. I fixed several bugs over the years, and always found a way to fix them (or, rather, the contributor whose patch I committed found a fix). OTOH, I never tried to extend the architecture (but never had a need to, either). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in the stdlib ([Python-checkins] r45510 - python/trunk/Lib/pkgutil.py python/trunk/Lib/pydoc.py)
Phillip J. Eby wrote: I assumed that it would be more controversial to merge setuptools into distutils, than to provide it as an enhanced alternative. It is this assumption in setuptools that seems to have guided many design decisions: that it is completely unacceptable to change implementation details, because somebody might rely on them. I firmly believe that this position is misguided, and that decisions resulting from it should be corrected (over time, of course). Beautiful is better than ugly: if you believe that the distutils code is wrong in some respect, then change it. Of course, things are more complicated in this approach: you have to actively consider the likelyhood of breakage, and you have to a) clearly document the potential for breakage: the more likely the breakage, the more visible the documentation should be b) try to come up with recommendations for users should the any code actually break: users then want to know how they should change their code so that it works with previous versions of Python still. c) ask for consent in advance to making a potentially-breaking change. 1. How to activate or deactivate backward compatibility for packages or people relying on intimate details of current distutils behaviors See above: on a case-by-case basis. 2. Maintaining the existing version of setuptools to work with Python 2.3 and 2.4, which would not have this integration with the distutils. One way would be to make another distutils release, and require setuptools users to install this distutils release as a prerequisite. Another solution would be to fork setuptools, in a pre-2.5 branch and a post-2.5 branch. Over time, the pre-2.5 branch could be abandoned. A third solution likely exists on a case-by-case basis: conditionalize code inside setuptools, depending on Python version (or other criteria). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
I don't understand it.
Have you read the manuals?
You mean,
http://peak.telecommunity.com/DevCenter/setuptools
Yes, I did. However, this would only enable me to use it (should I
ever find a need). What I really need to understand is how all this
stuff works inside.
Now, I haven't actually *tried* to learn it so far. Fredrik's question
was not who thinks he could learn it (I certainly could), I
read his question as who does understand it, as of today. I can't
faithfully answer yes to that question.
So far, I believe nobody has said yes, I do understand it, and could
maintain it if Phillip Eby became unavailable.
Please define magic.
Functionality with no obvious implementation, or functionality
implemented in an unobvious way.
Better, please point to the API functions or
classes, or the setup commands documented in the manual, to show me what
these things are that appear to be magic.
Here are some that I either already knew, or discovered just now:
1. setuptools manages to replace pydoc on earlier Python versions.
MAL assumed it does so by overwriting the files. This would be
bad practice, but is the obvious solution.
You then told him that this is not what actually happens (so
it's already magic); instead, you arrange that Python finds
your version before it finds its own.
Again, this is still magic: How does it do that? If you append
to sys.path, Python would still find its own version before
it finds yours. So perhaps you insert another path at
sys.path[0]? This would also be bad practice, but it would
be the next-obvious approach.
But I fully expect that you are now telling me that this is
still not how it works.
2. setuptools manages to install code deep inside a subdirectory
of site-python, and manages to install multiple versions
simultaneously.
How does that work? The first part (packages outside sys.path)
can be solved with a .pth file (which I already consider bad
practice, as it predates the package concept of Python);
but that can't work for multiple versions: magic.
I (think I) know the answer: there is a single .pth file,
and that is *edited* on package installation time. This
is a completely non-obvious approach. If Python needs
to support multiple versions of a package (and there
is apparently a need), that should be supported
properly, with a clear design, in a PEP.
3. namespace packages. From the documentation, I see that
you pass namespace_packages= to setup, and that you
put
__import__('pkg_resources').declare_namespace(__name__)
into the namespace package. How does that work?
The documentation only says
This code ensures that the namespace package machinery
is operating and that the current package is registered
as a namespace package.
Also, why not
import pkg_resources
pkg_resources.declare_namespace(__name__)
What does declare_namespace actually do?
4. zip_safe. Documentation says
If this argument is not supplied, the bdist_egg command
will have to analyze all of your project's contents
for possible problems each time it buids an egg.
How? Sounds like it is solving the halting problem, if
it manages to find possible problems.
There do exist implementation details that are not included in user
documentation, but this is going to be true of any system. These details
are sometimes necessarily complex due to distutils limitations, behavioral
quirks of deployed packages using distutils, and the sometimes baroque
variations in sys.path resolution across platforms, Python versions, and
invocation methods.
I trust that there is a good reason for each of them. However,
for inclusion in the standard library, some of these should go away:
- if you have distutils limitations, remove them
- differences in Python versions shouldn't matter: you always
know what the Python version is
- if there are baroque variations in sys.path resolution across
platforms, they should be removed, or modernized.
Not sure what invocation methods are.
This magic might do the right thing in many cases, and it might
indeed help the user that the magic is present, yet I still do believe
that magic is inherently evil: explicit is better than implicit.
Are documented defaults implicit or magic?
Documented defaults are explicit.
With respect to you and MAL, I think that some of your judgments regarding
setuptools may have perhaps been largely formed at a time last year when,
among other things:
* That documentation section I just referenced didn't exist yet
I don't think I ever complained about the lack of documentation. I would
(personally) not trust the documentation, anyway, but always look at the
code to find out how it actually works. I read documentation primarily
to find out what the *intent* was.
These are significant changes that are directly relevant to the objections
that you and he raised (regarding startup time, path
Re: [Python-Dev] setuptools in 2.5.
Anthony Baxter wrote: Not without a lot of the other stuff that's in setuptools. That is says from itself that it is version 0.7a1dev-r45536 doesn't help to reduce that fear. It's had _two_ _years_ of quite active development, so the version number of 0.7 is hardly a good indication. As far as all the other stuff on the end of the version string - well, right now Python's SVN trunk really could be considered 2.5a2dev-r45575. Right - I'm not concerned about the a1dev-r45536 part. The 0.7 part bothers me; this is two years of development from a single person, still. I have projects that are much older than that that I wouldn't suggest for inclusion in Python :-) Requiring Phillip to rewrite distutils instead is certainly unfair - but I'm still unhappy with the path events take. He's written code on _top_ _of_ _distutils_. How is this bad? It makes distutils an implementation detail of setuptools. What little development distutils has seen will stop; all fixes will go into setuptools directly. Users will be told that they should switch to setuptools. Please face it: setuptools *is* the death of distutils. This might not be that bad in the long run, but it does have the risk of repeating, when setuptools eventually is where distutils is today: complete, and unmaintained. See, I don't get the hostility thing. While I have some concerns about the state of distutils today, I still admire Greg Ward's efforts in producing the code, and Python is in a much better place than had he not done the work. Responding to an effort like Greg's, or Phillip's, with hostility is a fantastic way to discourage people from working further on Python or on the code in question. Well, I appreciate other contributions from other people, and I have always encouraged people to contribute to Python. It's just that I dislike this *specific* package, for several reasons, some of which I consider objective. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New-style icons, .desktop file
Fredrik Lundh wrote: you do wonderful stuff, and then you post the announcement as a followup to a really old message, to make sure that people using a threaded reader only stumbles upon this by accident... this should be on the python.org frontpage! I also wonder what the actions should be for the Windows release. Are these contributed to Python? With work of art, I'm particular cautious to include them without a specific permission of the artist, and licensing terms under which to use them. And then, technically: I assume The non-vista versions should be included the subversion repository, and the vista versions ignored? Or how else should that work? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raising objections
Phillip J. Eby wrote: How much any of that could have also applied to the distutils at one time, I don't know. My point is merely that setuptools has enough commercial value to make me believe that sponsorship for features shouldn't be that hard to come by, and there are certainly worse things I could do with my work days. I'm glad to hear that you seem to be finding support for ongoing work on setuptools (I'm *really* glad to hear that; it's always good when people get funded to work on open source). I'm also happy to hear that Neal wants to study the complete source base of setuptools. Perhaps that means I won't have to :-) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Anthony Baxter wrote: 4. .egg files. -1 As far as I understand it, an egg file is just a zipimport format zip file with some additional metadata. You can also install the egg files in an unpacked way, if you prefer that. I don't understand the objection here - it's no better or worse than countless packages in site-packages, and if it gives us multiple versions of the same code, all the better. It is worse: each .egg file makes an additional sys.path entry (if I understand correctly); every import statement will traverse every every package. I'm not sure precisely how all this works, but I wouldn't be surprised if the zip directory is read over and over again. Compare that to countless packages in site-packages: import foo will *just* look for foo.py, foo.so, and the directory foo. I understand there is a second API to importing, some kind of require() call. I consider that even worse: it obsoletes the language's mechanism for modules, and defines its own modularization. However, this isn't really my objection to .egg files. I dislike them because they compete with platform packages: .rpm, .msi, .deb. Package authors will refuse to produce them, putting the burden of package maintenance (what packages are installed, what are their dependencies, when should I remove a package) onto the the end user/system administrator. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Guido van Rossum wrote: This is another area where API standardization is important; as soon as modules are loaded out of a zip file, the traditional approach of looking relative to os.path.dirname(__file__) no longer works. Certainly. However, I get two conclusions out of this: 1. don't load packages out of .zip files. It's not that bad if software on the user's disk occupies multiple files, as long as there is a convenient way to get rid of them at once. Many problems go away if you just say no to zipimport. 2. standardize on file names, not API. If I want to deploy read-only data files, I put them into /usr/share. If I need read-write files, I put them into /var. I did not have such a problem yet on other systems, but I would also try to follow the conventions of these systems. With these combined, I can use any API I like to operate on the files. distutils already has support for that. Some libraries (not necessarily in Python) have gone the path of providing a unified API for all kinds of file stream access, e.g. in KDE, any tool can open files over many protocols, without the storage being mounted locally. I consider this approach flawed: once I leave the realm of KDE programs, this all stops working. It is the operating system's job to provide unified access to storage, not the programming language's or the job of a library. - I still fear that the code of setuptools will turn out to be a maintenance nightmare. AFAICT Phillip has explicitly promised he will maintain it (or if he doesn't, I expect that he would gladly do so if you asked). This has always been sufficient as a guarantee that a new module isn't orphaned. He has, and it is. Still, for whatever reason, I feel particularly uneasy here (and yes, that is my fear, my uncertainty, and my doubt). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Jérôme Laheurte wrote:
Sorry I'm late, but something like os.popen('taskkill.exe /F /IM
python_d.exe') would have worked; we use this at work.
Thanks, I didn't know about it. Is it available on Windows 2000,
too? (because the system in question is Windows 2000, and I see
it on a What's new in Windows XP page)
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Bob Ippolito wrote: They DO NOT compete any more than source packages do. eggs are packages plus metadata, nothing more. What eggs do and what rpm/msi/deb does are orthogonal. It's entirely reasonable that in the future rpm/msi/deb will simply be a delivery mechanism for eggs. That might be your view, but it apparently isn't the view of the inventor(s). From http://peak.telecommunity.com/DevCenter/setuptools Create Python Eggs - a single-file importable distribution format http://peak.telecommunity.com/DevCenter/PythonEggs 'Eggs are to Pythons as Jars are to Java...' 'There are several binary formats that embody eggs, but the most common is '.egg' zipfile format, because it's a convenient one for distributing projects.' '.egg files are a zero installation format for a Python package;' So the .egg inventors do view .egg files (i.e. the .egg zipfile format) as a distribution format, just like rpm/msi/deb *are* distribution formats (none of them zero installation, though, you always have to perform some deployment activity). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Ronald Oussoren wrote: As far as I understand the issues they compete up to a point, but should also make it easier to create platform packages that contain proper the proper dependencies because those are part of machine-readable meta-data instead of being written down in readme files. Oddly enough that was also the objection from one linux distribution maintainer: somehow his opinion was that the author of a package couldn't possibly know the right depedencies for it. What he can't possibly know is the *name* of the package he depends on. For example, a distutils package might be called 'setuptools', so developers of additional packages might depend on 'setuptools'. However, on Debian, the dependency should be different: The package should depend on either 'python-setuptools' or 'python2.3-setuptools', depending on details which are off-topic here. As for platform packages: not all platforms have useable packaging systems. MacOSX is one example of those, the system packager is an installer and doesn't include an uninstaller. Eggs make it a lot easier to manage python software in such an environment (and please don't point me to Fink or DarwinPorts on OSX, those have serious problems of their own). Isn't uninstallation just a matter of deleting a directory? If I know that I want to uninstall the Python package 'foo', I just delete its directory. Now, with setuptools, I might have multiple versions installed, so I have to chose (in Finder) which of them I want to delete. That doesn't require Eggs to be single-file zipfiles; deleting a directory would work just as will (and I believe it will work with ez_install, too). Package authors will refuse to produce them, putting the burden of package maintenance (what packages are installed, what are their dependencies, when should I remove a package) onto the the end user/system administrator. Philip has added specific support for them: it is possible to install packages in the tradition way but with some additional files that tell setuptools about installed packages. As a system administrator, I don't *want* to learn how to install Python packages. I know how to install RPMs (or MSIs, or whatever system I manage); this should be good enough. If this Python junk comes with its own installer procedure, I will hate it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Greg Ewing wrote: The resources name is actually quite a common meme; I believe it goes back to the original Macintosh, which was the first and only computer in the world to have files with something called a resource fork. The resource fork contained pieces of data called resources. I can believe that history. Still, I thought a resource is something you can exhaust; the fork should have been named data fork or just second fork. Then Microsoft stole the name, and before you knew, everyone was using it. It's all been downhill from there. :-) Right. I'm not asking that the name is changed in setuptools - I'm just complaining about the state of the world, and showing my lack of intuition for the English language. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Bob Ippolito wrote: 'There are several binary formats that embody eggs, but the most common is '.egg' zipfile format, because it's a convenient one for distributing projects.' '.egg files are a zero installation format for a Python package;' single modules are also such a zero installation format too. So what? You're simply reading things between the lines that aren't there. How about you describe exactly what parts of the documentation that lead you to believe that eggs are designed to compete with solutions like rpm/msi/deb so that it can be clarified? It's not just the documentation: I firmly believe that many people consider .egg files to be a distribution and package management format. People have commented that some systems (e.g. OSX) doesn't have a usable native packager, so setuptools fills a need here. This shows that people do believe that .egg files are to OSX what .deb files are to Debian. As .egg files work on Debian, too, it is natural that they compete with .deb. Phillip Eby once said (IIRC) that he doesn't want package authors to learn all the different bdist_* commands (which also require access to the target systems sometimes), and that they their life gets easier as they now only have to ship the native Python binary packages, namely .egg files. In this view, rpm/msi/deb have no place anymore, and are obsolete. I can readily believe that package authors indeed see this as a simplification, but I also see an increased burden on system admins in return. So if this attitude (Python Eggs are the preferred binary distribution format) is wrong, it is the attitude that has to change first. Changes to the documentation follow from that. If the attitude is right, I'll have to accept that I have a minority opinion. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools in 2.5.
Ian Bicking wrote: I don't think setuptools version requirements are enough to ensure the integrity of all pieces of a complex system will work together. Figuring out a self-consistent set of packages to work together is something that is rather independent of any particular package, and Setuptools doesn't have a facility for that. But it does provide the tools to build that kind of facility, and egg-based installations provide the sufficient metadata to report on what has been built. So I think it is a step in the right direction. Integrating packages from a wide variety of sources is hard. Indeed. Microsoft also thought that side-by-side installation of .NET assemblies will solve all versioning problems, as you can have as many versions installed simultaneously as you want to. It turns out that they also where wrong: at best, side-by-side installation can *help* to solve versioning problems. It might also contribute to increase them, as people stop worrying about backwards compatibility, trusting that side-by-side installation can replace proper engineering. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2.5alpha and naming schemes
Dennis Heuer wrote: Module names like hashlib are not python-like too (too c/lowlevel-like). I agree with Greg: hashlib is a Pythonic name for a module, just like httplib, mhlib, xmlrpclib, cookielib, contextlib, difflib, ... OTOH, it might be indeed that the ctypes name need to be aligned with naming conventions. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Ronald Oussoren wrote: That doesn't require Eggs to be single-file zipfiles; deleting a directory would work just as will (and I believe it will work with ez_install, too). Egg directories (which are basically just the same as packages using .pth files) also work for this and that's what I usually install. Setuptools can work with python extension inside .zip files, but I'm not entirely comfortable with that. It's primarily the .egg *files* that I dislike. I'm can accept the .egg directories. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Jérôme Laheurte wrote: Ah, no, it's only available in XP. There are some alternatives though: http://www.robvanderwoude.com/index.html Sure. Writing my own one wasn't that difficult, in the end, either (except that I overlooked that the API I used first exists in XP and later only). regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] magic in setuptools (Was: setuptools in the stdlib)
Greg Ewing wrote: Guido van Rossum wrote: You can't blame KDE for providing mechanisms that only work in the KDE world. It's fine for Python to provide Python-specific solutions for issues that have no cross-platform native solution. Also keep in mind that we're talking about resources used internally by the application. They don't *need* to be accessible outside the application. So I don't think the KDE argument applies here. They might need to be available outside Python. Two use cases: 1. The application embeds an sqlite database. Even though it knows how to get at the data, it can't use it, because the sqlite3 library won't accept .../foo-1.0.egg/resources/the_data (or some such) as a database name, if foo-1.0.egg is a ZIP file. If the installed application was a set of files, that would work. 2. The application embeds an SGML DTD (say, HTML). In order to perform validation, it invokes nsgmls on the host system. It cannot pass the SGML catalog to nsgmls (using the -C option) since you can't refer to DTD files inside ZIP files inside an SGML catalog. If this was a bunch of installed files, it would work. 3. The application includes an SSL certificate. It can't pass it to socket.ssl, since OpenSSL expects a host system file name, not a fragment of a zip file. If this was installed as files, it would work. This is precisely what happens in KDE: you have konqueror happily browse an SMB directory, double-click on a .xls file, OpenOffice starts and can't access the file it was started with. It doesn't matter to the user that there is a good reason for that. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2.5alpha and naming schemes
Thomas Heller wrote: Personally, I *like* the ctypes name. But I'm open for suggestions, and it might have intersting consequences if the Python core package would be renamed to something else. Any suggestions? Well, my only concern was py_object. I wondered whether the py_ prefix is really necessary, or, if it is meant to match PyObject* whether it should be called PyObject or PyObject_p. However, if every ctypes user knows what py_object is, there is probably little point in renaming it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Guido van Rossum wrote: Microsoft just announced that Visual Studio 2005 express will be free forever, including the IDE and the optimizing C++ compiler. (Not included in the forever clause are VS 2007 or later versions.) Does this make a difference for Python development for Windows? For future versions, perhaps. For 2.5, I think we now have settled on VS 2003, for several reasons: - I personally consider VS 2005 still verdant (crude? immature? unfledged?). They can't really mean the whole breakage they have done to the C library. Also, I expect another release of VS after Vista, to cover all the new .NET API, and I hope that we can skip VS 2005 (although Vista gets delays, and so gets VS 2007) - Fredrik Lundh points out that it would be nice if people producing extensions for multiple Python releases wouldn't need a separate compiler for each release. - Paul Moore has contributed a Python build procedure for the free version of the 2003 compiler. This one is without IDE, but still, it should allow people without a VS 2003 license to work on Python itself; it should also be possible to develop extensions with that compiler (although I haven't verified that distutils would pick that up correctly). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
Thomas Heller wrote: On XP (32-bit), I can compile python25.dll and python.exe for AMD64 now, after adding bufferoverflowU.lib to the linker options. On what project? There should be /GS- options on all projects that need it, which, in turn, should result in bufferoverflowU.lib not being needed. Or I forgot to check that change in... Will do Monday. Error : Could not create new temporary options response file I've never seen these. I will have to study the source again, and find out how that could happen. Do you have spaces in the directory leading to the working copy? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Giovanni Bajo wrote: It's been possible to compile distutils extensions with the VS 2003 toolkit for far longer than it's possible to compile Python itself: http://www.vrplumber.com/programming/mstoolkit/ Sure: with distutils modifications. In fact, it would be great if the patches provided here were reviewed and integrated into the official Python distutils. Something like this should already be in Python 2.5. I just haven't tested it in this configuration. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
Thomas Heller wrote: I forgot to mention that there are a lot of warnings about conversion betweem Py_ssize_t to int - if you want me to fix the obvious ones I'll offer to correct some of them from time to time and commit the changes. Right - they have been there ever since I started (in fact, I started this entire project *because* of these warnings). You can get them on x86, too, if you enable /Wp64. Please feel free to fix any of them that you feel comfortable about. I wonder why gcc doesn't warn about those. It just doesn't implement truncation warnings between variables of differently-sized integer types. This (typically) isn't undefined behaviour: the C standard mandates very well what should happen for these truncations. Warning about any and all truncations would lead to incredible noise. /Wp64 *only* works because they restrict themselves to int64-int32 warnings (essentially). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
Neal Norwitz wrote: Right - they have been there ever since I started (in fact, I started this entire project *because* of these warnings). You can get them on x86, too, if you enable /Wp64. In case it wasn't clear, the /Wp64 flag is available in icc (Intel's C compiler). It still isn't clear :-) The flags is also available in msvc: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/vchowWp64Detect64BitPortabilityProblems.asp There is even a checkbox for it in the project settings. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setuptools: past, present, future
Guido van Rossum wrote: Leaving aside the Perl vs. Py thing, opinions on CPAN seem to be diverse -- yes, I've heard people say that this is something that Python sorely lacks; but I've also heard from more than one person that CPAN sucks from a quality perspective. So I think we shouldn't focus on emulating CPAN; rather, we should solve the problems we actually have. I note that CPAN originated in an age before the web was mature. My personal problems with CPAN were always of the kind that it recorded too many/too stringent dependencies. I used it over a period of several years on Solaris, roughly two times a year. Each time, the package I wanted to installed depended on another package, this in turn on a third, and some of these eventually on a Perl version more recent than the one I had installed. So CPAN would always *first* install a new version of Perl for me. Sometimes, this would fail, because Perl wouldn't pass its test suite on Solaris. So I did huge downloads, long compilation times, and still didn't get the package installed. I always fixed it by installing the new Perl version manually, and then starting over with CPAN again. I'm not exactly sure why that happened, but I think there are two causes: - when installing a package, the automated download tool should not try to find the most recent version. Instead, it should try to find a version that causes the least amount of changes to my system. - CPAN shouldn't include Perl proper (likewise, the Cheesehop shouldn't include Python proper). If dependencies can't be resolved with the current version, but could be resolved with a later version, the download tool should give up and explain it all. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reducing memory overhead for dictionaries by removing me_hash
Kirat Singh wrote: Hi, this is my first python dev post, so please forgive me if this topic has already been discussed. To my knowledge, this hasn't been discussed before. It seemed to me that removing me_hash from a dict entry would save 2/3 of the space used by dictionaries and also improve alignment of the entries since they'd be 8 bytes instead of 12. And sets end up having just 4 byte entries. How did you compute the saving of 2/3? If one field goes and two fields stay, it saves 1/3, no? Also, the improved alignment isn't that much of an improvement on a 32-bit system, AFAICT. Likewise, on a 64-bit system, me_hash consumes 8 bytes (regardless of whether a long is 4 or 8 bytes), so the saving would be 1/3, and the alignment would change (not improve) from 8 to 16. I'm guessing that string dicts are the most common (hence the specialized lookupdict_string routine), and since strings already contain their hash, this would probably mitigate the performance impact. One could also add a hash to Tuples since they are immutable. You implicitly give the reason for the me_hash field: performance. This is a classical space-vs-time trade-off, currently leaning towards more-space-less-time. You want the trade-off be the other way: less-space-more-time. As you might not be aware of how much more time it will take, consider these uses of the me_hash field: - this is open addressing, so when performing a lookup 1. computes the hash of the key 2. looks at the address according to the hash 3. If there is no key at that address, lookup fails (and a free slot is found) 4. if the hash of the entry at that address is equal, it compares the keys. If they are equal, lookup succeeds. 5. If the hashes are not equal, or the keys are not equal, an alternative address is computed, and search continues with step 3. Under your algorithm, you would have additional object comparisons in case of hash values that translate to the same address. This might slow down dictionaries that are close to being full. - when the dictionary is resized, all entries need to be rehashed. So if me_hash is removed, you have many more PyObject_Hash calls on resizing. This would likely slow down resizing significantly. If this isn't a totally stupid idea, I'd be happy to volunteer to try the experiment and run any suggested tests. I don't see much value in saving a few bytes, and I expect that performance will go down noticeably. Still, only an experiment could show what the real impact is. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Win64 AMD64 (aka x64) binaries available64
Anthony Baxter wrote: On Saturday 22 April 2006 15:27, Neal Norwitz wrote: In case it wasn't clear, the /Wp64 flag is available in icc (Intel's C compiler). Is it worth turning this on for the icc ubuntu buildbot? Anyone got ideas on the best way to do this? Should I just set CFLAGS=-Wp64 before running the buildbot on the box (it's sitting 2 feet behind my head in the rack in my study(*)) Anthony (*) Yes, I have an almost-rack of machines in my house. And yes, this scares me. You should be scared what people are doing with your machines :-) From http://www.python.org/dev/buildbot/trunk/x86%20Ubuntu%20dapper%20%28icc%29%20trunk/builds/229/step-compile/0 'OPT': '-Wp64 -g -O3' icc -pthread -c -fno-strict-aliasing -Wp64 -g -O3 -I. -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] windows buildbot failures
Tim Peters wrote: I've never reported this as a Python bug If you do, I'll probably add a comment like the above ;-) because I've considered the antivirus SW likely to be the culprit. Could be. FWIW, Norton AV was running during the above. I see a similar phenomenon (sp?) on XP SP2: test_mailbox fails to me, with permission denied in some (random) test. I believe this is due to Tortoise SVN: test_mailbox creates a few directories, then Tortoise detects them (thanks to file change notifications) and tries to walk them, to find out whether that directory is under subversion control. Before Tortoise is done, test_mailbox tries to delete the directories. This (somewhat?) fails - there apparently is a delete pending state somehow in the system (sysinternals filemon sometimes shows DELETE PEND as the result of a call). Then, the next test will fail with permission denied. This is a heisenbug: different runs will fail in different tests, and letting sysinternals perform complete tracing make the failure go away. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reducing memory overhead for dictionaries by removing me_hash
Kirat Singh wrote: The reason I looked into this to begin with was that my code used up a bunch of memory which was traceable to lots of little objects with instance dicts, so it seemed that if instancedicts took less memory I wouldn't have to go and add __slots__ to a bunch of my classes, or rewrite things as tuples/lists, etc. Ah. In that case, I would be curious if tuning PyDict_MINSIZE could help. If you have many objects of the same type, am I right assuming they all have the same number of dictionary keys? If so, what is the dictionary size? Do they use ma_smalltable, or do they have an extra ma_table? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Builtin exit, good in interpreter, bad in code.
Sean Reifschneider wrote: Thoughts? In Python 2.5, exit(0) exits. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Martin v. Löwis wrote: - Paul Moore has contributed a Python build procedure for the free version of the 2003 compiler. This one is without IDE, but still, it should allow people without a VS 2003 license to work on Python itself; it should also be possible to develop extensions with that compiler (although I haven't verified that distutils would pick that up correctly). Apparently, the status of this changed right now: it seems that the 2003 compiler is not available anymore; the page now says that it was replaced with the 2005 compiler. Should we reconsider? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
Guido van Rossum wrote: Actually, what Nick describes is *exactly* how one should write code using a condition variable: LOCK while nothing to do: UNLOCK wait for the condition variable (or sleep, or whatever) LOCK # here we have something to do with the lock held remove the to-do item UNLOCK except that the outer LOCK/UNLOCK pair should be using a try/except and the inner UNLOCK/LOCK pair should too. I don't see how you can do this easily by rewriting the code; the rewrite would be considerably ugly (or requires a GOTO :-). I thought the trick is that the condition variable *atomically* releases the lock, waits for the condition, and then reacquires the condition variable. I.e. c = threading.Condition() c.lock() while nothing to do: c.wait() # here we have something to do with the lock held c.unlock() So the refactoring is to move the unlock/wait/lock sequence into the condition object. Using with, you could write this as with threading.Condition() as c: while nothing to do: c.wait() # do work So no need for an additional context manager here. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Neil Hodgson wrote: I expect Microsoft means that Visual Studio Express will be available free forever, not that you will always be able to download Visual Studio 2005 Express. They normally only provide a particular product version for a limited time after it has been superceded. Sure: they will remove download access to VS 2005 when VS 2007 comes available. Still, VS 2005 is available for download right now, and VS 2003 isn't (anymore). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
Alex Martelli wrote: For the Toolkit 2003: http://tinyurl.com/gv8wr When I go to this URL, I get redirected to http://www.microsoft.com/downloads/details.aspx?familyid=272BE09D-40BB-4displaylang=en This doesn't look right - it ought to be a UUID. Anyway, I get a page that reads The download you requested is unavailable. If you continue to see this message when trying to access this download, go to the Search for a Download area on the Download Center home page. both with Firefox and MSIE. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Visual studio 2005 express now free
John J Lee wrote: Actually, it's apparently still there, just at a different URL. Somebody posted the new URL on c.l.py a day or two back (Alex Martelli started the thread, IIRC). I'm off to the dentist, no time to Google for it! Please do. If you find the URL, please post it here. All URLs I found don't work (anymore). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gettext.py bug #1448060
Sylvain Thénault wrote: I've posted a patch (#1475523) for this and assigned it to Martin Von Loewis since he was the core developper who has made some followup on the original bug. Could someone (Martin or someone else) quick review this patch ? I really need a fix for this, so if anyone feels my patch is not correct, and explain why and what should be done, I can rework on it. If you need quick patch, you should just go ahead and use it (unreviewed). It will take several more months until either Python 2.4.4 or Python 2.5 is released. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
Tim Peters wrote: Does with cv: work to replace the outer (== only) acquire/try/finally/release dance? Indeed it does - although implemented in a somewhat convoluted way: A lock *is* a context (or is that context manager), i.e. it implements def __context__(self): return self __enter__=acquire def __exit__(self,*args): return self.release() #roughly A _Condition *has* a lock, so it could become the context (manager?) through def __context__(self): return self.lock However, instead of doing that, it does def __context__(self): return self # roughly: __enter__ is actually set in __init__ to self.lock.acquire def __enter__(self): return self.acquire() def __exit__(self): return self.release Looks somewhat redundant to me, but correct. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed addition to threading module - released
Guido van Rossum wrote: Tim is right, the UNLOCK/LOCK part is implied in the wait() call. However, the wait() implementation really *does* provide a use case for the primitive operation that Nick proposed, and it can't be refactored to remove the pattern Martin disapproves of (though of course the existing try/finally is fine). I'm not sure if the use case is strong enough to warrant adding it; I think it's fine not to support it directly. Ah, you mean that the wait implementation *itself* should use the unlocked() context (which calls release on enter, and acquire on exit). That wouldn't work, as _Condition.wait doesn't use release/enter, but _release_save/_acquire_restore. So the unlocked context couldn't be used there if it existed. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PY_FORMAT_SIZE_T warnings on OS X
Brett Cannon wrote: I created patch 1474907 with a fix for it. Checks if %zd works for size_t and if so sets PY_FORMAT_SIZE_T to z, otherwise just doesn't set the macro def. Assigned to Martin to make sure I didn't foul it up, but pretty much anyone could probably double-check it. Unfortunately, SF stopped sending emails when you get assigned an issue, so I didn't receive any message when you created that patch. I now reviewed it, and think the test might not fail properly on systems where %zd isn't supported. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 304 (Was: Addressing Outstanding PEPs)
[EMAIL PROTECTED] wrote: [...] Should be closed/rejected. [...] Neal S 304 Controlling Generation of Bytecode Files Montanaro Probably ditto. Rejected would be a wrong description, IMO; withdrawn describes it better. It's not that the feature is undesirable or the specific approach at solving the problem - just nobody is interested to work on it. So future contributors shouldn't get the impression that this was discussed and rejected, but that it was discussed and abandoned. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accessing DLL objects from other DLLs
Michael Hearne wrote: If I want to keep these classes as distinct modules, but retain this kind of module interdependency, how can I architect this with Python? Please understand that python-dev is for discussions about the development *of* Python, not for the development *with* Python. Use news:comp.lang.python (mailto:python-list@python.org) for the latter. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: My counter-proposal: to be considered a package, a directory must contain at least one module (which of course can be __init__). This allows the is it a package? question to be answered with only one directory read, as is the case now. Think of it also as a nudge in favor of flat is better than nested. I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. If there are no .py files in a tree, then there would be no point in importing it, so applications will typically not import an empty directory. Implementing an expensive test that will never give a positive result and causes no problems if skipped should be skipped. I can't see the problem this would cause to tools: they should assume any subdirectory can be a package, with all consequences this causes. If the consequences are undesirable, users should just stop putting non-package subdirectories into a package if they want to use the tool. However, I doubt there are undesirable consequences (although consequences might be surprising at first). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: At 11:50 AM 4/26/2006 -0700, Guido van Rossum wrote: I'm not sure what you mean by one directory read. You'd have to list the entire directory, which may require reading more than one block if the directory is large. You have to do this to find an __init__.py too, don't you? Technically, there's going to be a search for a .pyc or .pyo first, anyway. No. Python does stat(2) and open(2) to determine whether a file is present in a directory. Whether or not that causes a full directory scan depends on the operating system. On most operating systems, it is *not* a full directory scan: - on network file systems, the directory is read only on the server; a full directory read would also cause a network transmission of the entire directory contents - on an advanced filesystem (such as NTFS), a lookup operation is a search in a balanced tree, rather than a linear search, bypassing many directory blocks for a large directory - on an advanced operating system (such as Linux), a repeated directory lookup for the file will not go to the file system each time, but cache the result of the lookup in an efficient memory structure. In all cases, the directory contents (whether read from disk into memory or not) does not have to be copied into python's address space for stat(2), but does for readdir(3). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Phillip J. Eby wrote: I assume you want import x.y to fail if y is an empty directory (or non-empty, but without .py files). I don't see a value in implementing such a restriction. No, I'm saying that tools which are looking for packages and asking, Is this directory a package? should decide no in the case where it contains no modules. Ah. Tools are of course free to do that. It would slightly deviate from Python's implementation of import, but the difference wouldn't matter for all practical purposes. So from a language lawyers' point of view, I would specify: A sub-package is a sub-directory of a package that contains at least one module file. Python implementations MAY accept sub-directories as sub-packages even if they contain no module files as package, instead of raising ImportError on attempts to import that sub-package. (a module file, in that context, would be a file file which matches imp.get_suffixes(), in case that isn't clear) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
Dennis Heuer wrote: The reason why I'd like to use the long type as the base of my bitarray type is that the long type is already implemented as an array and working efficiently with large amounts of bytes. This is wrong; you are mixing up the is-a and has-a relationships. While it might be useful to have long as the *representation* of a bitarray, it's not the case that a bitarray *is* a long. A bitarray is a sequence type and should implement all aspects of the sequence protocol; long is a numeric type and should implement numeric operations. While the operators for these somewhat overlap (for + and *), the semantics of the operators never overlaps. So long and bitarray are completely distinct types, not subtypes of each other. IOW, you should do class bitarray: def __init__(self): self.value = 0L ... Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New-style icons, .desktop file
Simon Dahlbacka wrote: OTOH, the ETA for Vista is just after 2.5 release (end of 2006 for OEM:s, beginning of 2007 for customers), long before 2.6 That said, I don't have any strong preferences either way. (..but I do have a x64 Vista machine running ATM) Good to know, but unfortunately, not very helpful in the end: Even if I wanted to include these icons, I still had no clue whatsoever on how to do that. What files do I need to deploy into what locations for this to work, and how do I determine whether or not to use the Vista approach (assuming it is different from the pre-Vista approach)? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inheriting basic types more efficiently
Dennis Heuer wrote: The real misunderstanding lies somewhere else. I thought that the bitarray's instance would have more control over the long type's instance, like with the mutable types. I thought that the long type's superclass would provide methods similar to __setitem__ that would allow the bitarray instance to even *refresh* (or substitute) the long instance in place. The result would be a freshly created long instance substituting the old one. But the immuntable types do not offer such a feature because one cannot substitute the long instance without breaking the bitarray instance too. Maybe that's the misunderstanding: but then you are still left with the mis-design. Even if long was mutable, or even if you used a mutable type as the base type (such as array.array), you *still* shouldn't inherit from it - these types are not in an is-a relationship. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A better and more basic array type
Dennis Heuer wrote: I hope that somebody agrees and is already starting to implement this new array type. My best wishes are with you. This is off-topic for python-dev. Please post to comp.lang.python instead. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New-style icons, .desktop file
Simon Dahlbacka wrote: Given that, it does not really seem feasible to include them.. Ok, thanks for the investigation. Speaking of icons, do the bundled ico files have to be named py.ico and pyc.ico? No. I think I'll try to drop them altogether, getting the icons from python_icon.exe only. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Dropping __init__.py requirement for subpackages
Thomas Wouters wrote: Indeed! I hadn't realized that, although I might've if I'd been able to find where Modules is put on sys.path. And, likewise, I would do as you suggest (which feels like the right thing) if I could only find out where Modules is put on sys.path :) I don't have time to search for it today nor, probably, this weekend (which is a party weekend in the Netherlands.) I'll get to it eventually, although a helpful hint from an old-timer who remembers as far back as Modules/Setup would be welcome. :) With some debugging, I found it out: search_for_exec_prefix looks for the presence of Modules/Setup; if that is found, it strips off /Setup, leaving search_for_exec_prefix with -1. This then gets added to sys.path with /* Finally, on goes the directory for dynamic-load modules */ strcat(buf, exec_prefix); I wasn't following exactly, so I might have mixed something up, but... it appears that this problem here comes from site.py adding the build directory for the distutils dynamic objects even after Modules. The site.py code is # XXX This should not be part of site.py, since it is needed even when # using the -S option for Python. See http://www.python.org/sf/586680 def addbuilddir(): Append ./build/lib.platform in case we're running in the build dir (especially for Guido :-) from distutils.util import get_platform s = build/lib.%s-%.3s % (get_platform(), sys.version) s = os.path.join(os.path.dirname(sys.path[-1]), s) sys.path.append(s) I would suggest fixing #586680: Add build/lib.* before adding dynamic-load modules, by moving the code into Modules/getpath.c. You should be able to use efound 0 as an indication that this is indeed the build directory. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 open issues
Ronald Oussoren wrote: That's fine by me. I'll be adding them to the universal python 2.4 tree soon and to 2.5 when that's is done. Do have to wait for the contributor agreement, which the folks over at [EMAIL PROTECTED] say is good enough, to do that? If the artist has informally agreed to do that (so it is just the formal signing that takes time), go ahead. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 open issues
Georg Brandl wrote: As I posted here previously, I contacted the owner, and he said that he didn't care about specifying a license. I guess that means that we can pick one ;) Can you please ask whether he would be willing to fill out a contrib form (http://www.python.org/psf/contrib/contrib-form/)? Without some kind of explicit contribution, I hesitate to use it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] binary trees. Review obmalloc.c
Vladimir 'Yu' Stepanov wrote: * To adapt allocation of blocks of memory with other alignment. Now alignment is rigidly set on 8 bytes. As a variant, it is possible to use alignment on 4 bytes. And this value can be set at start of the interpreter through arguments/variable environments/etc. At testing with alignment on 4 or 8 bytes difference in speed of work was not appreciable. That depends on the hardware you use, of course. Some architectures absolutely cannot stand mis-aligned accesses, and programs will just crash if they try to perform such accesses. So Python should err on the safe side, and only use a smaller alignment when it is known not to hurt. OTOH, I don't see the *advantage* in reducing the alignment. * To expand the maximal size which can be allocated by means of the given module. Now the maximal size is 256 bytes. Right. This is somewhat deliberate, though; the expectation is that fragmentation will increase dramatically if even larger size classes are supported. * At the size of the allocated memory close to maximal, the allocation of blocks becomes inefficient. For example, for the sizesof blocks 248 and 256 (blocksize), values of quantity of elements (PAGESIZE/blocksize) it is equal 16. I.e. it would be possible to use for the sizes of the block 248 same page, as for the size of the block 256. Good idea; that shouldn't be too difficult to implement: for sizes above 215, pools need to be spaced apart only at 16 bytes. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3102: Keyword-only arguments
Terry Reedy wrote: Now, suppose you wanted to have 'key' be a keyword-only argument. Why? Why not let the user type the additional argument(s) without the parameter name? Are you asking why that feature (keyword-only arguments) is desirable? That's the whole point of the PEP. Or are you asking why the user shouldn't be allowed to pass keyword-only arguments by omitting the keyword? Because they wouldn't be keyword-only arguments then, anymore. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] methods on the bytes object
Josiah Carlson wrote: Specifically in the case of bytes.join(), the current common use-case of literal.join(...) would become something similar to bytes(literal).join(...), unless bytes objects got a syntax... Or maybe I'm missing something? I think what you are missing is that algorithms that currently operate on byte strings should be reformulated to operate on character strings, not reformulated to operate on bytes objects. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] methods on the bytes object
Josiah Carlson wrote: I think what you are missing is that algorithms that currently operate on byte strings should be reformulated to operate on character strings, not reformulated to operate on bytes objects. By character strings can I assume you mean unicode strings which contain data, and not some new character string type? I mean unicode strings, period. I can't imagine what unicode strings which do not contain data could be. I know I must have missed some conversation. I was under the impression that in Py3k: Python 1.x and 2.x str - mutable bytes object No. Python 1.x and 2.x str - str, Python 2.x unicode - str In addition, a bytes type is added, so that Python 1.x and 2.x str - bytes The problem is that the current string type is used both to represent bytes and characters. Current applications of str need to be studied, and converted appropriately, depending on whether they use str-as-bytes or str-as-characters. The default, in some sense of that word, is that str applications are assumed to operate on character strings; this is achieved by making string literals objects of the character string type. I was also under the impression that str.encode(...) - bytes, bytes.decode(...) - str Correct. and that there would be some magical argument to pass to the file or open open(fn, 'rb', magical_parameter).read() - bytes. I think the precise details of that are still unclear. But yes, the plan is to have two file modes: one that returns character strings (type 'str') and one that returns type 'bytes'. I mention this because I do binary data handling, some ''.join(...) for IO buffers as Guido mentioned (because it is the fastest string concatenation available in Python 2.x), and from this particular conversation, it seems as though Python 3.x is going to lose some expressiveness and power. You certainly need a concatenate list of bytes into a single bytes. Apparently, Guido assumes that this can be done through bytes().join(...); I personally feel that this is over-generalization: if the only practical application of .join is the empty bytes object as separator, I think the method should be omitted. Perhaps bytes(...) or bytes.join(...) could work? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3101: Advanced String Formatting
Aahz wrote: First of all, I recommend that you post this to comp.lang.python. This is the kind of PEP where wide community involvement is essential to success; be prepared for massive revision. Actually, *all* PEPs should be posted to c.l.p at some point; the PEP author is responsible for collecting all feedback, and either updating the specification, or at least, summarizing the discussion and the open issues (so that the same argument isn't made over and over again). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tkinter lockups.
Jeff Epler wrote:
However, on this system, I couldn't recreate the problem you reported
with either the using _tkinter directly instructions, or using this
C test program:
#include tcl.h
#include tk.h
int main(void) {
Tcl_Interp *trp;
unsetenv(DISPLAY);
trp = Tcl_CreateInterp();
printf(%d\n, Tk_Init(trp));
printf(%d\n, Tk_Init(trp));
return 0;
}
The problem only occurs when Tcl and Tk were compiled with
--enable-threads, and it occurs because Tk fails to unlock a mutex
in a few error cases. The patch below fixes the problem.
I'll report it to the Tcl people, and see whether I can work around
in _tkinter.
Regards,
Martin
diff -ur tk8.4.13/generic/tkWindow.c tk8.4.13.modified/generic/tkWindow.c
--- tk8.4.13/generic/tkWindow.c 2006-04-04 23:49:57.0 +0200
+++ tk8.4.13.modified/generic/tkWindow.c2006-04-30 21:41:37.0
+0200
@@ -3108,6 +3108,7 @@
Tcl_DStringFree(class);
if (code != TCL_OK) {
+ Tcl_MutexUnlock(windowMutex);
goto done;
}
Tcl_ResetResult(interp);
@@ -3124,6 +3125,7 @@
Tcl_SetVar(interp, geometry, geometry, TCL_GLOBAL_ONLY);
code = Tcl_VarEval(interp, wm geometry . , geometry, (char *) NULL);
if (code != TCL_OK) {
+ Tcl_MutexUnlock(windowMutex);
goto done;
}
geometry = NULL;
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] methods on the bytes object
Josiah Carlson wrote:
I mean unicode strings, period. I can't imagine what unicode strings
which do not contain data could be.
Binary data as opposed to text. Input to a array.fromstring(),
struct.unpack(), etc.
You can't/shouldn't put such data into character strings: you need
an encoding first. Neither array.fromstring nor struct.unpack will
produce/consume type 'str' in Python 3; both will operate on the
bytes type. So fromstring should probably be renamed frombytes.
Certainly it is the case that right now strings are used to contain
'text' and 'bytes' (binary data, encodings of text, etc.). The problem
is in the ambiguity of Python 2.x str containing text where it should
only contain bytes. But in 3.x, there will continue to be an ambiguity,
as strings will still contain bytes and text (parsing literals, see the
somewhat recent argument over bytes.encode('base64'), etc.).
No. In Python 3, type 'str' cannot be interpreted to contain bytes.
Operations that expect bytes and are given type 'str', and no encoding,
should raise TypeError.
We've not removed the problem, only changed it from being contained
in non-unicode
strings to be contained in unicode strings (which are 2 or 4 times larger
than their non-unicode counterparts).
We have removed the problem.
Within the remainder of this email, there are two things I'm trying to
accomplish:
1. preserve the Python 2.x string type
I would expect that people try that. I'm -1.
2. make the bytes object more pallatable regardless of #1
This might be good, but we have to be careful to not create a type
that people would casually use to represent text.
I do, however, believe that the Python 2.x string type is very useful
from a data parsing/processing perspective.
You have to explain your terminology somewhat better here: What
applications do you have in mind when you are talking about
parsing/processing? To me, parsing always means text, never
raw bytes. I'm thinking of the Chomsky classification of grammars,
EBNF, etc. when I hear parsing.
Look how successful and
effective it has been so far in the history of Python. In order to make
the bytes object be as effective in 3.x, one would need to add basically
all of the Python 2.x string methods to it
The precondition of this clause is misguided: the bytes type doesn't
need to be as effective, since the string type is as effective in 2.3,
so you can do all parsing based on strings.
(having some mechanism to use
slices of bytes objects as dictionary keys (if data[:4] in handler: ...
- if tuple(data[:4]) in handler: ... ?) would also be nice).
You can't use the bytes type as a dictionary key because it is
immutable. Use the string type instead.
So, what to do? Rename Python 2.x str to bytes. The name of the type
now confers the idea that it should contain bytes, not strings.
It seems that you want an immutable version of the bytes type. As I
don't understand what parsing is, I cannot see the need for it;
I think having two different bytes types is confusing.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3102: Keyword-only arguments
Terry Reedy wrote: Are you asking why that feature (keyword-only arguments) is desirable? That's the whole point of the PEP. Or are you asking why the user shouldn't be allowed to pass keyword-only arguments by omitting the keyword? Because they wouldn't be keyword-only arguments then, anymore. There are two subproposals: first, keyword-only args after a variable number of positional args, which requires allowing keyword parameter specifications after the *args parameter, and second, keyword-only args after a fixed number number of positional args, implemented with a naked '*'. To the first, I said The rationale for this is pretty obvious.. To the second, I asked, and still ask, Why?. One reason I see is to have keyword-only functions, i.e. with no positional arguments at all: def make_person(*, name, age, phone, location): pass which also works for methods: def make_person(self, *, name, age, phone, location): pass In these cases, you don't *want* name, age to be passed in a positional way. How else would you formulate that if this syntax wasn't available? (I know it is possible to formulate it elsehow, I'm asking what notation you would use) Again: if a function has a fixed number n of params, why say that the first k can be passed by position, while the remaining n-k *must* be passed by name? I see an important use case for k=0 functions, and k=1 methods (where the only positional argument is self). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3102: Keyword-only arguments
Terry Reedy wrote: Now, suppose you wanted to have 'key' be a keyword-only argument. Why? Why not let the user type the additional argument(s) without the parameter name? Like Martin, you clipped most of the essential context of my question: Talin's second proposal. I clipped it because I couldn't understand your question: Why what? (the second question only gives Why not) I then assumed that the question must have applied to the text that immediately preceded the question - hence that's the text that I left. Now I understand, though. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tkinter lockups.
Thomas Wouters wrote: It seems that, on my platform at least, Tk_Init() doesn't like being called twice even when the first call resulted in an error. That's Tcl and Tk 8.4.12. Tkapp_Init() (which is the Tkinter part that calls Tk_Init()) does its best to guard against calling Tk_Init() twice when the first call was succesful, but it doesn't remember failure cases. I don't know enough about Tcl/Tk or Tkinter how this is best handled, but it would be mightily convenient if it were. ;-) I've created a bugreport on it, and I hope someone with Tkinter knowledge can step in and fix it. (It looks like SF auto-assigned it to Martin already, hmm.) I have now reported the underlying Tk bug at http://sourceforge.net/tracker/index.php?func=detailaid=1479587group_id=12997atid=112997 and worked around it in _tkinter.c. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3102: Keyword-only arguments
Edward Loper wrote: Martin v. Löwis wrote: One reason I see is to have keyword-only functions, i.e. with no positional arguments at all: def make_person(*, name, age, phone, location): pass But is it necessary to syntactically *enforce* that the arguments be used as keywords? This really challenges the whole point of the PEP: keyword-only arguments (at least, it challenges the title of the PEP, although probably not the specified rationale). I.e., why not just document that the arguments should be used as keyword arguments, and leave it at that. Because they wouldn't be keyword-only arguments, then. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3102: Keyword-only arguments
Terry Reedy wrote: This is not a reason for subproposal two, but a special case, as you yourself note below, and hence does say why you want to have such. def make_person(*, name, age, phone, location): pass You weren't asking for a reason, you were asking for an example: this is one. And again, why would you *make* me, the user-programmer, type make_person(name=namex, age=agex, phone=phonex, location = locationx) #instead of make_person(namex,agex,phonex,locationx) ? Because there should be preferably only one obvious way to call that function. Readers of the code should not need to remember the order of parameters, instead, the meaning of the parameters should be obvious in the call. This is the only sane way of doing functions with many arguments. PS. I see that Guido finally gave a (different) use case for bare * that does make sense to me. It's actually the same use case: I don't *want* callers to pass these parameters positionally, to improve readability. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] introducing the experimental pyref wiki
Guido van Rossum wrote: Agreed. Is it too late to also attempt to bring Doc/ref/*.tex completely up to date and remove confusing language from it? Ideally that's the authoritative Language Reference -- admittedly it's been horribly out of date but needn't stay so forever. It's never too late to update the specification. I really think there should be a specification, and I really think it should be as precise as possible - where possible takes both of these into account: - it may get out of date due to lack of contributors. This is free software, and you don't always get what you want unless you do it yourself (and even then, sometimes not). - it might be deliberately vague to allow for different implementation strategies. Ideally, it would be precise in pointing out where it is deliberately vague. So I think the PEPs all should be merged into the documentation, at least their specification parts (rationale, history, examples might stay in the PEPs). To some degree, delivery of documentation can be enforced by making acceptance of the PEP conditional upon creation of documentation patches. Once the feature is committed, we can only hope for (other) volunteers to provide the documentation, or keep nagging the author of the code to produce it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] more pyref: continue in finally statements
Guido van Rossum wrote: Strange. I thought this was supposed to be fixed? (But I can confirm that it isn't.) Perhaps you were confusing it with this HISTORY entry? - A 'continue' statement can now appear in a try block within the body of a loop. It is still not possible to use continue in a finally clause. This was added as r19261 | jhylton | 2001-02-01 23:53:15 +0100 (Do, 01 Feb 2001) | 2 lines Geänderte Pfade: M /python/trunk/Misc/NEWS continue now allowed in try block r19260 | jhylton | 2001-02-01 23:48:12 +0100 (Do, 01 Feb 2001) | 3 lines Geänderte Pfade: M /python/trunk/Doc/ref/ref7.tex M /python/trunk/Include/opcode.h M /python/trunk/Lib/dis.py M /python/trunk/Lib/test/output/test_exceptions M /python/trunk/Lib/test/output/test_grammar M /python/trunk/Lib/test/test_exceptions.py M /python/trunk/Lib/test/test_grammar.py M /python/trunk/Python/ceval.c M /python/trunk/Python/compile.c Allow 'continue' inside 'try' clause SF patch 102989 by Thomas Wouters Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] more pyref: continue in finally statements
Fredrik Lundh wrote: the language reference says: continue may only occur syntactically nested in a for or while loop, but not nested in a function or class definition or finally statement within that loop. /.../ It may occur within an except or else clause. The restriction on occurring in the try clause is implementor's laziness and will eventually be lifted. and it looks like the new compiler still has the same issue: $ python test.py File test.py, line 5: continue SyntaxError: 'continue' not supported inside 'finally' clause how hard would it be to fix this ? (shouldn't the try clause in the note read finally clause, btw? continue within the try suite seem to work just fine...) For the latter: the documentation apparently wasn't fully updated in r19260: it only changed ref7.tex, but not ref6.tex. IOW, it really means to say in the try clause, and it is out-of-date in saying so. As for continue in the 'finally' clause: What would that mean? Given def f(): raise Exception while 1: try: f() finally: g() continue then what should be the meaning of continue here? The finally block *eventually* needs to re-raise the exception. When should that happen? So I would say: It's very easy to fix, just change the message to SyntaxError: 'continue' not allowed inside 'finally' clause :-) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] methods on the bytes object
Josiah Carlson wrote:
Certainly that is the case. But how would you propose embedded bytes
data be represented? (I talk more extensively about this particular
issue later).
Can't answer: I don't know what embedded bytes data are.
Ok. I think I would use base64, of possibly compressed content. It's
more compact than your representation, as it only uses 1.3 characters
per byte, instead of the up-to-four bytes that the img2py uses.
If ease-of-porting is an issue, img2py should just put an
.encode(latin-1) at the end of the string.
return zlib.decompress(
'x\xda\x01\x14\x02\xeb\xfd\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00 \
[...]
That data is non-textual. It is bytes within a string literal. And it
is embedded (within a .py file).
In Python 2.x, it is that, yes. In Python 3, it is a (meaningless)
text.
I am apparently not communicating this particular idea effectively
enough. How would you propose that I store parsing literals for
non-textual data, and how would you propose that I set up a dictionary
to hold some non-trivial number of these parsing literals?
An operationX literal is a symbol that describes how to interpret the
subsequent or previous data. For an example of this, see the pickle
module (portions of which I include below).
I don't think there can be, or should be, a general solution for
all operationX literals, because the different applications of
operationX all have different requirements wrt. their literals.
In binary data, integers are the most obvious choice for
operationX literals. In text data, string literals are.
I described before how you would use this kind of thing to perform
operationX on structured information. It turns out that pickle (in
Python) uses a dictionary of operationX symbols/literals - unbound
instance methods to perform operationX on the pickled representation of
Python objects (literals where = '...' are defined, and symbols
using the names). The relevant code for unpickling is the while 1:
section of the following.
Right. I would convert the top of pickle.py to read
MARK= ord('(')
STOP= ord('.')
...
def load(self):
Read a pickled object representation from the open file.
Return the reconstituted object hierarchy specified in the file.
self.mark = object() # any new unique object
self.stack = []
self.append = self.stack.append
read = self.read
dispatch = self.dispatch
try:
while 1:
key = read(1)
and then this to
key = ord(read(1))
dispatch[key](self)
except _Stop, stopinst:
return stopinst.value
For an example of where people use '...' to represent non-textual
information in a literal, see the '# Protocol 2' section of pickle.py ...
Right.
# Protocol 2
PROTO = '\x80' # identify pickle protocol
This should be changed to
PROTO = 0x80 # identify pickle protocol
etc.
The point of this example was to show that operationX isn't necessarily
the processing of text, but may in fact be the interpretation of binary
data. It was also supposed to show how one may need to define symbols
for such interpretation via literals of some kind. In the pickle module,
this is done in two parts: XXX = literal; dispatch[XXX] = fcn. I've
also seen it as dispatch = {literal: fcn}
Yes. For pickle, the ordinals of the type code make good operationX
literals.
See any line-based socket protocol for where .find() is useful.
Any line-based protocol is textual, usually based on ASCII.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] more pyref: a better term for string conversion
Fredrik Lundh wrote: for some reason, the language reference uses the term string con- version for the backtick form of repr: http://docs.python.org/ref/string-conversions.html any suggestions for a better term ? should backticks be deprecated, and documented in terms of repr (rather than the other way around) ? I vaguely recall that they are deprecated, but I can't remember the details. The one obvious way to invoke that functionality is the repr() builtin. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python for Windows CE
Luke Dunstan wrote: 1. Is there any reason in principle why patches for Windows CE support would be rejected? No, not in principle. Of course, - the patch shouldn't break anything - you should state an explicit commitment to support the port for some years (or else it might get removed at the slightest problem) 2. Should I submit unified diffs or context diffs? Yes, please :-) Actually, unified diffs are common these days, in particular as subversion generates them. 3. The page http://www.python.org/dev/patches/style/ says that platform-specific code should use preprocessor symbols documented for the compiler, but existing Python code uses e.g. the invented MS_WIN32 symbol instead of the standard _WIN32. Should we create a symbol MS_WINCE for consistency or use the more common _WIN32_WCE? Depends on who defines it. If the compiler defines it on its own, it is best to use that. If some header file that gets unconditionally included defines it, that is just as well. If you explicitly define something, try to follow the conventions you like best. 4. The latest existing patch set uses os.name = ce, and this can be used to distinguish Windows CE from other operating systems in Python code. The existing patches also set sys.platform = Pocket PC, but I am not so keen on keeping this, mainly because Windows CE is equivalent to Win32 in many cases. Any comments? I would guide the decision based on the number of API changes you need to make to the standard library (e.g. distutils). In any case, the platform module should be able to reliably report the specific system (including the specific processor architecture). (b) Instead we could use #ifdef HAVE_DIRECT_H, requiring patches to #define HAVE_DIRECT_H for other platforms That would be best. Python generally uses autoconf methodology, which implies that conditionally-existing headers should be detected using HAVE_*_H. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python long command line options
Fredrik Lundh wrote: I'm +1 on adding --help and --version, +1 on adding -? and /? for windows only, -0=slightly sceptical if adding a generic long option machinery is worth it, and -1 on a guessing machinery. I also agree with all these judgments. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python sprint mechanics
Paul Moore wrote: Is it possible to create a branch in the main Python svn, and grant commit privs to that branch only, for sprint participants? I seem to recall something like mod_authzsvn being involved, but I don't know much more... We couldn't technically enforce it - but I'm sure sprint participants would restrict checkins to a branch if they were told to. However, I don't see how that would help. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python sprint mechanics
Benji York wrote: I'm not familiar with the mechanics, recent versions of Subversion allow per-directory security. We do this to give some customers read access to parts of the repo, and read-write to others. It shouldn't be difficult (given a recent enough Subversion) to set up a sprint area in the repo. It works fine for http(s), but not for svn+ssh. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] binary trees. Review obmalloc.c
Vladimir 'Yu' Stepanov wrote: Yes. I understood it when resulted a set example. However, as I just said, people usually don't remove items from just-sorted lists, they tend to iterate over them via 'for i in list:' . Such problem arises at creation of the list of timers. For a list of timers/timeout events, a priority queue is often the best data structure, which is already available through the heapq module. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python sprint mechanics
Greg Ewing wrote: Tim Peters wrote: Instead it would make best sense for each sprint project to work in its own branch, something SVN makes very easy, but only for those who _can_ commit. There's no way of restricting commit privileges to a particular branch? In the current setup, not easily. However, sprinters would certainly restrain themselves to the branches they are told to use. IOW, technical mechanisms for fine-tuned write access are often besides the point: you first need to give these people any kind of write access at all, and then you find that further restricting that can readily be done without any technical enforcement. Or, to put it yet in a different way: whether or not commit privileges are restricted, you need to add the sprinters to the committers list first, unless you want to allow anonymous commits to these branches. Just to not be mistaken: it is technically fairly easy to add somebody to the committers list. So technical, there is no problem to add all sprinters. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
