Re: [Python-Dev] PEP 3145 (With Contents)
Please stop copying me on this thread. Thanks, Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3145 (With Contents)
Please stop copying me on this thread. Thanks, Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] readd u'' literal support in 3.3?
On 09:37 pm, tjre...@udel.edu wrote: On 12/13/2011 10:54 AM, Vinay Sajip wrote: I started writing a tool today, tentatively called '2to23', which aims to do this. It's basically 2to3, but with a package of custom fixers in a package 'lib2to23.fixers' adapted from the corresponding fixers in lib2to3. When, some year in the future, people want to drop Python 2 compatibility from their Python23 code, they will need a 23to3 tool. No, they will not. They only need a 2to3 or 2to6 tool because Python 2 and Python 3 are not compatible with each other, but they want one program to be valid in Python 2 and Python 3 simultaneously. When they decide they no longer care about Python 2, they can just stop taking care to keep their program valid as Python 2 and only take care to keep it a valid Python 3 program. There's no specific change to make, just a different approach to take with future maintenance. You might say that they will *want* to immediately discard all of their legacy Python 2 support code. I suspect many of them will not want this; but either way it's a want, not a need. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Deprecation policy
On 12:14 pm, st...@pearwood.info wrote: Xavier Morel wrote: Not being too eager to kill APIs is good, but giving rise to this kind of living-dead APIs is no better in my opinion, even more so since Python has lost one of the few tools it had to manage them (as DeprecationWarning was silenced by default). Both choices are harmful to users, but in the long run I do think zombie APIs are worse. I would much rather have my code relying on zombie APIs and keep working, than to have that code suddenly stop working when the zombie is removed. Working code should stay working. Unless the zombie is actively harmful, what's the big deal if there is a newer, better way of doing something? If it works, and if it's fast enough, why force people to fix it? It is a good thing that code or tutorials from Python 1.5 still (mostly) work, even when there are newer, better ways of doing something. I see a lot of newbies, and the frustration they suffer when they accidentally (carelessly) try following 2.x instructions in Python3, or vice versa, is great. It's bad enough (probably unavoidable) that this happens during a major transition like 2 to 3, without it also happening during minor releases. Unless there is a good reason to actively remove an API, it should stay as long as possible. I don't like this and it should go is not a good reason, nor is but there's a better way you should use. When in doubt, please don't break people's code. +1 Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] unittest missing assertNotRaises
On 27 Sep, 11:58 pm, ckay...@zindagigames.com wrote: On Tue, Sep 27, 2011 at 4:43 PM, Steven D'Aprano st...@pearwood.info wrote: But I can't see this being a useful test. As written, exceptions are still treated as errors, except for excClass, which is treated as a test failure. I can't see the use-case for that. assertRaises is useful: IOError is allowed, but any other exception is a bug. makes perfect sense. assertNotRaises doesn't seem sensible or useful to me: IOError is a failed test, but any other exception is a bug. What's the point? When would you use that? I've run across a few cases where this is the correct behavior. The most recent one that comes to mind is while testing some code which has specific silencing options: specifically, writing a main file and a backup file, where failure to write the backup is not an error, but failure to write the main is. As such, the test suite should have the following tests: - Failure to write the main should assert that the code raises the failure error. No error is a failure, any other error is an error, that error is a success. (it may also check that the backup was written) This is assertRaises, not assertNotRaises. - Failure to write the backup should assert that the code does not raise the failure error. No error is a success, that error is a failure, any other error is a error. (it may also check that the main was written) This is calling the function and asserting something about the result. - Both succeeding should assert that the files were actually written, and that no error was raised. Any other result is an error. Now, the difference between a Failure and an Error is more or less a mute point, however I would expect an Error to be any unexpected result, while a Failure is a predicted (either via forethought or prior tests) but incorrect result. assertNotRaises doesn't make anything possible that isn't possible now. It probably doesn't even make anything easier - but if it does, it's so obscure (and I've read and written thousands of tests for all kinds of libraries over the years) that it doesn't merit a dedicated helper in the unittest library. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should we move to replace re with regex?
On 26 Aug, 09:45 pm, gu...@python.org wrote: I just made a pass of all the Unicode-related bugs filed by Tom Christiansen, and found that in several, the response was this is fixed in the regex module [by Matthew Barnett]. I started replying that I thought that we should fix the bugs in the re module (i.e., really in _sre.c) but on second thought I wonder if maybe regex is mature enough to replace re in Python 3.3. It would mean that we won't fix any of these bugs in earlier Python versions, but I could live with that. However, I don't know much about regex -- how compatible is it, how fast is it (including extreme cases where the backtracking goes crazy), how bug-free is it, and so on. Plus, how much work would it be to actually incorporate it into CPython as a complete drop-in replacement of the re package (such that nobody needs to change their imports or the flags they pass to the re module). We'd also probably have to train some core developers to be familiar enough with the code to maintain and evolve it -- I assume we can't just volunteer Matthew to do so forever... :-) What's the alternative? Is adding the requested bug fixes and new features to _sre.c really that hard? What about other Python implementations (ie, PEP 399)? For this to be seriously considered, shouldn't there also be a pure Python implementation of the functionality? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add from __experimental__ import bla [was: Should we move to replace re with regex?]
On 07:57 pm, digitalx...@gmail.com wrote: In the thread about replacing re with regex someone mentioned adding to __future__ which isnt a great idea as future APIs are already solidified, they just live there to give developer time to adapt their code. The idea of a __experimental__ area is good for any pep's or stliib additions that are somewhat controversial (API isnt agreed on, code may take a while to integrate properly, developer wants some time to hash out any edge case bugs or API clarifications that may come up in large scale testing, etc). __experimental__ should emit a warning on import that says anything in here may change or be removed at any time and should not be used in stable code. __experimental__ features should behave the same as __future__ in that they can add new keywords or semantics to the existing language __experimental__ features can move directly to the stlib or builtins if they do not add new keywords and/or are backwards compatible with the feature they are replacing. Otherwise they move into __future__ for how ever many releases are deemed reasonable time for developers to adapt their code. Hi Dj, As a developer of Python libraries and applications, I don't see how this would make my life easier. A warning in a module docstring that a module may not be long-lived if it is not well received tells me just as much as a warning emitted at runtime. And a warning emitted at runtime is likely to scare my users into thinking something is broken, leading to spurious or misleading bug reports. There also does not appear to be general consensus that modules should be added to stdlib if they are not widely used and demanded, so I don't know when a module would be added to __experimental__, anyway. The normal deprecation procedures (rarely used as they are) seem to cover this, anyway. Adding a new namespace separate from __future__ also just gives me another thing to remember. Was the feature added to __experimental__ or __future__? Also, it seems even less common that language features are added on an experimental basis. When a language feature (new syntax or semantics) goes in to the language, it is there for a long, long time. If new features are added first to __experimental__ and then to __future__ or the non-__experimental__ stdlib namespace, then I just have to update all my code to keep using it. So I'm guaranteed extra work whether the feature is successful and is adopted or if it fails and is later removed. I'd rather not have to do the extra work in the success case, at least, which is what the existing add-it-and-then-maybe -(but-probably-not-)deprecate it approach gives me. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python- dev/exarkun%40twistedmatrix.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Snow Leopard buildbot failing again...
On 05:01 pm, jans...@parc.com wrote: I see that parc-snowleopard-1 went down again. I've done a software update, rebooted, and installed the latest buildslave, 0.8.4. I can ping dinsdale.python.org successfully from the machine. However, when I start the buildslave, I get this: [snip] The buildslave took more than 10 seconds to start, so we were unable to confirm that it started correctly. Please 'tail twistd.log' and look for a line that says 'configuration update complete' to verify correct startup. Did you look at twistd.log? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.x and bytes
On 12:35 am, ncogh...@gmail.com wrote: On Tue, Jun 14, 2011 at 9:40 AM, P.J. Eby p...@telecommunity.com wrote: You can still do it one at a time: CHAR, = b'C' INT, �= b'I' ... etc. �I just tried it with Python 3.1 and it works there. I almost mentioned that, although it does violate one of the unwritten rules of the Zen (in this case, syntax shall not look like grit on Tim's monitor) [CHAR] = b'C' [INT] = b'I' ... Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The socket HOWTO
On 5 Jun, 10:35 pm, mar...@v.loewis.de wrote: First, Twisted doesn't always use the BSD sockets API; the Windows IOCP reactor, especially, starts off with the socket() function, but things go off in a different direction pretty quickly from there. Hmm. Are you saying it doesn't use listen, connect, bind, send, recv? To me, that's the core of BSD sockets. I can understand it doesn't use select(2). Yes, that's correct. Those aren't the best APIs to use on Windows, so they aren't necessarily used on Windows. So it's perfectly fine to introduce yourself to networking via Twisted, and many users have done just that. If you're using it idiomatically, you should never encounter a socket object or file descriptor poking through the API anywhere. And that's all fine. I still claim that you have to *understand* sockets in order to use it properly. By this, I mean stuff like what is a TCP connection? how is it established?, how is UDP different from TCP?, when data arrives, what layers of software does it go through?, what is a port number?, etc. These may be good things to understand. The current socket howto doesn't explain them, though. Second, it makes me a little sad that it appears to be folk wisdom that Twisted is only for servers. A lot of work has gone into making it equally appropriate for clients. This is especially true if your client has a GUI, where Twisted is often better than a protocol-specific library, which may either be blocking or have its own ad-hoc event loop. I think that's because many of the problems that Twisted solves don't exist in many of the client applications - in particular, you often don't have many simultaneous connections. GUI apps may be the interesting special case, but I expect that people dealing with these rather use separate threads. On the contrary, many of the problems do exist in client applications (every time I have to use virt-manager I want to throw it out a window). Some people certainly would rather use threading, but that doesn't say anything about whether Twisted solves problems relevant to clients, only about the fact that a lot of people like to use threads. I don't have an opinion on the socket HOWTO per se, only on the possibility of linking to Twisted as an alternate implementation mechanism. It really would be better to say go use Twisted rather than reading any of the following than read the following, which will help you understand Twisted. Wouldn't you agree that Twisted is very difficult to learn, and thus much heavier than sockets? And I don't blame the Twisted API for that, but rather the mental model of overlapping activities that people have severe problems with. This discussion has a significant problem, in taking Twisted as a monolithic all-or-nothing entity. Restricting the scope to merely the lowest-level socket replacement APIs - ie, the bare TCP, UDP, etc functionality - no, Twisted is not very difficult to learn. Expanding the scope to include the higher level functionality, it is much easier to learn than reimplementing line parsing and concurrency and so forth. However, does that really have anything to do with improving the socket howto? The Python documentation can include a clear explanation of what functionality the socket module provides - without forcing you to read Stevens _or_ use Twisted, but it can still refer you to both, since it is very likely that you'll need at least one of them in addition to the socket module. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why are there no 'set' and 'frozenset' types in the 'types' module?
On 02:01 pm, ha...@interia.pl wrote: Because there's no reason to include them, since they are already in the root (builtins) namespace. You'll notice that in Python 3, the types module only contains types which are not obviously accessed through easier means: OK, makes sense, but in this case it would be handy to have some list of all possible built-in types. I was just creating one and I nearly missed sets. If an entry in 'types' module is too much, there should be some comprehensive list in the documentation at least. Maybe this is what you're after? pprint([t for t in object.__subclasses__() if t.__module__ == '__builtin__']) [type 'type', type 'weakref', type 'weakcallableproxy', type 'weakproxy', type 'int', type 'basestring', type 'bytearray', type 'list', type 'NoneType', type 'NotImplementedType', type 'traceback', type 'super', type 'xrange', type 'dict', type 'set', type 'slice', type 'staticmethod', type 'complex', type 'float', type 'buffer', type 'long', type 'frozenset', type 'property', type 'tuple', type 'enumerate', type 'reversed', type 'code', type 'frame', type 'builtin_function_or_method', type 'instancemethod', type 'function', type 'classobj', type 'dictproxy', type 'generator', type 'getset_descriptor', type 'wrapper_descriptor', type 'instance', type 'ellipsis', type 'member_descriptor', type 'EncodingMap', type 'module', type 'classmethod', type 'file'] Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #11223: Add threading._info() function providing informations about the
On 08:20 am, victor.stin...@haypocalc.com wrote: Hi, Le mardi 19 avril 2011 � 22:42 -0400, Terry Reedy a �crit : On 4/19/2011 5:59 PM, victor.stinner wrote: Issue #11223: Add threading._info() function providing informations about the thread implementation. Since this is being documented, making it part of the public api, why does it have a leading underscore? Can I propose something wildly radical? Maybe the guarantees made about whether an API will be available in future versions of Python (ostensibly what public vs private is for) should not be tightly coupled to the decision about whether to bother to explain what an API does? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython: Issue #11223: Add threading._info() function providing informations about the
On 01:11 pm, benja...@python.org wrote: 2011/4/20 exar...@twistedmatrix.com: On 08:20 am, victor.stin...@haypocalc.com wrote: Hi, Le mardi 19 avril 2011 � 22:42 -0400, Terry Reedy a �crit : On 4/19/2011 5:59 PM, victor.stinner wrote: Issue #11223: Add threading._info() function providing informations about the thread implementation. Since this is being documented, making it part of the public api, why does it have a leading underscore? Can I propose something wildly radical? Maybe the guarantees made about whether an API will be available in future versions of Python (ostensibly what public vs private is for) should not be tightly coupled to the decision about whether to bother to explain what an API does? With what criteria would you propose to replace it with? I'm not sure what kind of criteria you're thinking of. I'm only suggesting that: 1) Document whatever you want (preferably as much as possible) 2) Make privateness defined by whether there is a leading underscore It is a big mistake to think that documentation isn't necessary for things just because you don't want application developers to use them. Maintainers benefit from it just as much. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements
On 16 Apr, 11:03 pm, st...@pearwood.info wrote: Brett Cannon wrote: In the grand python-dev tradition of silence means acceptance, I consider this PEP finalized and implicitly accepted. How long does that silence have to last? I didn't notice a definition of what counts as 100% branch coverage. Apologies if I merely failed to notice it, but I think it should be explicitly defined. Presumably it means that any time you have an explicit branch (if...elif...else, try...except...else, for...else, etc.) you need a test that goes down each branch. But it isn't clear to me whether it's sufficient to test each branch in isolation, or whether you need to test all combinations. That is, if you have five branches, A or B, C or D, E or F, G or H, I or J, within a single code unit (function? something else?), is it sufficient to have at least one test that goes down each of A...J, or do you need to explicitly test each of: A-C-E-G-I A-C-E-G-J A-C-E-H-I A-C-E-H-J A-C-F-G-I ... B-D-F-H-J (10 tests versus 32 tests). If the latter, this could become impractical *very* fast. But if not, I don't see how we can claim 100% coverage when there are code paths which are never tested. The mostly commonly used definition of branch coverage is that each outcome of each individual branch is executed, not that all possible combinations of all branches in a unit are executed. I haven't heard anyone in this thread propose the latter, only the former. 100% coverage by itself is certainly ambiguous. At the very least, I think you need to explicitly define what you mean by 100% branch coverage. Possibly this will assist in the disagreement between you and Antoine re 100% versus comprehensive coverage. I suspect that everyone who has said branch coverage in this thread has intended the definition given above (and encourage anyone who meant something else to clarify their position). Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 396, Module Version Numbers
On 04:02 am, p...@telecommunity.com wrote: At 08:52 AM 4/10/2011 +1000, Ben Finney wrote: This is an often-overlooked case, I think. The unspoken assumption is often that ``setup.py`` is a suitable place for the overall version string, but this is not the case when that string must be read by non-Python programs. If you haven't used the distutils a lot, you might not realize that you can do this: $ python setup.py --version 0.6c12 (The --name option also works, and they can be used together -- the answers will be on two separate lines.) This only works as long as setup.py is around - which it typically no longer is after installation is complete. And though it's common and acceptable enough to launch a child process in a shell script in order to get some piece of information, it isn't as pleasant in a Python program. Can you get this version information out of setup.py without running a child process and without monkey-patching sys.argv and sys.stdout? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot problems again...
On 08:31 pm, jans...@parc.com wrote: My Intel Snow Leopard 2 build slave has gone into outer-space again. When I look at it, I see buildslave taking up most of a CPU (80%), and nothing much else going on. The twistd log says: [... much omitted ...] 2011-04-04 08:35:47-0700 [-] sending app-level keepalive 2011-04-04 08:45:47-0700 [-] sending app-level keepalive 2011-04-04 08:55:47-0700 [-] sending app-level keepalive 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] Lost connection to dinsdale.python.org:9020 2011-04-04 09:03:15-0700 [Broker,client] twisted.internet.tcp.Connector instance at 0x101629ab8 will retry in 3 seconds 2011-04-04 09:03:15-0700 [Broker,client] Stopping factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:18-0700 [-] Starting factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:18-0700 [-] Connecting to dinsdale.python.org:9020 2011-04-04 09:03:18-0700 [Uninitialized] Connection to dinsdale.python.org:9020 failed: Connection Refused 2011-04-04 09:03:18-0700 [Uninitialized] twisted.internet.tcp.Connector instance at 0x101629ab8 will retry in 8 seconds 2011-04-04 09:03:18-0700 [Uninitialized] Stopping factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:27-0700 [-] Starting factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:27-0700 [-] Connecting to dinsdale.python.org:9020 So it's been spinning its wheels for 3 days. Does this mean that the 2011-04-04 09:03:27-0700 [-] Connecting to dinsdale.python.org:9020 message in the logs is the last one you see until you restart the slave? Or does it mean that the logs go on and on for three days with these Connecting to dinsdale / Connection Refused / ... will retry in N seconds cycles, thousands and thousands of times? What does the buildmaster's info page for this slave say when the slave is in this state? In particular, what does it say about connects/hour? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] funky buildbot problems again...
On 12:07 am, jans...@parc.com wrote: exar...@twistedmatrix.com wrote: On 08:31 pm, jans...@parc.com wrote: My Intel Snow Leopard 2 build slave has gone into outer-space again. When I look at it, I see buildslave taking up most of a CPU (80%), and nothing much else going on. The twistd log says: [... much omitted ...] 2011-04-04 08:35:47-0700 [-] sending app-level keepalive 2011-04-04 08:45:47-0700 [-] sending app-level keepalive 2011-04-04 08:55:47-0700 [-] sending app-level keepalive 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] lost remote 2011-04-04 09:03:15-0700 [Broker,client] Lost connection to dinsdale.python.org:9020 2011-04-04 09:03:15-0700 [Broker,client] twisted.internet.tcp.Connector instance at 0x101629ab8 will retry in 3 seconds 2011-04-04 09:03:15-0700 [Broker,client] Stopping factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:18-0700 [-] Starting factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:18-0700 [-] Connecting to dinsdale.python.org:9020 2011-04-04 09:03:18-0700 [Uninitialized] Connection to dinsdale.python.org:9020 failed: Connection Refused 2011-04-04 09:03:18-0700 [Uninitialized] twisted.internet.tcp.Connector instance at 0x101629ab8 will retry in 8 seconds 2011-04-04 09:03:18-0700 [Uninitialized] Stopping factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:27-0700 [-] Starting factory buildslave.bot.BotFactory instance at 0x1016299e0 2011-04-04 09:03:27-0700 [-] Connecting to dinsdale.python.org:9020 So it's been spinning its wheels for 3 days. Does this mean that the 2011-04-04 09:03:27-0700 [-] Connecting to dinsdale.python.org:9020 message in the logs is the last one you see until you restart the slave? Yes, that's the last line in the file. Or does it mean that the logs go on and on for three days with these Connecting to dinsdale / Connection Refused / ... will retry in N seconds cycles, thousands and thousands of times? Well, it's doing something, chewing up cycles, but there's only one Connecting line at the end of the log file. That's very interesting. It may be worth doing some gdb or dtrace investigation next time it gets into this state. What does the buildmaster's info page for this slave say when the slave is in this state? In particular, what does it say about connects/hour? Ah, good question. Too bad I restarted the slave after I sent out my info. Is there some way to recover that from earlier? If not, it will undoubtedly fail again in a few days. If the master logs are available, that would provide some information. Otherwise, I think waiting for it to happen again is the thing to do. Since there were no other messages in the log file, I expect the connects/hour value will be low - perhaps 0. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Supporting Visual Studio 2010
On 09:55 am, mar...@v.loewis.de wrote: Am 05.04.2011 00:21, schrieb Antoine Pitrou: On Mon, 04 Apr 2011 23:40:33 +0200 Martin v. L�wis mar...@v.loewis.de wrote: - users have expressed concerns that they constantly need to upgrade VS releases when developing for Python. Isn't that kind of a misguided argument? It's not Python who decides the lifecycle of MSVC releases, it's Microsoft. We can't be blamed for the churn. But we *can* be blamed for closely following the MS release cycle (if we actually did that). For Python 3.2, we resisted. If getting old (Microsoft-unsupported) MSVC releases is difficult, then I think switching to the newest MSVC as soon as possible is the best strategy, since it minimizes the annoyance for people wanting to build extensions several years after a release is made. OTOH, the very same people will have to buy licenses for all MSVC releases. If we manage to skip some of them, the zoo of products you need to install to support Python gets smaller. Recent Visual Studio Express editions are available as free downloads. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Supporting Visual Studio 2010
On 09:58 am, mar...@v.loewis.de wrote: Won't that still be an issue despite the stable ABI? Extensions on Windows should be linked to the same version of MSVCRT used to compile Python Not if they use the stable ABI. There still might be issues if you mix CRTs, but none related to the Python ABI - in particular, none of those crashing conditions can arise from the stable ABI. Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously? That would be great. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Supporting Visual Studio 2010
On 5 Apr, 07:58 pm, mar...@v.loewis.de wrote: Does this mean new versions of distutils let you build_ext with any C compiler, instead of enforcing the same compiler as it has done previously? No, it doesn't. distutils was considered frozen, and changes to it to better support the ABI where rejected. How about distutils2 then? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Core Mentorship program
On 12:03 pm, jnol...@gmail.com wrote: Hello everyone: The new list will also have a closed, members-only archive. After consulting with other core developers, we believe it's easier to ask questions when you don't have to worry about Google picking up your words from a public archive. Boggle. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 395: Module Aliasing
On 03:30 pm, ncogh...@gmail.com wrote: Fixing dual imports of the main module -- Two simple changes are proposed to fix this problem: 1. In ``runpy``, modify the implementation of the ``-m`` switch handling to install the specified module in ``sys.modules`` under both its real name and the name ``__main__``. (Currently it is only installed as the latter) 2. When directly executing a module, install it in ``sys.modules`` under ``os.path.splitext(os.path.basename(__file__))[0]`` as well as under ``__main__``. With the main module also stored under its real name, imports will pick it up from the ``sys.modules`` cache rather than reimporting it under a new name. Something to consider here is how this will interact with Python files which are _not_ modules. I'm a little uneasy about having sys.modules[trial] refer to the module defined by /usr/bin/trial. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding buildbot failures
On 01:16 pm, fuzzy...@voidspace.org.uk wrote: On 25/02/2011 19:00, exar...@twistedmatrix.com wrote: On 06:47 pm, fuzzy...@voidspace.org.uk wrote: On 25/02/2011 18:10, Vinay Sajip wrote: What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser? That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though. Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from. That would be a great improvement to the Python buildbot infrastructure. (Probably a bit small for a GSOC project on its own.) I've had great success using pyjunitxml to generate Hudson compatible reports from unittest test runs (extremely easy to use), and if buildbot *can* handle junit xml format reports then it may be the path of least resistance: https://launchpad.net/pyjunitxml I tried searching (both google and the buildbot docs) for ways to generate more complete reports or to integrate junitxml with builbot, but failed. Can you point me at anything? I think this is the relevant pages in the buildbot manual for custom reporting: http://buildbot.net/buildbot/docs/latest/BuildStep-LogFiles.html #BuildStep-LogFiles There's also http://buildbot.net/buildbot/docs/latest/SubunitShellCommand.html#SubunitShellCommand which is basically the same idea as junitxml, but not actually junitxml itself, so I'd probably go that way. However if you really need junitxml for something, there are tools for converting between subunit and junitxml. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Finding buildbot failures
On 06:47 pm, fuzzy...@voidspace.org.uk wrote: On 25/02/2011 18:10, Vinay Sajip wrote: What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser? That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though. Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 14 Feb, 10:15 pm, greg.ew...@canterbury.ac.nz wrote: Giampaolo Rodol� wrote: for me it should also fit one crucial requirement: it should be *simple* and reflect the simplicity and taste of all other stdlib modules, and to fulfill such a requirement I think Twisted probably needs to be adapted a bit. My thoughts exactly -- from a bird's eye view, Twisted appears to be very far from simple. While there may be some good ideas to adopt from it, I suspect that finding them will require just as much careful thought as designing an API from scratch. Can you try to be more constructive? Generalizations like this don't help the process at all. They don't explain why Twisted's APIs shouldn't be adopted in the stdlib and they don't explain what APIs _should_ be adopted. It's basically just stop energy. I'm not picking on Giampaolo because despite the small portion of his message you quoted, his full email actually contained quite a bit of useful technical information. It wasn't just an unsupported snipe. As far as the difficulties of finding the good ideas in Twisted goes, there are several people familiar with Twisted already contributing to this thread. Between us all, I'm sure we can dig out the insidiously buried secrets. As I mentioned before, I've also started a PEP myself to lay bare the mysteries. I may try working on it some more, since there seems to be some interest. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 08:06 pm, greg.ew...@canterbury.ac.nz wrote: exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: I was thinking of something lighter-weight than that. Twisted Core I just had a look at the docs for Twisted Core, and it lists 10 sub-modules. The only one that really looks core to me is twisted.internet. Drilling into that reveals another 39 public sub-sub-modules and 10 private ones. Sorry, but you'll have to chop it back quite a bit more than that before it's focused enough to be a stlib module, I think. Excluding stuff is not hard, seriously. It's not hard to see that wxPython integration doesn't belong in the stdlib. There are more useful aspects of the task to discuss. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. I was thinking of something lighter-weight than that. Twisted Core Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 12:13 am, p.f.mo...@gmail.com wrote: On 12 February 2011 23:10, exar...@twistedmatrix.com wrote: On 10:46 pm, greg.ew...@canterbury.ac.nz wrote: Antoine Pitrou wrote: On Sun, 13 Feb 2011 11:19:06 +1300 Greg Ewing greg.ew...@canterbury.ac.nz wrote: So maybe it's time to design a new module with a better API and deprecate the old one? That's called Twisted. I was thinking of something lighter-weight than that. Twisted Core Is anyone willing to package up Twisted Core for stdlib inclusion, then? Paul. Do people want to seriously consider deprecating asyncore and adding a replacement for it to the stdlib? (Hey, PyCon is coming up. How convenient. :) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88395 - python/branches/py3k/Lib/asyncore.py
On 12:34 am, stutzb...@google.com wrote: On Sat, Feb 12, 2011 at 4:22 PM, exar...@twistedmatrix.com wrote: Do people want to seriously consider deprecating asyncore and adding a replacement for it to the stdlib? (Hey, PyCon is coming up. How convenient. :) The desire is there, but it's a hard problem. There was a similar discussion before PyCon 2009, but not much came of it: http://mail.python.org/pipermail/python-dev/2009-March/086678.html I started working on a PEP last year, but I didn't get very far partly because I doubted the desire. What part do you think is a hard problem? Convincing people to switch to a new API? *Defining* the new API doesn't seem very hard to me. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
On 12:43 pm, ncogh...@gmail.com wrote: On Wed, Feb 9, 2011 at 10:11 PM, Mark Shannon ma...@dcs.gla.ac.uk wrote: OK, so UnicodeError_xxx is important for codecs, but surely this sort of argument could be made for lots of things. Don't forget that for each function added to the API, all other implementations have to support it forever. Other implementations that want to support CPython extensions should focus their efforts on the limited API defined in PEP 384. That will not only be a lot easier, it will also be less of a moving target. And will produce what kind of results? How many extension libraries work with this subset? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
On 01:59 pm, ncogh...@gmail.com wrote: On Wed, Feb 9, 2011 at 11:03 PM, exar...@twistedmatrix.com wrote: On 12:43 pm, ncogh...@gmail.com wrote: On Wed, Feb 9, 2011 at 10:11 PM, Mark Shannon ma...@dcs.gla.ac.uk wrote: OK, so UnicodeError_xxx is important for codecs, but surely this sort of argument could be made for lots of things. Don't forget that for each function added to the API, all other implementations have to support it forever. Other implementations that want to support CPython extensions should focus their efforts on the limited API defined in PEP 384. That will not only be a lot easier, it will also be less of a moving target. And will produce what kind of results? How many extension libraries work with this subset? Right now? Very few, given the changes to the way types need to be created. But prioritising it will speed convergence over time as more extension modules cut over to it for the stable ABI benefits. And, since the C API has never been anywhere near as tightly controlled as the language definition, alternative implementations are going to garner more sympathy if they restrict their concerns to the growth of the stable ABI rather than worrying about an implementation detail of CPython. Sympathy, perhaps. But that doesn't mean people will drop everything and rewrite their extension modules. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import and unicode: part two
On 09:22 am, catch-...@masklinn.net wrote:
On 2011-01-25, at 04:26 , Toshio Kuratomi wrote:
* If you can pick a set of encodings that are valid (utf-8 for Linux
and
MacOS
HFS+ uses UTF-16 in NFD (actually in an Apple-specific variant of NFD).
Right here you've already broken Python modules on OSX.
Are you sure about the UTF-16 part? Evidence strongly points towards
UTF-8:
$ python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple
Inc. build 5646)] on darwin
Type help, copyright, credits or license for more information.
import unicodedata, os
file(u'\N{SNOWMAN}', 'w').close()
os.listdir('.')
['\xe2\x98\x83']
unicodedata.name('\xe2\x98\x83'.decode('utf-8'))
'SNOWMAN'
Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add sendfile() to core?
On 9 Jan, 08:09 pm, g.rod...@gmail.com wrote: A strong +1. Projects such as Twisted would certainly benefit from such an addiction. Eh. There would probably be some benefits, but I don't think they would be very large in the majority of cases. Also, since adding it to 2.x would be prohibited, it will be at least several years before Twisted actually benefits from its addition to the standard library. Plus, Pavel Pergamenshchik wrapped sendfile for Twisted already many years ago, but no one was interested enough to actually land the change in trunk. However, if it would help, I'm sure Pavel's code can be contributed to CPython. If anyone would like to take a look: http://twistedmatrix.com/trac/browser/branches/sendfile-585-4/twisted/python/test/test_sendfile.py http://twistedmatrix.com/trac/browser/branches/sendfile-585-4/twisted/python/_sendfile.c Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fault handler updated, now disabled by default
On 25 Dec, 10:31 pm, mer...@netwok.org wrote: faulthandler is a module: enable the handler is simple as import faulthandler. That sounds like a source of unwanted behavior (aka problems) if the handler is enabled by 1Cpydoc faulthandler 1D or by a pkgutil walk. You may want to consider using a function to enable the functionality (and add one to disable it). Enormous +1. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues
On 02:00 pm, solip...@pitrou.net wrote: On Thu, 16 Dec 2010 15:52:19 +0200 Dimitrios Pritsos dprit...@extremepro.gr wrote: Hello Michael, OK I will do sent it to the bug tracker. But what about the last issue i.e. that even if the class is transfered-and-pickled-unpickled it raises an exception if the class is defined into the __main__ script and not imported from an other one. (see below) could you please guide me where should I post this for someone to help me fix it! The likely explanation is that the __main__ module in the parent process isn't the same as in the child process, so fetching the class from that module doesn't work. If that's really important for you you could open a separate issue. And another option is to use the simple, pleasant fix of not defining any functions or classes in __main__. Define them in a real module and import them for use in __main__. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On 05:02 pm, solip...@pitrou.net wrote: On Fri, 17 Dec 2010 00:52:14 +0800 Senthil Kumaran orsent...@gmail.com wrote: Actually, it is turning out to be true: http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response According to HTTP 1.0, When a request is Simple-Request, it means a VERB URL (without a version) and it generally corresponds to HTTP 0.9 And when a server receives such a Simple-Request, it sends a Simple-Response where it does not send the headers back. I think, the same is exhibited by other Servers as well www.google.com, www.mozilla.org where for Invalid Request without version, you are sending a Simple-Request (HTTP 0.9) style and getting the corresponding response. Yes, but only error or redirect responses: $ nc www.google.fr 80 GET / HTTP/1.0 302 Found Location: http://www.google.fr/ [etc.] Note that by using `nc` to test this, you're already generating an illegal request (unless you're doing something very special with your keyboard ;). When you hit return, nc will send \n. However, lines are delimited by \r\n in HTTP. I doubt this makes a difference to the point being discussed, but it _could_. I suggest performing your tests with telnet, instead. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OpenSSL Voluntarily (openssl-1.0.0a)
On 08:02 am, solip...@pitrou.net wrote: Le mardi 23 novembre 2010 � 20:56 -0500, Glyph Lefkowitz a �crit : On Nov 23, 2010, at 9:02 AM, Antoine Pitrou wrote: On Tue, 23 Nov 2010 00:07:09 -0500 Glyph Lefkowitz gl...@twistedmatrix.com wrote: On Mon, Nov 22, 2010 at 11:13 PM, Hirokazu Yamamoto ocean-c...@m2.ccsnet.ne.jp wrote: Hello. Does this affect python? Thank you. http://www.openssl.org/news/secadv_20101116.txt No. Well, actually it does, but Python links against the system OpenSSL on most platforms (except Windows), so it's up to the OS vendor to apply the patch. It does? If so, I must have misunderstood the vulnerability. Can you explain how it affects Python? If I believe the link above: 1CAny OpenSSL based TLS server is vulnerable if it is multi-threaded and uses OpenSSL's internal caching mechanism. Servers that are multi-process and/or disable internal session caching are NOT affected. 1D So, you just have to create a multithreaded TLS server which doesn't disable server-side session caching (it is enabled by default according to http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html ) Hm. The session cache is enabled by default, but nothing will ever use it unless the server specifies a session id using SSL_set_session_id_context or SSL_CTX_set_session_id_context. Python doesn't expose these, so I don't think any Python SSL server can set them. The vulnerability announcement isn't 100% clear on this, but I took a look at the patch which fixes the issue and it /appears/ as though if a client never tries to re-use a session then you will be safe from this bug. However, perhaps this only means that only malicious clients (which send a session id even when they can't actually have one) will be able to trigger the bug. Or I may misunderstand how SSL sessions work in OpenSSL entirely. The documentation for them is on par with that for most of the rest of OpenSSL. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OpenSSL Vulnerability (openssl-1.0.0a)
On 03:11 pm, solip...@pitrou.net wrote: On Wed, 24 Nov 2010 15:01:06 - exar...@twistedmatrix.com wrote: If I believe the link above: 1CAny OpenSSL based TLS server is vulnerable if it is multi-threaded and uses OpenSSL's internal caching mechanism. Servers that are multi-process and/or disable internal session caching are NOT affected. 1D So, you just have to create a multithreaded TLS server which doesn't disable server-side session caching (it is enabled by default according to http://www.openssl.org/docs/ssl/SSL_CTX_set_session_cache_mode.html ) Hm. The session cache is enabled by default, but nothing will ever use it unless the server specifies a session id using SSL_set_session_id_context or SSL_CTX_set_session_id_context. Python doesn't expose these, so I don't think any Python SSL server can set them. Well, Python calls SSL_CTX_set_session_id_context() implicitly, starting from 3.2 (precisely so that the session cache gets used). The documentation I've found about the session id context seems to suggest that a process-wide constant is enough. Ah. Okay, then Python 3.2 would be vulnerable. Good thing it isn't released yet. ;) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r86633 - in python/branches/py3k: Doc/library/inspect.rst Doc/whatsnew/3.2.rst Lib/inspect.py Lib/test/test_inspect.py Misc/NEWS
On 04:24 pm, solip...@pitrou.net wrote: On Mon, 22 Nov 2010 17:08:36 +0100 Hrvoje Niksic hrvoje.nik...@avl.com wrote: On 11/22/2010 04:37 PM, Antoine Pitrou wrote: +1. The problem with int constants is that the int gets printed, not the name, when you dump them for debugging purposes :) Well, it's trivial to subclass int to something with a nicer __repr__. PyGTK uses that technique for wrapping C enums: Nice. It might be useful to add a private _Constant class somewhere for stdlib purposes. http://www.python.org/dev/peps/pep-0354/ Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python- dev/exarkun%40twistedmatrix.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 03:48 pm, gu...@python.org wrote: On Tue, Nov 16, 2010 at 7:16 AM, Alexander Belopolsky alexander.belopol...@gmail.com wrote: What this thread has shown is that there is no consensus on what public names are and what rules should be followed when changing names that can be imported from a module. �I have opened an issue at http://bugs.python.org/issue10434 to address this. �My vote is to adopt the definition spelled out in the language reference, copy it to the library manual and add some discussion of the deprecation policies. Hm. Apart from the specific semantics assigned by the language to single and double leading (and trailing) underscores, I still think this belongs in a style guide, not in the library manual. When reading the library manual, one should always assume that undocumented features are subject to change at any time. I don't think it belongs only in PEP 8 (that's a style guide you're referring to, correct?). It needs to be front and center. This is information that every single user of the stdlib needs in order to use the stdlib correctly. Imagine trying to use a dictionary without knowing about alphabetical ordering. Or driving a car without knowing what lane markers indicate. No matter how many times we discuss this policy on this list (I know it's come up here before), the majority of python users still won't learn about it. PEP 8 isn't nearly visible enough, either. Whatever the rule is, it needs to be presented with the information itself. If the rule is that things not documented in the library manual have no compatibility guarantees, then all of the means of getting documentation *other* than looking at the library manual need to indicate this somehow (alternatively, the information shouldn't be duplicated, but I doubt I'll convince anyone of that). Here's a stupid proposal. What if the top of pydoc output said (for stdlib modules only) The library manual is the canonical reference. Refer to it before using APIs you find in this documentation. Still inconvenient, but inconvenient is better than secret/impossible. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 05:21 pm, fuzzy...@voidspace.org.uk wrote: On 16/11/2010 17:16, 1ukasz Langa wrote: Am 16.11.2010 18:06, schrieb Antoine Pitrou: On Tue, 16 Nov 2010 16:34:54 - exar...@twistedmatrix.com wrote: Imagine trying to use a dictionary without knowing about alphabetical ordering. You mean an ordered dictionary, right? He meant the ones with actual paper pages. But given that we are particularly talking about how to handle undocumented APIs, a more apropos comparison would be to ask how dictionary readers are supposed to look up words that aren't in the dictionary... No, this isn't an appropriate comparison. The dictionary was an example of something that presents information but is very hard to use without knowing the rules. We're not talking about undocumented APIs. We're talking about APIs that are documented somewhere other than in the library manual. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 11:53 am, solip...@pitrou.net wrote: On Tue, 09 Nov 2010 02:03:23 - exar...@twistedmatrix.com wrote: I wonder if there are any actual technical arguments to be made against something like `deprecatedModuleAttribute`? For example, does it work well with import hacks such as Mercurial's demandimport? I haven't tried before, but a quick experiment suggests that the two happily co-exist (aside from demandimport getting the blame instead of the true offending code, but that's really a problem with the warnings module): import mercurial.demandimport as di di.enable() import twisted.python.threadpool as tp tp.ThreadSafeList /usr/lib/pymodules/python2.6/mercurial/demandimport.py:76: DeprecationWarning: twisted.python.threadpool.ThreadSafeList was deprecated in Twisted 10.1.0: This was an internal implementation detail of support for Jython 2.1, which is now obsolete. return getattr(self._module, attr) class twisted.python.threadpool.ThreadSafeList at 0xb746decc Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backward incompatible API changes in the pydoc module
On 11:44 am, ncogh...@gmail.com wrote: All, I was about to commit the patch for issue 2001 (the improvements to the pydoc web server and the removal of the Tk GUI) when I realised that pydoc.serve() and pydoc.gui() are technically public standard library APIs (albeit undocumented ones). Currently the patch switches serve() to start the new server implementation and gui() to start the server and open a browser window for it. It occurred to me that, despite the it's an application feel to the pydoc web server APIs, it may be a better idea to leave the two existing functions alone (aside from adding DeprecationWarning), and using new private function names to start the new server and the web browser. Is following the standard deprecation procedure the better course here, or am I being overly paranoid? Following the deprecation procedure here sounds awesome to me. Thanks for considering it, I hope you'll choose to go that way. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Buildbot for AIX
On 05:50 pm, solip...@pitrou.net wrote: Le lundi 08 novembre 2010 � 18:46 +0100, S�bastien Sabl� a �crit : xlc: 1501-216 (W) command option - -qmaxmem=18000 is not recognized - passed to ld Is -qmaxmem really necessary to build Python? If so, you could try passing it in CFLAGS. However running 2 different slaves per host in order to distinguish xlc and gcc would be OK; though I would appreciate if they could run sequentially rather than in parallel as that would limit the host load. If there are two separate slaves, I can't think of any simple way to run builds sequentially. Perhaps you can assign both of them to a single CPU (assuming AIX allows that). A master lock will allow this. Although just having a single slave and using a slave lock would be simpler. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 07:58 pm, br...@python.org wrote: I don't think a strict don't remove without deprecation policy is workable. �For example, is trace.rx_blank constant part of the trace module API that needs to be preserved indefinitely? �I don't even know if it is possible to add a deprecation warning to it, but CoverageResults._blank_re would certainly be a better place for it. The deprecation policy obviously cannot apply to module-level attributes. I'm not sure why this is. Can you elaborate? The main reason I have said that non-underscore names should be properly deprecated (assuming they are not contained in an underscored-named module) is that dir() and help() do not distinguish. If you are perusing a module from the interpreter prompt you have no way to know whether something is public or private if it lacks an underscore. Is it reasonable to assume that any API found through dir() or help() must be checked with the official docs before you can consider using it, even if you have no explicit need to read the official docs? I (unfortunately) say no, which is why I have argued that non-underscored names need to be properly deprecated. This obviously places a nasty burden on us, though, so I don't like taking this position. Unless we can make it clearly known through help() or something that the official docs must be checked to know what can and cannot be reliably used I don't think it is reasonable to force users to not be able to rely on help() (we should probably change help() to print a big disclaimer for anything with a leading underscore, though). But that doesn't mean we can't go through, fix up our names, and deprecate the old public names; that's fair game in my book. +1 Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 09:25 pm, br...@python.org wrote: On Mon, Nov 8, 2010 at 13:03, exar...@twistedmatrix.com wrote: On 07:58 pm, br...@python.org wrote: I don't think a strict don't remove without deprecation policy is workable. �For example, is trace.rx_blank constant part of the trace module API that needs to be preserved indefinitely? I don't even know if it is possible to add a deprecation warning to it, but CoverageResults._blank_re would certainly be a better place for it. The deprecation policy obviously cannot apply to module-level attributes. I'm not sure why this is. �Can you elaborate? There is no way to directly trigger a DeprecationWarning for an attribute. We can still document it, but there is just no way to programmatically enforce it. What about `deprecatedModuleAttribute` (http://twistedmatrix.com/documents/current/api/twisted.python.deprecate.html) or zope.deprecation (http://docs.zope.org/zope3/Book/deprecation/show.html) which inspired it? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 09:57 pm, br...@python.org wrote: On Mon, Nov 8, 2010 at 13:45, exar...@twistedmatrix.com wrote: On 09:25 pm, br...@python.org wrote: On Mon, Nov 8, 2010 at 13:03, �exar...@twistedmatrix.com wrote: On 07:58 pm, br...@python.org wrote: I don't think a strict don't remove without deprecation policy is workable. For example, is trace.rx_blank constant part of the trace module API that needs to be preserved indefinitely? I don't even know if it is possible to add a deprecation warning to it, but CoverageResults._blank_re would certainly be a better place for it. The deprecation policy obviously cannot apply to module-level attributes. I'm not sure why this is. �Can you elaborate? There is no way to directly trigger a DeprecationWarning for an attribute. We can still document it, but there is just no way to programmatically enforce it. What about `deprecatedModuleAttribute` (http://twistedmatrix.com/documents/current/api/twisted.python.deprecate.html) or zope.deprecation (http://docs.zope.org/zope3/Book/deprecation/show.html) which inspired it? Just checked the code and it looks like it substitutes the module for some proxy object? To begin that break subclass checks. After that I don't know the ramifications without really digging into the ModuleType code. That could be fixed if ModuleType allowed subclassing. :) For what it's worth, no one has complained about problems caused by `deprecatedModuleAttribute`, but we've only been using it for about two and a half years. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Breaking undocumented API
On 12:50 am, gu...@python.org wrote: On Mon, Nov 8, 2010 at 3:55 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This seems like a pretty clear case of practicality beats purity. Not only has nobody complained about deprecatedModuleAttribute, but there are tons of things which show up in sys.modules that aren't modules in the sense of 'instances of ModuleType'. The Twisted reactor, for example, is an instance, and we've been doing *that* for about 10 years with no complaints. But the Twisted universe is only a subset of the Python universe. The Python stdlib needs to move more carefully. I think that Twisted developers are pretty careful to consider the consequences of changes they make to Twisted. We have an explicit, documented backwards compatibility policy, for example. We also have mandatory code review for all changes, with a documented set of guidelines outlining the minimum things a reviewer should be considering. I wonder if there are any actual technical arguments to be made against something like `deprecatedModuleAttribute`? Also, it turns out that ModuleType can be subclassed these days. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Snakebite, buildbot and low hanging fruit -- feedback wanted! (Was Re: SSH access against buildbot boxes)
On 11:24 am, tr...@snakebite.org wrote: 2. Address the second problem of the buildbot web interface sucking for non-standard branches. I'm thinking along the lines of a hack to buildbot, such that upon creation of new per-activity branches off a mainline, something magically runs in the background and sets up a complete buildbot view at python.snakebite.org/dev/buildbot/your- branch-name, just as if you were looking at a trunk buildbot page. This is basically trivial. I gave #python-dev a tool for forcing builds, dunno if anyone still has a copy, but it's easy to reconstruct from http://twistedmatrix.com/trac/browser/sandbox/exarkun/force- builds.py (which is what the Twisted project uses). Plus, you can add ?branch=name to most BuildBot views to limit display of results to just builds for the named branch. Titus, for example, alluded to some nifty way for a committer to push his local hg branch/changes somewhere, such that it would kick off builds on multiple platforms in the same sorta' vein as point 2, but able to leverage cloud resources like Amazon's EC2, not just Snakebite hardware. BuildBot supports managing EC2 instance lifetimes to run builds. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)
On 06:28 am, techto...@gmail.com wrote: On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: This is the strongest reason why I recommend to everyone I know that they not use pickle for storage they'd like to keep working after upgrades [not just of stdlib, but other 3rd party software or their own software]. :) +1. Twisted actually tried to preserve pickle compatibility in the bad old days, but it was impossible. Pickles should never really be saved to disk unless they contain nothing but lists, ints, strings, and dicts. But what is alternative in stdlib? Don't you think that Python doesn't provide any? Persistence is a very hard problem. Lots and lots of trade-offs need to be made, and you generally want to tailor those trade-offs to the particular application at hand. This probably means that the stdlib isn't a suitable place to try to solve the problem. Look outside the stdlib and you'll find an extremely vibrant and diverse collection of software which is aimed at solving this problem, though. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)
On 12:21 am, m...@gsites.de wrote:
Am 04.11.2010 17:15, schrieb anatoly techtonik:
pickle is insecure, marshal too.
If the transport or storage layer is not save, you should
cryptographically sign the data anyway::
def pickle_encode(data, key):
msg = base64.b64encode(pickle.dumps(data, -1))
sig = base64.b64encode(hmac.new(key, msg).digest())
return sig + ':' + msg
def pickle_decode(data, key):
if data and ':' in data:
sig, msg = data.split(':', 1)
if sig == base64.b64encode(hmac.new(key, msg).digest()):
return pickle.loads(base64.b64decode(msg))
raise pickle.UnpicklingError(Wrong or missing signature.)
Bottle (a web framework) uses a similar approach to store non-string
data in client-side cookies. I don't see a (security) problem here.
Your pickle_decode leaks information about the key. An attacker will
eventually (a few seconds to a few minutes, depending on how they have
access to this system) be able to determine your key and send you
arbitrary pickles (ie, execute arbitrary code on your system).
Oops.
This stuff is hard. If you're going to mess around with it, make sure
you're *serious* (better approach: don't mess around with it).
Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cleaning-up the new unittest API
On 04:29 pm, fuzzy...@voidspace.org.uk wrote: On 02/11/2010 16:23, Terry Reedy wrote: On 11/2/2010 10:05 AM, C. Titus Brown wrote: ...but, as someone who has to figure out how to teach stuff to CSE undergrads (and biology grads) I hate the statement ...any programmer should expect this... And indeed I (intentionally) did not say that. People who are ignorant and inexperienced about something should avoid making expectations in any direction until they have read the doc and experimented a bit. Expectations come from consistent behaviour. sorted behaves consistently for *most* of the built-in types and will also work for custom types that provide a 'standard' (total ordering) implementation of __lt__. It is very easy to *not realise* that a consequence of sets (and frozensets) providing partial ordering through operator overloading is that sorting is undefined for them. Perhaps. The documentation for sets says this, though: Since sets only define partial ordering (subset relationships), the output of the list.sort() method is undefined for lists of sets. Particularly as it still works for other mutable collections. Worth being aware that custom implementations of standard operators will break expectations of users who aren't intimately aware of the problem domains that the specific type may be created for. I can't help thinking that most of this confusion is caused by using for determining subsets. If were not defined for sets and people had to use set.issubset (which exists already), then sorting a list with sets would raise an exception, a much more understandable failure mode than getting back a list in arbitrary order. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/
On 12:47 am, ben+pyt...@benfinney.id.au wrote: Antoine Pitrou solip...@pitrou.net writes: I don't agree with this. Until it's documented, it's an implementation detail and should be able to change without notice. If it's an implementation detail, shouldn't it be named as one (i.e. with a leading underscore)? If someone wants to depend on some undocumented detail of the directory layout it's their problem (like people depending on bytecode and other stuff). I would say that names without a single leading underscore are part of the public API, whether documented or not. And if that isn't the rule, then what the heck is? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
On 02:51 am, br...@python.org wrote: 2010/10/28 Kristj�n Valur J�nsson krist...@ccpgames.com: Hi all. This has been a lively discussion. My desire to keep 2.x alive in some sense is my own and I don't know if anyone shares it but as a member of this community I think I'm allowed to voice it. So, just to clarify my particular position, let me explain where all this comes from. [snip] And as everyone has said so far (and with which I agree), that's fine. As long as it is not called Python 2.8 -- EVE-Python 2.8 or some Monty Python reference -- then that's fine. And as pointed out by folks, once Hg kicks in and we have user repos you can even host it on hg.python.org yourself to give it some semblance of authority if you want. In case anyone was discouraged by the idea that a 2.x continuation would not be allowed to bear the name Python as Brett suggests here, I want to make a clarification. Brett is speaking for himself here (and he never claimed otherwise!). However, decisions about where to allow the use of the Python trademark are made by the Python Software Foundation. The PSF has not decided to reject any request by a 2.x continuation project to use the name Python. Of course, this does not mean they would necessarily allow such a use. I just wanted to point out that they have not categorically rejected it, as one might be tempted to infer from Brett's message. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
On 04:04 pm, ba...@python.org wrote: I'd *much* rather this enthusiasm be spent on making Python 3 rock, and in porting third party code to Python 3. Enthusiasm isn't fungible. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 10194 - Adding a gc.remap() function
On 26 Oct, 11:31 pm, pinge...@yahoo.com wrote: --- On Tue, 10/26/10, exar...@twistedmatrix.com exar...@twistedmatrix.com wrote: This can be implemented with ctypes right now (I half did it several years ago). Jean-Paul Is there a trick to doing it this way, or are you suggesting building a ctypes wrapper for each C type in the Python library, and then effectively reimplementing tp_traverse in Python? That's the idea, yes. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python- dev/exarkun%40twistedmatrix.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MemoryError... how much memory?
On 07:09 pm, facundobati...@gmail.com wrote: On Wed, Oct 27, 2010 at 12:05 PM, Benjamin Peterson benja...@python.org wrote: Isn't this usually when you do something like [None]*2**300? In that case, wouldn't you know how much memory you're requesting? It could happen on any malloc. It depends on how much you have free. Don't think on getting a MemoryError on a python you just opened in the console. Think about a server with a month of uptime, where you have all the memory fragmented, etc. Also, why is that useful? It helps to determine why we're having some Memory Errors on our long-lived server, how is the behaviour when that happens, etc. But... If you allocated all of your memory to some garbage, and then a 5 byte string can't be allocated, you don't really care about the 5 byte string, you care about the garbage that's wasting your memory. Tools like heapy will give you a lot of information. Maybe it wouldn't hurt anyone to have more information in a MemoryError. But I don't think it's going to help a lot either. It's not the information that you're really interested in. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issue 10194 - Adding a gc.remap() function
On 08:28 pm, pinge...@yahoo.com wrote: --- On Tue, 10/26/10, Martin v. L�wis mar...@v.loewis.de wrote: I think this then mandates a PEP; I'm -1 on the feature also. I am happy to write up a PEP for this feature. I'll start that process now, though if anyone feels that this idea has no chance of acceptance please let me know. This can be implemented with ctypes right now (I half did it several years ago). Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SVN rev. 85392 broke module handling in py3k
On 02:13 pm, stefan...@behnel.de wrote: Benjamin Peterson, 22.10.2010 16:03: 2010/10/22 Stefan Behnel: since SVN rev. 85392, Cython's installation fails on the py3k branch with a weird globals error. I think it is related to some sys.modules magic that we do in order to support running Cython in Python 3 using lib2to3. Basically, what we do is, we import some parts of Cython at the beginning that are Py3 clean, specifically some distutils build_ext replacement for building Cython modules. Then we start up distutils, which first runs lib2to3 on Cython's sources to convert them into Py3 code. When it then gets to building the binary modules, we remove all Cython modules and packages from sys.modules and reimport their 2to3-ed sources so that we can run the complete compiler during the installation (to bootstrap parts of Cython into binary modules). Since the above revision, this process bails out with an error when accessing os.path because os is None. The os module is imported globally in our early-imported build_ext module, more or less like this: import os from distutils.command import build_ext as _build_ext class build_ext(_build_ext.build_ext): def build_extensions(self): print(os) # prints None! I suspect that the fact that we remove the modules from sys.modules somehow triggers the cleanup of these modules while there are still objects from these modules alive that refer to their globals. So, what I think is happening is that the module cleanup sets the module's globals to None before the objects from that module that refer to these globals have actually gone out of scope. Instances of classes don't refer to the module their class is defined in. It seems more likely that the reason the module is garbage collected is that there really is nothing which refers to it anymore. The behavior of setting the attributes of a module being freed to None has been in place for a long time, r85392 only restored it after a brief absence. Perhaps Cython itself should keep the modules alive that it wants kept alive. Alternatively, if Cython owns the code that's running into the zapped global, you could change it to not use globals. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for async read/write
On 04:50 pm, j...@jcea.es wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Current Python lacks support for aio_* syscalls to do async IO. I think this could be a nice addition for python 3.3. Adding more platform wrappers is always nice. Keep in mind that the quality of most (all?) aio_* implementations is spotty at best, though. On Linux, they will sometimes block (for example, if you fail to align buffers properly, or open a file without O_DIRECT, or if there are too many other aio operations active on the system at the time, etc). Thus, these APIs are finicky at best, and the Python bindings will be similarly fragile. Jean-Paul If you agree, I will create an issue in the tracker. If you think the idea is of no value, please say so for me to move on. Do you have an application in mind? Maybe an 3th party module, but I think this functionality sould be available in core python. Starting off as a 3rd party library to try to develop some interest and users (and thus experience) before adding it to the standard library might make sense (as it frequently does). Thanks!. PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read, aio_return, aio_write. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for async read/write
On 01:37 am, gl...@twistedmatrix.com wrote: On Oct 19, 2010, at 8:09 PM, James Y Knight wrote: There's a difference. os._exit is useful. os.open is useful. aio_* are *not* useful. For anything. If there's anything you think you want to use them for, you're wrong. It either won't work properly or it will worse performing than the simpler alternatives. I'd like to echo this sentiment. This is not about providing a 'safe' wrapper to hide some powerful feature of these APIs: the POSIX aio_* functions are really completely useless. To quote the relevant standard http://www.opengroup.org/onlinepubs/95399/basedefs/aio.h.html: APPLICATION USAGE None. RATIONALE None. FUTURE DIRECTIONS None. Not only is the performance usually worse than expected, the behavior of aio_* functions require all kinds of subtle and mysterious coordination with signal handling, which I'm not entirely sure Python would even be able to pull off without some modifications to the signal module. (And, as Jean-Paul mentioned, if your OS kernel runs out of space in a queue somewhere, completion notifications might just never be delivered at all.) Just to be clear, James corrected me there. I thought Jesus was talking about the mostly useless Linux AIO APIs, which have the problems I described. He was actually talking about the POSIX AIO APIs, which have a different set of problems making them a waste of time. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] We should be using a tool for code reviews
On 02:47 pm, jnol...@gmail.com wrote: On Wed, Sep 29, 2010 at 2:32 PM, Guido van Rossum gu...@python.org wrote: I would like to recommend that the Python core developers start using a code review tool such as Rietveld or Reviewboard. I don't really care which tool we use (I'm sure there are plenty of pros and cons to each) but I do think we should get out of the stone age and start using a tool for the majority of our code reviews. While I would personally love to see Rietveld declared the official core Python code review tool, I realize that since I wrote as a Google engineer and it is running on Google infrastructure (App Engine), I can't be fully objective about the tool choice -- even though it is open source, has several non-Googler maintainers, and can be run outside App Engine as well. But I do think that using a specialized code review tool rather than unstructured email plus a general-purpose issue tracker can hugely improve developer performance and also increase community participation. (A code review tool makes it much more convenient for a senior reviewer to impart their wisdom to a junior developer without appearing judgmental or overbearing.) See also this buzz thread: http://www.google.com/buzz/115212051037621986145/At6Rj82Kret/When- will-the-Python-dev-community-start-using -- --Guido van Rossum (python.org/~guido) Regardless of the tool(s) used, code reviews are a fantastic equalizer. If you have long time, experienced developers submitting to the same rules that newer contributors have to follow then it helps remove the idea that there is special treatment occurring. Of course, this is only true if the core developers *do* submit to the same rules. Is anyone proposing that current core committers have all their work reviewed before it is accepted? (I am strongly in favor of this, but I don't think many core committers are.) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Atlassian and bitbucket merge
On 01:13 am, st...@holdenweb.com wrote: I see that Atlassian have just taken over BitBucket, the Mercurial hosting company. IIRC Atlassian offered to host our issue tracking on JIRA, but in the end we decided to eat our own dog food and went with roundup. I'm wondering if they'd be similarly interested in supporting our Hg server. Or is self-hosting the only acceptable solution? From recent mail it looks likes we may be up and running on Hg fairly soon. I know of two medium sized projects (smaller than CPython) that are switching away from BitBucket's free services because of their poor reliability. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Volunteer help with porting
On 01:33 pm, p...@phd.pp.ru wrote: Hello. Thank you for the offer! On Tue, Sep 07, 2010 at 06:36:10PM +0530, Prashant Kumar wrote: My name is Prashant Kumar and I wish to contribute to the Python development process by helping convert certain existing python over to python3k. Is there anyway I could obtain a list of libraries which need to be ported over to python3k, sorted by importance(by importance i mean packages which serve as a dependency for larger number of packages being more important). As there is already Python 3.2 alpha, the core of Python has already been ported - and this mailing list is for discussion of development of the very Python, not about working on 3rd-party libraries. How about the email package? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Volunteer help with porting
On 02:34 pm, p...@phd.pp.ru wrote: On Tue, Sep 07, 2010 at 02:02:59PM -, exar...@twistedmatrix.com wrote: On 01:33 pm, p...@phd.pp.ru wrote: As there is already Python 3.2 alpha, the core of Python has already been ported How about the email package? What about email? It is a core library, right? It has been ported AFAIK. Or have I missed something? Are you just assuming that because there have been 3.x releases, everything in the standard library works properly in 3.x? Or do you know something about the email package specifically? Last I checked, there were a number of outstanding issues with email in 3.x. And as Michael Foord pointed out, there are other standard library modules in the same state. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Internal counter to debug leaking file descriptors
On 05:22 pm, gl...@twistedmatrix.com wrote: On Aug 31, 2010, at 10:03 AM, Guido van Rossum wrote: On Linux you can look somewhere in /proc, but I don't know that it would help you find where a file was opened. /dev/fd is actually a somewhat portable way of getting this information. I don't think it's part of a standard, but on Linux it's usually a symlink to /proc/self/fd, and it's available on MacOS and most BSDs (based on a hasty and completely-not-comprehensive investigation). But it won't help you find out when the FDs were originally opened, no. ___ On OS X and Solaris, dtrace and ustack will tell you exactly when and where the FDs were originally opened, though. On Linux, SystemTap might give you the same information (but I know much less about SystemTap). If http://bugs.python.org/issue4111 is resolved, then this may even be possible without using a patched version of Python. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Looking after the buildbots (in general)
On 02:51 pm, ba...@python.org wrote: On Aug 04, 2010, at 11:16 AM, Antoine Pitrou wrote: I think the issue is that many core developers don't have the reflex to check buildbot state after they commit some changes (or at least on a regular, say weekly, basis), and so gradually the buildbots have a tendency to turn from green to red, one after another. I'd classify this as a failure of the tools, not of the developers. These post-commit verification steps should be proactive, and scream really loud (or even prevent future commits) until everything is green again. Buildbots themselves can be unstable, so this may or may not be workable, and changing any of this will take valuable volunteer time. It's also unsexy work. How hard is it to look at a web page? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Looking after the buildbots (in general)
On 03:17 pm, fuzzy...@voidspace.org.uk wrote: On 04/08/2010 16:15, exar...@twistedmatrix.com wrote: On 02:51 pm, ba...@python.org wrote: On Aug 04, 2010, at 11:16 AM, Antoine Pitrou wrote: I think the issue is that many core developers don't have the reflex to check buildbot state after they commit some changes (or at least on a regular, say weekly, basis), and so gradually the buildbots have a tendency to turn from green to red, one after another. I'd classify this as a failure of the tools, not of the developers. These post-commit verification steps should be proactive, and scream really loud (or even prevent future commits) until everything is green again. Buildbots themselves can be unstable, so this may or may not be workable, and changing any of this will take valuable volunteer time. It's also unsexy work. How hard is it to look at a web page? The hard part is remembering. Seems to be setting the bar awfully low for Python developers. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Looking after the buildbots (in general)
On 03:31 pm, ba...@python.org wrote: On Aug 04, 2010, at 03:15 PM, exar...@twistedmatrix.com wrote: On 02:51 pm, ba...@python.org wrote: On Aug 04, 2010, at 11:16 AM, Antoine Pitrou wrote: I think the issue is that many core developers don't have the reflex to check buildbot state after they commit some changes (or at least on a regular, say weekly, basis), and so gradually the buildbots have a tendency to turn from green to red, one after another. I'd classify this as a failure of the tools, not of the developers. These post-commit verification steps should be proactive, and scream really loud (or even prevent future commits) until everything is green again. Buildbots themselves can be unstable, so this may or may not be workable, and changing any of this will take valuable volunteer time. It's also unsexy work. How hard is it to look at a web page? That's not the right question :) The real questions are: how hard is it to remember how to find the appropriate web page Oh, come on. I don't believe this. how hard is it to know which buildbots are *actually* stable enough to rely on, This is more plausible. But it's not the tools' fault that the test suite has intermittent failures. Developers choose to add new features or change existing ones instead of fixing bugs in existing code or tests. I'd call that a developer failure. how hard is it to decipher the results to know what they're telling you? Red bad, green good. A much more plausible explanation, to me, is that most developers don't really care if things are completely working most of the time. They're happy to push the work onto other developers and onto the release team. And as long as other developers let them get away with that, it's not likely to stop. But perhaps the people picking up the slack here don't mind and are happy to keep doing it, in which case nothing needs to change. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Looking after the buildbots (in general)
On 03:53 pm, g.bra...@gmx.net wrote: Am 04.08.2010 17:15, schrieb exar...@twistedmatrix.com: On 02:51 pm, ba...@python.org wrote: On Aug 04, 2010, at 11:16 AM, Antoine Pitrou wrote: I think the issue is that many core developers don't have the reflex to check buildbot state after they commit some changes (or at least on a regular, say weekly, basis), and so gradually the buildbots have a tendency to turn from green to red, one after another. I'd classify this as a failure of the tools, not of the developers. These post-commit verification steps should be proactive, and scream really loud (or even prevent future commits) until everything is green again. Buildbots themselves can be unstable, so this may or may not be workable, and changing any of this will take valuable volunteer time. It's also unsexy work. How hard is it to look at a web page? The hard part is to know *when* to look. As you might have noticed, the Python test suite does not run in ten seconds, especially on some of the buildbots -- it can take 1-2 there to complete. So if you look too soon, you won't see all the results, and usually the slow systems are the interesting ones. Now we could of course have a commit hook that counts down two hours and then sends an email to the committer Now look at the buildbot!... I don't think it's that hard to take a look at the end of the day (or before starting anything else the next morning). All it really takes is a choice on the part of each developer to care whether or not their changes are correct. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 proposed changes for basic plugins support
On 12:21 pm, m...@egenix.com wrote: Tarek Ziad� wrote: On Mon, Aug 2, 2010 at 3:06 AM, P.J. Eby p...@telecommunity.com wrote: .. So without specific examples of why this is a problem, it's hard to see why a special Python-specific set of configuration files is needed to resolve it, vs. say, encouraging application authors to use the available alternatives for doing plugin directories, config files, etc. I don't have a specific example in mind, and I must admit that if an application does the right thing (provide the right configuration file), this activate feature is not useful at all. So it seems to be a bad idea. I propose that we drop the PLUGINS file idea and we add a new metadata field called Provides-Plugin in PEP 345, which will contain the info I've described minus the state field. This will allow us to expose plugins at PyPI. IOW, have entry points like setuptools provides, but in a metadata field instead of a entry_points.txt file. Do we really need to make Python packaging even more complicated by adding support for application-specific plugin mechanisms ? Packages can already work as application plugins by simply defining a plugins namespace package and then placing the plugin packages into that namespace. See Zope for an example of how well this simply mechanism works out in practice: it simply scans the Products namespace for sub-packages and then loads each sub-package it finds to have it register itself with Zope. This is also roughly how Twisted's plugin system works. One drawback, though, is that it means potentially executing a large amount of Python in order to load plugins. This can build up to a significant performance issue as more and more plugins are installed. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 proposed changes for basic plugins support
On 01:27 pm, m...@egenix.com wrote: exar...@twistedmatrix.com wrote: On 12:21 pm, m...@egenix.com wrote: See Zope for an example of how well this simply mechanism works out in practice: it simply scans the Products namespace for sub-packages and then loads each sub-package it finds to have it register itself with Zope. This is also roughly how Twisted's plugin system works. One drawback, though, is that it means potentially executing a large amount of Python in order to load plugins. This can build up to a significant performance issue as more and more plugins are installed. I'd say that it's up to the application to deal with this problem. An application which requires lots and lots of plugins could define a registration protocol that does not require loading all plugins at scanning time. It's not fixable at the application level, at least in Twisted's plugin system. It sounds like Zope's system has the same problem, but all I know of that system is what you wrote above. The cost increases with the number of plugins installed on the system, not the number of plugins the application wants to load. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 proposed changes for basic plugins support
On 03:08 pm, mer...@netwok.org wrote: Le 02/08/2010 14:31, exar...@twistedmatrix.com a �crit : On 12:21 pm, m...@egenix.com wrote: Do we really need to make Python packaging even more complicated by adding support for application-specific plugin mechanisms ? Packages can already work as application plugins by simply defining a plugins namespace package and then placing the plugin packages into that namespace. [...] This is also roughly how Twisted's plugin system works. One drawback, though, is that it means potentially executing a large amount of Python in order to load plugins. This can build up to a significant performance issue as more and more plugins are installed. If namespace packages make it into Python, they would indeed solve a part of the problem in a nice, generic way. I don't think this solves the problem. Twisted's plugin system already uses namespace packages. It helps slightly, by spreading out your plugins, but you can still end up with lots of plugins in a particular namespace. Regarding the performance issue, I wonder if functions in pkgutil or importlib could allow one to iterate over the plugins (i.e. submodules and subpackages of the namespace package) without actually loading then. We would get only their names though, not their description or any other information useful to decide to activate them or not. The trick is to continue to provide enough information so that the code iterating over the data can make a correct decision. It's not clear that names are enough. Maybe importing is the way to go, with a doc recommendation that people make their plugins subpackages with an __init__ module containing only a docstring. Regards Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] EINVAL
On 10:33 am, solip...@pitrou.net wrote: On Thu, 22 Jul 2010 17:50:00 +0900 Stephen J. Turnbull step...@xemacs.org wrote: I think that's Antoine's PEP 3151. Interestingly, he doesn't mention EINVAL at all. http://www.python.org/dev/peps/pep-3151/ That's right. It is based on a survey of existing exception-catching code in the stdlib. There's only one match in the whole Lib/ subtree: $ grep -r EINVAL Lib/ Lib/plat-sunos5/STROPTS.py:968:EINVAL = 22 I guess EINVAL would most often indicate a programming error, which is why it doesn't get handled specifically in except clauses. For setgroups it means you exceeded a platform-specific limit. On Windows, for non-blocking connect, it means wait a little longer. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting an optional parameter instead of creating a socket internally
On 12:30 pm, thebra...@brasse.org wrote: On Sun, Jul 11, 2010 at 9:06 PM, exar...@twistedmatrix.com wrote: It's still little more than an outline. You can see it here: http://twistedmatrix.com/trac/wiki/ProtocolPEP And if you're interested in helping, we can figure out a way to do that (you can have edit permission on the wiki or we can move the document elsewhere, whatever). Jean-Paul This seems like an interesting idea to me. I would like to figure out some way I could help with the PEP. If you move the document, could you please keep me updated on the new location? Sure. If it moves, there'll be a pointer from its existing location to the new one. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting an optional parameter instead of creating a socket internally
On 03:11 pm, j...@jcea.es wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/04/10 04:03, exar...@twistedmatrix.com wrote: On 12 Apr, 11:19 pm, j...@jcea.es wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/13/2010 12:47 AM, Antoine Pitrou wrote: Jesus Cea jcea at jcea.es writes: PS: socket.setdefaulttimeout() is not enough, because it could shutdown a perfectly functional connection, just because it was idle for too long. The socket timeout doesn't shutdown anything. It just puts a limit on how much time recv() and send() can block. Then it's up to you to detect whether the server is still alive (for example by pinging it through whatever means the application protocol gives you). A regular standard library (let say, poplib) would abort, after getting the timeout exception. 4. Modify client libraries to accept a new optional socket-like object as an optional parameter. This would allow things like transparent compression or encryption, or to replace the socket connection by anything else (read/write to shared memory or database, for example). This could be useful too. I have been thinking about this for years. Do you actually think this could be formally proposed?. Every once in a while I make a little bit more progress on the PEP I'm working on for this. If you want to talk more about this, you can find me in #python-dev or #twisted on freenode. Jean-Paul Jean-Paul, I would like to have this for 3.2. How is the PEP going?. It's still little more than an outline. You can see it here: http://twistedmatrix.com/trac/wiki/ProtocolPEP And if you're interested in helping, we can figure out a way to do that (you can have edit permission on the wiki or we can move the document elsewhere, whatever). Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 05:24 am, mar...@v.loewis.de wrote: Seems to work fine. So this I don't understand. Any ideas, anyone? Didn't we discuss this before? The buildbot slave has no controlling terminal anymore, hence it cannot open /dev/tty. If you are curious, just patch your checkout to output the exact errno (e.g. to stdout), and trigger a build through the web. Could the test be rewritten (or supplemented) to use a pty? Most or perhaps all of the same operations should be supported. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 04:44 pm, solip...@pitrou.net wrote: On Wed, 30 Jun 2010 09:00:09 PDT Bill Janssen jans...@parc.com wrote: So, my question then is, why are these skips unexpected? Seems to me that if this is the case, this test will never run on any platform. You can change the value of the usepty option in your buildbot.tac. (you will also have to restart the buildslave process) But don't do this. The usepty option is completely unrelated to the suggestion I was making. Flipping it to True will only cause other things to break and have no impact on this test. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 04:26 pm, jans...@parc.com wrote:
exar...@twistedmatrix.com wrote:
Could the test be rewritten (or supplemented) to use a pty? Most or
perhaps all of the same operations should be supported.
Buildbot seems to be explicitly not using a PTY. From the the top of
the test output:
make buildbottest
in dir /Users/buildbot/buildarea/trunk.parc-leopard-1/build (timeout
1800 secs)
watching logfiles {}
argv: ['make', 'buildbottest']
[...]
closing stdin
using PTY: False
This output is telling you that the build slave isn't giving the child
processes it creates a pty. What I had in mind was writing the test to
create a new pty, instead of trying to use the controlling tty. So
basically, the two things are completely unrelated and this buildbot
configuration isn't hurting anything (and in fact is likely helping
quite a few things, so I suggest leaving it alone).
I believe this is specified by the build master.
This test seems to work on Ubuntu and FreeBSD, though.
That's interesting. I wonder if those slaves are able to open /dev/tty
for some reason? The slave is supposed to detach from the controlling
terminal when it daemonizes. There could be a bug in that code, I
suppose, or the slaves could be running without daemonization for some
reason. The operators would have to tell us about that, I think. Or,
another possibility is that /dev/tty doesn't work how I expect it to and
on Ubuntu and FreeBSD it can be opened even if you don't have a
controlling terminal. Hopefully not, though.
Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 05:29 pm, mar...@v.loewis.de wrote: Am 30.06.2010 13:32, schrieb exar...@twistedmatrix.com: On 05:24 am, mar...@v.loewis.de wrote: Seems to work fine. So this I don't understand. Any ideas, anyone? Didn't we discuss this before? The buildbot slave has no controlling terminal anymore, hence it cannot open /dev/tty. If you are curious, just patch your checkout to output the exact errno (e.g. to stdout), and trigger a build through the web. Could the test be rewritten (or supplemented) to use a pty? Most or perhaps all of the same operations should be supported. I'm not sure. It uses TIOCGPGRP, basically to establish that ioctl can also put results into a Python array (IIUC). This goes back to http://bugs.python.org/555817 Somebody rewriting it would need to make sure the original test purpose is still met. Absolutely. And even so, it may still make sense to run the test against both /dev/tty and a pty (or whatever subset of those things can be acquired in the testing environment). You can do a TIOCGPGRP on a new pty (created by os.openpty) but it produces somewhat less interesting results than doing it on /dev/tty. FIONREAD might be a nice alternative. It produces interesting (ie, non- zero) values in an easily predictable/controllable way (it tells you how many bytes are in the read buffer). Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] OS X buildbots: why am I skipping these tests?
On 06:46 pm, exar...@twistedmatrix.com wrote: On 04:44 pm, solip...@pitrou.net wrote: On Wed, 30 Jun 2010 09:00:09 PDT Bill Janssen jans...@parc.com wrote: So, my question then is, why are these skips unexpected? Seems to me that if this is the case, this test will never run on any platform. You can change the value of the usepty option in your buildbot.tac. (you will also have to restart the buildslave process) But don't do this. The usepty option is completely unrelated to the suggestion I was making. Flipping it to True will only cause other things to break and have no impact on this test. Ah, sorry. I confused myself. The option is related. But it will also break other things, so I still would recommend looking for other solutions. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Library Support in 3.x (Was: email package status in 3.X)
On 10:59 am, arcri...@gmail.com wrote: You mean Twisted support, because library support is at the point where there are fewer actively maintained packages not yet ported than those which are. Of course if your Python experience is hyper-focused to one framework that isn't ported yet, it will certainly seem like a lot, and you guys who run #Python are clearly hyper-focused on Twisted. Arc, This isn't about Twisted. Let's not waste everyone's time by trying to make it into a conflict between Twisted users and the rest of the Python community. You listed six other major packages that you yourself use that aren't available on Python 3 yet, so why are you trying to say here that this is all about Twisted? [snip] This anti-Py3 rhetoric is damaging to the community and needs to stop. We're moving forward toward Python 3.2 and beyond, complaining about it only saps valuable developer time (including your own) from getting these libraries you need ported faster. No, it's not damaging. Critical self-evaluation is a useful tool. Trying to silence differing perspectives is what's damaging to the community. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python Library Support in 3.x (Was: email package status in 3.X)
On 01:09 pm, arcri...@gmail.com wrote: [snip] It is not critical self-evaluation to repeat Python 3 is not ready as litany in #Python and your supporting website. I use the word litany here because #Python refers users to what appears to be a religious website http://python-commandments.org/python3.html It's not my website. I don't own the domain, I don't control the hosting, I didn't generate the content, I have no access to change anything on it. I've barely even frequent #python in the last three years. Perhaps you were directing those comments at Stephen Thorne though (although I don't know if he's any more involved in it than I am so don't take this as anything but idle speculation). I have further witnessed (and even been the other party to) you and other ops in #Python telling package developers, who have clearly said that they are working to port their legacy package to Py3, that Python 3 is not ready. I'm not going to condone or condemn events which I didn't observe. However you've never witnessed me discouraging developers who were actively porting software to Python 3 because I've never done it. I'm sure this was an honest mistake and you simply confused me with someone else. Besides rally against it what have you, as a Twisted developer, done regarding the Python 3 migration process? This, however, I find extremely insulting. I don't answer to you. The only reason I'm replying at all is to correct the two pieces of misinformation in your message. I don't see how this discussion can go anywhere productive, so I'll do my best to make this my last post on the subject. Obviously I made a mistake posting to the thread at all. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ssl
On 08:34 am, krist...@ccpgames.com wrote: Hello there. I wanted to do some work on the ssl module, but I was a bit daunted at the prerequisites. Is there anywhere that I can get at precompiled libs for the openssl that we use? In general, gettin all those external projects seem to be complex to build. Is there a fast way? I take it the challenge is that you want to do development on Windows? If so, this might help: http://www.slproweb.com/products/Win32OpenSSL.html It's what I use for any Windows pyOpenSSL development I need to do. What I want to do, is to implement a separate BIO for OpenSSL, one that calls back into python for writes and reads. This is so that I can use my own sockets implementation for the actual IO, in particular, I want to funnel the encrypted data through our IOCompletion-based stackless sockets. For what it's worth, Twisted's IOCP SSL support is implemented using pyOpenSSL's support of OpenSSL memory BIOs. This is a little different from your idea: memory BIOs are a built-in part of OpenSSL, and just give you a buffer from which you can pull whatever bytes OpenSSL wanted to write (or a buffer into which to put bytes for OpenSSL to read). I suspect this would work well enough for your use case. Being able to implement an actual BIO in Python would be pretty cool, though. If successful, I think this would be a useful addition to ssl. You would do something like: class BIO(): def write(): pass def read(): pass from ssl.import bio = BIO() ssl_socket = ssl.wrap_bio(bio, ca_certs=...) Hopefully this would integrate more nicely with the recent work Antoine has done with SSL contexts. The preferred API for creating an SSL connection is now more like this: import ssl ctx = ssl.SSLContext(...) conn = ctx.wrap_socket(...) So perhaps you want to add a wrap_bio method to SSLContext. In fact, this would be the more general API, and could supercede wrap_socket: after all, socket support is just implemented with the socket BIOs. wrap_socket would become a simple wrapper around something like wrap_bio(SocketBIO(socket)). I am new to OpenSSL, I haven't even looked at what a BIO looks like, but I read this: http://marc.info/?l=openssl- usersm=99909952822335w=2 which indicates that this ought to be possible. And before I start experimenting, I need to get my OpenSSL external ready. Any thoughts? It should be possible. One thing that's pretty tricky is getting threading right, though. Python doesn't have to deal with this problem yet, as far as I know, because it never does something that causes OpenSSL to call back into Python code. Once you have a Python BIO implementation, this will clearly be necessary, and you'll have to solve this. It's certainly possible, but quite fiddly. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] robots exclusion file on the buildbot pages?
On 05:48 pm, solip...@pitrou.net wrote: Hello, The buildbots are sometimes subject to a flood of svn exception errors. It has been conjectured that these errors are caused by Web crawlers pressing force build buttons without filling any of the fields (of course, the fact that we get such ugly errors in the buildbot results, rather than a clean error message when pressing the button, is a buildbot bug in itself). Couldn't we simply exclude all crawlers from the buildbot Web pages? Most (all?) legitimate crawlers won't submit forms. Do you think there's a non-form link to the force build URL (which _will_ accept a GET request to mean the same thing as a POST)? One thing I have noticed is that spammers find these forms and submit them with garbage. We can probably suppose that such people are going to ignore a robots.txt file. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] robots exclusion file on the buildbot pages?
On 08:32 pm, mar...@v.loewis.de wrote: I'd find it useful if the branch field was a choice pull-down listing valid branches, rather than a plain text field, and if the revision field always defaulted to HEAD. Seems to me that since the form is coming from the buildmaster, that should be possible. Unfortunately, these forms are deeply hidden in the buildbot code. So I'd rather avoid editing them, or else upgrading to the next buildbot version becomes even more tedious. Someone sufficiently interested in this feature could work with buildbot upstream to get the feature added to an upcoming release, though (obviously). Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] configuring the buildbot to skip some tests?
On 03:17 am, jans...@parc.com wrote: I've got parc-tiger-1 up and running again. It's failing on test_tk, which makes sense, because it's running as a background twisted process, and thus can't access the window server. I should configure that out. You can run it in an xvfb. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anyone can do patch reviews (was: Enhanced tracker privileges...)
On 01:38 pm, rdmur...@bitdance.com wrote: On Tue, 27 Apr 2010 11:15:49 +1000, Steven D'Aprano st...@pearwood.info wrote: No, of course not. There are always other reasons, the biggest is too many things to do and not enough time to do it. If I did review patches, would they be accepted on the strength on my untrusted reviews? It is very very helpful for *anyone* to review patches. Let's see if I can clarify the process a little. (This is, of course, my take on it, others can chime in if they think I got anything wrong.) Someone submits a bug. Someone submits a patch to fix that bug (or add the enhancement). Is that patch ready for commit? No. Is it ready for *commit review* (ie: someone with commit privileges to look at it with an eye toward committing it)? Probably not. What makes a patch ready for commit review? The patch should: 1) conform to pep 7/8 2) have unit tests that fail before the patch and succeed after 3) have documentation updates if needed 4) have a py3k port *if and only if* the port is non-trivial (well, if someone wants to add one when it is trivial that's OK, but it probably won't get used) 5) if it is at all likely to have system dependencies, be tested on at least linux and windows This list would make a good addition to one of the cpython development pages. If potential contributors could find this information, then they'd be much more likely to participate by doing reviews. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Enhanced tracker privileges for dangerjim to do triage.
On 09:39 pm, solip...@pitrou.net wrote: skip at pobox.com writes: Sean However, I will step up for him and say that I've known him a Sean decade, and he's very trustworthy. He has been the president (we Sean call that position Maximum Leader) of our Linux Users Group here Sean for 5 years or so. Given that Sean is vouching for him I'm fine with it. I'm not sure I agree. Of course it could be argued the risk is minimal, but I think it's better if all people go through the same path of proving their motivation and quality of work. And if there's something wrong with that process we'd better address it than give random privileges to people we like :) +1 As others have said, I don't have any problem with Sean's judgement. That's not what's being questioned here. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Enhanced tracker privileges for dangerjim to do triage.
On 25 Apr, 11:18 pm, st...@holdenweb.com wrote: Tres Seaver wrote: Antoine Pitrou wrote: skip at pobox.com writes: Sean However, I will step up for him and say that I've known him a Sean decade, and he's very trustworthy. He has been the president (we Sean call that position Maximum Leader) of our Linux Users Group here Sean for 5 years or so. Given that Sean is vouching for him I'm fine with it. I'm not sure I agree. Of course it could be argued the risk is minimal, but I think it's better if all people go through the same path of proving their motivation and quality of work. And if there's something wrong with that process we'd better address it than ^^^ Don't overlook this part of Antoine's post. give random privileges to people we like :) I think there is a definite unpriced externality to keeping the process barriers high here. I don't belive from conversations at the language summit / PyCon that the community is being overrun with hordes of unworthies clamoring to triage Python bugs: rather the opposite, in fact. It seems to me that backing from an established community member ought to be enough to get a prospective triageur at least provisional roles to do the work, with the caveat that it might be revoked it it didn't turn out well. If it does turn out well, then look to *expand* that user's roles in the community, with a nice helping of public acclaim to go with it. I am not arguing for making exceptions for friends here; rather that the acknowledged issues with inclusiveness / espansion of the developer community require making changes to the rules to encourage more participation. BTW, language like prov[ing] their motivation is itself demotivating, and likely contributes to the status quo ante. With my PSF hat on I'd like to support Tres here (and, by extension, Sean's proposal). Lowering the barriers of entry is a desirable goal. If adding people created work for already-busy developers then I'd be against it*, but with Sean offering to mentor his new protege and ensure that he limits his role to triage initially that doesn't seem to be an issue. Maybe it's time to review the way people prove their motivation and the quality of their work? Sounds good. Why is the barrier for this permission any higher than someone asking for it? Is there really a need to protect against contributors with malicious intent? I think there should be a page on python.org that says all contributors are welcome, and one way to become a contributor is to wrangle the issue tracker, and explains what this involves (I don't really have any idea, actually; I assume it's things like setting the owner of new tickets to someone who might actually fix it, things that would happen automatically if roundup had the right information), and then anyone who steps up gets the necessary access. Jean-Paul regards Steve * I'd be against it, but I'd fight to change the development process so that adding new people *didn't* create work. We should, in my opinion, be looking for a continual influx of new worker bees. -- Steve Holden +1 571 484 6266 +1 800 494 3119 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] MSDN licenses available for python-dev
On 02:56 pm, techto...@gmail.com wrote: Twisted folks will surely appreciate any help and may be able to contribute back. http://twistedmatrix.com/trac/wiki/Windows Extra Windows and VS licenses would certainly be helpful for Twisted development, and might lead indirectly to CPython/Windows improvements. Is this the kind of use for which it is appropriate to request an MSDN license via the PSF? Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Getting an optional parameter instead of creating a socket internally (was: Re: stdlib socket usage and quot; keepalivequot; )
On 12 Apr, 11:19 pm, j...@jcea.es wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/13/2010 12:47 AM, Antoine Pitrou wrote: Jesus Cea jcea at jcea.es writes: PS: socket.setdefaulttimeout() is not enough, because it could shutdown a perfectly functional connection, just because it was idle for too long. The socket timeout doesn't shutdown anything. It just puts a limit on how much time recv() and send() can block. Then it's up to you to detect whether the server is still alive (for example by pinging it through whatever means the application protocol gives you). A regular standard library (let say, poplib) would abort, after getting the timeout exception. 4. Modify client libraries to accept a new optional socket-like object as an optional parameter. This would allow things like transparent compression or encryption, or to replace the socket connection by anything else (read/write to shared memory or database, for example). This could be useful too. I have been thinking about this for years. Do you actually think this could be formally proposed?. Every once in a while I make a little bit more progress on the PEP I'm working on for this. If you want to talk more about this, you can find me in #python-dev or #twisted on freenode. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ffi junk messages
On 01:29 pm, mar...@v.loewis.de wrote: Mark Dickinson wrote: On Wed, Apr 7, 2010 at 1:39 PM, Jeroen Ruigrok van der Werven asmo...@in-nomine.org wrote: Before I file a bug report, is anyone else seeing this (in my case on FreeBSD 8): Modules/_ctypes/libffi/src/x86/sysv.S:360: Error: junk at end of line, first unrecognized character is `@' Modules/_ctypes/libffi/src/x86/sysv.S:387: Error: junk at end of line, first unrecognized character is `@' Modules/_ctypes/libffi/src/x86/sysv.S:423: Error: junk at end of line, first unrecognized character is `@' It's on the buildbots, too. See: http://www.python.org/dev/buildbot/builders/x86%20FreeBSD%20trunk/builds/208/steps/compile/logs/stdio Instead of submitting a bug report, it would be better to submit a In *addition* to submitted a bug report, surely. :) patch, though. Can you try having the build process use freebsd.S instead of sysv.S? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python- dev/exarkun%40twistedmatrix.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On 08:56 pm, digitalx...@gmail.com wrote: On Mon, Mar 8, 2010 at 12:04 PM, Dj Gilcrease digitalx...@gmail.com wrote: A style I have used in my own code in the past is a Singleton class with register and create methods, where the register takes a name(string) and the class and the create method takes the name and *args, **kwargs and acts as a factory. So I decided to play with this design a little and since I made it a singleton I decided to place all the thread/process tracking and exit handle code in it instead of having the odd semi-global scoped _shutdown, _thread_references, _remove_dead_thread_references and _python_exit objects floating around in each executor file, seems to work well. The API would be from concurrent.futures import executors executor = executors.create(NAME, *args, **kwargs) # NAME is 'process' or 'thread' by default To create your own executor you create your executor class and add the following at the end Getting rid of the process-global state like this simplifies testing (both testing of the executors themselves and of application code which uses them). It also eliminates the unpleasant interpreter shutdown/module globals interactions that have plagued a number of stdlib systems that keep global state. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On 02:10 am, br...@sweetapp.com wrote: On 7 Mar 2010, at 03:04, Phillip J. Eby wrote: At 05:32 AM 3/6/2010, Brian Quinlan wrote: Using twisted (or any other asynchronous I/O framework) forces you to rewrite your I/O code. Futures do not. Twisted's Deferred API has nothing to do with I/O. I see, you just mean the API and not the underlying model. We discussed the Deferred API on the stdlib-sig and I don't think that anyone expressed a preference for it over the one described in the PEP. Do you have any concrete criticism? From reading some of the stdlib-sig archives, it sounds like there is general agreement that Deferreds and Futures can be used to complement each other, and that getting code that is primarily Deferred-based to integrate with Future-based code or vice versa should eventually be possible. Do I have the right sense of people's feelings? And relatedly, once Futures are accepted and implemented, are people going to use them as an argument to exclude Deferreds from the stdlib (or be swayed by other people making such arguments)? Hopefully not, given what I read on stdlib-sig, but it doesn't hurt to check... Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On 05:06 pm, c...@hagenlocher.org wrote: On Fri, Mar 5, 2010 at 8:35 AM, Jesse Noller jnol...@gmail.com wrote: On Fri, Mar 5, 2010 at 11:21 AM, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: On Fri, Mar 5, 2010 at 12:03 AM, Brian Quinlan br...@sweetapp.com wrote: import futures +1 on the idea, -1 on the name. It's too similar to from __future__ import Futures is a common term for this, and implemented named this in other languages. I don't think we should be adopting things that are common, and found elsewhere and then renaming them. Another common term for this is a promise. Promises aren't exactly the same. This would be a particularly bad name to apply here. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
On 07:10 pm, gu...@python.org wrote: On Fri, Mar 5, 2010 at 10:30 AM, exar...@twistedmatrix.com wrote: On 05:06 pm, c...@hagenlocher.org wrote: On Fri, Mar 5, 2010 at 8:35 AM, Jesse Noller jnol...@gmail.com wrote: On Fri, Mar 5, 2010 at 11:21 AM, Daniel Stutzbach dan...@stutzbachenterprises.com wrote: On Fri, Mar 5, 2010 at 12:03 AM, Brian Quinlan br...@sweetapp.com wrote: import futures +1 on the idea, -1 on the name. It's too similar to from __future__ import Futures is a common term for this, and implemented named this in other languages. I don't think we should be adopting things that are common, and found elsewhere and then renaming them. Another common term for this is a promise. Promises aren't exactly the same. This would be a particularly bad name to apply here. Please explain. Even the Wikipedia article (http://en.wikipedia.org/wiki/Futures_and_promises), despite promising to explain the difference, didn't explain it. The explicit futures on the wikipedia page seems to cover what is commonly referred to as a future. For example, Java's futures look like this. The implicit futures are what is generally called a promise. For example, E's promises look like this. Though the difference is mainly one of API, it turns out to make a significant difference in what you can accomplish. Promises are much more amenable to the pipelining optimization, for example. They're also much harder to implement in Python without core language changes. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
On 07:48 pm, gu...@python.org wrote: On Fri, Feb 12, 2010 at 7:49 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: My *hope* is that we provide a general solution, possibly based on all or part of Test Resources, with an easy mechanism for the setUpClass and setUpModule but also solves the more general case of sharing fixtures between tests. If that doesn't turn out to be possible then we'll go for a straight implementation of setUpClass / setUpModule. I'm hoping I can get this together in time for the PyCon sprints... Do you have a reference for Test Resources? http://pypi.python.org/pypi/testresources/0.2.2 [snip] However from this example I *cannot* guess whether those resources are set up and torn down per test or per test class. Also the notation The idea is that you're declaring what the tests need in order to work. You're not explicitly defining the order in which things are set up and torn down. That is left up to another part of the library to determine. One such other part, OptimisingTestSuite, will look at *all* of your tests and find an order which involves the least redundant effort. You might have something else that breaks up the test run across multiple processes and uses the resource declarations to run all tests requiring one thing in one process and all tests requiring another thing somewhere else. You might have still something else that wants to completely randomize the order of tests, and sets up all the resources at the beginning and tears them down at the end. Or you might need to be more memory/whatever conscious than that, and do each set up and tear down around each test. The really nice thing here is that you're not constrained in how you group your tests into classes and modules by what resources you want to use in them. You're free to group them by what they're logically testing, or in whatever other way you wish. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
On 08:27 pm, gu...@python.org wrote: On Fri, Feb 12, 2010 at 12:20 PM, exar...@twistedmatrix.com wrote: The idea is that you're declaring what the tests need in order to work. You're not explicitly defining the order in which things are set up and torn down. �That is left up to another part of the library to determine. One such other part, OptimisingTestSuite, will look at *all* of your tests and find an order which involves the least redundant effort. So is there a way to associate a cost with a resource? I may have one resource which is simply a /tmp subdirectory (very cheap) and another that requires starting a database service (very expensive). I don't think so. From the docs, This TestSuite will introspect all the test cases it holds directly and if they declare needed resources, will run the tests in an order that attempts to minimise the number of setup and tear downs required.. You might have something else that breaks up the test run across multiple processes and uses the resource declarations to run all tests requiring one thing in one process and all tests requiring another thing somewhere else. I admire the approach, though I am skeptical. We have a thing to split up tests at Google which looks at past running times for tests to make an informed opinion. Have you thought of that? You might have still something else that wants to completely randomize the order of tests, and sets up all the resources at the beginning and tears them down at the end. Or you might need to be more memory/whatever conscious than that, and do each set up and tear down around each test. How does your code know the constraints? To be clear, aside from OptimisingTestSuite, I don't think testresources implements any of the features I talked about. They're just things one might want to and be able to implement, given a test suite which uses testresources. The really nice thing here is that you're not constrained in how you group your tests into classes and modules by what resources you want to use in them. You're free to group them by what they're logically testing, or in whatever other way you wish. I guess this requires some trust in the system. :-) Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] setUpClass and setUpModule in unittest
On 02:41 pm, ole...@gmail.com wrote: On Thu, Feb 11, 2010 at 7:41 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: On 11/02/2010 12:30, Nick Coghlan wrote: Michael Foord wrote: I'm not sure what response I expect from this email, and neither option will be implemented without further discussion - possibly at the PyCon sprints - but I thought I would make it clear what the possible directions are. I'll repeat what I said in the python-ideas thread [1]: with the advent of PEP 343 and context managers, I see any further extension of the JUnit inspired setUp/tearDown nomenclature as an undesirable direction for Python to take. Instead, I believe unittest should be adjusted to allow appropriate definition of context managers that take effect at the level of the test module, test class and each individual test. For example, given the following method definitions in unittest.TestCase for backwards compatibility: � def __enter__(self): � � self.setUp() � def __exit__(self, *args): � � self.tearDown() The test framework might promise to do the following for each test: � with get_module_cm(test_instance): # However identified � � with get_class_cm(test_instance): # However identified � � � with test_instance: # ** � � � � test_instance.test_method() What Nick pointed out is the right direction (IMHO), and the one I had Why? Change for the sake of change is not a good thing. What are the advantages of switching to context managers for this? Perhaps the idea was more strongly justified in the python-ideas thread. Anyone have a link to that? in mind since I realized that unittest extensibility is the key feature that needs to be implemented . I even wanted to start a project using this particular architecture to make PyUnit extensible. What makes you think it isn't extensible now? Lots of people are extending it in lots of ways. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
