Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Mon, 2005-03-21 at 17:32 +1200, Greg Ewing wrote: > > On 18 March 2005, Donovan Baarda said: > > >>Many Python library methods and classes like select.select(), os.popen2(), > >>and subprocess.Popen() return and/or operate on builtin file objects. > >>However even simple applications of these methods and classes require the > >>files to be in non-blocking mode. > > I don't agree with that. There's no need to use non-blocking > I/O when using select(), and in fact things are less confusing > if you don't. You would think that... and the fact that select, popen2 etc all use file objects encourage you to think that. However, this is a trap that can catch you out badly. Check the attached python scripts that demonstrate the problem. Because staller.py outputs and flushes a fragment of data smaller than selector.py uses for its reads, the select statement is triggered, but the corresponding read blocks. A similar thing can happen with writes... if the child process consumes a fragment smaller than the write buffer of the selector process, then the select can trigger and the corresponding write can block because there is not enough space in the file buffer. The only ways to ensure that a select process does not block like this, without using non-blocking mode, are; 1) use a buffer size of 1 in the select process. 2) understand the child process's read/write behaviour and adjust the selector process accordingly... ie by making the buffer sizes just right for the child process, Note that it all interacts with the file objects buffer sizes too... making for some extremely hard to debug intermittent behaviour. > >>The read method's current behaviour needs to be documented, so its actual > >>behaviour can be used to differentiate between an empty non-blocking read, > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > >>non-blocking read. > > Isn't that unix-specific? The file object is supposed to > provide a more or less platform-independent interface, I > thought. I think the fread/fwrite and read/write behaviour is posix standard and possibly C standard stuff... so it _should_ be the same on other platforms. -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/ selector.py Description: application/python staller.py Description: application/python ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Mon, 21 Mar 2005, Donovan Baarda wrote: > > I don't agree with that. There's no need to use non-blocking > > I/O when using select(), and in fact things are less confusing > > if you don't. > > You would think that... and the fact that select, popen2 etc all use > file objects encourage you to think that. However, this is a trap that > can catch you out badly. Check the attached python scripts that > demonstrate the problem. This is no "trap". When select() indicates that you can write or read, it means that you can write or read at least one byte. The .read() and .write() file methods, however, always writes and reads *everything*. These works, basically, just like fread()/fwrite(). > The only ways to ensure that a select process does not block like this, > without using non-blocking mode, are; > > 1) use a buffer size of 1 in the select process. > > 2) understand the child process's read/write behaviour and adjust the > selector process accordingly... ie by making the buffer sizes just right > for the child process, 3) Use os.read / os.write. > > >>The read method's current behaviour needs to be documented, so its actual > > >>behaviour can be used to differentiate between an empty non-blocking read, > > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > > >>non-blocking read. > > > > Isn't that unix-specific? The file object is supposed to > > provide a more or less platform-independent interface, I > > thought. > > I think the fread/fwrite and read/write behaviour is posix standard and > possibly C standard stuff... so it _should_ be the same on other > platforms. Sorry if I've misunderstood your point, but fread()/fwrite() does not return EAGAIN. /Peter Åstrand <[EMAIL PROTECTED]> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] docstring before function declaration
Rule #1: If the docstring is the first line of a module, it's the module's docstring. Rule #2: If the docstring comes right before a class/function, it's that class/function's docstring. > How do you distinguish between a docstring at the > top of a module > that's immediately followed by a function? Is it > the module docstring > or the function docstring? It's both. The docstring would be assigned to both the module and the function. This is a *good* thing when there is a module with only one function in it. i.e. there should only be one docstring for both, and this saves repetition of that docstring. If a programmer wanted a docstring for the function but not the module, a blank first line would do the trick. A docstring for the module but not the function? Put a blank line between the module's docstring and the function. --Nick Jacobson __ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
G'day, From: "Peter Astrand" <[EMAIL PROTECTED]> > On Mon, 21 Mar 2005, Donovan Baarda wrote: [...] > This is no "trap". When select() indicates that you can write or read, it > means that you can write or read at least one byte. The .read() and > .write() file methods, however, always writes and reads *everything*. > These works, basically, just like fread()/fwrite(). yep, which is why you can only use them reliably in a select loop if you read/write one byte at a time. > > The only ways to ensure that a select process does not block like this, > > without using non-blocking mode, are; > > > > 1) use a buffer size of 1 in the select process. > > > > 2) understand the child process's read/write behaviour and adjust the > > selector process accordingly... ie by making the buffer sizes just right > > for the child process, > > 3) Use os.read / os.write. [...] but os.read / os.write will block too. Try it... replace the file read/writes in selector.py. They will only do partial reads if the file is put into non-blocking mode. > > I think the fread/fwrite and read/write behaviour is posix standard and > > possibly C standard stuff... so it _should_ be the same on other > > platforms. > > Sorry if I've misunderstood your point, but fread()/fwrite() does not > return EAGAIN. no, fread()/fwrite() will return 0 if nothing was read/written, and ferror() will return EAGAIN to indicated that it was a "would block" condition at least I think it does... the man page simply says ferror() returns a non-zero value. Looking at the python implementation of file.read(), for an empty fread() where ferror() is non zero, it only raises IOError if errno is not EAGAIN or EWOULDBLOCK. It blindly clearerr()'s for any other partial read. The implementation of file.write() raises IOError whenever there is an incomplete write. So it looks, as I pointed out in the draft PEP, that the current file.read() supports non-blocking mode, but file.write() doesn't... a bit asymmetric :-) Donovan Baardahttp://minkirri.apana.org.au/~abo/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blockingmode.
G'day, From: "Greg Ward" <[EMAIL PROTECTED]> > On 18 March 2005, Donovan Baarda said: [...] > > Currently the built in file type does not support non-blocking mode very > > well. Setting a file into non-blocking mode and reading or writing to it > > can only be done reliably by operating on the file.fileno() file descriptor. > > This requires using the fnctl and os module file descriptor manipulation > > methods. > > Is having to use fcntl and os really so awful? At least it requires > the programmer to prove he knows what he's doing putting this file > into non-blocking mode, and that he really wants to do it. ;-) It's not that bad I guess... but then I'm proposing a very minor change to fix it. The bit that annoys me is popen2() and select() give this false sense of "File Object compatability", when in reality you can't use them reliably with file objects. It is also kind of disturbing that file.read() actually does work in non-blocking mode, but file.write() doesn't. The source for file.read() shows a fair bit of effort towards making it work for non-blocking mode... why not do the same for file.write()? > > Details > > === > > > > The documentation of file.read() warns; "Also note that when in non-blocking > > mode, less data than what was requested may be returned, even if no size > > parameter was given". An empty string is returned to indicate an EOF > > condition. It is possible that file.read() in non-blocking mode will not > > produce any data before EOF is reached. Currently there is no documented > > way to identify the difference between reaching EOF and an empty > > non-blocking read. > > > > The documented behaviour of file.write() in non-blocking mode is undefined. > > When writing to a file in non-blocking mode, it is possible that not all of > > the data gets written. Currently there is no documented way of handling or > > indicating a partial write. > > That's more interesting and a better motivation for this PEP. The other solution to this of course is to simply say "file.read() and file.write() don't work in non-blocking mode", but that would be a step backwards for the current file.read(). > > file.read([size]) Changes > > -- > > > > The read method's current behaviour needs to be documented, so its actual > > behaviour can be used to differentiate between an empty non-blocking read, > > and EOF. This means recording that IOError(EAGAIN) is raised for an empty > > non-blocking read. > > > > > > file.write(str) Changes > > > > > > The write method needs to have a useful behaviour for partial non-blocking > > writes defined, implemented, and documented. This includes returning how > > many bytes of "str" are successfully written, and raising IOError(EAGAIN) > > for an unsuccessful write (one that failed to write anything). > > Proposing semantic changes to file.read() and write() is bound to > raise hackles. One idea for soothing such objections: only make these > changes active when setblocking(False) is in effect. I.e., a > setblocking(True) file (the default, right?) behaves as you described > above, warts and all. (So old code that uses fcntl() continues to > "work" as before.) But files that have had setblocking(False) called > could gain these new semantics that you propose. There is nothing in this proposal that would break or change the behaviour of any existing code, unless it was relying on file.write() returning None. or checking that file objects don't have a "setblocking" method. Note that the change for file.read() is simply to document the current behaviour... not to actually change it. The change for file.write() is a little more dramatic, but I really can't imagine anyone relying on file.write() returning None. A compromise would be to have file.write() return None in blocking mode, and a count in non-blocking mode... but I still can't believe people will rely on it returning None :-) It would be more useful to always return a count, so that methods using them could handle both modes easily. Note that I did consider some more dramatic changes that would have made them even easier to use. Things like raising an exception for EOF instead of EAGAIN would actually make a lot of things easier to code... but it would be too big a change. Donovan Baardahttp://minkirri.apana.org.au/~abo/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Mon, 21 Mar 2005, Donovan Baarda wrote: > > > The only ways to ensure that a select process does not block like this, > > > without using non-blocking mode, are; > > 3) Use os.read / os.write. > [...] > > but os.read / os.write will block too. No. >Try it... replace the file > read/writes in selector.py. They will only do partial reads if the file is > put into non-blocking mode. I've just tried it; I replaced: data = o.read(BUFF_SIZE) with: data = os.read(o.fileno(), BUFF_SIZE) Works for me without any hangs. Another example is the subprocess module, which does not use non-blocking mode in any way. (If you are using pipes, however, you shouldn't write more than PIPE_BUF bytes in each write.) > > > I think the fread/fwrite and read/write behaviour is posix standard and > > > possibly C standard stuff... so it _should_ be the same on other > > > platforms. > > > > Sorry if I've misunderstood your point, but fread()/fwrite() does not > > return EAGAIN. > > no, fread()/fwrite() will return 0 if nothing was read/written, and ferror() > will return EAGAIN to indicated that it was a "would block" condition at > least I think it does... the man page simply says ferror() returns a > non-zero value. fread() should loop internally on EAGAIN, in blocking mode. /Peter Åstrand <[EMAIL PROTECTED]> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Mon, 2005-03-21 at 11:42 +0100, Peter Astrand wrote: > On Mon, 21 Mar 2005, Donovan Baarda wrote: > > > > > The only ways to ensure that a select process does not block like this, > > > > without using non-blocking mode, are; > > > > 3) Use os.read / os.write. > > [...] > > > > but os.read / os.write will block too. > > No. [...] Hmmm... you are right... that changes things. Blocking vs non-blocking becomes kinda moot if read/write will do partial writes in blocking mode. > fread() should loop internally on EAGAIN, in blocking mode. Yeah, I was talking about non-blocking mode... -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Mon, 21 Mar 2005 17:32:36 +1200, Greg Ewing <[EMAIL PROTECTED]> wrote: > > On 18 March 2005, Donovan Baarda said: > >>The read method's current behaviour needs to be documented, so its actual > >>behaviour can be used to differentiate between an empty non-blocking read, > >>and EOF. This means recording that IOError(EAGAIN) is raised for an empty > >>non-blocking read. > > Isn't that unix-specific? The file object is supposed to > provide a more or less platform-independent interface, I > thought. The whole thing is, I believe, highly Unix-specific. I say this because I am essentially a Windows programmer, and the proposal means almost nothing to me :-) More seriously, non-blocking IO and select-type readability checks are VERY different on Windows, and so I would expect the implementation of this chance to be completely different as well. The C standard says nothing about non-blocking IO. While POSIX might, that doesn't apply to Windows. Oh, and in case it's not obvious, I'm -1 on something "Unix-only" here. Python file objects are supposed to be cross-platform, in general. Paul. PS Donovan's sample code seems to be process-related - if so, isn't that what the new subprocess module was supposed to resolve (process-related communication in a platform-independent way)? If the only use case is with subprocesses, then is this change needed at all? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] docstring before function declaration
On Monday 21 March 2005 20:08, Nicholas Jacobson wrote: > > How do you distinguish between a docstring at the > > top of a module > > that's immediately followed by a function? Is it > > the module docstring > > or the function docstring? > > It's both. The docstring would be assigned to both > the module and the function. This is a *good* thing > when there is a module with only one function in it. > i.e. there should only be one docstring for both, and > this saves repetition of that docstring. > > If a programmer wanted a docstring for the function > but not the module, a blank first line would do the > trick. A docstring for the module but not the > function? Put a blank line between the module's > docstring and the function. Yuk. This is magic taken to a ridiculous level. Note that "blank lines" currently have no meaning in Python, and adding a meaning to them is not my idea of a good thing. -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] docstring before function declaration
Nicholas Jacobson wrote: IIRC, Guido once mentioned that he regretted not setting function docstrings to come before the function declaration line, instead of after. He did, but I don't know how strong that regret is. i.e. """This describes class Bar.""" class Bar: ... Or with a decorator: """This describes class Bar.""" @classmethod class Bar: ... Versus the current method: class Bar: """This describes class Bar.""" def foo: ... I am going to be -42 on this one. I personally love having the docstring below the definition line. So much so, in fact, that in personal C code I use the same style for documentation. I find it easier to browse the source since where a definition starts is much cleaner (yes, syntax highlighting and searching for ``\s*def `` works as well, but I am thinking when you are just scrolling). Beyond that I can't really rationalize it beyond just aesthetics at the moment. But I definitely prefer the current style. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Fwd: [Python-Dev] docstring before function declaration
Nicholas Jacobson wrote: IIRC, Guido once mentioned that he regretted not setting function docstrings to come before the function declaration line, instead of after. [ examples deleted ] I think that commenting the function before its declaration, at the same tabbed point, increases the code's readability. What do you think about making this change at some point (e.g. Python 3000)? Or if not, then having the option to toggle to this layout? Please don't. All class attributes are declared within the class. Pulling out the docstring would make it an exception. The current situation is very clear and easy to explain to newbies. If you feel the current position of the docstring may be the first attribute's docstring, insert a newline at the proper positions to separate the two. It works for me. --eric ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [AST] A somewhat less trivial patch than the last one. . .
I've put a first cut at generator expressions for the AST branch on Sourceforge. It's enough to get test_grammar to pass, and tinkering at the interactive prompt appears to work. The patch also fixes a problem with displaying interim results for functions entered at the interactive prompt (I noticed it when trying to get the AST branch genexp bytecode to roughly match that produced by Python 2.4). I didn't check if classes suffer from the same problem, though. Finally, the patch fixes asdl_seq_new to avoid allocating large chunks of memory when passed a size of zero. (This one bit me when trying to get generator expressions to work as arguments - a function call may end up with zero length asdl sequences for either the positional or the keyword arguments). Patch number is 1167628. Cheers, Nick. P.S. Could the powers-that-be add me to the developer list on Sourceforge? I'm interested in better access to the SF trackers, rather than CVS access, though. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils/tests test_dist.py, 1.1, 1.2
[EMAIL PROTECTED] writes: > PEP 314 implementation (client side): I'm not sure where I should post this, but shouldn't there be a way to specify the encoding of the metadata? There are people (not me, fortunately), with umlauts in their names, for example. Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib/distutils/tests test_dist.py, 1.1, 1.2
On Mon, 21 Mar 2005 16:08:57 +0100, Thomas Heller <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] writes: > > > PEP 314 implementation (client side): > > I'm not sure where I should post this, but shouldn't there be a way to > specify the encoding of the metadata? There are people (not me, > fortunately), with umlauts in their names, for example. UTF-8 as the default would accommodate most uses, but it does seem essential to have some way of specifying an encoding. Jeremy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] distutils/PyPI package metadata fields
On Monday 21 March 2005 10:08, Thomas Heller wrote: > I'm not sure where I should post this, but shouldn't there be a way to > specify the encoding of the metadata? There are people (not me, > fortunately), with umlauts in their names, for example. Agreed. I think there are a number of additional metadata fields which would be valuable, but which don't exist. Given that PEP 314 is close to being completely implemented, that's probably not the place to add them. I think a new PEP should be written to describe the new fields, and call that Metadata-Version 1.2. Some sort of Metadata-Encoding field should be included. There should also be a field for the version of Python that's required. -Fred -- Fred L. Drake, Jr. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [AST] question about marshal_write_*() fxns
For those of you who don't know, I am sprinting on the AST branch here on PyCon. Specifically, I am fleshing out Python/compile.txt so that it can act as a good intro to new users and as a design doc. But one of things I am not sure of is what the marshal_write_*() functions in Python/Python-ast.c are used for. I assume they output to the marshal format, but there is mention of a byte stream format and so I thought it might be that as well. Anyone know which one it is? -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [AST] question about marshal_write_*() fxns
On Mon, Mar 21, 2005 at 11:53:04AM -0500, Brett C. wrote: > But one of things I am not sure of is what the marshal_write_*() functions > in Python/Python-ast.c are used for. I assume they output to the marshal > format, but there is mention of a byte stream format and so I thought it > might be that as well. I believe the idea is that they can be used to move an AST back and forth between Python "space" (e.g. you could build an AST using Python code and then marshal it and send it to the C compiler). Neil ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: docstring before function declaration
Brett C. wrote: I am going to be -42 on this one. I personally love having the docstring below the definition line I can't really rationalize > it beyond just aesthetics at the moment I completely agree that the current form is better. It reduces the temptation to use boilerplate docstrings. No comment is better than an uninformative comment. If you don't want to spend the time to write a comment, step back and let me read the code itself. If the docstring is below the declaration, you needn't repeat the declaration in the comment (and people are less tempted to do so). Documentation and code should come from a human mind, and should communicate to another human mind. Attempting to automate the task of documentation creates hard-to-read code, interrupted by large meaningless comments which, often as not, are copied from a template and incompletely editted to be appropriate to the given function. Sorry about the rant, but this is another of my hot buttons. The single most disappointing thing I encountered on one project in a large corporation was an operating system requirements document that was being developped. The group had, at one point, a twenty-two page document with no real content. Really, the twenty two pages included an introduction, conclusion, table of contents, appendix, and index. It just didn't have anything but section headings. It was a thrilling triumph of form over function; a real Suahuab aesthetic, to coin a term. --Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [AST] question about marshal_write_*() fxns
Neil Schemenauer wrote: On Mon, Mar 21, 2005 at 11:53:04AM -0500, Brett C. wrote: But one of things I am not sure of is what the marshal_write_*() functions in Python/Python-ast.c are used for. I assume they output to the marshal format, but there is mention of a byte stream format and so I thought it might be that as well. I believe the idea is that they can be used to move an AST back and forth between Python "space" (e.g. you could build an AST using Python code and then marshal it and send it to the C compiler). Jeremy emailed me and said the same thing. Name should be changed, though, to prevent this confusion. Jeremy thought maybe the name should be changed. Here are some ideas for a different name (thinking in terms of what the name would be changed to for the prefix of the function names): - byte_encode - linear_form - zephyr_encoding - flat_form - flat_prefix - prefix_form - scheme_form =) For those that don't know and it wasn't obvious from the names, the format is a prefix-based, linear form. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [AST] Procedure for AST Branch patches
Grant Olson wrote: Make sure "AST" is used in the subject line; e.g., "[AST]" at the beginning. Unfortunately the AST group is only available for patches; not listed for bug reports (don't know why; can this be fixed?). Other than that, just assign it to me since I will most likely be doing AST work in the near future. Brett, I sent a few outstanding ones your way. I hate to complain again, I know everyone involved are volunteers, etc, etc; but it'd be really nice if someone could take a look at '[ 742621 ] ast-branch: msvc project sync'. The patch is almost two years old now, has been updated for VC7.1 and still hasn't been checked in. Without this, any Windows developers who check out the ast-branch will experience a broken build. OK, thanks to John Ehresman here at PyCon sprint I got logistix's patch applied. Beyond a warning that a warning that decode_unicode() is never called and the parser module failing to compile under Windows everything should be fine for compiling the AST branch. Passing the test suite, though, is another question. =) -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Deprecating 2.2 old bugs
Going on with the old bugs checking, here are the results for 2.2. When I'll finish this will be put in an informational PEP. When I verified the bug, I filled two fields: - Summary: the same subject as in SF - Group: the bug's group at verifying time. - Bug #: the bug number - Verified: is the date when I checked the bug. - Action: is what I did then. If the bug survived the verification, the next two fields are applicable (if not, I put a dash, the idea is to keep this info easily parseable): - Final: is the action took by someone who eliminated the bug from that category (closed, moved to Py2.4, etc). - By: is the someone who did the final action. So, here's the info... Summary: docs need to discuss // and __future__.division Group:2.2 Bug #:449093 Verified: 25-Nov-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Final:Group changed to 2.4. By: facundobatista Summary: new int overflow handling needs docs Group:2.2 Bug #:454446 Verified: 25-Nov-2004 Action: Closed: Won't fix. Behaviour changed. Final:- By: - Summary: urllib2 proxyhandle won't work. Group:2.2 Bug #:487471 Verified: 25-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. For a comment is not clear to me if it's a bug or not. Final:Closed: Won't fix. By: facundobatista Summary: BasHTTPServer IE Mac 5.1 size problem Group:2.2 Bug #:812340 Verified: 08-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final:Closed: Won't fix. By: facundobatista Summary: Section 11.20: xmlrpclib Details Group:2.2 Bug #:791067 Verified: 11-Nov-2004 Action: Deprecation alerted. Doc changed a lot, don't know enough about the subject to be clear to me if the problem is solved or not. Final:Closed: Won't fix. By: facundobatista Summary: socket.inet_aton on 64bit platform Group:2.2 Bug #:767150 Verified: 11-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. Final:Closed: Won't fix. By: facundobatista Summary: Error using Tkinter embeded in C++ Group:2.2 Bug #:699068 Verified: 11-Nov-2004 Action: Deprecation alerted. I can not try it, don't have that context. For a comment is not clear to me if the problem actually existed. Final:Closed: Won't fix. By: facundobatista Summary: raw_input defers alarm signal Group:2.2 Bug #:685846 Verified: 11-Nov-2004 Action: Changed to Group Py2.3, there's a patch: #706406 Final:- By: - Summary: repr.repr not always safe Group:2.2 Bug #:666958 Verified: 15-Nov-2004 Action: Changed to Group Py2.3: it seems a bug to me. Final:- By: - Summary: win32 os.path.normpath not correct for leading slash cases Group:2.2 Bug #:665336 Verified: 16-Nov-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final:- By: - Summary: memory leaks when importing posix module Group:2.2 Bug #:613222 Verified: 21-Mar-2005 Action: Changed to Group Py2.3, considering original post. Final:- By: - Summary: xml.sax second time file loading problem Group:2.2 Bug #:606692 Verified: 15-Nov-2004 Action: Deprecation alerted. Final:Closed: Won't fix. By: facundobatista Summary: urllib ftp broken if Win32 proxy Group:2.2 Bug #:532007 Verified: 15-Nov-2004 Action: Deprecation alerted. Final:Closed: Won't fix. By: facundobatista Summary: Bugs in rfc822.parseaddr() Group:2.2 Bug #:531205 Verified: 24-Nov-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final:Closed. Invalid. By: loewis Summary: shelve update fails on "large"; entry Group:2.2 Bug #:523425 Verified: 25-Nov-2004 Action: Deprecation alerted. Can't discern if it's really a bug or not. Final:Closed: Fixed (the original submitter posted that is fixed in later version) By: facundobatista Summary: exception cannot be new-style class Group:2.2 Bug #:518846 Verified: 24-Nov-2004 Action: Changed to Group Py2.3: it seems a bug to me. Final:- By: - Summary: Missing docs for module imputil Group:2.2 Bug #:515751 Verified: 25-Nov-2004 Action: Changed to Group Py2.4: it seems a bug to me. Final:- By: - Regards, .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://pyar.decode.com.ar/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] docstring before function declaration
Nicholas Jacobson wrote: If a programmer wanted a docstring for the function but not the module, a blank first line would do the trick. A docstring for the module but not the function? Put a blank line between the module's docstring and the function. -1 on all this making of blank lines significant. Currently I can leave some space before/after a docstring without breaking anything. This can help readability, especially for class docstrings. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
Donovan Baarda wrote: Consider the following. This is pretty much the only way you can use popen2 reliably without knowing specific behaviours of the executed command; > ... fcntl.fcntl(child_in, fcntl.F_SETFL, flags | os.O_NONBLOCK) # \ ... # / fcntl.fcntl(child_out, fcntl.F_SETFL, flags | os.O_NONBLOCK)# \ I still don't believe you need to make these non-blocking. When select() returns a fd for reading/writing, it's telling you that the next os.read/os.write call on it will not block. Making the fd non-blocking as well is unnecessary and perhaps even undesirable. For 1) and 2), note that popen2 returns file objects, but as they cannot be reliably used as file objects, we ignore them and grab their fileno(). Why does popen2 return file objects if they cannot reliably be used? I would go along with giving file objects alternative read/write methods which behave more like os.read/os.write, maybe called something like readsome() and writesome(). That would eliminate the need to extract and manipulate the fds, and might make it possible to do some of this stuff in a more platform-independent way. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
Donovan Baarda wrote: On Mon, 2005-03-21 at 17:32 +1200, Greg Ewing wrote: I don't agree with that. There's no need to use non-blocking I/O when using select(), and in fact things are less confusing if you don't. Because staller.py outputs and flushes a fragment of data smaller than selector.py uses for its reads, the select statement is triggered, but the corresponding read blocks. Your selector.py is using file object read/write methods, not os.read and os.write. I fully agree that you can't reliably use stdio-style I/O in conjunction with select(). But as long as you use os-level I/O, there shouldn't be any need to make anything non-blocking. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Tue, 2005-03-22 at 12:49 +1200, Greg Ewing wrote: > Donovan Baarda wrote: > > > Consider the following. This is pretty much the only way you can use > > popen2 reliably without knowing specific behaviours of the executed > > command; > > > > ... > > fcntl.fcntl(child_in, fcntl.F_SETFL, flags | os.O_NONBLOCK) # \ > > ... # / > > fcntl.fcntl(child_out, fcntl.F_SETFL, flags | os.O_NONBLOCK)# \ > > I still don't believe you need to make these non-blocking. > When select() returns a fd for reading/writing, it's telling > you that the next os.read/os.write call on it will not block. > Making the fd non-blocking as well is unnecessary and perhaps > even undesirable. Yeah... For some reason I had it in my head that os.read/os.write would not do partial/incomplete reads/writes unless the file was in non-blocking mode. > > For 1) and 2), note that popen2 returns file objects, but as they cannot > > be reliably used as file objects, we ignore them and grab their > > fileno(). Why does popen2 return file objects if they cannot reliably be > > used? > > I would go along with giving file objects alternative read/write > methods which behave more like os.read/os.write, maybe called > something like readsome() and writesome(). That would eliminate > the need to extract and manipulate the fds, and might make it > possible to do some of this stuff in a more platform-independent > way. The fact that partial reads/writes are possible without non-blocking mode changes things a fair bit. Also, the lack of fnctl support in Windows needs to be taken into account too. I still think the support for partial reads in non-blocking mode on file.read() is inconsistent with the absence of partial write support in file.write(). I think this PEP still has some merit for cleaning up this inconsistency, but otherwise doesn't gain much... just adding a return count to file.write() and clearing up the documented behaviour is enough to do this. The lack of support on win32 for non-blocking mode, combined with the reduced need for it, makes adding a "setblocking" method undesirable. I don't know what the best thing to do now is... I guess the readsome/writesome is probably best, but given that os.read/os.write is not that bad, perhaps it's best to just forget I even suggested this PEP :-) -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Draft PEP to make file objects support non-blocking mode.
On Mon, 2005-03-21 at 23:31 +1100, Donovan Baarda wrote: > On Mon, 2005-03-21 at 11:42 +0100, Peter Astrand wrote: > > On Mon, 21 Mar 2005, Donovan Baarda wrote: > > > > > > > The only ways to ensure that a select process does not block like > > > > > this, > > > > > without using non-blocking mode, are; > > > > > > 3) Use os.read / os.write. > > > [...] > > > > > > but os.read / os.write will block too. > > > > No. > [...] > > Hmmm... you are right... that changes things. Blocking vs non-blocking > becomes kinda moot if read/write will do partial writes in blocking > mode. > > > fread() should loop internally on EAGAIN, in blocking mode. > > Yeah, I was talking about non-blocking mode... Actually, in blocking mode you never get EAGAIN read() only gets EAGAIN on an empty non-blocking read(). In non-blocking mode, EAGAIN is considered an error by fread(), so it will return a partial read. The python implementation of file.read() will return this partial read, and clear the EAGAIN error, or raise IOError if it was an empty read (to differentiate between an empty read and EOF). -- Donovan Baarda <[EMAIL PROTECTED]> http://minkirri.apana.org.au/~abo/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Patch review: all webbrowser.py related patches up to 2005-03-20
On 20 March 2005, [EMAIL PROTECTED] said: > 1166780 Fix _tryorder in webbrowser.py > --- > > This is my own patch. > > At the present time (Py2.4) documentation says: > """ > Under Unix, if the environment variable BROWSER exists, > it is interpreted to override the platform default list of > browsers,... > """ > (extract from Python-2.4/Doc/html/lib/module-webbrowser.html) > > But, when the environment variable BROWSER is messed up, > there is no reason not to try the other detected browser alternatives. I like it. Short, simple, and obvious. Can you think of a way to unit-test it? I took a brief look at the code, and it wasn't obvious to me. Might require factoring out a lazy initializer, and even then it might not work. And that would certainly call for more unit tests! Still, it's worth a shot. Greg -- Greg Ward <[EMAIL PROTECTED]> http://www.gerg.ca/ Predestination was doomed from the start. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [AST] Procedure for AST Branch patches
Brett C. wrote: OK, thanks to John Ehresman here at PyCon sprint I got logistix's patch applied. Beyond a warning that a warning that decode_unicode() is never called and the parser module failing to compile under Windows everything should be fine for compiling the AST branch. Under Linux (Suse 9.1), the parser module compiles with a couple of warnings about implicit declaration of PyParser_SimpleParseString, but the .so fails to load due to a missing symbol PyNode_Compile. Passing the test suite, though, is another question. =) Heh. I managed to get test_grammar to pass 100% last night, which is at least a start! And I must say that the new system is rather nice to work with - it took me a while to work out what was going on, but the multi-pass 'build AST', 'build symbol table', 'compile AST' is much cleaner than it is with the lists-of-lists arrangement on the trunk (which is the entire point of the AST-branch, I guess. . .) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [AST] question about marshal_write_*() fxns
Brett C. wrote: For those of you who don't know, I am sprinting on the AST branch here on PyCon. Ah, so that's why it's quiet this week :) But one of things I am not sure of is what the marshal_write_*() functions in Python/Python-ast.c are used for. I assume they output to the marshal format, but there is mention of a byte stream format and so I thought it might be that as well. Given Neil & Jeremy's answers, perhaps "linear_ast_*" or "bytestream_ast_*" would work as a new prefix? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
