[Python-Dev] Adding Python-Native Threads
There are some very serious problems with threads today. Events are
often proposed as a solution to these problems, but they have their own
issues. A detailed analysis of the two options and some solutions
(most of which are used in my proposal) can be found in Why Events Are
A Bad Idea [0]. I'm going to skip the event aspects and sum up the
thread problems here:
* Expensive (resident memory, address space, time to create and to
switch between)
* Finely-grained atomicity (python is coarser than C, but still finer
than we need)
* Unpredictable (switching between them is not deterministic)
* Uninterruptible (no way to kill them, instead you must set a flag and
make sure they check it regularly)
* Fail silently (if they die with an exception it gets printed to the
console but the rest of the program is left uninformed)
To resolve these problems I propose adding lightweight cooperative
threads to Python. They can be built around a Frame object, suspending
and resuming like generators do.
That much is quite easy. Avoiding the C stack is a bit harder.
However, it can be done if we introduce a new operator for threaded
calls (which I refer to as non-atomic calls), which I propose be
func@(). As a bonus this prevents any operation which doesn't use
the non-atomic call syntax from triggering a thread switch accidentally.
After that you need a way of creating new threads. I propose we use a
sibling func@() statement (note that the @() would be parsed as
part of the statement itself, and not as an expression). It would
create a new thread and run func in it, but more importantly it would
also prejoin the new thread to the current function. This means that
for the current function to return to its parent, all of the threads it
created must first return themselves. It also means that an exception
in any of them will be propagated to the parent of the creator, after
interrupting all of its siblings (with an Interrupted exception) so
that it is permitted to return.
Now for an example of how this would work in practice, using an echo
server. Note that I am assuming a con module (short for concurrent)
that would contain standard functions to support these python-native
threads.
from con import bootstrap, tcplisten, ipv6addr, ipv4addr
def main():
main_s = tcplisten@(ipv6addr('::1', 6839), ipv4addr('127.0.0.1', 6839))
while 1:
sibling echo@(main_s.accept@())
def echo(s):
try:
try:
while 1:
s.write@(s.read@())
except (EOFError, IOError):
pass
finally:
s.close()
if __name__ == '__main__':
try:
bootstrap(main)
except KeyboardInterrupt:
pass
And here is a diagram of the resulting cactus stack, assuming three
clients, two of which are reading and the other is writing.
bootstrap - main - accept
|
echo - read
|
echo - read
|
echo - write
Some notes:
* An idle@() function would be used for all thread switches. I/O
functions such as read@() and write@() would use it internally, and
idle would internally call a function to do poll() on all file
descriptors.
* idle is the function that will raise Interrupted
* It should be illegal to attempt a non-atomic call once your thread is
in an interrupted state. This ensures you cannot get interrupted
twice, by having something further up the call stack fail and
interrupt you while you are already handling an interruption.
* Being a cooperative system, it does not allow you to use multiple
CPUs simultaneously. I don't regard this as a significant problem,
and in any case there are better ways use multiple CPUs if you need to.
References:
[0]
http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/index.html
--
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Python-Native Threads
On 26-jun-2005, at 11:34, Adam Olsen wrote: To resolve these problems I propose adding lightweight cooperative threads to Python. They can be built around a Frame object, suspending and resuming like generators do. That much is quite easy. Avoiding the C stack is a bit harder. However, it can be done if we introduce a new operator for threaded calls (which I refer to as non-atomic calls), which I propose be func@(). As a bonus this prevents any operation which doesn't use the non-atomic call syntax from triggering a thread switch accidentally. Have a look at stackless python. http://www.stackless.com/ Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Python-Native Threads
Also look at greenlets, they are in the py lib http://codespeak.net/py Regards, Florian Schulze ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Python-Native Threads
On 6/26/05, Ronald Oussoren [EMAIL PROTECTED] wrote: Have a look at stackless python. http://www.stackless.com/ On 6/26/05, Florian Schulze [EMAIL PROTECTED] wrote: Also look at greenlets, they are in the py lib http://codespeak.net/py While internally Stackless and Greenlets may (or may not) share a lot with my proposed python-native threads, they differ greatly in their intent and the resulting interface they expose. Stackless and Greenlets are both tools for building an architecture that uses threads, while my python-native threads IS that resulting architecture. For example, with Greenlets you would use the .switch() method of a specific greenlet instance to switch to it, and with my python-native threads you would use the global idle() function which would decide for itself which thread to switch to. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Python-Native Threads
At 05:04 AM 6/26/2005 -0600, Adam Olsen wrote: On 6/26/05, Ronald Oussoren [EMAIL PROTECTED] wrote: Have a look at stackless python. http://www.stackless.com/ On 6/26/05, Florian Schulze [EMAIL PROTECTED] wrote: Also look at greenlets, they are in the py lib http://codespeak.net/py While internally Stackless and Greenlets may (or may not) share a lot with my proposed python-native threads, they differ greatly in their intent and the resulting interface they expose. Stackless and Greenlets are both tools for building an architecture that uses threads, while my python-native threads IS that resulting architecture. For example, with Greenlets you would use the .switch() method of a specific greenlet instance to switch to it, and with my python-native threads you would use the global idle() function which would decide for itself which thread to switch to. See PEP 342 for a competing proposal, that includes a short (50 lines) co-operative threading example. It uses the 'yield' keyword instead of your '@' proposal, and already has a Sourceforge patch implementing it. It does not propose a specific built-in idler or I/O handler, but such things could be added later. In the short term, I think it's enough that Python be *able* to have such things (without using Stackless or Greenlets), and maybe include an example of a run loop. I don't currently plan to push for a standard or official run loop implementation in Python 2.5, however, unless all the relevant stakeholders just happen to come to some kind of agreement that makes it happen. Note that while having a universal run loop is a worthy goal, actually implementing one is hard. The experts in this are definitely in the Twisted camp, as they have implemented run loops for numerous platforms to support compatibility w/various GUI frameworks' event loops. To some extent, this means that the run loop needs to be an interface with multiple implementations, and I think that we would need a standard library implementation of PEP 246 in order to support it properly. Anyway, my point is that even deciding on standards for a universal run loop is a fairly big project, one that possibly even deserves its own SIG to put together. By contrast, the implementation of simulated threads and non-blocking function calls was a weekend project. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Some RFE for review
Hi, while bugs and patches are sometimes tricky to close, RFE can be very easy to decide whether to implement in the first place. So what about working a bit on this front? Here are several RFE reviewed, perhaps some can be closed (should is always from submitter's point of view): 1193128: str.translate(None, delchars) should be allowed and only delete delchars from the string. Review: may be a good feature, although string._idmap can be passed as the first parameter too. 1226256: The path module by Jason Orendorff should be in the standard library. http://www.jorendorff.com/articles/python/path/ Review: the module is great and seems to have a large user base. On c.l.py there are frequent praises about it. 1216944: urllib(2) should gain a dict mapping HTTP status codes to the correspondig status/error text. Review: I can't see anything speaking against it. 1214675: warnings should get a removefilter() method. An alternative would be to fully document the filters attribute to allow direct tinkering with it. Review: As mwh said in a comment, this isn't Java, so the latter may be the way to go. 1205239: Shift operands should be allowed to be negative integers, so e.g. a -2 is the same as a 2. Review: Allowing this would open a source of bugs previously well identifiable. 1152248: In order to read records separated by something other than newline, file objects should either support an additional parameter (the separator) to (x)readlines(), or gain an additional method which does this. Review: The former is a no-go, I think, because what is read won't be lines. The latter is further complicating the file interface, so I would follow the principle that not every 3-line function should be builtin. 1110010: A function attrmap should be introduced which is used as follows: attrmap(x)['att'] == getattr(x, 'att') The submitter mentions the use case of new-style classes without a __dict__ used at the right of %-style string interpolation. Review: I don't know whether this is worth it. 1052098: An environment variable should be supported to set the default encoding. Review: If one wants this for a single application, he can still implement it himself. 985094: getattr should support a callable as the second argument, used as follows: getattr(obj, func) == func(obj) Review: Highly unintuitive to me. That's all for today; sorry if it was too much ;) Reinhold -- Mail address is perfectly valid! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 06:57 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote:
1226256:
The path module by Jason Orendorff should be in the standard library.
http://www.jorendorff.com/articles/python/path/
Review: the module is great and seems to have a large user base. On c.l.py
there are frequent praises about it.
I would note that there are some things in the interface that should be
cleaned up before it becomes a stdlib module. It has many ways to do the
same thing, and many of its property and method names are confusing because
they either do the same thing as a standard function, but have a different
name (like the 'parent' property that is os.path.dirname in disguise), or
they have the same name as a standard function but do something different
(like the 'listdir()' method that returns full paths rather than just
filenames). I'm also not keen on the fact that it makes certain things
properties whose value can change over time; i.e. ctime/mtime/atime and
size really shouldn't be properties, but rather methods. I'm also not sure
how I feel about all the read/write methods that hide the use of file
objects; these seem like they should remain file object methods, especially
since PEP 343 will allow easy closing with something like:
with closing(some_path.open('w')) as f:
f.write(data)
Granted, this is more verbose than:
some_path.write_bytes(data)
But brevity isn't always everything. If these methods are kept I would
suggest using different names, like set_bytes(), set_text(), and
set_lines(), because write sounds like something you do on an ongoing
basis to a stream, while these methods just replace the file's entire contents.
Aside from all these concerns, I'm +1 on adding the module.
Here's my list of suggested changes:
* path.joinpath(*args) - path.subpath(*args)
* path.listdir() - path.subpaths()
* path.splitall()- path.parts()
* path.parent- path.dirname (and drop dirname() method)
* path.name - path.filename (and drop basename() method)
* path.namebase - path.filebase (maybe something more descriptive?)
* path.atime/mtime/ctime - path.atime(), path.mtime(), path.ctime()
* path.size - path.filesize()
* drop getcwd(); it makes no sense on a path instance
* add a samepath() method that compares the absolute, case and
path-normalized versions of two paths, and a samerealpath() method that
does the same but with symlinks resolved.
And, assuming these file-content methods are kept:
* path.bytes() - path.get_file_bytes()
* path.write_bytes() - path.set_file_bytes() and path.append_file_bytes()
* path.text() - path.get_file_text()
* path.write_text()- path.set_file_text() and path.append_file_text()
* path.lines() - path.get_file_lines()
* path.write_lines() - path.set_file_lines() and path.append_file_lines()
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Python-Native Threads
On 6/26/05, Adam Olsen [EMAIL PROTECTED] wrote: ... To resolve these problems I propose adding lightweight cooperative threads to Python. Speaking of lightweight cooperative threads - has anyone recently tried to build Python with the pth option? It doesn't quite work out of the box. How much maintenance would be required to make it work again? Oren ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Phillip J. Eby wrote: At 06:57 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote: 1226256: The path module by Jason Orendorff should be in the standard library. http://www.jorendorff.com/articles/python/path/ Review: the module is great and seems to have a large user base. On c.l.py there are frequent praises about it. [...] Aside from all these concerns, I'm +1 on adding the module. Here's my list of suggested changes: [...] I agree with your changes list. One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? Reinhold -- Mail address is perfectly valid! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
On Sun, 26 Jun 2005, Phillip J. Eby wrote: * drop getcwd(); it makes no sense on a path instance Personally I use path.getcwd() as a class method all the time. It makes as much sense as fromkeys() does on a dict instance, which is technically possible but non-sensical. And, assuming these file-content methods are kept: * path.bytes() - path.get_file_bytes() * path.write_bytes() - path.set_file_bytes() and path.append_file_bytes() * path.text() - path.get_file_text() * path.write_text()- path.set_file_text() and path.append_file_text() * path.lines() - path.get_file_lines() * path.write_lines() - path.set_file_lines() and path.append_file_lines() I don't know how often these are used. I don't use them myself. I am mainly interested in this module so that I don't have to use os.path anymore. Reinhold Birkenfeld wrote: One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? I would argue for Path. It fits with the recent cases of: from sets import Set from decimal import Decimal -- Michael Hoffman [EMAIL PROTECTED] European Bioinformatics Institute ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Phillip It has many ways to do the same thing, and many of its property Phillip and method names are confusing because they either do the same Phillip thing as a standard function, but have a different name (like Phillip the 'parent' property that is os.path.dirname in disguise), or Phillip they have the same name as a standard function but do something Phillip different (like the 'listdir()' method that returns full paths Phillip rather than just filenames). To the extent that the path module tries to provide a uniform abstraction that's not saddled with a particular way of doing things (e.g., the Unix way or the Windows way), I don't think this is necessarily a bad thing. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.17, 1.18 bgenObjectDefinition.py, 1.29, 1.30 bgenType.py, 1.15, 1.16 bgenVariable.py, 1.6, 1.7 scantools.py, 1.37
On 24-jun-2005, at 21:46, [EMAIL PROTECTED] wrote: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18095/Tools/ bgen/bgen Modified Files: bgenGenerator.py bgenObjectDefinition.py bgenType.py bgenVariable.py scantools.py Log Message: Normalize whitespace to avoid offending Bug Day volunteers. Argh, sorry... I thought I had all my machines educated to do tab expansion for Python, but apparently not... -- Jack Jansen, [EMAIL PROTECTED], http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Phillip J. Eby wrote: [...] I'm also not keen on the fact that it makes certain things properties whose value can change over time; i.e. ctime/mtime/atime and size really shouldn't be properties, but rather methods. I think ctime, mtime and atime should be (or return) datetime.datetime objects instead of integer timestamps. Bye, Walter Dörwald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] python/dist/src/Lib Cookie.py, 1.17, 1.18
[EMAIL PROTECTED] Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4891/Lib Modified Files: Cookie.py Log Message: bug [ 1108948 ] Cookie.py produces invalid code Index: Cookie.py === RCS file: /cvsroot/python/python/dist/src/Lib/Cookie.py,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- Cookie.py 20 Oct 2003 14:01:49 - 1.17 +++ Cookie.py 26 Jun 2005 21:02:49 - 1.18 @@ -470,9 +470,9 @@ def js_output(self, attrs=None): # Print javascript return -SCRIPT LANGUAGE=JavaScript +script type=text/javascript !-- begin hiding -document.cookie = \%s\ +document.cookie = \%s\; // end hiding -- /script % ( self.OutputString(attrs), ) I assume this accounts for the current failure of test_cookie: test_cookie test test_cookie produced unexpected output: ** *** mismatch between line 19 of expected output and line 19 of actual output: - SCRIPT LANGUAGE=JavaScript + script type=text/javascript *** mismatch between line 21 of expected output and line 21 of actual output: - document.cookie = Customer=WILE_E_COYOTE; Path=/acme; Version=1; + document.cookie = Customer=WILE_E_COYOTE; Path=/acme; Version=1;; ? + *** mismatch between line 26 of expected output and line 26 of actual output: - SCRIPT LANGUAGE=JavaScript + script type=text/javascript *** mismatch between line 28 of expected output and line 28 of actual output: - document.cookie = Customer=WILE_E_COYOTE; Path=/acme; + document.cookie = Customer=WILE_E_COYOTE; Path=/acme;; ? + ** ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE forreview)
[Reinhold Birkenfeld] One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? [Michael Hoffman] I would argue for Path. Granted path is actually os.path, but I don't think it's wise to have stdlib modules whose names are differentiated only by case, especially on Windows (and other case-insensitive filesystems). =Tony.Meyer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 09:00 PM 6/26/2005 +0200, Reinhold Birkenfeld wrote: One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? I was thinking os.Path, myself. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 08:19 PM 6/26/2005 +0100, Michael Hoffman wrote: On Sun, 26 Jun 2005, Phillip J. Eby wrote: * drop getcwd(); it makes no sense on a path instance Personally I use path.getcwd() as a class method all the time. It makes as much sense as fromkeys() does on a dict instance, which is technically possible but non-sensical. It's also duplication with os.path; I'm -1 on creating a new staticmethod for it. Reinhold Birkenfeld wrote: One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? I would argue for Path. It fits with the recent cases of: from sets import Set from decimal import Decimal I like it too, as a class in the os module. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 02:31 PM 6/26/2005 -0500, Skip Montanaro wrote: Phillip It has many ways to do the same thing, and many of its property Phillip and method names are confusing because they either do the same Phillip thing as a standard function, but have a different name (like Phillip the 'parent' property that is os.path.dirname in disguise), or Phillip they have the same name as a standard function but do something Phillip different (like the 'listdir()' method that returns full paths Phillip rather than just filenames). To the extent that the path module tries to provide a uniform abstraction that's not saddled with a particular way of doing things (e.g., the Unix way or the Windows way), I don't think this is necessarily a bad thing. I'm confused by your statements. First, I didn't notice the path module providing any OS-abstractions that aren't already provided by os.path. Second, using inconsistent and confusing names is pretty much always a bad thing. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 12:22 AM 6/27/2005 +0200, Dörwald Walter wrote: Phillip J. Eby wrote: [...] I'm also not keen on the fact that it makes certain things properties whose value can change over time; i.e. ctime/mtime/atime and size really shouldn't be properties, but rather methods. I think ctime, mtime and atime should be (or return) datetime.datetime objects instead of integer timestamps. With what timezone? I don't think that can be done portably and unambiguously, so I'm -1 on that. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE forreview)
At 12:08 PM 6/27/2005 +1200, Tony Meyer wrote: [Reinhold Birkenfeld] One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? [Michael Hoffman] I would argue for Path. Granted path is actually os.path, but I don't think it's wise to have stdlib modules whose names are differentiated only by case, especially on Windows (and other case-insensitive filesystems). This is the name of a *class*, not a module. I.e., we are discussing adding a Path class to the 'os' module, that implements the interface of the path module. We can't call it path (as a top-level module) because the interface will not be backward-compatible with current uses and installations of the path module. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE forreview)
[Reinhold Birkenfeld] One more issue is open: the one of naming. As path is already the name of a module, what would the new object be called to avoid confusion? pathobj? objpath? Path? [Michael Hoffman] I would argue for Path. [Tony Meyer Granted path is actually os.path, but I don't think it's wise to have stdlib modules whose names are differentiated only by case, especially on Windows (and other case-insensitive filesystems). [Phillip J. Eby] This is the name of a *class*, not a module. Sorry - it sounded like the idea was to put this class in a module by itself (i.e. the class would be os.Path.Path). I.e., we are discussing adding a Path class to the 'os' module, that implements the interface of the path module. In that case, I would argue against Path as the name of the class because it's confusing to have os.path be the path module, and os.Path be an class that provides an interface to that module. I think differentiating things solely on the case of the name is a bad idea. =Tony.Meyer ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Walter I think ctime, mtime and atime should be (or return) Walter datetime.datetime objects instead of integer timestamps. +1 Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Phillip ... but have a different name (like the 'parent' property that Phillip is os.path.dirname in disguise) ... Phillip ... (like the 'listdir()' method that returns full paths rather Phillip than just filenames). Skip To the extent that the path module tries to provide a uniform Skip abstraction that's not saddled with a particular way of doing Skip things (e.g., the Unix way or the Windows way), I don't think this Skip is necessarily a bad thing. Phillip I'm confused by your statements. First, I didn't notice the Phillip path module providing any OS-abstractions that aren't already Phillip provided by os.path. Second, using inconsistent and confusing Phillip names is pretty much always a bad thing. :) Sorry, let me be more explicit. dirname is the Unix name for return the parent of this path. In the Windows and Mac OS9 worlds (ignore any possible Posix compatibility for a moment), my guess would be it's probably something else. I suspect listdir gets its return individual filenames instead of full paths from the semantics of the Posix opendir/readdir/ closedir functions. If it makes more sense to return strings that represent full paths or new path objects that have been absolute-ified, then the minor semantic change going from os.path.listdir() to the listdir method of Jason's path objects isn't a big problem to me. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
On Jun 26, 2005, at 8:54 PM, Phillip J. Eby wrote: At 12:22 AM 6/27/2005 +0200, Dörwald Walter wrote: Phillip J. Eby wrote: [...] I'm also not keen on the fact that it makes certain things properties whose value can change over time; i.e. ctime/mtime/atime and size really shouldn't be properties, but rather methods. I think ctime, mtime and atime should be (or return) datetime.datetime objects instead of integer timestamps. With what timezone? I don't think that can be done portably and unambiguously, so I'm -1 on that. That makes no sense, timestamps aren't any better, and datetime objects have no time zone set by default anyway. datetime.fromtimestamp(time.time()) gives you the same thing as datetime.now(). -bob ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 08:29 PM 6/26/2005 -0500, Skip Montanaro wrote: Phillip ... but have a different name (like the 'parent' property that Phillip is os.path.dirname in disguise) ... Phillip ... (like the 'listdir()' method that returns full paths rather Phillip than just filenames). Skip To the extent that the path module tries to provide a uniform Skip abstraction that's not saddled with a particular way of doing Skip things (e.g., the Unix way or the Windows way), I don't think this Skip is necessarily a bad thing. Phillip I'm confused by your statements. First, I didn't notice the Phillip path module providing any OS-abstractions that aren't already Phillip provided by os.path. Second, using inconsistent and confusing Phillip names is pretty much always a bad thing. :) Sorry, let me be more explicit. dirname is the Unix name for return the parent of this path. In the Windows and Mac OS9 worlds (ignore any possible Posix compatibility for a moment), my guess would be it's probably something else. I suspect listdir gets its return individual filenames instead of full paths from the semantics of the Posix opendir/readdir/ closedir functions. If it makes more sense to return strings that represent full paths or new path objects that have been absolute-ified, then the minor semantic change going from os.path.listdir() to the listdir method of Jason's path objects isn't a big problem to me. The semantics aren't the issue; it's fine and indeed quite useful to have a method that returns path objects. I'm just saying it shouldn't be called listdir(), since that's confusing when compared to what the existing listdir() function does. If you look at my original post, you'll see I suggested it be called 'subpaths()' instead, to help reflect that it returns paths, rather than filenames. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
At 09:26 PM 6/26/2005 -0400, Bob Ippolito wrote: On Jun 26, 2005, at 8:54 PM, Phillip J. Eby wrote: At 12:22 AM 6/27/2005 +0200, Dörwald Walter wrote: Phillip J. Eby wrote: I'm also not keen on the fact that it makes certain things properties whose value can change over time; i.e. ctime/mtime/atime and size really shouldn't be properties, but rather methods. I think ctime, mtime and atime should be (or return) datetime.datetime objects instead of integer timestamps. With what timezone? I don't think that can be done portably and unambiguously, so I'm -1 on that. That makes no sense, timestamps aren't any better, Sure they are, if what you want is a timestamp. In any case, the most common use case I've seen for mtime and friends is just comparing against a previous value, or the value on another file, so it doesn't actually matter most of the time what the type of the value is. and datetime objects have no time zone set by default anyway. datetime.fromtimestamp(time.time()) gives you the same thing as datetime.now(). In which case, it's also easy enough to get a datetime if you really want one. I personally would rather do that than complicate the use cases where a datetime isn't really needed. (i.e. most of the time, at least in my experience) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Some RFE for review
1193128: str.translate(None, delchars) should be allowed and only delete delchars from the string. I had agreed to this one and it's on my todo list to implement. 1214675: warnings should get a removefilter() method. An alternative would be to fully document the filters attribute to allow direct tinkering with it. I'm concerned that removefilter() may not work well in the presence of multiple modules that use the warnings module. It may be difficult to make sure the one removed wasn't subsequently added another module. Also, the issue is compounded because the order of filter application is important. 1205239: Shift operands should be allowed to be negative integers, so e.g. a -2 is the same as a 2. Review: Allowing this would open a source of bugs previously well identifiable. The OP is asking why it is different for negative sequence indicies (why the added convenience was thought to outweigh the loss of error detection). 1152248: In order to read records separated by something other than newline, file objects should either support an additional parameter (the separator) to (x)readlines(), or gain an additional method which does this. Review: The former is a no-go, I think, because what is read won't be lines. Okay, call it a record then. The OPs request is not a non-starter. There is a proven precedent in AWK which allows programmer specifiable record separators. The latter is further complicating the file interface, so I would follow the principle that not every 3-line function should be builtin. This is not a design principle. UserDict.Mixin shows that most of the mapping API is easily expressible in terms of a few lines and a few primitives; however, the mapping API has long been proven as valuable for its expressiveness. Likewise, Guido's any() and all() builtins can be expressed in a single line but were accepted anyway. A more nuanced version of the principle is: if a proposal can be easily expressed with a small grouping of existing constructs, then must meet much higher standards of use frequency and expressiveness in order to be accepted. 1110010: A function attrmap should be introduced which is used as follows: attrmap(x)['att'] == getattr(x, 'att') The submitter mentions the use case of new-style classes without a __dict__ used at the right of %-style string interpolation. Review: I don't know whether this is worth it. While potentially useful, the function is entirely unintuitive (it has to be studied a bit before being able to see what it is for). Also, the OP is short on use cases (none were presented). IMO, this belongs as a cookbook recipe. Raymond Hettinger ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] python/dist/src/Lib Cookie.py, 1.17, 1.18
Tim Peters wrote: [EMAIL PROTECTED] Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4891/Lib Modified Files: Cookie.py Log Message: bug [ 1108948 ] Cookie.py produces invalid code [...] I assume this accounts for the current failure of test_cookie: You're right, I forgot to checkin the change to the output file. Thanks, Reinhold -- Mail address is perfectly valid! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
