Re: [Python-Dev] Proposed Python 3.0 schedule
On Tue, 2008-10-07 at 11:30 +0200, Victor Stinner wrote: - I'd think find . -type f -print0 | xargs -0 python -c 'pass' ought to work (with files with bad bytes being returned by find), First, fix your home directory :-) There are good tools (convmv?) to fix invalid filenames. Fixing the home directory doesn't help in the long run because files with non-UTF-8 file names on a nominally UTF-8 system are not that exceptional, they crop up all over the place in non-ASCII countries. One can obtain them simply by copying stuff from a DVD someone else burned, by downloading a Japanese-released torrent, or by copying files from a shared hard drive. which means that Python shouldn't blow up and refuse to start when there's a non-properly-encoding argv (Could not convert argument 1 to string and exiting isn't appropriate behavior) Why not? It's a good idea to break compatibility to refuse invalid bytes sequences. You can still uses the command line, an input file or a GUI to read raw bytes sequences. Maybe I am misunderstanding you, but if python blows up at startup when unable to encode argv to Unicode, then how can you still use the command line to access the actual file name? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Filename as byte string in python 2.6 or 3.0?
On Tue, 2008-09-30 at 07:26 -0700, Guido van Rossum wrote: I am not convinced that a word processor can just ignore files with (what it thinks are) undecodable file names. In countries with a history of incompatible national encodings, such file names crop up very often, sometimes as a natural consequence of data migrating from older systems to newer ones. You can and do encounter invalid file names in the filesystems of mainstream users even without them using buggy or obsolete software. This is a quality of implementation issue. Either the word processor is written to support undecodable files, or it isn't. If it isn't, there's nothing that can be done about it (short of buying another wordprocessor) I agree with this. I just believe the underlying python APIs shouldn't make it impossible (or unnecessarily hard) for the word processor to implement showing of files with undecodable names. For example, implementing os.listdir to return the file names as Unicode subclasses with ability to access the underlying bytes (automatically recognized by open and friends) sounds like a good compromise that allows the word processor to both have the cake and eat it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Any PEP about 2.6 - 3000 code transition?
On Wed, 2008-08-13 at 22:20 +1000, Anthony Baxter wrote: Last time I looked at it, the C API wasn't nailed down yet. That's why I passed over it entirely for my tutorial. The only advice I was able to give was that if your extension is just a wrapper around existing C code, you might be better off rewriting it using ctypes. Given that ctypes doesn't work on a number of popular architectures, including x86_64 (the last time I looked), I'd hardly call that better off. :-( ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] xmlrpclib.{True, False} (was Re: Assignment to None)
On Mon, 2008-06-16 at 22:27 +0200, M.-A. Lemburg wrote: On 2008-06-15 16:47, Georg Brandl wrote: Thomas Lee schrieb: Georg Brandl wrote: Remember that it must still be possible to write (in 2.6) True = 0 assert not True Ah of course. Looks like I should just avoid optimizations of Name(True) and Name(False) all together. That's a shame! We can of course decide to make assignment to True and False illegal in 2.7 :) Raising a run-time exception would be fine, but not a SyntaxError at compile time - this would effectively make it impossible to keep code compatible to Python 2.1. Maybe it wouldn't. Something like: try: True, False except NameError: globals()['True'] = 1 globals()['False'] should still work for compatibility. It would break *existing* code that tries to be compatible with old releases, but that is unavoidable in making something illegal that was previously legal. In this case, the price is justified by being able to optimize access to None, True, and False without having to resort to dirty tricks such as none=None (previously None=None) in functions. It should also enable the compiler to optimize while True: the way it optimizes while 1:. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python parallel benchmark
On Thu, 2008-05-15 at 21:02 -0400, Tom Pinckney wrote: I found some other references where people were expressing concern over numpy releasing the GIL due to the fact that other C extensions could call numpy and unexpectedly have the GIL released on them (or something like that). Could you please post links to those? I'm asking because AFAIK that concern doesn't really stand. Any (correct) code that releases the GIL is responsible for reacquiring it before calling *any* Python code, in fact before doing anything that might touch a Python object or its refcount. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python parallel benchmark
On Fri, 2008-05-16 at 08:04 -0400, Tom Pinckney wrote: Here's one example, albeit from a few years ago http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/1625465 Thanks for the pointer. I'm not sure I fully understand Konrad Hinsen's concerns, but maybe the problem is that Numpy's number-crunching needs to call back into Python frequently. The notion of releasing the GIL for number-crunching assumes that the code is structured like this: 1. code that works with python objects ... 2. acquire pointer to a C struct/array ... 3. release GIL 4. work with C objects (crunch the numbers) without calling any Python code and without touching Python objects or refcounts 5. reacquire GIL If step 4 needs to call into Python frequently, then this strategy won't really work. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Warn about mktemp once again?
On Tue, 2008-04-29 at 15:34 +0200, Giovanni Bajo wrote: As of 2.6, Greg's use case is addressed by the new 'delete' parameter on tempfile.NamedTemporaryFile. Then I personally don't have any objection to the removal of os.mkstemp. os.mkstemp is still useful when you *don't* need the file object, but the actual file descriptor, for passing to C code or to the child processes, or to mmap.mmap and such. It is also familiar to the Unix/C hackers, and it should cost very little to keep it around. Since we're at it, a common pattern I use is to create temporary file to atomically replace files: I create a named temporary file in the same directory of the file I want to replace; write data into it; close it; and move it (POSIX move: rename with silent overwrite) to the destination file. AFAIK, this is allows an atomic file replacemente on most filesystems. I believe this is a common useful pattern that could be handled in module tmpfile (especially since the final rename step requires a little care to be truly multiplatform). I agree, having a tempfile class with rename-on-close semantics would be very useful. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyArg_ParseTuple and Py_BuildValue question
On Thu, 2008-04-10 at 09:23 +0800, Alvin Delagon wrote: I'm currently writing a simple python SCTP module in C. So far it works sending and receiving strings from it. [...] I'm going to construct an SS7 packet in python using struct.pack(). Here's the question, how am I going to pass the packet I wrote in python to my module and back? I already asked this question in comp.lang.python but so far no responses yet. Have you tried posting to the C API mailing list? http://mail.python.org/mailman/listinfo/capi-sig ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pickle format machine independence
Are pickle files intended to be readable across different machine architectures? The documentation states unequivocally that they are compatible across Python versions, but compatibility across machine architectures (wrt to differences in size and layout of primitive C types) is not explicitly addressed. One example where I stumbled upon the incompatibility is the pickling of arrays. While pickle is normally very careful to write out numbers in a platform-independent way, arrays are written out in tostring format. This is filed under http://bugs.python.org/issue2389. I can work around this issue in my application, but if this is considered a bug, I'd prefer to fix it in Python instead. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GC Changes
On Tue, 2007-10-02 at 10:50 +0100, Gustavo Carneiro wrote: Correct. And that reminds me of the limitation of the the Python GC: it doesn't take into account how much memory is being indirectly retained by a Python Object. Like in the example I already gave, gtk.gdk.Pixbuf can easily hold hundreds of megabytes, yet the GC gives it as much consideration as it does to a simple python integer object which is several orders of magnitude smaller. That sounds like a case for the Pixbuf object to have a close method (not necessarily called that) that releases the resources. The point of GC is that you normally don't care if memory is released sooner or later; for stuff you do care about, such as files, shared memory, or even large memory objects, there is always explicit management. cStringIO's close method provides a precedent. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] GC Changes
On Tue, 2007-10-02 at 11:30 +0100, Gustavo Carneiro wrote: even large memory objects, there is always explicit management. cStringIO's close method provides a precedent. I think close in real files is needed not so much because you want to free memory, but that you want to prevent data loss by flushing the file buffer into the actual file at a certain point in time. You can also do that with flush. What is specific for close is that it frees up the system resources occupied by the open file. Calling it at a known point in time ensures that these external resources aren't occupied any longer than necessary, regardless of the object deallocator's policies. That is why open(filename).read() is discouraged, despite being a useful idiom in throwaway scripts and working just fine in CPython. The fact that programmers need Pixbuf's memory released sooner rather than later might indicate that Pixbuf would benefit from a close-like method. In any case, I trust your judgment on Pixbufs, but merely point out that in general it is unwise to rely on the deallocator to release resources where timing of the release matters. And I suspect that cStringIO just added a close() method for compatibility with real files. But of course I speculating here... That could be the case, but then the method could also be a no-op. I only brought up cStringIO as a precedent for a close method that works with a memory-only object. (StringIO's close also tries to arrange for the buffer to be freed immediately, inasmuch as that is possible to achieve in pure Python.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing the GIL (Me, not you!)
On Thu, 2007-09-13 at 18:38 -0500, [EMAIL PROTECTED] wrote: Hrvoje More precisely, Python will call the deallocator appropriate for Hrvoje the object type. If that deallocator does nothing, the object Hrvoje continues to live. Such objects could also start out with a Hrvoje refcount of sys.maxint or so to ensure that calls to the no-op Hrvoje deallocator are unlikely. Maybe sys.maxint/2? You'd hate for the first incref to invoke the deallocator even if it was a no-op. ob_refcnt is signed. :-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing the GIL (Me, not you!)
On Thu, 2007-09-13 at 13:15 +0200, Martin v. Löwis wrote: To put it another way, would it actually matter if the reference counts for such objects became hopelessly wrong due to non-atomic adjustments? If they drop to zero (which may happen due to non-atomic adjustments), Python will try to release the static memory, which will crash the malloc implementation. More precisely, Python will call the deallocator appropriate for the object type. If that deallocator does nothing, the object continues to live. Such objects could also start out with a refcount of sys.maxint or so to ensure that calls to the no-op deallocator are unlikely. The part I don't understand is how Python would know which objects are global/static. Testing for such a thing sounds like something that would be slower than atomic incref/decref. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] working with Python threads from C extension module?
On Fri, 2007-09-07 at 16:20 -0700, Bill Janssen wrote:
#define SSL_ALLOW_THREADS {if (_ssl_locks != NULL) { Py_BEGIN_ALLOW_THREADS
}}
#define SSL_DISALLOW_THREADS {if (_ssl_locks != NULL) {
Py_BEGIN_ALLOW_THREADS }}
I'd forgotten how convoluted Py_BEGIN_ALLOW_THREADS and
Py_END_ALLOW_THREADS were. Anyone have any other suggestions about
how to do this?
Be convoluted yourself and do this:
#define PySSL_BEGIN_ALLOW_THREADS { if (_ssl_locks) { Py_BEGIN_ALLOW_THREADS
#define PySSL_END_ALLOW_THREADS Py_END_ALLOW_THREADS } }
(Untested, but I think it should work.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Segfault
On Fri, 2007-08-24 at 09:01 -0700, Neal Norwitz wrote:
Right. I looked at this with Jeffrey Yasskin and agree that a lock is
needed for f_fp. The fix is ugly because it's needed around all
accesses to f_fp and there are a ton of them. Some are with the GIL
held and others when it isn't.
The way I see it, when the GIL is held, we're in the clear because
f-f_fp can only disappear with the GIL held. The troublesome cases are
those ones where f_fp is accessed inside an allow threads section.
To fix this, it shouldn't be necessary to change every individual place
that accesses f_fp. It is sufficient to protect the sections that
specifically allow threads, changing the semantics from allow threads
to allow threads, locking out the ones operating on the same
fileobject.
For example, instead of Py_BEGIN_ALLOW_THREADS, the fileobject code
could use a macro such as:
#define PyFile_BEGIN_PROTECT(f) \
PyThread_acquire_lock(f-f_lock, 1); \
if (!f-f_fp) \
;\
else {
#define PyFile_END_PROTECT(f)\
} \
PyThread_release_lock(f-f_lock);
That change might be somewhat easier than changing each individual line
that accesses f-f_fp.
I would be fine with the much simpler approach in the patch I sent
(assuming the patch is finished). It doesn't completely address the
problem, but does make the race condition much, much smaller. Part of
the reason I'm ok with this is that in 3.0, all this code has already
been removed.
That is a good point. Does your patch make the race crash go away
entirely in the tests like the one the OP posted?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Segfault
On Thu, 2007-08-23 at 11:57 -0700, Neal Norwitz wrote: allow threads section that runs with the GIL released, file_close might acquire the GIL and be running in parallel to this code. If file_close sets f_fp to NULL after the if condition evaluates, but before the call to _portable_fseek completes, we still get a segfault. However, the setting of f_fp to NULL happens with the GIL set, ie while only one thread is running. I thought other IO-bound threads (the sections that allow threads) could also run in parallel. Otherwise what is the point of BEGIN_ALLOW_THREADS? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Segfault
On Tue, 2007-08-21 at 09:14 -0700, Neal Norwitz wrote: The patch is insufficient to prevent all types of crashes that occur when accessing a file from 2 threads (closing in one and doing whatever in another). You are right. I wouldn't go so far to say the file object thread-unsafe, but it certainly has lurking bugs with calling close while other threads are running. BTW your mail doesn't seem to contain the actual patch. Almost every place that accesses f_fp is a problem. I think we need a reliable mechanism to prevent file_close messing with f_fp while other operations are being performed. Since each FILE has an internal lock associated with it, flockfile could be used to lock out the sections that access f_fp (falling back to a mutex in PyFileObject on platforms without flockfile) without affecting other threads that operate on other FILE's. It's questionnable whether such an overhaul would pay off, given the Py3k rewrite of the IO subsystem. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Segfault
On Wed, 2007-08-22 at 10:35 +0200, Hrvoje Nikšić wrote: I think we need a reliable mechanism to prevent file_close messing with f_fp while other operations are being performed. Since each FILE has an internal lock associated with it, flockfile could be used to lock out the sections that access f_fp (falling back to a mutex in PyFileObject on platforms without flockfile) without affecting other threads that operate on other FILE's. Scrap this suggestion, it won't work as written. flockfile must not be called after the FILE is deallocated, which is exactly what fclose does. The mutex would need to be in PyFileObject itself. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Segfault
On Mon, 2007-08-20 at 20:27 +0200, Maciej Fijalkowski wrote:
import thread
while 1:
f = open(/tmp/dupa, w)
thread.start_new_thread(f.close, ())
f.close()
file_close inadvertently allows fclose to be called twice on the same
stdio file. This patch should fix the problem:
Python-2.5.1/Objects/fileobject.c
--- Python-2.5.1.orig/Objects/fileobject.c 2007-01-23 14:54:30.0
+0100
+++ Python-2.5.1/Objects/fileobject.c 2007-08-21 10:04:18.0 +0200
@@ -440,13 +440,14 @@
{
int sts = 0;
if (f-f_fp != NULL) {
+ FILE *fp = f-f_fp;
+ f-f_fp = NULL;
if (f-f_close != NULL) {
Py_BEGIN_ALLOW_THREADS
errno = 0;
- sts = (*f-f_close)(f-f_fp);
+ sts = (*f-f_close)(fp);
Py_END_ALLOW_THREADS
}
- f-f_fp = NULL;
}
PyMem_Free(f-f_setbuf);
f-f_setbuf = NULL;
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
On Wed, 2007-06-20 at 19:26 +0200, Martin v. Löwis wrote: As Hrvoje says: try python-list (aka comp.lang.python). If you don't get an answer, you didn't phrase your question interestingly enough, or nobody knows the answer, or nobody has the time to tell you. The thing with comp.lang.python is that it is followed by a large number of Python users, but a much smaller number of the C API users -- which is only natural, since the group is about Python, not about C. For most users the Python/C API is an implementation detail which they never have to worry about. Futrhermore, questions about the C API often concern CPython implementation details and so they don't feel like they would belong in comp.lang.python. As an experiment, it might make sense to open a mailing list dedicated to the Python C API. It could become a useful support forum for extension writers (a group very useful to Python) and maybe even a melting pot for new ideas regarding CPython, much like comp.lang.python historically provided ideas for Python the language. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
On Wed, 2007-06-20 at 00:21 +1000, Campbell Barton wrote:
I want to add my own call's before and after PyLists standard functions
but have a proplem with functons that use keywords and have no API
equivalent.
For example, I cant use the API's PyList_Sort because that dosnt support
keywords like...
ls.sort(key=lambda a: a.foo))
And the Problem with PyObject_CallMethod is that it dosnt accept keywords.
Note that you can always simply call PyObject_Call on the bound method
object retrieved using PyObject_GetAttrString. The hardest part is
usually constructing the keywords dictionary, a job best left to
Py_BuildValue and friends. When I need that kind of thing in more than
one place, I end up with a utility function like this one:
/* Equivalent to PyObject_CallMethod but accepts keyword args. The
format... arguments should produce a dictionary that will be passed
as keyword arguments to obj.method.
Usage example:
PyObject *res = call_method(lst, sort, {s:O}, key, keyfun));
*/
PyObject *
call_method(PyObject *obj, const char *methname, char *format, ...)
{
va_list va;
PyObject *meth = NULL, *args = NULL, *kwds = NULL, *ret = NULL;
args = PyTuple_New(0);
if (!args)
goto out;
meth = PyObject_GetAttrString(obj, methname);
if (!meth)
goto out;
va_start(va, format);
kwds = Py_VaBuildValue(format, va);
va_end(va);
if (!kwds)
goto out;
ret = PyObject_Call(meth, args, kwds);
out:
Py_XDECREF(meth);
Py_XDECREF(args);
Py_XDECREF(kwds);
return ret;
}
It would be nice for the Python C API to support a more convenient way
of calling objects and methods with keyword arguments.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
[ Note that this discussion, except maybe for the suggestion to add a simpler way to call a method with keyword args, is off-topic to python-dev. ] On Wed, 2007-06-20 at 20:17 +1000, Campbell Barton wrote: I dont think I can use PyObject_GetAttrString because the subtype would return a reference to this function - rather then the lists original function, Id need an instance of a list and dont have one at that point. Note that PyList_Type is a full-fledged PyObject, so PyObject_GetAttrString works on it just fine. Of course, you would also need to add the self argument before the keywords, but that's a trivial change to the function. Calling PyObject_GetAttrString feels cleaner than accessing tp_dict directly, and most importantly call_method as written delegates creation of the dictionaty to Py_BuildValue. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Calling Methods from Pythons C API with Keywords
On Wed, 2007-06-20 at 22:12 +1000, Campbell Barton wrote: Hrvoje Nikšić wrote: [ Note that this discussion, except maybe for the suggestion to add a simpler way to call a method with keyword args, is off-topic to python-dev. ] Is there a list for this kind of discussion? I believe the appropriate list would be the general Python list/newsgroup. I agree that response about the Python/C API tends to be sparse on general-purpose lists, though. If there is a forum dedicated to discussing the *use* of Python at the C level, I'd like to know about it as well. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Tracker Issues
On Wed, 2007-05-16 at 22:17 -0700, Talin wrote: Here's a simple method: Put up a free porn site [...] Is it known that someone actually implemented this? It's a neat trick, but as far as I know, it started as a thought experiment of what *could* be done to fairly easily defeat the captchas, as well as all other circumvention methods that make use of human intelligence. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] \u and \U escapes in raw unicode string literals
On Fri, 2007-05-11 at 13:06 -0700, Guido van Rossum wrote: attribution_pattern = re.compile(ur'(---?(?!-)|\u2014) *(?=[^ \n])') But wouldn't it be just as handy to teach the re module about \u and \U, just as it already knows about \x (and \123 octals)? And \n, \r, etc. Implementing \u in re is not only useful, but also consistent. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] deprecate commands.getstatus()
On Thu, 2007-03-22 at 13:38 -0700, Guido van Rossum wrote: Sounds good to me. In 3.0 we should probably not have os.popen*(), nor the popen2 module at all, and do everything via the subprocess module. I wonder if we should even get rid of os.system(); then there should be a subprocess.system() instead. And do we even need os.fork(), os.exec*(), os.spawn*()? Please don't remove os.fork and os.exec*. Some people need to fine-tune process creation and don't need portability to non-Unix OS'es. For them, the functions that call the underlying system API and little or nothing else are a god-send. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Interning string subtype instances
On Mon, 2007-02-12 at 12:29 -0800, Josiah Carlson wrote: Hrvoje Nikšic [EMAIL PROTECTED] wrote: I propose modifying PyString_InternInPlace to better cope with string subtype instances. Any particular reason why the following won't work for you? [... snipped a simple intern implementation ...] Because Python internals don't use that; they call PyString_InternInPlace directly. Another reason is that Python's interning mechanism is much better than such a simple implementation: it stores the interned state directly in the PyString_Object structure, so you can find out that a string is already interned without looking it up in the dictionary. This information can (and is) used by both Python core and by C extensions. Another advantage is that, as of recently, interned strings can be garbage collected, which is typically not true of simple replacements (although it could probably be emulated by using weak references, it's not trivial.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Interning string subtype instances
On Wed, 2007-02-07 at 15:39 +0100, Hrvoje Nikšić wrote: The patch could look like this. If there is interest in this, I can produce a complete patch. The complete patch is now available on SourceForge: http://sourceforge.net/tracker/index.php?func=detailaid=1658799group_id=5470atid=305470 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Interning string subtype instances
I propose modifying PyString_InternInPlace to better cope with string
subtype instances.
Current implementation of PyString_InternInPlace does nothing and
returns if passed an instance of a subtype of PyString_Type. This is a
problem for applications that need to support string subtypes, but also
must intern the strings for faster equivalence testing. Such an
application, upon receiving a string subtype, will silently fail to
work.
There is good reason for PyString_InternInPlace not accepting string
subtypes: since a subtype can have modified behavior, interning it can
cause problems for other users of the interned string. I agree with the
reasoning, but propose a different solution: when interning an instance
of a string subtype, PyString_InternInPlace could simply intern a copy.
This should be a fully backward compatible change because: 1) code that
passes PyString instances (99.99% cases) will work as before, and 2)
code that passes something else silently failed to intern the string
anyway. Speed should be exactly the same as before, with the added
benefit that interning PyString subtype instances now does something,
but without the problems that interning the actual instance can produce.
The patch could look like this. If there is interest in this, I can
produce a complete patch.
@@ -5,10 +5,6 @@
PyObject *t;
if (s == NULL || !PyString_Check(s))
Py_FatalError(PyString_InternInPlace: strings only please!);
- /* If it's a string subclass, we don't really know what putting
- it in the interned dict might do. */
- if (!PyString_CheckExact(s))
- return;
if (PyString_CHECK_INTERNED(s))
return;
if (interned == NULL) {
@@ -25,6 +21,18 @@
*p = t;
return;
}
+ /* Make sure we don't intern a string subclass, since we don't
+ really know what putting it in the interned dict might do. */
+ if (!PyString_CheckExact(s)) {
+ PyObject *copy;
+ copy = PyString_FromStringAndSize(PyString_AS_STRING(*p),
+ PyString_GET_SIZE(*p));
+ if (!copy)
+ return;
+ Py_DECREF(*p);
+ *p = copy;
+ s = (PyStringObject *) copy;
+ }
if (PyDict_SetItem(interned, (PyObject *)s, (PyObject *)s) 0) {
PyErr_Clear();
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
