Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
Hi, I fixed various bugs in the implementation of the (new) PEP 446: http://hg.python.org/features/pep-446 At revision da685bd67524, the full test suite pass on: - Fedora 18 (Linux 3.9), x86_64 - FreeBSD 9.1, x86_64 - Windows 7 SP1, x86_64 - OpenIndiana (close to Solaris 11), x86_64 Some tests are failing, but these failures are unrelated to the PEP 446 (same tests are failing in the original Python): - Windows: test_signal, failure related to faulthandler (issue already fixed in default) - OpenIndiana: test_locale, test_uuid Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
2013/8/12 Victor Stinner victor.stin...@gmail.com: I fixed various bugs in the implementation of the (new) PEP 446: http://hg.python.org/features/pep-446 At revision da685bd67524, the full test suite pass on: (...) I also checked the usage of atomic flags. There was a minor bug on Linux, it is now fixed (remove an useless call to fcntl to check if SOCK_CLOEXEC works). open(): On Linux, FreeBSD and Solaris 11, O_CLOEXEC flag is used. fcntl(F_GETFD) is only called once for all file descriptors, to check if O_CLOEXEC works. On Windows, O_NOINHERIT is used. socket.socket(): On Linux, SOCK_CLOEXEC flag is used, no extra syscall is required. os.pipe(): On Linux, pipe2() is used with O_CLOEXEC. On other platforms, os.set_inheritable() must be called to make the new file descriptors non-inheritables. On Windows, the atomic flag WSA_FLAG_NO_HANDLE_INHERIT is not used to create a socket. I don't know the Windows well enough to make such change. My OpenIndiana VM looks to be older than Solaris 11: O_CLOEXEC flag is missing. I regenerated the patch in the isssue: http://bugs.python.org/issue18571 Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
Also the socket library creates sockets with inheritable handles by default. Apparently there isn't a reliable way to make sockets non-inheritable because anti-virus/firewall software can interfere: http://stackoverflow.com/questions/12058911/can-tcp-socket-handles-be-set-not-inheritable Recent versions of Windows provide an atomic flag to create a non-inheritable socket. I hope that the falg is respected even with antivirus/firewall. For older versions of Windows, I don't see what Python can do. Is it a blocker point for the PEP? Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
2013/8/6 Victor Stinner victor.stin...@gmail.com: Oh, the summary table is wrong for the subprocess, default line: all inheritable handles are inherited if at least one standard stream is replaced. I updated the PEP: - add a new section Performances of Closing All File Descriptors - mention a previous attempt in 2007 to add open_noinherit - complete the summary table of the status of python 3.3 to mention the subprocess, replace stdout case - Windows creates non-inheritable *handles* (not fds) by default See the history: http://hg.python.org/peps/log/tip/pep-0446.txt Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
On 06/08/2013 1:23am, Victor Stinner wrote: Each operating system handles the inheritance of file descriptors differently. Windows creates non-inheritable file descriptors by default, whereas UNIX creates inheritable file descriptors by default. The Windows API creates non-inheritable *handles* by default. But the C runtime creates inheritable fds by default. Also the socket library creates sockets with inheritable handles by default. Apparently there isn't a reliable way to make sockets non-inheritable because anti-virus/firewall software can interfere: http://stackoverflow.com/questions/12058911/can-tcp-socket-handles-be-set-not-inheritable -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
On 06/08/2013 1:23am, Victor Stinner wrote: Each operating system handles the inheritance of file descriptors differently. Windows creates non-inheritable file descriptors by default, whereas UNIX creates inheritable file descriptors by default. The Windows API creates non-inheritable *handles* by default. But the C runtime creates inheritable fds by default. Also the socket library creates sockets with inheritable handles by default. Apparently there isn't a reliable way to make sockets non-inheritable because anti-virus/firewall software can interfere: http://stackoverflow.com/questions/12058911/can-tcp-socket-handles-be-set-not-inheritable -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (New) PEP 446: Make newly created file descriptors non-inheritable
On Windows, the ``subprocess`` closes all handles and file descriptors in the child process by default. If at least one standard stream (stdin, stdout or stderr) is replaced (ex: redirected into a pipe), all inheritable handles are inherited in the child process. Summary: === = == = Module FD on UNIX Handles on Windows FD on Windows === = == = subprocess, default STD, pass_fds noneSTD Oh, the summary table is wrong for the subprocess, default line: all inheritable handles are inherited if at least one standard stream is replaced. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP
On Thu, Mar 22, 2012 at 00:39, Huan Do dob...@gmail.com wrote: Tell me what you guys think. I don't really want to add more things to the language, so I hate to say this: It makes sense to me. However, the syntax is very close to the syntax for function annotations. But that's when defining, and this is when calling, so it might work anyway, I don't have the knowledge necessary to know. So put it up on python-ideas. I'm not on that list, but people who know more about this are, so they can tell you if this is feasible or not and if it is a good idea or not. //Lennart ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP
On 03/21/2012 07:39 PM, Huan Do wrote: *Hi, I am a graduating Berkeley student that loves python and would like to propose an enhancement to python. My proposal introduces a concept of slicing generator. For instance, if one does x[:] it returns a list which is a copy of x. Sometimes programmers would want to iterate over a slice of x, but they do not like the overhead of constructing another list. Instead we can create a similar operator that returns a generator. My proposed syntax is x(:). The programmers are of course able to set lower, upper, and step size like the following. x(1::-1) This would make code much cleaner in a lot of instances, one example lets say we have a very large list x and we want to sum all the numbers but the last 20, and we only want to loop through the even indices. We would have to do something like this. sum(x[:-20:2]) or we can do a workaround to save space for time and do something like this. sum( value for i, value in enumerate(x) if i -20 and not i % 2 ) But with my proposal we are able do the following. sum(x(:-20:2)) Which affords space without sacrificing expressiveness. For another example lets say we have a problem that we want to check a condition is true for every pairwise element in a list x. def allfriends(x): for i in range(len(x)): for j in range(i+1, len(x)): if not friends(x[i], x[j]): return False return True A more pythonic way is to actually loop through the values instead of the indices like this. def allfriends(x): for i, a in enumerate(x): for j, b in enumerate(x[i+1:]): if not friends(a, b): return False return True This however bring a lot of overhead because we have to construct a new list for every slice call. With my proposal we are able to do this. def allfriends(x): for i, a in enumerate(x): for j, b in enumerate(x(i+1:)): if not friends(a, b): return False return True This proposal however goes against one heuristic in the zen of python, namely “Special cases aren’t special enough to break the rules.” The way that the proposal breaks this rule is because the syntax x(:), uses a function call syntax but would be a special case here. I chose using parenthesis because I wanted this operation to be analogous to the generator syntax in list comprehensions. ListGenerators Comprehension [ x for x in L ]( x for x in L ) Slicing L[a:b:c]L(a:b:c) Tell me what you guys think. Thanks!* ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/animelovin%40gmail.com Hi, I'm not sure i get it.. Assuming your PEP is accepted, what should happens then to the lambda op and standard function calls ? Or Is this merely another case of metaprogramming, which obviously should not be confused with languages such as lisp? Thank you, Etienne ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP
My proposed syntax is x(:) Change the Python syntax is not a good start. You can already experiment your idea using the slice() type. We would have to do something like this. sum(x[:-20:2]) Do you know the itertools module? It looks like itertools.islice(). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP
@Ethan Furman each call to x(:) would return a different iterator, so both sides will have their own information about where they are. Also it is the case that checking for equality of generators does not make the generators to expand out, so checking for equality becomes to checking if they are the same generator object. The following example shows this Python3 (x for x in range(10)) == (x for x in range(10)) False @Etienne lambda is a keyword and would get captured by the lexer, so this should conflict with adding the grammar that would make this work. This is different than function calls because currently arguments of function calls cannot have :, causing `x(:)` to be a syntax error. The grammar that would have to be added would be mutually exclusive from current functionality. @Victor I was not completely familiar with itertools but itertools.islice() seems to have the functionality that I propose. It is great that there already exist a solution that does not change python's syntax. Unless anyone wants to pursue this proposal I will drop it next week. Thanks for your feedback guys On Wed, Mar 21, 2012 at 5:09 PM, Ethan Furman et...@stoneleaf.us wrote: Huan Do wrote: *Hi, I am a graduating Berkeley student that loves python and would like to propose an enhancement to python. My proposal introduces a concept of slicing generator. For instance, if one does x[:] it returns a list which is a copy of x. Sometimes programmers would want to iterate over a slice of x, but they do not like the overhead of constructing another list. Instead we can create a similar operator that returns a generator. My proposed syntax is x(:). The programmers are of course able to set lower, upper, and step size like the following. x(1::-1) The biggest problem with your proposal is that generators don't remember what they have already yielded, so x(:) != x(:) # first time gets everything, second time gets nothing ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP
Huan Do wrote: *Hi, I am a graduating Berkeley student that loves python and would like to propose an enhancement to python. My proposal introduces a concept of slicing generator. For instance, if one does x[:] it returns a list which is a copy of x. Sometimes programmers would want to iterate over a slice of x, but they do not like the overhead of constructing another list. Instead we can create a similar operator that returns a generator. My proposed syntax is x(:). The programmers are of course able to set lower, upper, and step size like the following. x(1::-1) The biggest problem with your proposal is that generators don't remember what they have already yielded, so x(:) != x(:) # first time gets everything, second time gets nothing ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP
On Thu, Mar 22, 2012 at 10:28 AM, Huan Do dob...@gmail.com wrote: I was not completely familiar with itertools but itertools.islice() seems to have the functionality that I propose. It is great that there already exist a solution that does not change python's syntax. Unless anyone wants to pursue this proposal I will drop it next week. Just as a further follow-up on the recommended approach for making suggestions: for initial concepts like this one, the python-ideas mailing list is the preferred venue. It's intended for initial validation and refinement of suggestions to see if they're a reasonable topic for the main development list. Many ideas don't make it past the python-ideas stage (either because they have too many problems, they get redirected to a third party PyPI project, or existing alternatives are pointed out, as happened in this case). Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP numbering scheme
On Mar 12, 2012, at 07:33 PM, Brett Cannon wrote: It came up at the sprints about how to choose new PEP numbers. It was agreed that the newest, *lowest* number should be used (e.g. 418) and not the next highest number (e.g. 3156). I have already updated PEP 1 to reflect this. +1 -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new pep index
On Fri, Jan 9, 2009 at 09:50, Benjamin Peterson benja...@python.org wrote: This is just a note that the PEP index (PEP 0) is now automatically generated, so you need not bother to update any more. Thanks for getting this done! -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Neal Norwitz wrote: On 1/10/06, M.-A. Lemburg [EMAIL PROTECTED] wrote: We'd also have to make sure that old extensions don't just import with a warning, since the change will introduce buffer overflows and seg faults in extensions that are not aware of the change. I agree that on 64-bit platforms we should prevent the import. In the past we only provided a warning and the users were on their own. This is different. If you read my massive checkin to check the return results of Py_InitModule*(), you'll realize this isn't as simple as just failing in Py_InitMethod*(). I was hoping to just modify Py_InitModule4() in Python/modsupport.c to fail and return NULL. That doesn't seem practical given that we didn't check return results. We will just crash the interpreter with standard python 2.4 modules. ISTM we need to modify _PyImport_LoadDynamicModule() in Python/importdl.c before calling the init function (line 56, (*p)()) to check for some magic symbol that is defined only when compiling 2.5 and above. For example we could add a static int _64_bit_clean = 1; in modsupport.h. Without some trickery we will get this defined in every .o file though, not just modules. Other ideas? We could explicitly break binary compatibility for Python 2.5 on 64-bit platforms, by changing the name of an often used API, e.g. the Py_InitModule*() APIs. This is how Unicode does it - we map the various APIs to either ...UCS2 or ...UCS4, so that you cannot import an extension compiled for e.g. UCS2 into a Python interpreter compiled for UCS4. If we didn't, you'd get seg faults and buffer overflows the same way you would with the ssize_t change on 64-bit platforms. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 19 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Neal Norwitz wrote: Other ideas? We could rename essential function symbols, like PyInitModule4 (to, say, PyInitModule4b). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
On 1/10/06, M.-A. Lemburg [EMAIL PROTECTED] wrote: We'd also have to make sure that old extensions don't just import with a warning, since the change will introduce buffer overflows and seg faults in extensions that are not aware of the change. I agree that on 64-bit platforms we should prevent the import. In the past we only provided a warning and the users were on their own. This is different. If you read my massive checkin to check the return results of Py_InitModule*(), you'll realize this isn't as simple as just failing in Py_InitMethod*(). I was hoping to just modify Py_InitModule4() in Python/modsupport.c to fail and return NULL. That doesn't seem practical given that we didn't check return results. We will just crash the interpreter with standard python 2.4 modules. ISTM we need to modify _PyImport_LoadDynamicModule() in Python/importdl.c before calling the init function (line 56, (*p)()) to check for some magic symbol that is defined only when compiling 2.5 and above. For example we could add a static int _64_bit_clean = 1; in modsupport.h. Without some trickery we will get this defined in every .o file though, not just modules. Other ideas? n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
M.-A. Lemburg wrote: Sure. However, most users will compile it on 32-bit systems. If they find they cannot get it to work on a 64-bit system, they should ask the author for help, or just use it in 32-bit mode (as 64-bit mode won't gain them anything, anyway). I wonder how you are going to import a 32-bit extension into a 64-bit binary of Python. It simply doesn't work. No. They will need to run the 32-bit binary of Python on the 64-bit system. I wouldn't say that PyString_AsStringAndSize() is rarely used and neither is PyArg_ParseTuple(). I can't find much usage of PyString_AsStringAndSize. Even in the Python core, it is only called 8 times outside string and strop. I agree that other APIs are certainly more domain specific and can be dealt with in the extension, but those two APIs need special care and so do the type slot functions. For PyArg_ParseTuple, there is special care, because there won't be compiler warnings otherwise. For PyString_AsStringAndSize, the compiler will warn about incorrect pointer types. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Martin v. Löwis wrote: M.-A. Lemburg wrote: If it were this easy, I wouldn't have objections. But it's not. It is so easy. Can you should me an example where it isn't? The variables you use with these APIs tend to propagate through the extension, you use them in other calls, make assignments, etc. They only propage if you make them. You don't have to, if you don't want to. If you implement extension types, you end up having to convert all the length related struct variables to Py_ssize_t. Only if you want to. If not, the module will work (nearly) unmodified. Of course, it will be limited to 32-bit indices. See my other reply on this topic. All this is fine, but it's also a lot of work which can be made easier. Recompiling an extension is well within range of many Python users, manually checking, fixing and porting it to the new API is certainly not. Sure. However, most users will compile it on 32-bit systems. If they find they cannot get it to work on a 64-bit system, they should ask the author for help, or just use it in 32-bit mode (as 64-bit mode won't gain them anything, anyway). I wonder how you are going to import a 32-bit extension into a 64-bit binary of Python. It simply doesn't work. The set of functions that will require Py_ssize_t is getting larger in your branch which is why I started this discussion. How so? I did not add a single function that has int* output values, AFAICT. No, but there are quite a few APIs with Py_ssize_t* output values. I am talking about the entirety of these functions, and claim that they are rarely used (including the Unicode and buffer APIs). I wouldn't say that PyString_AsStringAndSize() is rarely used and neither is PyArg_ParseTuple(). I agree that other APIs are certainly more domain specific and can be dealt with in the extension, but those two APIs need special care and so do the type slot functions. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 12 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Tim Peters wrote: This reminded me that I still owe you a reply about s# and t# format codes. It occurred to me that I've never used them, and probably never will, so I really don't care how they work: I'm only really worried about widespread ugliness, meaning wide enough that it touches me ;-) Ok, given that Armin also gave in, I think I keep that at a macro. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Martin v. Löwis wrote: Armin Rigo wrote: This would do the right thing for = 2.4, using ints everywhere; and the Python.h version 2.5 would detect the #define and assume it's a 2.5-compatible module, so it would override the #define with the real thing *and* turn on the ssize_t interpretation of the '#' format character. This would be very similar to the PY_SIZE_T_CLEAN approach, except that it would also help to detect spelling mistakes. From an implementation point of view, the real challenge is to give PyArg_ParseTuple a different meaning; I do this be #defining it to PyArg_ParseTupleSsize_t (to preserve binary compatibility for the original interpretation of ParseTuple). Putting additional flags arguments in the entire code is also quite hackish. I still don't like the idea of a magic #define that changes the behavior of '#include Python.h', but I admit I don't find any better solution. I suppose I'll just blame C. More precisely, the printf style of function calling, and varargs functions. ISO C is pretty type safe, but with varargs functions, you lose that completely. I still hope I can write a C parser some day that does ParseTuple/BuildValue checking. At least gcc does check the parameter types and generates warnings for wrong vararg parameters passed to printf() et al. We definitely need a clean solution for this. Otherwise, authors who want to support more than just the latest Python release will run into deep trouble. Note that the branch also has other changes of output parameter types which will cause problems in extensions (not only extensions implementing the sequence protocol as the PEP suggests, but also ones using it as well as extensions implementing or using the buffer protocol and the slicing protocol). These are not as easily avoidable as the PyArg_ParseTuple() problem which could be dealt with by a a new parser marker for ssize_t lengths (with the '#' marker casting the argument to an int). I don't see a good solution for these other than introducing a set of new APIs (and maybe doing some macro magic as Martin did for PyArg_ParseTuple()). Due to the fact that changes in the types of output parameters require changes in the extension variable type layout itself, they introduce a large number of type changes in the extension and make writing backwards compatible extensions harder than necessary. Furthermore, all extensions for Python 2.4 would have to be ported to the new Python API and porting is not going to be a simple recompile, but will require C skills. We'd also have to make sure that old extensions don't just import with a warning, since the change will introduce buffer overflows and seg faults in extensions that are not aware of the change. Martin, please add the above points to the PEP. I'd also like to see it published, because it's hard to track a PEP in the mailing Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 10 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
M.-A. Lemburg wrote: I don't see a good solution for these other than introducing a set of new APIs (and maybe doing some macro magic as Martin did for PyArg_ParseTuple()). Due to the fact that changes in the types of output parameters require changes in the extension variable type layout itself, they introduce a large number of type changes in the extension and make writing backwards compatible extensions harder than necessary. That's not true. It is very easy to write extensions that receive such values and are still backwards-compatible. Suppose you had int pos; PyObject *k, *v; PyDict_Next(dict, pos, k, v); You just change this to /* beginning of file */ #ifdef Py_HEX_VERSION 2.5 typedef int Py_ssize_t; #endif /* later */ Py_ssize_t pos; PyObject *k, *v; PyDict_Next(dict, pos, k, v); That's it! Furthermore, all extensions for Python 2.4 would have to be ported to the new Python API and porting is not going to be a simple recompile, but will require C skills. Not all extensions. Only those that call functions that expect int* output parameters - which is fairly uncommon. Martin, please add the above points to the PEP. I'd also like to see it published, because it's hard to track a PEP in the mailing It's very difficult to get a PEP number assigned. I wrote [EMAIL PROTECTED] with no response. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Martin v. Löwis wrote: M.-A. Lemburg wrote: I don't see a good solution for these other than introducing a set of new APIs (and maybe doing some macro magic as Martin did for PyArg_ParseTuple()). Due to the fact that changes in the types of output parameters require changes in the extension variable type layout itself, they introduce a large number of type changes in the extension and make writing backwards compatible extensions harder than necessary. That's not true. It is very easy to write extensions that receive such values and are still backwards-compatible. Suppose you had int pos; PyObject *k, *v; PyDict_Next(dict, pos, k, v); You just change this to /* beginning of file */ #ifdef Py_HEX_VERSION 2.5 typedef int Py_ssize_t; #endif /* later */ Py_ssize_t pos; PyObject *k, *v; PyDict_Next(dict, pos, k, v); That's it! If it were this easy, I wouldn't have objections. But it's not. The variables you use with these APIs tend to propagate through the extension, you use them in other calls, make assignments, etc. If you implement extension types, you end up having to convert all the length related struct variables to Py_ssize_t. If you're writing against 3rd party APIs which don't use ssize_t or size_t, you have to convert Py_ssize_t to int where necessary. All this is fine, but it's also a lot of work which can be made easier. Recompiling an extension is well within range of many Python users, manually checking, fixing and porting it to the new API is certainly not. Furthermore, all extensions for Python 2.4 would have to be ported to the new Python API and porting is not going to be a simple recompile, but will require C skills. Not all extensions. Only those that call functions that expect int* output parameters - which is fairly uncommon. The set of functions that will require Py_ssize_t is getting larger in your branch which is why I started this discussion. In the first checkin you only had the rarely used slice APIs converted. In the meantime the buffer APIs, the Unicode APIs and others have been added to the list. These APIs are used a lot more often than the slice APIs. I'm not saying that it's a bad idea to adjust these to Py_ssize_t, it's just the backwards incompatible way this is done which bothers me. Martin, please add the above points to the PEP. I'd also like to see it published, because it's hard to track a PEP in the mailing It's very difficult to get a PEP number assigned. I wrote [EMAIL PROTECTED] with no response. Would it be possible to host the PEP in the python.org wiki or maybe in the sandbox on svn.python.org ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 10 2006) Python/Zope Consulting and Support ...http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
M.-A. Lemburg wrote: If it were this easy, I wouldn't have objections. But it's not. It is so easy. Can you should me an example where it isn't? The variables you use with these APIs tend to propagate through the extension, you use them in other calls, make assignments, etc. They only propage if you make them. You don't have to, if you don't want to. If you implement extension types, you end up having to convert all the length related struct variables to Py_ssize_t. Only if you want to. If not, the module will work (nearly) unmodified. Of course, it will be limited to 32-bit indices. All this is fine, but it's also a lot of work which can be made easier. Recompiling an extension is well within range of many Python users, manually checking, fixing and porting it to the new API is certainly not. Sure. However, most users will compile it on 32-bit systems. If they find they cannot get it to work on a 64-bit system, they should ask the author for help, or just use it in 32-bit mode (as 64-bit mode won't gain them anything, anyway). The set of functions that will require Py_ssize_t is getting larger in your branch which is why I started this discussion. How so? I did not add a single function that has int* output values, AFAICT. I am talking about the entirety of these functions, and claim that they are rarely used (including the Unicode and buffer APIs). Would it be possible to host the PEP in the python.org wiki or maybe in the sandbox on svn.python.org ? I pinged the PEP editors again, and they assigned the PEP number. Hosting it in a Wiki would be besides the point of the PEP process. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
[Martin v. Löwis] ... I am talking about the entirety of these functions, and claim that they are rarely used (including the Unicode and buffer APIs). This reminded me that I still owe you a reply about s# and t# format codes. It occurred to me that I've never used them, and probably never will, so I really don't care how they work: I'm only really worried about widespread ugliness, meaning wide enough that it touches me ;-) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
[I just noticed that I sent this mail to just Martin when I meant it for the list. Sorry Martin!] On 1/5/06, Martin v. Löwis [EMAIL PROTECTED] wrote: More precisely, the printf style of function calling, and varargs functions. ISO C is pretty type safe, but with varargs functions, you lose that completely. I still hope I can write a C parser some day that does ParseTuple/BuildValue checking. I put together a non-parsing checker last month to help me feel more secure after http://python.org/sf/1365916. It's awful code, but the simple things are easy to change or extend. Fixing the false positives and other misinterpretations is harder. http://www.tortall.net/mu/static/fmtcheck.py?raw - it takes a list of directories to os.walk for c source files. Michael -- Michael Urman http://www.tortall.net/mu/blog ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
On 1/6/06, Michael Urman [EMAIL PROTECTED] wrote: I put together a non-parsing checker last month to help me feel more secure after http://python.org/sf/1365916. It's awful code, but the simple things are easy to change or extend. Fixing the false positives and other misinterpretations is harder. Perhaps you could use Synopsis to parse the C code? http://synopsis.fresco.org/ I'm hoping to look into this more. IIRC, it generates an AST from the C source which you can iterate over to find problems. It's got Python APIs. I was hoping it could provide the basis for a static analysis tool to help find local memory (ref) leaks among other things. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Hi Martin, On Fri, Dec 30, 2005 at 11:26:44AM +0100, Martin v. L?wis wrote: Hum. It would be much cleaner to introduce a new format character to replace '#' and deprecate '#'... That would certainly be clearer. What character would you suggest? I see two drawbacks with that approach: 1. writing backwards-compatible modules will become harder. Users have to put ifdefs around the ParseTuple calls (or atleast around the format strings) Ok, I see your point. In theory we could reuse a macro-based trick in C extensions: #include Python.h #ifndef Py_SIZE_CHR typedef int Py_Ssize_t; #define Py_SIZE_CHR # #endif And then we can replace -- say -- the format string is#s# with is Py_SIZE_CHR s Py_SIZE_CHR But it's rather cumbersome. An equally strange alternative would be to start C modules like this: #define Py_Ssize_t int /* compatibility with Python = 2.4 */ #include Python.h This would do the right thing for = 2.4, using ints everywhere; and the Python.h version 2.5 would detect the #define and assume it's a 2.5-compatible module, so it would override the #define with the real thing *and* turn on the ssize_t interpretation of the '#' format character. Not that I think that this is a good idea. Just an idea. I still don't like the idea of a magic #define that changes the behavior of '#include Python.h', but I admit I don't find any better solution. I suppose I'll just blame C. A bientot, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Armin Rigo wrote: This would do the right thing for = 2.4, using ints everywhere; and the Python.h version 2.5 would detect the #define and assume it's a 2.5-compatible module, so it would override the #define with the real thing *and* turn on the ssize_t interpretation of the '#' format character. This would be very similar to the PY_SIZE_T_CLEAN approach, except that it would also help to detect spelling mistakes. From an implementation point of view, the real challenge is to give PyArg_ParseTuple a different meaning; I do this be #defining it to PyArg_ParseTupleSsize_t (to preserve binary compatibility for the original interpretation of ParseTuple). Putting additional flags arguments in the entire code is also quite hackish. I still don't like the idea of a magic #define that changes the behavior of '#include Python.h', but I admit I don't find any better solution. I suppose I'll just blame C. More precisely, the printf style of function calling, and varargs functions. ISO C is pretty type safe, but with varargs functions, you lose that completely. I still hope I can write a C parser some day that does ParseTuple/BuildValue checking. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Martin v. Löwis wrote: Please let me know what you think. Regards, Martin PEP: XXX Title: Using ssize_t as the index type Version: $Revision$ Last-Modified: $Date$ Author: Martin v. Löwis [EMAIL PROTECTED] Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 18-Dec-2005 Post-History: Abstract In Python 2.4, indices of sequences are restricted to the C type int. On 64-bit machines, sequences therefore cannot use the full address space, and are restricted to 2**31 elements. This PEP proposes to change this, introducing a platform-specific index type Py_ssize_t. An implementation of the proposed change is in http://svn.python.org/projects/python/branches/ssize_t. Sounds wonderful. Would love to see this in Python 2.5. This will fix important 64-bit issues. Perhaps someone could explain to me what the difference between Py_ssize_t and Py_intptr_t would be? Is this not a satisfactory Py_ssize_t already? Rationale = 64-bit machines are becoming more popular, and the size of main memory increases beyond 4GiB. On such machines, Python currently is limited, in that sequences (strings, unicode objects, tuples, lists, array.arrays, ...) cannot contain more than 2GiElements. Today, very few machines have memory to represent larger lists: as each pointer is 8B (in a 64-bit machine), one needs 16GiB to just hold the pointers of such a list; with data in the list, the memory consumption grows even more. However, there are three container types for which users request improvements today: * strings (currently restricted to 2GiB) * mmap objects (likewise; plus the system typically won't keep the whole object in memory concurrently) * Numarray objects (from Numerical Python) scipy_core objects are the replacement for both Numarray and Numerical Python and support 64-bit clean objects *except* for the sequence protocol and the buffer protocol. Fixing this problem will clean up a lot of unnecessary code. Looking forward to it... -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Fredrik Lundh wrote:
well, one thing seems to missing from your PEP: in several modules, you've
changed the cast used in the type table. e.g.
...
is this change backwards compatible ?
See the section Conversion guidelines. I prefer the approach taken in
the patch below, i.e. remove the casts in initializing the type object.
Adding more typedefs for the function types would be the other option.
Regards,
Martin
Index: Modules/_elementtree.c
===
--- Modules/_elementtree.c (Revision 41837)
+++ Modules/_elementtree.c (Arbeitskopie)
@@ -92,6 +92,9 @@
#endif
/* compatibility macros */
+#if (PY_VERSION_HEX 0x0205)
+typedef int Py_ssize_t;
+#endif
#if (PY_VERSION_HEX 0x0204)
#define PyDict_CheckExact PyDict_Check
#if (PY_VERSION_HEX 0x0202)
@@ -916,8 +919,9 @@
}
static PyObject*
-element_getitem(ElementObject* self, Py_ssize_t index)
+element_getitem(PyObject* _self, Py_ssize_t index)
{
+ElementObject* self = (ElementObject*)_self;
if (!self-extra || index 0 || index = self-extra-length) {
PyErr_SetString(
PyExc_IndexError,
@@ -931,8 +935,9 @@
}
static PyObject*
-element_getslice(ElementObject* self, Py_ssize_t start, Py_ssize_t end)
+element_getslice(PyObject* _self, Py_ssize_t start, Py_ssize_t end)
{
+ElementObject* self = (ElementObject*)_self;
Py_ssize_t i;
PyObject* list;
@@ -1158,8 +1163,9 @@
}
static int
-element_setslice(ElementObject* self, Py_ssize_t start, Py_ssize_t end,
PyObject* item)
+element_setslice(PyObject* _self, Py_ssize_t start, Py_ssize_t end,
PyObject* item)
{
+ElementObject* self = (ElementObject*)_self;
int i, new, old;
PyObject* recycle = NULL;
@@ -1228,8 +1234,9 @@
}
static int
-element_setitem(ElementObject* self, Py_ssize_t index, PyObject* item)
+element_setitem(PyObject* _self, Py_ssize_t index, PyObject* item)
{
+ElementObject* self = (ElementObject*)_self;
int i;
PyObject* old;
@@ -1371,10 +1378,10 @@
(inquiry) element_length,
0, /* sq_concat */
0, /* sq_repeat */
-(ssizeargfunc) element_getitem,
-(ssizessizeargfunc) element_getslice,
-(ssizeobjargproc) element_setitem,
-(ssizessizeobjargproc) element_setslice,
+element_getitem,
+element_getslice,
+element_setitem,
+element_setslice,
};
statichere PyTypeObject Element_Type = {
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Travis E. Oliphant wrote: Sounds wonderful. Would love to see this in Python 2.5. This will fix important 64-bit issues. Perhaps someone could explain to me what the difference between Py_ssize_t and Py_intptr_t would be? Is this not a satisfactory Py_ssize_t already? Practically, yes. Conceptually, intptr_t and size_t are different things: sizeof(void*) might differ from sizeof(size_t) (e.g. when you have segment and offset in pointers). I believe in flat-address space machines, they are always the same. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Tim Peters wrote: Better to use a new gibberish character and deprecate the old one? Otherwise I'm afraid we'll be stuck supporting PY_SIZE_T_CLEAN forever, and it's not good to have the meaning of a format string depend on the presence or absence of a #define far away in the file. See my response to Armin also; I'm concerned that writing code that uses that character will not easily be backwards compatible. What character would you suggest? I would not expect that we are stuck with PY_SIZE_T_CLEAN: In P3k, Py_ssize_t should become the only type to do indexing, at which point PY_SIZE_T_CLEAN could go. That can be done in one gulp via: some_int = Py_SAFE_DOWNCAST(some_value, Py_ssize_t, int); Thanks, added (I didn't know it existed). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
+1. I think this is long overdue. While I can't judge the amount of code breakage, 2.5 is as good an opportunity as any. --Guido On 12/29/05, Martin v. Löwis [EMAIL PROTECTED] wrote: Please let me know what you think. Regards, Martin PEP: XXX Title: Using ssize_t as the index type Version: $Revision$ Last-Modified: $Date$ Author: Martin v. Löwis [EMAIL PROTECTED] Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 18-Dec-2005 Post-History: Abstract In Python 2.4, indices of sequences are restricted to the C type int. On 64-bit machines, sequences therefore cannot use the full address space, and are restricted to 2**31 elements. This PEP proposes to change this, introducing a platform-specific index type Py_ssize_t. An implementation of the proposed change is in http://svn.python.org/projects/python/branches/ssize_t. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Not sure what I think of the proposal (though I guess I'm overall +0 -- needs to be done sometime and no time like the present). However, I think this PEP should be held up as an example of how to write a good PEP. Aside from my inability to follow some of the arcane points due to lack of C programming skill, this PEP was extremely readable and well-organized. Nice job! -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ Given that C++ has pointers and typecasts, it's really hard to have a serious conversation about type safety with a C++ programmer and keep a straight face. It's kind of like having a guy who juggles chainsaws wearing body armor arguing with a guy who juggles rubber chickens wearing a T-shirt about who's in more danger. --Roy Smith ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Aahz wrote: However, I think this PEP should be held up as an example of how to write a good PEP. Aside from my inability to follow some of the arcane points due to lack of C programming skill, this PEP was extremely readable and well-organized. Nice job! Thanks! Part of it probably stems from thinking about this stuff a lot. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Hi Martin, On Thu, Dec 29, 2005 at 03:04:30PM +0100, Martin v. L?wis wrote: New conversion functions PyInt_FromSsize_t, PyInt_AsSsize_t, PyLong_AsSsize_t are introduced. PyInt_FromSsize_t will transparently return a long int object if the value exceeds the MAX_INT. I guess you mean LONG_MAX instead of MAX_INT, in the event that ssize_t is larger than a long. Also, distinguishing between PyInt_AsSsize_t() and PyLong_AsSsize_t() doesn't seem to be useful (a quick look in your branch makes me guess that they both accept an int or a long object anyway). The conversion codes 's#' and 't#' will output Py_ssize_t if the macro PY_SIZE_T_CLEAN is defined before Python.h is included, and continue to output int if that macro isn't defined. Hum. It would be much cleaner to introduce a new format character to replace '#' and deprecate '#'... Compatibility with previous Python versions can be achieved with the test:: #if PY_VERSION_HEX 0x0205 typedef int Py_ssize_t; #endif and then using Py_ssize_t in the rest of the code. Nice trick :-) As far as I can tell you have done as much as possible to ensure compatibility, short of adding new slots duplicating the existing ones with the new signature -- which would make abstract.c/typeobject.c a complete nightmare. I'm +1 on doing this in 2.5. A bientot, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
[Martin v. Löwis] ... PEP: XXX Title: Using ssize_t as the index type +1, and for Python 2.5, and the sooner done the less painful for all. Same concerns as Armin, where this is especially unattractive: The conversion codes 's#' and 't#' will output Py_ssize_t if the macro PY_SIZE_T_CLEAN is defined before Python.h is included, and continue to output int if that macro isn't defined. Better to use a new gibberish character and deprecate the old one? Otherwise I'm afraid we'll be stuck supporting PY_SIZE_T_CLEAN forever, and it's not good to have the meaning of a format string depend on the presence or absence of a #define far away in the file. A suggestion: ... In these cases, three strategies are available: * statically determine that the size can never exceed an int (e.g. when taking the sizeof a struct, or the strlen of a file pathname). In this case, add a debug assert() that the value is indeed smaller than INT_MAX, and cast the value to int. That can be done in one gulp via: some_int = Py_SAFE_DOWNCAST(some_value, Py_ssize_t, int); In a debug build that will assert-fail if some_value loses info when cast from Py_ssize_t to int. If people had been careful, this would already be in use when casting from size_t to int; there's even one place in unicodeobject.c that does so ;-). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
On 12/29/05, Martin v. Löwis [EMAIL PROTECTED] wrote: Please let me know what you think. Regards, Martin PEP: XXX Title: Using ssize_t as the index type [SNIP] +1 from me. As everyone else is saying, this has to happen at some point and 2.5 is as good as any. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP: Using ssize_t as the index type
Martin v. Löwis wrote:
Please let me know what you think.
+1. in honor of the Bike Shed Effect, I'm going to assume that you've
thought of everything.
:::
well, one thing seems to missing from your PEP: in several modules, you've
changed the cast used in the type table. e.g.
--- python/branches/ssize_t/Modules/_elementtree.c (original)
+++ python/branches/ssize_t/Modules/_elementtree.c Tue Dec 20 09:52:16 2005
@@ -1228,7 +1228,7 @@
}
static int
-element_setitem(ElementObject* self, size_t index, PyObject* item)
+element_setitem(ElementObject* self, Py_ssize_t index, PyObject* item)
{
int i;
PyObject* old;
@@ -1373,7 +1373,7 @@
0, /* sq_repeat */
(ssizeargfunc) element_getitem,
(ssizessizeargfunc) element_getslice,
-(sizeobjargproc) element_setitem,
+(ssizeobjargproc) element_setitem,
(ssizessizeobjargproc) element_setslice,
};
is this change backwards compatible ?
/F
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP 342 suggestion: result() and allow return with arguments in generators (was Re: PEP 342 suggestion: start(), __call__() and unwind_call() methods)
Greg Ewing wrote: Nick Coghlan wrote: Sometimes I miss the obvious. There's a *much*, *much* better place to store the return value of a generator than on the StopIteration exception that it raises when it finishes. Just save the return value in the *generator*. I'm not convinced that this is better, because it would make value-returning something specific to generators. On the other hand, raising StopIteration(value) is something that any iterator can easily do, whether it's implemented as a generator, a Python class, a C type, or whatever. Besides, it doesn't smell right to me -- sort of like returning a value from a function by storing it in a global rather than using a return statement. Yeah, the various responses have persuaded me that having generators resemble threads in that they don't have a defined return value isn't a bad thing at all. Although that means I've gone all the way back to preferring the status quo - if you want to pass data back from a generator when it terminates, just use StopIteration(result). I'm starting to think we want to let PEP 342 bake for at least one release cycle before deciding what (if any) additional behaviour should be added to generators. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP 342 suggestion: result() and allow return with arguments in generators (was Re: PEP 342 suggestion: start(), __call__() and unwind_call() methods)
On 10/10/05, Nick Coghlan [EMAIL PROTECTED] wrote: I'm starting to think we want to let PEP 342 bake for at least one release cycle before deciding what (if any) additional behaviour should be added to generators. Yes please! -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP 342 suggestion: result() and allow return with arguments in generators (was Re: PEP 342 suggestion: start(), __call__() and unwind_call() methods)
On 10/9/05, Nick Coghlan [EMAIL PROTECTED] wrote: Sometimes I miss the obvious. There's a *much*, *much* better place to store the return value of a generator than on the StopIteration exception that it raises when it finishes. Just save the return value in the *generator*. And then provide a method on generators that is the functional equivalent of: def result(): # Finish the generator if it isn't finished already for step in self: pass return self._result # Return the result saved when the block finished It doesn't matter that a for loop swallows the StopIteration exception any more, because the return value is retrieved directly from the generator. Actually, I don't like this at all. It harks back to earlier proposals where state was stored on the generator (e.g. PEP 288). I also like that this interface could still be used even if the work of getting the result is actually farmed off to a separate thread or process behind the scenes. That seems an odd use case for generators, better addressed by creating an explicit helper object when the need exists. I bet that object will need to exist anyway to hold other information related to the exchange of information between threads (like a lock or a Queue). Looking at your example, I have to say that I find the trampoline example from PEP 342 really hard to understand. It took me several days to get it after Phillip first put it in the PEP, and that was after having reconstructed the same functionality independently. (I have plans to replace or augment it with a different set of examples, but haven't gotten the time. Old story...) I don't think that something like that ought to be motivating generator extensions. I also think that using a thread for async I/O is the wrong approach -- if you wanted to use threads shou should be using threads and you wouldn't be dealing with generators. There's a solution that uses select() which can handle as many sockets as you want without threads and without the clumsy polling (is it ready yet? is it ready yet? is it ready yet?). I urge you to leave well enough alone. There's room for extensions after people have built real systems with the raw material provided by PEP 342 and 343. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New PEP 342 suggestion: result() and allow return with arguments in generators (was Re: PEP 342 suggestion: start(), __call__() and unwind_call() methods)
Nick Coghlan wrote: Sometimes I miss the obvious. There's a *much*, *much* better place to store the return value of a generator than on the StopIteration exception that it raises when it finishes. Just save the return value in the *generator*. I'm not convinced that this is better, because it would make value-returning something specific to generators. On the other hand, raising StopIteration(value) is something that any iterator can easily do, whether it's implemented as a generator, a Python class, a C type, or whatever. Besides, it doesn't smell right to me -- sort of like returning a value from a function by storing it in a global rather than using a return statement. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new PEP type: Process
[Aahz] Go ahead and make PEP 6 a Process PEP. Done! -- David Goodger http://python.net/~goodger signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new PEP type: Process
On Fri, Aug 12, 2005, David Goodger wrote: Barry Warsaw and I, the PEP editors, have been discussing the need for a new PEP type lately. Martin von L?wis' PEP 347 was a prime example of a PEP that didn't fit into the existing Standards Track and Informational categories. We agreed upon a new Process PEP type. For more information, please see PEP 1 (http://www.python.org/peps/pep-0001.html) -- the type of which has also been changed to Process. Go ahead and make PEP 6 a Process PEP. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The way to build large Python applications is to componentize and loosely-couple the hell out of everything. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
