[Python-Dev] nonlocal x = value
Hi, it seems that Py3 doesn't support setting a "nonlocal" value as part of the "nonlocal" command Python 3.2a4+ (py3k:86480, Nov 16 2010, 16:43:22) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> def x(): ... y = 5 ... def f(): ... nonlocal y = 6 File "", line 4 nonlocal y = 6 ^ SyntaxError: invalid syntax even though the PEP clearly describes this feature. http://www.python.org/dev/peps/pep-3104/#proposed-solution Either the PEP or the implementation should be updated. Personally, I think the PEP should be changed as I think that the syntax complicates the grammar more than it's worth. Also, the moratorium applies here, given that Py3.1 does not implement this. Comments? Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
Senthil Kumaran wrote: Given these, any assumption that servers no longer support HTTP/0.9 becomes false. But as long as httplib only sends requests with a version number >= 1.0, it should be able to expect headers in the response, shouldn't it? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thu, Dec 16, 2010 at 1:30 PM, wrote: > I doubt this makes a difference to the point being discussed, but it > _could_. I suggest performing your tests with telnet, instead. I received similar results using telnet earlier today. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
* Fred Drake wrote:
> On Thu, Dec 16, 2010 at 10:52 AM, André Malo wrote:
> > I'd vote for removing it from the client code and keeping it in the
> > server.
>
> If it must be maintained anywhere, it should be in the client,
> according to the basic principle of "accept what you can, generate
> carefully."
*scratching head* exactly why I would keep support in the server.
nd
--
package Hacker::Perl::Another::Just;print
q...@{[reverse split/::/ =>__PACKAGE__]}~;
# André Malo # http://www.perlig.de #
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On 05:02 pm, solip...@pitrou.net wrote: On Fri, 17 Dec 2010 00:52:14 +0800 Senthil Kumaran wrote: Actually, it is turning out to be true: http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response According to HTTP 1.0, When a request is Simple-Request, it means a VERB URL (without a version) and it generally corresponds to HTTP 0.9 And when a server receives such a Simple-Request, it sends a Simple-Response where it does not send the headers back. I think, the same is exhibited by other Servers as well www.google.com, www.mozilla.org where for Invalid Request without version, you are sending a Simple-Request (HTTP 0.9) style and getting the corresponding response. Yes, but only error or redirect responses: $ nc www.google.fr 80 GET / HTTP/1.0 302 Found Location: http://www.google.fr/ [etc.] Note that by using `nc` to test this, you're already generating an illegal request (unless you're doing something very special with your keyboard ;). When you hit return, nc will send \n. However, lines are delimited by \r\n in HTTP. I doubt this makes a difference to the point being discussed, but it _could_. I suggest performing your tests with telnet, instead. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
All this talk of modern servers that also still support HTTP/0.9 is irrelevant. Unless anybody knows of a server that *only* supports HTTP 0.9 (and that's relevant to users of httplib) let's please kill support in the client. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Fri, 17 Dec 2010 00:52:14 +0800 Senthil Kumaran wrote: > Actually, it is turning out to be true: > > http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response > > According to HTTP 1.0, When a request is Simple-Request, it means a > VERB URL (without a version) and it generally corresponds to HTTP 0.9 > And when a server receives such a Simple-Request, it sends a > Simple-Response where it does not send the headers back. > > I think, the same is exhibited by other Servers as well > www.google.com, www.mozilla.org where for Invalid Request without > version, you are sending a Simple-Request (HTTP 0.9) style and getting > the corresponding response. Yes, but only error or redirect responses: $ nc www.google.fr 80 GET / HTTP/1.0 302 Found Location: http://www.google.fr/ [etc.] $ nc www.mozilla.org 80 GET / 403 Forbidden Forbidden You don't have permission to access /error/noindex.html on this server. That's quite understandable, since most HTTP servers will expect a "host" header to know which site is actually desired. So a HTTP 0.9 client sending Simple-Requests has very little chance of being useful these days. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thu, Dec 16, 2010 at 04:52:43PM +0100, André Malo wrote: > HTTP/0.9 doesn't *have* a version string. > > GET /foo > > is a HTTP/0.9 request. > > GET /foo HTTP/0.9 > > isn't actually (it's a paradoxon, alright ;). It simply isn't a valid HTTP > request, which would demand a 505 response. Yes, this is an important point. Many webservers seem to exhibit the HTTP 0.9 response behaviors when you don't specify the version. -- Senthil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thu, Dec 16, 2010 at 11:21:37AM -0500, James Y Knight wrote: > > Even HTTP 0.9 says that response SHOULD start with status line, but > > gives a suggestion that clients can "tolerate" bad server server > > behaviors when they don't send the status line and in that the case > > response is the body. > > > > http://www.w3.org/Protocols/HTTP/Response.html > > > > So, It cannot be associated with the behavior "most webservers", back > > then and even more so now. > > Actually no. That document is describing almost-HTTP 1.0. Yeah. I know it was almost-HTTP 1.0, but the same docs say that if protocol version was not specified, it is assumed to be 0.9. So, I thought it was a good reference point to understand the behavior. > Here is the actual document you were looking for: > http://www.w3.org/Protocols/HTTP/AsImplemented.html > > HTTP 0.9 had no headers, no status line, nothing but "GET $url " and a > stream of data in response. Actually, you are right. I seems be be actual defined behavior of HTTP 0.9. As explained in that above doc and also in RFC 1945 Request section. -- Senthil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Dec 16, 2010, at 3:14 AM, Senthil Kumaran wrote: > Even HTTP 0.9 says that response SHOULD start with status line, but > gives a suggestion that clients can "tolerate" bad server server > behaviors when they don't send the status line and in that the case > response is the body. > > http://www.w3.org/Protocols/HTTP/Response.html > > So, It cannot be associated with the behavior "most webservers", back > then and even more so now. Actually no. That document is describing almost-HTTP 1.0. Here is the actual document you were looking for: http://www.w3.org/Protocols/HTTP/AsImplemented.html HTTP 0.9 had no headers, no status line, nothing but "GET $url " and a stream of data in response. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thu, Dec 16, 2010 at 02:20:37PM +0100, Antoine Pitrou wrote: > > > Try e.g. www.mozilla.org or www.google.com or www.msn.com. > > > (but www.python.org or www.apache.org still have the legacy behaviour) > > > > What legacy behavior did you observe in these? > > -> Request: > xyzzy > > -> Response: > > (notice how the response has no headers) > Well, Error response without headers was observed in www.mozilla.org and www.google.com for Invalid requests. But, I observed something surprising at www.apache.org If you do GET / HTTP/1.0 You do get the valid Response with headers. But if you do GET / You get a valid response Without headers. I was afraid if this behavior was to support HTTP 0.9 style where the the reponse is sent without the headers. Actually, it is turning out to be true: http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#Response According to HTTP 1.0, When a request is Simple-Request, it means a VERB URL (without a version) and it generally corresponds to HTTP 0.9 And when a server receives such a Simple-Request, it sends a Simple-Response where it does not send the headers back. I think, the same is exhibited by other Servers as well www.google.com, www.mozilla.org where for Invalid Request without version, you are sending a Simple-Request (HTTP 0.9) style and getting the corresponding response. Given these, any assumption that servers no longer support HTTP/0.9 becomes false. So nuking it will require some thought. -- Senthil -- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87310 - in python/branches/release27-maint: Doc/library/compileall.rst Lib/compileall.py
Hi, Thanks for double-checking. When I first looked into compileall, I opened http://bugs.python.org/issue10454 where I state that I find the description of those options unclear or even not understandable, so your diagnosis that I just copied text is right. A rewrite to fully cover the module functionality in clear English is needed. Your email was very helpful; can you turn it into a patch and post it to the bug? Thanks in advance. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thu, Dec 16, 2010 at 10:52 AM, André Malo wrote: > I'd vote for removing it from the client code and keeping it in the server. If it must be maintained anywhere, it should be in the client, according to the basic principle of "accept what you can, generate carefully." Python.org's HTTP/0.9 responses appear to be in response to HTTP/0.9 requests only. A request claiming to be HTTP 1.0, but without a Host: header, gets a redirect to the same page. I'm still in favor of removing HTTP 0.9 support entirely. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87310 - in python/branches/release27-maint: Doc/library/compileall.rst Lib/compileall.py
On Thu, 16 Dec 2010 07:15:02 +0100, eric.araujo wrote: > Modified: python/branches/release27-maint/Doc/library/compileall.rst > == > --- python/branches/release27-maint/Doc/library/compileall.rst > (original) > +++ python/branches/release27-maint/Doc/library/compileall.rstThu Dec > 16 07:15:02 2010 > @@ -1,4 +1,3 @@ > - > :mod:`compileall` --- Byte-compile Python libraries > === > > @@ -50,14 +49,14 @@ > > Expand list with its content (file and directory names). I realize you didn't write this line, but note that '-' is accepted as an argument and means "read the list from stdin". > -.. versionadded:: 2.7 > - The ``-i`` option. > +.. versionchanged:: 2.7 > + Added the ``-i`` option. > > > Public functions > > > -.. function:: compile_dir(dir[, maxlevels[, ddir[, force[, rx[, quiet]) > +.. function:: compile_dir(dir[, maxlevels[, ddir[, force[, rx[, quiet]) > > Recursively descend the directory tree named by *dir*, compiling all > :file:`.py` > files along the way. The *maxlevels* parameter is used to limit the > depth of > @@ -72,6 +71,23 @@ > If *quiet* is true, nothing is printed to the standard output in normal > operation. > > + > +.. function:: compile_file(fullname[, ddir[, force[, rx[, quiet) > + > + Compile the file with path *fullname*. If *ddir* is given, it is used as > the > + base path from which the filename used in error messages will be > generated. > + If *force* is true, modules are re-compiled even if the timestamp is up to > + date. Although this is copied from the other descriptions of *ddir*, it and the other instances (and the description of the -d option) should all really be fixed. As I discovered when writing the tests for the -d option, what ddir is is the path that is "baked in" to the .pyc file. In very old versions of Python that meant it was the path that would show up in tracebacks as the path to the source file. In modern Pythons the ddir path shows up if and only if the .py file does not exist and the .pyc file is being run directly. In 3.2, this means it will never show up normally, since you can't even run the .pyc file without moving it out of __pycache__. Which means 'ddir' is henceforth useful only to those people who want to package sourceless distributions of the python code. (If you want to see this in action check out the -d tests in test_compileall.) So, 'in error messages' really means 'in tracebacks if the .py file doesn't exist'. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thursday 16 December 2010 15:23:05 Antoine Pitrou wrote: > On Thu, 16 Dec 2010 07:42:08 +0100 > > André Malo wrote: > > * Antoine Pitrou wrote: > > > Hello, > > > > > > I would like to remove HTTP 0.9 support from http.client and > > > http.server. I've opened an issue at http://bugs.python.org/issue10711 > > > for that. Would anyone think it's a bad idea? > > > > > > (HTTP 1.0 was devised in 1996) > > > > HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still > > current). > > > > I'm wondering, why you would consider touching that at all. Is it broken? > > Does it stand in the way of anything? If not, why throw away a feature? > > Well, it complicates maintenance and makes fixing issues such as > http://bugs.python.org/issue6791 less likely. I'd vote for removing it from the client code and keeping it in the server. > Note that the patch still accepts servers and clients which advertise > themselves as 0.9 (using "HTTP/0.9" as a version string). HTTP/0.9 doesn't *have* a version string. GET /foo is a HTTP/0.9 request. GET /foo HTTP/0.9 isn't actually (it's a paradoxon, alright ;). It simply isn't a valid HTTP request, which would demand a 505 response. nd ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues
On 02:00 pm, solip...@pitrou.net wrote: On Thu, 16 Dec 2010 15:52:19 +0200 Dimitrios Pritsos wrote: Hello Michael, OK I will do sent it to the bug tracker. But what about the last issue i.e. that even if the class is transfered-and-pickled-unpickled it raises an exception if the class is defined into the __main__ script and not imported from an other one. (see below) could you please guide me where should I post this for someone to help me fix it! The likely explanation is that the __main__ module in the parent process isn't the same as in the child process, so fetching the class from that module doesn't work. If that's really important for you you could open a separate issue. And another option is to use the simple, pleasant fix of not defining any functions or classes in __main__. Define them in a real module and import them for use in __main__. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Thu, 16 Dec 2010 07:42:08 +0100 André Malo wrote: > * Antoine Pitrou wrote: > > > Hello, > > > > I would like to remove HTTP 0.9 support from http.client and > > http.server. I've opened an issue at http://bugs.python.org/issue10711 > > for that. Would anyone think it's a bad idea? > > > > (HTTP 1.0 was devised in 1996) > > HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still > current). > > I'm wondering, why you would consider touching that at all. Is it broken? > Does it stand in the way of anything? If not, why throw away a feature? Well, it complicates maintenance and makes fixing issues such as http://bugs.python.org/issue6791 less likely. Note that the patch still accepts servers and clients which advertise themselves as 0.9 (using "HTTP/0.9" as a version string). It just removes support for the style of "simple response" without headers that 0.9 allowed. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues
On Thu, 16 Dec 2010 15:52:19 +0200 Dimitrios Pritsos wrote: > > Hello Michael, > > OK I will do sent it to the bug tracker. But what about the last issue > i.e. that even if the class is transfered-and-pickled-unpickled it > raises an exception if the class is defined into the __main__ script and > not imported from an other one. (see below) could you please guide me > where should I post this for someone to help me fix it! The likely explanation is that the __main__ module in the parent process isn't the same as in the child process, so fetching the class from that module doesn't work. If that's really important for you you could open a separate issue. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues
On 12/16/2010 02:29 PM, Michael Foord wrote: On 16/12/2010 11:09, Dimitrios Pritsos wrote: Hello Core Developers, My name is Dimitrios and I am newbie in python. I am working on a Project (part of my PhD) that is called Synergeticprocessing module. Initially is imitating the multiprocessing built in module but the processes are distributed on a LAN and not Locally. The main issue I have is with Pickle module. And I think I found some kind of BUG in the built in multiprocessing module. Hello Dimitrios, Please post your bug report to the Python bug tracker. As you think you have a fix for the issue it is much more likely to be applied quickly if you can package it in the form of a test that demonstrates the issue and a patch that fixes it. http://bugs.python.org/ All the best, Michael Foord Hello Michael, OK I will do sent it to the bug tracker. But what about the last issue i.e. that even if the class is transfered-and-pickled-unpickled it raises an exception if the class is defined into the __main__ script and not imported from an other one. (see below) could you please guide me where should I post this for someone to help me fix it! Best Regards, Dimitrios (Synergeticprocessing module is located at GitHub: https://github.com/dpritsos/synergeticprocessing) Starting with the "BUG". In case someone uses the multiprocessing.Pool of processes he/she has to face the problem of types.MehtodType Impossible pickling. That is you cannot dispatch an class instance method to the to the Process Pool. However, digging in to the Source Code of the module there are few lines that resolve this issue however are not activated or they are faultily activated so they do not work. This is the 'BUG' _@ ../multiprocessing/forking.py_ . . . # # Try making some callable types picklable # from pickle import Pickler class ForkingPickler(Pickler): dispatch = Pickler.dispatch.copy() @classmethod def register(cls, type, reduce): def dispatcher(self, obj): rv = reduce(obj) self.save_reduce(obj=obj, *rv) cls.dispatch[type] = dispatcher def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) ForkingPickler.register(type(ForkingPickler.save), _reduce_method) def _reduce_method_descriptor(m): return getattr, (m.__objclass__, m.__name__) ForkingPickler.register(type(list.append), _reduce_method_descriptor) ForkingPickler.register(type(int.__add__), _reduce_method_descriptor) #def _reduce_builtin_function_or_method(m): #return getattr, (m.__self__, m.__name__) #ForkingPickler.register(type(list().append), _reduce_builtin_function_or_method) #ForkingPickler.register(type(int().__add__), _reduce_builtin_function_or_method) . . . The RED lines are not doing the job, for some reason they are not managing to register the GREEN function as a global reduce/pickling function even if you call the registration function into you __main__ script. The solution I found is just to do this * import copy_reg import types* def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) *copy_reg.pickle(types.MethodType, _reduce_method)* . . . Doing that everything works FINE. But ONLY for local methods i.e. the ones that their class is defined on the __main__ script or other import-ed. In case you want to send something remotely (in an other machine) or to an other __main__ script running separately then you get a message like this: 'module' object has no attribute '' The only way to resolve this is firstly to import a script that has defined there and everything works fine. SO the problems it seems to be that the *m.im_class* (look code above) has some attribute __module__ defined as __module__ = '__main__' or something like that. And this is the reason why remote script cannot execute the function. I mean that the _reduce_method() above DOES is pickling the whole CLASS object so there is no reason not to be executed at the remote script. Besides it does as mentioned above in you just import this the user defined class form an other script. I have already spent about 12 weeks working on building my synergeticPool and resolve the issue of Pickling and only 2 days needed for the code of the Pool and the rest of the time was spent for the Pickling issues, and study all the Class related mechanics of python. That was the reason I ve started digging the multipocessessing module and found this say 'BUG', and finally sent this email. Best Regards, Dimitrios ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:http://mail.python.org/mailman/options/python-dev/fuzzy
Re: [Python-Dev] Remove HTTP 0.9 support
Le jeudi 16 décembre 2010 à 16:14 +0800, Senthil Kumaran a écrit : > On Wed, Dec 15, 2010 at 11:29:27PM +0100, Antoine Pitrou wrote: > > Well, I think the "most web servers" comment itself is outdated. > > Try e.g. www.mozilla.org or www.google.com or www.msn.com. > > (but www.python.org or www.apache.org still have the legacy behaviour) > > What legacy behavior did you observe in these? -> Request: xyzzy -> Response: 405 Method Not Allowed Method Not Allowed The requested method xyzzy is not allowed for the URL /. Apache/2.3.8 (Unix) mod_ssl/2.3.8 OpenSSL/1.0.0a Server at www.apache.org Port 80 (notice how the response has no headers) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues
On 16/12/2010 11:09, Dimitrios Pritsos wrote: Hello Core Developers, My name is Dimitrios and I am newbie in python. I am working on a Project (part of my PhD) that is called Synergeticprocessing module. Initially is imitating the multiprocessing built in module but the processes are distributed on a LAN and not Locally. The main issue I have is with Pickle module. And I think I found some kind of BUG in the built in multiprocessing module. Hello Dimitrios, Please post your bug report to the Python bug tracker. As you think you have a fix for the issue it is much more likely to be applied quickly if you can package it in the form of a test that demonstrates the issue and a patch that fixes it. http://bugs.python.org/ All the best, Michael Foord (Synergeticprocessing module is located at GitHub: https://github.com/dpritsos/synergeticprocessing) Starting with the "BUG". In case someone uses the multiprocessing.Pool of processes he/she has to face the problem of types.MehtodType Impossible pickling. That is you cannot dispatch an class instance method to the to the Process Pool. However, digging in to the Source Code of the module there are few lines that resolve this issue however are not activated or they are faultily activated so they do not work. This is the 'BUG' _@ ../multiprocessing/forking.py_ . . . # # Try making some callable types picklable # from pickle import Pickler class ForkingPickler(Pickler): dispatch = Pickler.dispatch.copy() @classmethod def register(cls, type, reduce): def dispatcher(self, obj): rv = reduce(obj) self.save_reduce(obj=obj, *rv) cls.dispatch[type] = dispatcher def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) ForkingPickler.register(type(ForkingPickler.save), _reduce_method) def _reduce_method_descriptor(m): return getattr, (m.__objclass__, m.__name__) ForkingPickler.register(type(list.append), _reduce_method_descriptor) ForkingPickler.register(type(int.__add__), _reduce_method_descriptor) #def _reduce_builtin_function_or_method(m): #return getattr, (m.__self__, m.__name__) #ForkingPickler.register(type(list().append), _reduce_builtin_function_or_method) #ForkingPickler.register(type(int().__add__), _reduce_builtin_function_or_method) . . . The RED lines are not doing the job, for some reason they are not managing to register the GREEN function as a global reduce/pickling function even if you call the registration function into you __main__ script. The solution I found is just to do this * import copy_reg import types* def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) *copy_reg.pickle(types.MethodType, _reduce_method)* . . . Doing that everything works FINE. But ONLY for local methods i.e. the ones that their class is defined on the __main__ script or other import-ed. In case you want to send something remotely (in an other machine) or to an other __main__ script running separately then you get a message like this: 'module' object has no attribute '' The only way to resolve this is firstly to import a script that has defined there and everything works fine. SO the problems it seems to be that the *m.im_class* (look code above) has some attribute __module__ defined as __module__ = '__main__' or something like that. And this is the reason why remote script cannot execute the function. I mean that the _reduce_method() above DOES is pickling the whole CLASS object so there is no reason not to be executed at the remote script. Besides it does as mentioned above in you just import this the user defined class form an other script. I have already spent about 12 weeks working on building my synergeticPool and resolve the issue of Pickling and only 2 days needed for the code of the Pool and the rest of the time was spent for the Pickling issues, and study all the Class related mechanics of python. That was the reason I ve started digging the multipocessessing module and found this say 'BUG', and finally sent this email. Best Regards, Dimitrios ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.pytho
[Python-Dev] Multiprocessing module - Synergeticprocessing (my custom module) - Pickling unPickling issues
Hello Core Developers, My name is Dimitrios and I am newbie in python. I am working on a Project (part of my PhD) that is called Synergeticprocessing module. Initially is imitating the multiprocessing built in module but the processes are distributed on a LAN and not Locally. The main issue I have is with Pickle module. And I think I found some kind of BUG in the built in multiprocessing module. (Synergeticprocessing module is located at GitHub: https://github.com/dpritsos/synergeticprocessing) Starting with the "BUG". In case someone uses the multiprocessing.Pool of processes he/she has to face the problem of types.MehtodType Impossible pickling. That is you cannot dispatch an class instance method to the to the Process Pool. However, digging in to the Source Code of the module there are few lines that resolve this issue however are not activated or they are faultily activated so they do not work. This is the 'BUG' _@ ../multiprocessing/forking.py_ . . . # # Try making some callable types picklable # from pickle import Pickler class ForkingPickler(Pickler): dispatch = Pickler.dispatch.copy() @classmethod def register(cls, type, reduce): def dispatcher(self, obj): rv = reduce(obj) self.save_reduce(obj=obj, *rv) cls.dispatch[type] = dispatcher def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) ForkingPickler.register(type(ForkingPickler.save), _reduce_method) def _reduce_method_descriptor(m): return getattr, (m.__objclass__, m.__name__) ForkingPickler.register(type(list.append), _reduce_method_descriptor) ForkingPickler.register(type(int.__add__), _reduce_method_descriptor) #def _reduce_builtin_function_or_method(m): #return getattr, (m.__self__, m.__name__) #ForkingPickler.register(type(list().append), _reduce_builtin_function_or_method) #ForkingPickler.register(type(int().__add__), _reduce_builtin_function_or_method) . . . The RED lines are not doing the job, for some reason they are not managing to register the GREEN function as a global reduce/pickling function even if you call the registration function into you __main__ script. The solution I found is just to do this * import copy_reg import types* def _reduce_method(m): if m.im_self is None: return getattr, (m.im_class, m.im_func.func_name) else: return getattr, (m.im_self, m.im_func.func_name) *copy_reg.pickle(types.MethodType, _reduce_method)* . . . Doing that everything works FINE. But ONLY for local methods i.e. the ones that their class is defined on the __main__ script or other import-ed. In case you want to send something remotely (in an other machine) or to an other __main__ script running separately then you get a message like this: 'module' object has no attribute '' The only way to resolve this is firstly to import a script that has defined there and everything works fine. SO the problems it seems to be that the *m.im_class* (look code above) has some attribute __module__ defined as __module__ = '__main__' or something like that. And this is the reason why remote script cannot execute the function. I mean that the _reduce_method() above DOES is pickling the whole CLASS object so there is no reason not to be executed at the remote script. Besides it does as mentioned above in you just import this the user defined class form an other script. I have already spent about 12 weeks working on building my synergeticPool and resolve the issue of Pickling and only 2 days needed for the code of the Pool and the rest of the time was spent for the Pickling issues, and study all the Class related mechanics of python. That was the reason I ve started digging the multipocessessing module and found this say 'BUG', and finally sent this email. Best Regards, Dimitrios ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
* Antoine Pitrou wrote: > Hello, > > I would like to remove HTTP 0.9 support from http.client and > http.server. I've opened an issue at http://bugs.python.org/issue10711 > for that. Would anyone think it's a bad idea? > > (HTTP 1.0 was devised in 1996) HTTP/0.9 support is still recommended (RFC 2616 is from 1999, but still current). I'm wondering, why you would consider touching that at all. Is it broken? Does it stand in the way of anything? If not, why throw away a feature? nd -- Already I've seen people (really!) write web URLs in the form: http:\\some.site.somewhere [...] How soon until greengrocers start writing "apples $1\pound" or something? -- Joona I Palaste in clc ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Wed, Dec 15, 2010 at 11:29:27PM +0100, Antoine Pitrou wrote: > Well, I think the "most web servers" comment itself is outdated. > Try e.g. www.mozilla.org or www.google.com or www.msn.com. > (but www.python.org or www.apache.org still have the legacy behaviour) What legacy behavior did you observe in these? -- Senthil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Remove HTTP 0.9 support
On Wed, Dec 15, 2010 at 02:20:54PM -0800, Glenn Linderman wrote: > On 12/15/2010 10:39 AM, Antoine Pitrou wrote: > I would like to remove HTTP 0.9 support from http.client and > http.server. I've opened an issue at > http://bugs.python.org/issue10711 > for that. Would anyone think it's a bad idea? > > (HTTP 1.0 was devised in 1996) > > Please address the following comment from the server.py source: > > # The default request version. This only affects responses up > until > # the point where the request line is parsed, so it mainly > decides what > # the client gets back when sending a malformed request line. > # Most web servers default to HTTP 0.9, i.e. don't send a status > line. > default_request_version = "HTTP/0.9" > > What do you mean by "address"? The patch changes this to 1.0. > And, as the comment says, this only affects what happens when the > client sends a syntactically invalid request line, so whether the server > does a 0.9-style or 1.0-style response is unimportant. > > > Just what you did... justify the unimportance of not changing it :) Since now > it is different than "most web servers". +1 to removing HTTP 0.9 related code in http.client and http.server. Until today, I hadn't cared to read any details of HTTP 0.9 except that I knew that some code was present in http library to support it. Reading a bit about the HTTP 0.9 'Internet Draft' written 1991 is enough to convince that, if we have to fall back on any old behavior falling back to HTTP 1.0 is perfectly fine. Regarding this comment # Most web servers default to HTTP 0.9, i.e. don't send a status line. Even HTTP 0.9 says that response SHOULD start with status line, but gives a suggestion that clients can "tolerate" bad server server behaviors when they don't send the status line and in that the case response is the body. http://www.w3.org/Protocols/HTTP/Response.html So, It cannot be associated with the behavior "most webservers", back then and even more so now. -- Senthil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
