Re: [IronPython] socket, _socket, and socket.py
That's good enough for me. I'm going to give it a go. On 5/17/2011 6:48 PM, Dino Viehland wrote: I think manual ref count tracking (like _make_filerefs) + defining __del__ on the socket objects would fix it. But there may be issues w/ that - when we first encountered this I pinged python-dev and I recall Guido saying that there were problems getting anything else working on Windows. So there may be some oddity that prevents this from working but I don't know what that is off the top of my head. -Original Message- From: users-boun...@lists.ironpython.com [mailto:users- boun...@lists.ironpython.com] On Behalf Of Zachary Gramana Sent: Tuesday, May 17, 2011 3:38 PM To: Discussion of IronPython Subject: Re: [IronPython] socket, _socket, and socket.py I noticed this, with a little disappointment. I also noticed that they implemented a sort of ersatz/platform agnostic reference tracking system in ssl.py. Seeing the _makefile_refs system got me thinking about the whole ssl.py/_ssl.pyd paradigm and sockets.cs, since I remembered you tackled this issue once before (or one very similar to it: http://lists.ironpython.com/pipermail/users-ironpython.com/2007- May/004946.html). Perhaps something along the lines of _makefile_refs and GC.SuppressFinalize might get us closer? On 5/17/2011 6:06 PM, Dino Viehland wrote: There is actually more of a story to this though - Cpython's socket.py relies upon reference counting for handling dup. See this comment from socket.py: # Wrapper around platform socket objects. This implements # a platform-independent dup() functionality. The # implementation currently relies on reference counting # to close the underlying socket object. class _socketobject(object): Because IronPython doesn't have a reference counting GC we cannot simply use socket.py. So someone would need to patch both IronPython and socket.py so this would work. For the record it looks like Jython also has a custom socket.py. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] socket, _socket, and socket.py
Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern. On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana zgram...@pottsconsultinggroup.com wrote: Hi: I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL. I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7). The socket module, however, does not. I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else? The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations. Thanks! Regards, Zack P.S. Any advice on tackling issue #26852 is very warmly appreciated. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] socket, _socket, and socket.py
There is actually more of a story to this though - Cpython's socket.py relies upon reference counting for handling dup. See this comment from socket.py: # Wrapper around platform socket objects. This implements # a platform-independent dup() functionality. The # implementation currently relies on reference counting # to close the underlying socket object. class _socketobject(object): Because IronPython doesn't have a reference counting GC we cannot simply use socket.py. So someone would need to patch both IronPython and socket.py so this would work. For the record it looks like Jython also has a custom socket.py. From: users-boun...@lists.ironpython.com [mailto:users-boun...@lists.ironpython.com] On Behalf Of Curt Hagenlocher Sent: Tuesday, May 17, 2011 2:59 PM To: Discussion of IronPython Subject: Re: [IronPython] socket, _socket, and socket.py Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern. On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana zgram...@pottsconsultinggroup.commailto:zgram...@pottsconsultinggroup.com wrote: Hi: I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL. I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7). The socket module, however, does not. I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else? The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations. Thanks! Regards, Zack P.S. Any advice on tackling issue #26852 is very warmly appreciated. ___ Users mailing list Users@lists.ironpython.commailto:Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] socket, _socket, and socket.py
Thanks Curt. That's encouraging news to hear. I've noticed that some Python projects have a dependency on being able to standard library methods and types. For example, Mercurial replaces buffer() with their own fakebuffer(), but ipy socket reads still return the original buffer. I ended up patching around their monkey patch, which, understandably, they weren't too keen on importing back into their repo. An eventual move to _socket + socket.py wrapper could help improve compatibility with those projects. These type of hidden dependencies on vm-specific behaviors seem to the biggest source of issues I've come across so far (in 2.7, anyway). I'm continually amazed at how big of a step forward the team made from 2.6 - 2.7 with respect to compatibility. On 5/17/2011 5:58 PM, Curt Hagenlocher wrote: Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern. On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana zgram...@pottsconsultinggroup.com mailto:zgram...@pottsconsultinggroup.com wrote: Hi: I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL. I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7). The socket module, however, does not. I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else? The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations. Thanks! Regards, Zack P.S. Any advice on tackling issue #26852 is very warmly appreciated. ___ Users mailing list Users@lists.ironpython.com mailto:Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] socket, _socket, and socket.py
I noticed this, with a little disappointment. I also noticed that they implemented a sort of ersatz/platform agnostic reference tracking system in ssl.py. Seeing the _makefile_refs system got me thinking about the whole ssl.py/_ssl.pyd paradigm and sockets.cs, since I remembered you tackled this issue once before (or one very similar to it: http://lists.ironpython.com/pipermail/users-ironpython.com/2007-May/004946.html). Perhaps something along the lines of _makefile_refs and GC.SuppressFinalize might get us closer? On 5/17/2011 6:06 PM, Dino Viehland wrote: There is actually more of a story to this though – Cpython’s socket.py relies upon reference counting for handling dup. See this comment from socket.py: # Wrapper around platform socket objects. This implements # a platform-independent dup() functionality. The # implementation currently relies on reference counting # to close the underlying socket object. class _socketobject(object): Because IronPython doesn’t have a reference counting GC we cannot simply use socket.py. So someone would need to patch both IronPython and socket.py so this would work. For the record it looks like Jython also has a custom socket.py. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] socket, _socket, and socket.py
I think manual ref count tracking (like _make_filerefs) + defining __del__ on the socket objects would fix it. But there may be issues w/ that - when we first encountered this I pinged python-dev and I recall Guido saying that there were problems getting anything else working on Windows. So there may be some oddity that prevents this from working but I don't know what that is off the top of my head. -Original Message- From: users-boun...@lists.ironpython.com [mailto:users- boun...@lists.ironpython.com] On Behalf Of Zachary Gramana Sent: Tuesday, May 17, 2011 3:38 PM To: Discussion of IronPython Subject: Re: [IronPython] socket, _socket, and socket.py I noticed this, with a little disappointment. I also noticed that they implemented a sort of ersatz/platform agnostic reference tracking system in ssl.py. Seeing the _makefile_refs system got me thinking about the whole ssl.py/_ssl.pyd paradigm and sockets.cs, since I remembered you tackled this issue once before (or one very similar to it: http://lists.ironpython.com/pipermail/users-ironpython.com/2007- May/004946.html). Perhaps something along the lines of _makefile_refs and GC.SuppressFinalize might get us closer? On 5/17/2011 6:06 PM, Dino Viehland wrote: There is actually more of a story to this though - Cpython's socket.py relies upon reference counting for handling dup. See this comment from socket.py: # Wrapper around platform socket objects. This implements # a platform-independent dup() functionality. The # implementation currently relies on reference counting # to close the underlying socket object. class _socketobject(object): Because IronPython doesn't have a reference counting GC we cannot simply use socket.py. So someone would need to patch both IronPython and socket.py so this would work. For the record it looks like Jython also has a custom socket.py. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] socket, _socket, and socket.py
Ah, right, I'd forgotten that -- we would have had to have distributed a modified socket.py, and that was infeasible even after we were delivering the standard library. This feels like a place where IronPython could collaborate on a joint change with Jython and have the result distributed inside the standard library using a test for sys.platform. On Tue, May 17, 2011 at 3:06 PM, Dino Viehland di...@microsoft.com wrote: There is actually more of a story to this though – Cpython’s socket.py relies upon reference counting for handling dup. See this comment from socket.py: # Wrapper around platform socket objects. This implements # a platform-independent dup() functionality. The # implementation currently relies on reference counting # to close the underlying socket object. class _socketobject(object): Because IronPython doesn’t have a reference counting GC we cannot simply use socket.py. So someone would need to patch both IronPython and socket.py so this would work. For the record it looks like Jython also has a custom socket.py. *From:* users-boun...@lists.ironpython.com [mailto: users-boun...@lists.ironpython.com] *On Behalf Of *Curt Hagenlocher *Sent:* Tuesday, May 17, 2011 2:59 PM *To:* Discussion of IronPython *Subject:* Re: [IronPython] socket, _socket, and socket.py Originally, we weren't allowed to redistribute the Python standard library with IronPython. So it made sense to implement the socket module directly. When IronPython started shipping the standard lib, it could have been changed, but never was. I think it makes sense to follow the CPython pattern. On Tue, May 17, 2011 at 2:33 PM, Zachary Gramana zgram...@pottsconsultinggroup.com wrote: Hi: I've been working on adapting Mercurial to run on IronPython 2.7, and ran into http://ironpython.codeplex.com/workitem/26852 which has stopped me from getting `hg clone` working over SSL. I noticed that for the ssl module, the IPY team mirrored the CPython pattern of placing the platform-specific code in the _ssl compiled module, and then wrapped the platform-independent around it in ssl.py (almost entirely shared with CPython 2.7). The socket module, however, does not. I admit I have a limited understanding of the code, but at first blush, it appears that adopting the _socket.cs/socket.py isn't out-of-the-question. Is there a story behind this, or am I missing something obvious to everyone else? The immediate benefit would be getting a free implementation of _socketobject, _dummy, and dup(); it also improves DRY conformance, and would help to limit behavioral differences with respect to other implementations. Thanks! Regards, Zack P.S. Any advice on tackling issue #26852 is very warmly appreciated. ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com ___ Users mailing list Users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
