Interesting. I have seen the following errors (randomly) testing a tornado
app. Could this be related?
tornado.general: DEBUG: Error deleting fd from IOLoop
Traceback (most recent call last):
File "tornado/ioloop.py", line 482, in remove_handler
self._impl.unregister(fd)
IOError: [Errno 2] No such file or directory
tornado.application: ERROR: Exception in I/O handler for fd 14
Traceback (most recent call last):
File "tornado/ioloop.py", line 628, in start
self._handlers[fd](fd, events)
KeyError: 14
Victor, any plans for Windows support? I guess porting "_winapi" would be
the biggest issue. Compiling "overlapped.c" is possible with small changes
(I could send the patch if you are interested).
On Tuesday, January 7, 2014 6:56:17 AM UTC+1, Ben Darnell wrote:
>
> With Trollius 0.1.1, all the core Tornado tests are passing. It's now
> possible to run some of the integration tests that couldn't be run with
> asyncio proper because they're py2-only. The twisted tests (i.e. twisted
> on top of tornado on top of asyncio) work fine now. With pycurl, I'm
> seeing some puzzling issues. It's actually looking like a bug in Tornado,
> but I don't know why this hasn't been seen before.
>
> The issue is that in between tests, tornado destroys the IOLoop and closes
> any remaining file descriptors with os.close. If the socket objects for
> those FDs still exist and are later garbage collected, they will try to
> close the FD a second time. If the FD has been reassigned, this will stomp
> on an unrelated file handle. The asyncio-backed IOLoop seems to be somehow
> delaying the GC of these socket objects (making them a part of a cycle so
> they are not promptly dereferenced at the end of the test).
>
> I'm not sure how this is happening; remember that asyncio never even
> touches the socket objects directly (Tornado only uses the fd-based
> add_reader/add_writer interface). Tornado should ultimately be more
> careful about closing its socket objects through sock.close() instead of
> clobbering them with os.close(fd), but that doesn't explain why we have
> only seen this with the asyncio-based IOLoop.
>
>
> On Fri, Jan 3, 2014 at 11:13 PM, Ben Darnell <[email protected]<javascript:>
> > wrote:
>
>> On Fri, Jan 3, 2014 at 9:12 PM, Victor Stinner
>> <[email protected]<javascript:>
>> > wrote:
>>
>>> 2013/12/31 Ben Darnell <[email protected] <javascript:>>:
>>> > I just tried running the Tornado test suite with this backported
>>> package and
>>> > it mostly works.
>>>
>>> Wow, great!
>>>
>>> > There are two issues:
>>> >
>>> > * Tornado uses hasattr(ssl, 'SSLContext') to determine whether or not
>>> it can
>>> > use the APIs introduced in Python 3.2; the incomplete version of
>>> SSLContext
>>> > patched in by this package confuses it. I had to replace the hasattr
>>> calls
>>> > with explicit sys.version_info checks.
>>>
>>> Which issue do you get?
>>>
>>
>> An AttributeError for SSLContext.load_verify_locations (we also use
>> verify_mode, load_cert_chain and set_ciphers, but load_verify_locations is
>> the first to fail). If ssl.SSLContext exists, Tornado will use it;
>> otherwise it uses the corresponding keyword arguments for ssl.wrap_socket
>> (when corresponding options exist).
>> https://github.com/facebook/tornado/blob/v3.2.0b1/tornado/netutil.py#L358
>>
>>
>>
>>>
>>> It's not easy to drop the "backported" SSLContext class, because
>>> asyncio API is based on it, and I'm trying to keep Trollius API close
>>> as possible to Tulip API.
>>>
>>
>> Can you just change all the internal references to use asyncio.SSLContext
>> instead of ssl.SSLContext?
>>
>>
>>>
>>> > * One corner case is handled differently than in other implementations
>>> > (TestIOStreamSSL.test_inline_read_error). I haven't traced it all the
>>> way
>>> > through (this test is deliberately doing weird things to
>>> deterministically
>>> > trigger an error at a particular spot). The failure may be
>>> kqueue-specific
>>> > (it has to do with the fact that if you close an fd without the event
>>> loop's
>>> > knowledge with os.close(fd), subsequently unregistering that fd will
>>> fail).
>>>
>>> The Trollius project now tracks updates from Tulip and so should work
>>> correctly. Can you try Trollius 0.1 with Tornado?
>>>
>>
>> This issue is fixed now; with the hasattr(ssl, 'SSLContext') checks
>> changed all of Tornado's tests are passing. I did, however, have to patch
>> trollius's time_monotonic.py to get it working on a mac (it needs to import
>> ctypes.util, not just ctypes, and there are unqualified references to
>> ctypes.POINTER and ctypes.byref below).
>>
>> -Ben
>>
>>
>>> Victor
>>>
>>
>>
>
