With Trollius 0.1.1, all the core Tornado tests are passing. It's now possible to run some of the integration tests that couldn't be run with asyncio proper because they're py2-only. The twisted tests (i.e. twisted on top of tornado on top of asyncio) work fine now. With pycurl, I'm seeing some puzzling issues. It's actually looking like a bug in Tornado, but I don't know why this hasn't been seen before.
The issue is that in between tests, tornado destroys the IOLoop and closes any remaining file descriptors with os.close. If the socket objects for those FDs still exist and are later garbage collected, they will try to close the FD a second time. If the FD has been reassigned, this will stomp on an unrelated file handle. The asyncio-backed IOLoop seems to be somehow delaying the GC of these socket objects (making them a part of a cycle so they are not promptly dereferenced at the end of the test). I'm not sure how this is happening; remember that asyncio never even touches the socket objects directly (Tornado only uses the fd-based add_reader/add_writer interface). Tornado should ultimately be more careful about closing its socket objects through sock.close() instead of clobbering them with os.close(fd), but that doesn't explain why we have only seen this with the asyncio-based IOLoop. On Fri, Jan 3, 2014 at 11:13 PM, Ben Darnell <[email protected]> wrote: > On Fri, Jan 3, 2014 at 9:12 PM, Victor Stinner > <[email protected]>wrote: > >> 2013/12/31 Ben Darnell <[email protected]>: >> > I just tried running the Tornado test suite with this backported >> package and >> > it mostly works. >> >> Wow, great! >> >> > There are two issues: >> > >> > * Tornado uses hasattr(ssl, 'SSLContext') to determine whether or not >> it can >> > use the APIs introduced in Python 3.2; the incomplete version of >> SSLContext >> > patched in by this package confuses it. I had to replace the hasattr >> calls >> > with explicit sys.version_info checks. >> >> Which issue do you get? >> > > An AttributeError for SSLContext.load_verify_locations (we also use > verify_mode, load_cert_chain and set_ciphers, but load_verify_locations is > the first to fail). If ssl.SSLContext exists, Tornado will use it; > otherwise it uses the corresponding keyword arguments for ssl.wrap_socket > (when corresponding options exist). > https://github.com/facebook/tornado/blob/v3.2.0b1/tornado/netutil.py#L358 > > > >> >> It's not easy to drop the "backported" SSLContext class, because >> asyncio API is based on it, and I'm trying to keep Trollius API close >> as possible to Tulip API. >> > > Can you just change all the internal references to use asyncio.SSLContext > instead of ssl.SSLContext? > > >> >> > * One corner case is handled differently than in other implementations >> > (TestIOStreamSSL.test_inline_read_error). I haven't traced it all the >> way >> > through (this test is deliberately doing weird things to >> deterministically >> > trigger an error at a particular spot). The failure may be >> kqueue-specific >> > (it has to do with the fact that if you close an fd without the event >> loop's >> > knowledge with os.close(fd), subsequently unregistering that fd will >> fail). >> >> The Trollius project now tracks updates from Tulip and so should work >> correctly. Can you try Trollius 0.1 with Tornado? >> > > This issue is fixed now; with the hasattr(ssl, 'SSLContext') checks > changed all of Tornado's tests are passing. I did, however, have to patch > trollius's time_monotonic.py to get it working on a mac (it needs to import > ctypes.util, not just ctypes, and there are unqualified references to > ctypes.POINTER and ctypes.byref below). > > -Ben > > >> Victor >> > >
