On Tue, Jan 7, 2014 at 6:01 AM, Marc Schlaich <[email protected]>wrote:
> Interesting. I have seen the following errors (randomly) testing a tornado > app. Could this be related? > > tornado.general: DEBUG: Error deleting fd from IOLoop > Traceback (most recent call last): > File "tornado/ioloop.py", line 482, in remove_handler > self._impl.unregister(fd) > IOError: [Errno 2] No such file or directory > These logs are common if you're using curl_httpclient; that's why they're DEBUG, so they don't normally show up (and they should perhaps be removed entirely). If you're not using curl_httpclient they're less common but still pretty benign. > > tornado.application: ERROR: Exception in I/O handler for fd 14 > Traceback (most recent call last): > File "tornado/ioloop.py", line 628, in start > self._handlers[fd](fd, events) > KeyError: 14 > We used to get reports of errors like this, but it's been a long time. Neither of these are obviously related to the GC issue here (the most common error is a "bad file descriptor" error on write()), but it's hard to say for sure. -Ben > > Victor, any plans for Windows support? I guess porting "_winapi" would be > the biggest issue. Compiling "overlapped.c" is possible with small changes > (I could send the patch if you are interested). > > On Tuesday, January 7, 2014 6:56:17 AM UTC+1, Ben Darnell wrote: > >> With Trollius 0.1.1, all the core Tornado tests are passing. It's now >> possible to run some of the integration tests that couldn't be run with >> asyncio proper because they're py2-only. The twisted tests (i.e. twisted >> on top of tornado on top of asyncio) work fine now. With pycurl, I'm >> seeing some puzzling issues. It's actually looking like a bug in Tornado, >> but I don't know why this hasn't been seen before. >> >> The issue is that in between tests, tornado destroys the IOLoop and >> closes any remaining file descriptors with os.close. If the socket objects >> for those FDs still exist and are later garbage collected, they will try to >> close the FD a second time. If the FD has been reassigned, this will stomp >> on an unrelated file handle. The asyncio-backed IOLoop seems to be somehow >> delaying the GC of these socket objects (making them a part of a cycle so >> they are not promptly dereferenced at the end of the test). >> >> I'm not sure how this is happening; remember that asyncio never even >> touches the socket objects directly (Tornado only uses the fd-based >> add_reader/add_writer interface). Tornado should ultimately be more >> careful about closing its socket objects through sock.close() instead of >> clobbering them with os.close(fd), but that doesn't explain why we have >> only seen this with the asyncio-based IOLoop. >> >> >> On Fri, Jan 3, 2014 at 11:13 PM, Ben Darnell <[email protected]> wrote: >> >>> On Fri, Jan 3, 2014 at 9:12 PM, Victor Stinner <[email protected]>wrote: >>> >>>> 2013/12/31 Ben Darnell <[email protected]>: >>>> >>>> > I just tried running the Tornado test suite with this backported >>>> package and >>>> > it mostly works. >>>> >>>> Wow, great! >>>> >>>> > There are two issues: >>>> > >>>> > * Tornado uses hasattr(ssl, 'SSLContext') to determine whether or not >>>> it can >>>> > use the APIs introduced in Python 3.2; the incomplete version of >>>> SSLContext >>>> > patched in by this package confuses it. I had to replace the hasattr >>>> calls >>>> > with explicit sys.version_info checks. >>>> >>>> Which issue do you get? >>>> >>> >>> An AttributeError for SSLContext.load_verify_locations (we also use >>> verify_mode, load_cert_chain and set_ciphers, but load_verify_locations is >>> the first to fail). If ssl.SSLContext exists, Tornado will use it; >>> otherwise it uses the corresponding keyword arguments for ssl.wrap_socket >>> (when corresponding options exist). >>> https://github.com/facebook/tornado/blob/v3.2.0b1/tornado/ >>> netutil.py#L358 >>> >>> >>>> >>>> It's not easy to drop the "backported" SSLContext class, because >>>> asyncio API is based on it, and I'm trying to keep Trollius API close >>>> as possible to Tulip API. >>>> >>> >>> Can you just change all the internal references to use >>> asyncio.SSLContext instead of ssl.SSLContext? >>> >>> >>>> >>>> > * One corner case is handled differently than in other implementations >>>> > (TestIOStreamSSL.test_inline_read_error). I haven't traced it all >>>> the way >>>> > through (this test is deliberately doing weird things to >>>> deterministically >>>> > trigger an error at a particular spot). The failure may be >>>> kqueue-specific >>>> > (it has to do with the fact that if you close an fd without the event >>>> loop's >>>> > knowledge with os.close(fd), subsequently unregistering that fd will >>>> fail). >>>> >>>> The Trollius project now tracks updates from Tulip and so should work >>>> correctly. Can you try Trollius 0.1 with Tornado? >>>> >>> >>> This issue is fixed now; with the hasattr(ssl, 'SSLContext') checks >>> changed all of Tornado's tests are passing. I did, however, have to patch >>> trollius's time_monotonic.py to get it working on a mac (it needs to import >>> ctypes.util, not just ctypes, and there are unqualified references to >>> ctypes.POINTER and ctypes.byref below). >>> >>> -Ben >>> >>> >>>> Victor >>>> >>> >>> >>
