masterhugo opened a new pull request, #3725:
URL: https://github.com/apache/tinkerpop/pull/3725
## Summary (Part 1 of 4: Native `asyncio` for `gremlin-python` 4.0)
This is **PR 1 of 4** introducing native `asyncio` support to
`gremlin-python` for TinkerPop 4.0 over HTTP streaming, **without**
`nest_asyncio`, background `ThreadPoolExecutor` pools, or per-connection
private event loops (`asyncio.new_event_loop()`).
### Why native `asyncio` without `nest_asyncio`?
Currently, `AiohttpHTTPTransport` creates a private event loop per
connection and drives `aiohttp` synchronously via `run_until_complete()`. When
called from inside an already-running event loop (FastAPI/`uvloop`, Jupyter
notebooks, async AWS Lambda handlers), it fails with `RuntimeError: Cannot run
the event loop while another loop is running`, and `nest_asyncio` cannot patch
Cython-based loops like `uvloop`.
### What is included in PR 1 (Foundation layer)
-
[`gremlin_python/driver/aiohttp/async_transport.py`](gremlin-python/src/main/python/gremlin_python/driver/aiohttp/async_transport.py):
- `AsyncBufferedStream`: sliding-window `bytearray` buffer over
`aiohttp.ClientResponse.content` with `checkpoint()`, `restore()`, `compact()`,
`fill_more()`, and `read_exactly()`. Translates `aiohttp.ServerTimeoutError`
into `ReadTimeoutError`.
- `AsyncAiohttpHTTPTransport`: native `aiohttp.ClientSession` transport
bound directly to `asyncio.get_running_loop()`.
-
[`gremlin_python/driver/async_resultset.py`](gremlin-python/src/main/python/gremlin_python/driver/async_resultset.py):
- `AsyncResultSet`: single-item streaming result set supporting `await
rs.one()` (returning `_EXHAUSTED` on stream end so `None` result items are
preserved), `await rs.all()`, and `async for item in rs`, using
`asyncio.wait(..., return_when=FIRST_COMPLETED)` instead of polling loops.
-
[`gremlin_python/driver/serializer.py`](gremlin-python/src/main/python/gremlin_python/driver/serializer.py):
- Added
`GraphBinarySerializersV4.deserialize_async_response_stream(stream)` to
incrementally decode GraphBinary v4 (`0x84`) objects and end-of-stream trailers
across arbitrary HTTP chunk boundaries.
-
[`tests/unit/driver/test_async_http_streaming.py`](gremlin-python/src/main/python/tests/unit/driver/test_async_http_streaming.py):
- 27 unit tests covering `AsyncResultSet`, `AsyncBufferedStream`,
`deserialize_async_response_stream` (including 2-byte chunk fragmentation
across object/trailer boundaries), and `AsyncAiohttpHTTPTransport`.
### Roadmap of the 4-PR Series
- **PR 1 (this PR)**: `AsyncBufferedStream`, `AsyncAiohttpHTTPTransport`,
`AsyncResultSet`, `deserialize_async_response_stream`
- **PR 2**: `AsyncConnection` & `AsyncClient` (`asyncio.Queue` connection
pool)
- **PR 3**: `AsyncDriverRemoteConnection` & `AsyncGraphTraversal`
(`async_traversal()`)
- **PR 4**: `AsyncTransaction` (`g.tx()`, `execute_in_tx`), integration
tests, examples, and documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
