contrueCT opened a new issue, #3166:
URL: https://github.com/apache/hugegraph/issues/3166
### Bug Type (问题类型)
logic / resource lifecycle
### Before submit
- [x] I searched the existing issues, pull requests, and FAQ and did not
find a report for this exact `ShortestPathTraverser` iterator leak.
### Environment (环境信息)
- Server source: current `apache/hugegraph` `master`
(`431f6e6b431014feb5b50d27ea67180b4610f10b`)
- Backend: HStore, one PD + one Store + one Server on one machine
- OS: Ubuntu 24.04, 16 vCPU, about 78 GiB RAM, swap disabled during the
reproduction
- Data size: 41,652,230 vertices and 1,468,365,182 directed edges (SNAP
Twitter-2010)
- Workload: REST `shortestpath`, concurrency 12, 180-second warmup +
600-second measurement
### Expected & Actual behavior (期望与实际表现)
Every edge iterator returned by `edgesOfVertex()` should be closed on normal
completion, early return, and exception paths. For HStore, closing the iterator
is also the signal that allows the remote scanner stream to be cancelled and
cleaned up.
`ShortestPathTraverser.Traverser` does not currently guarantee that
ownership contract:
1. `forward()` returns immediately when it finds the requested path without
closing the current edge iterator.
2. `backward()` has the equivalent early-return path.
3. `superNode()` passes an edge iterator to `IteratorUtils.count()` without
closing it explicitly.
In the HStore reproduction, successful requests left Store threads waiting
in the scanner delivery path (`HgChannel.send`). The blocked frames accumulated
from 11 shortly after launch to 38, 75, 108, and finally 113. The measured
phase completed 5,817 of 5,818 requests; one request returned HTTP 500 after
approximately 61 seconds. Swap remained zero, so the failure was not caused by
swapping.
As a diagnostic control, reducing the Store client-take wait caused
abandoned scanner senders to be reaped instead of accumulating, but it also
produced 107 timeout warnings and reduced throughput. This confirms the scanner
lifecycle path; increasing or reducing timeouts is not a fix.
A focused regression test uses close-tracking edge iterators while keeping
the path assertions unchanged. On unpatched `master`, all three path results
are correct and all three close assertions fail:
```text
Tests run: 3, Failures: 3
```
The proposed fix retains the original iterator returned by `edgesOfVertex()`
and closes it with `CloseableIterator.closeIterator()` in `finally` blocks. The
original iterator must be retained because `skipSuperNodeIfNeeded()` may
consume and replace it. `superNode()` receives the same explicit `finally`
cleanup. Query semantics, limits, and timeouts do not change.
With the patch, the focused test passes 3/3. A full-graph HStore validation
then completed all 1,516 warmup and 5,791 measured requests with zero errors,
zero swap, stable service PIDs, and zero `HgChannel.send` frames at every
sampled point from 0 through 600 seconds.
### Vertex/Edge example (问题点 / 边数据举例)
The problem is independent of a particular vertex or edge value. It is
triggered when bounded bidirectional shortest-path traversal finds a path
before the current backend iterator has been exhausted.
### Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)
```groovy
schema.vertexLabel("user")
.useCustomizeNumberId()
.ifNotExist()
.create()
schema.edgeLabel("link")
.sourceLabel("user")
.targetLabel("user")
.ifNotExist()
.create()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]