NotHimmel opened a new issue, #2567:
URL: https://github.com/apache/age/issues/2567
**Describe the bug**
`manage_GRAPH_global_contexts()` (`age_global_graph.c:1024`) builds a full
in-memory copy
of every vertex and edge of a graph in `TopMemoryContext` (`:1033`), once
per backend.
There is no memory ceiling and no GUC, and the load ignores the label scope
of the calling
query. A 3-hop `shortest_path()` on a 500k-vertex graph takes ~3.3 s the
first time, of
which ~1 ms is the search and the rest is the load. On a 3M-vertex / 9M-edge
graph the
cache is ~2.3 GB per backend, so a few concurrent graph queries OOM a modest
server.
Three separate observations:
1. The load is not shared between backends. A second session pays the full
cost again.
2. The load ignores the query's label scope. `load_vertex_hashtable()`
(`:715`) and
`load_edge_hashtable()` (`:817`) call `table_beginscan(..., snapshot, 0,
NULL)` at
`:748` and `:850` — zero scan keys. Passing `edge_types` does not reduce
the load;
the filter is applied inside the traversal, after the load.
3. There is no upper bound. The context grows to whatever the graph size
demands and is
released only when the graph version changes (`is_ggctx_invalid()`,
`:235`) or on an
explicit `age_delete_global_graphs()`. A read-only workload holds it for
the life of
the session, and nothing stops the OOM killer.
This is not a regression, and not a defect in the new shortest-path code:
`sp_run_bfs()`
(`age_vle.c:2983`) and the `SP_MAX_RESULT_PATHS` cap (`:3215`) are fine. But
`age_shortest_path` calls `manage_GRAPH_global_contexts()` at
`age_vle.c:3632`, so it
inherits this from VLE and makes it much easier to hit — one
`shortest_path()` call now
pays the whole load.
**How are you accessing AGE (Command line, driver, etc.)?**
- psql (command line)
**What data setup do we need to do?**
Two graphs with identical vertices and an identical `E1` ring. `sp3`
additionally has
`E2`/`E3`, which every query below explicitly excludes. Takes about a minute
to build.
```pgsql
LOAD 'age'; SET search_path = ag_catalog, public;
SET maintenance_work_mem = '1GB';
-- sp3: 500k vertices, THREE edge labels x 500k edges
SELECT create_graph('sp3');
SELECT create_vlabel('sp3','V');
SELECT create_elabel('sp3','E1');
SELECT create_elabel('sp3','E2');
SELECT create_elabel('sp3','E3');
INSERT INTO sp3."V"(id, properties)
SELECT _graphid(3,i::bigint), ('{"k":'||i||'}')::agtype FROM
generate_series(1,500000) i;
-- E1 is a ring i -> i+1, so vertex 1 reaches vertex 4 in exactly 3 hops
INSERT INTO sp3."E1"(id, start_id, end_id, properties)
SELECT _graphid(4,i::bigint), _graphid(3,i::bigint),
_graphid(3,(i%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;
-- E2/E3 are chords, irrelevant to any E1-only query
INSERT INTO sp3."E2"(id, start_id, end_id, properties)
SELECT _graphid(5,i::bigint), _graphid(3,i::bigint),
_graphid(3,((i::bigint*104729)%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;
INSERT INTO sp3."E3"(id, start_id, end_id, properties)
SELECT _graphid(6,i::bigint), _graphid(3,i::bigint),
_graphid(3,((i::bigint*15485863)%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;
-- sp1: same vertices, ONLY the E1 ring
SELECT create_graph('sp1');
SELECT create_vlabel('sp1','V');
SELECT create_elabel('sp1','E1');
INSERT INTO sp1."V"(id, properties)
SELECT _graphid(3,i::bigint), ('{"k":'||i||'}')::agtype FROM
generate_series(1,500000) i;
INSERT INTO sp1."E1"(id, start_id, end_id, properties)
SELECT _graphid(4,i::bigint), _graphid(3,i::bigint),
_graphid(3,(i%500000)+1), '{}'::agtype
FROM generate_series(1,500000) i;
ANALYZE sp3."V"; ANALYZE sp3."E1"; ANALYZE sp3."E2"; ANALYZE sp3."E3";
ANALYZE sp1."V"; ANALYZE sp1."E1";
```
**What is the necessary configuration info needed?**
- Nothing beyond a default install. `shared_buffers = 1GB` here; no
concurrent writers,
so the cache stays valid across the measurements.
- On PG 18, `detect_version_mode()` selects `VERSION_MODE_DSM`, so
snapshot-based
invalidation is not involved.
**What is the command that caused the error?**
There is no error message — the symptom is time and memory. Run this in a
**fresh
session** each time, changing the graph name for step 5:
```pgsql
LOAD 'age'; SET search_path = ag_catalog, public;
SELECT pg_backend_pid() AS pid \gset
\setenv PID :pid
\! grep VmRSS /proc/$PID/status
\timing on
SELECT count(*) FROM age_shortest_path('"sp3"'::agtype,
_graphid(3,1)::agtype, _graphid(3,4)::agtype,
'"E1"'::agtype, NULL, NULL, 3::agtype);
\timing off
\! grep VmRSS /proc/$PID/status
```
Every step returns the one correct 3-hop path. `VmRSS` is the serving
backend's.
| step | session | graph | time | backend RSS |
|---|---|---|---|---|
| 1 | A | - | baseline after `LOAD 'age'` | 15.7 MB |
| 2 | A | sp3 | **3296 ms** | 15.7 MB -> **439 MB** |
| 3 | A | sp3 | **1.15 ms** (identical call again) | 439 MB |
| 4 | B (new) | sp3 | **3751 ms** | 19.9 MB -> **439 MB** |
| 5 | C (new) | sp1 | **1252 ms** | 15.6 MB -> **307 MB** |
Step 4 repeated in three further fresh sessions: 3162 / 3383 / 3437 ms,
438.6 MB each
time.
- **2 vs 3** — the search costs 1.15 ms; the other ~3295 ms is the cache
build.
- **2 vs 4** — a second backend pays it again. The cache is backend-private.
- **4 vs 5** — same query, same answer, `edge_types = '"E1"'` in both. `sp3`
merely *also
contains* `E2`/`E3`; those 1M excluded edges still cost **+132 MB and
+2.50 s**.
Same build at 3M vertices + 3 x 3M edges, `max_hops = 4`: first call 22.4 s
/ **2.27 GB**,
identical call in the same session 3.0 ms, a new session 24.8 s / 2.27 GB
again, and
`edge_types = '"E1"'` in a fresh session still 23.4 s / 2.27 GB. A user
reported a backend
killed with `anon-rss 3.3 GB` + `signal 9` on a graph of this shape (their
report, not my
measurement).
**Expected behavior**
Two things, in order of how badly they are needed:
1. A graph query should not be able to drive the backend into the OOM killer
with no
configurable limit. Exceeding a bound should fail the query with a clear
message.
2. A 3-hop query on a graph with a start vertex should not require the
entire graph to be
resident, should not re-pay the load in every backend, and should not
load edge labels
the query has explicitly excluded.
**Environment (please complete the following information):**
- Version: AGE 1.8.0 (`PG18/v1.8.0-rc0`, commit `e43dc1a1`)
- PostgreSQL 18.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
