This is an automated email from the ASF dual-hosted git repository. MuhammadTahaNaveed pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/age.git
commit 00bb09ce31c2465856f424e625720a500002c992 Author: John Gemignani <[email protected]> AuthorDate: Thu Oct 8 05:08:19 2026 +0000 Keep VLE graph contexts alive for the whole query A traversal points into a backend-local graph context that the next acquisition could free. A context loaded through RLS is never valid for a new acquisition, so when two traversals ran in one query the second one freed the first one's context. Interleaved traversals lost results, and materializing a path after its context was freed dereferenced a NULL pointer (review issue 1 on #2561). Pin a traversal's graph context to the query's per-query memory context. A pinned context that becomes invalid is retired: it is no longer offered to new acquisitions, and a memory-context callback frees it when the last query holding it ends, including after an error. Deleting the caches retires pinned contexts the same way. A VLE path is materialized from a context that holds all of its vertices and edges; when there is none, the materializers raise an error instead of dereferencing a missing context. Within one query, reuse a pinned context under the rules PostgreSQL gives a statement's scans, so that the policies run once for each row a load scans instead of again for every traversal. The 400-edge workload from the review evaluated its policy 160,000 times and now evaluates it 400 times, for stable and volatile policies alike: - A load made with another role, enforcement setting or snapshot is a different scan. - A later command id means that a traversal which has not loaded the graph yet must see the statement's own writes. The traversal that loaded it keeps it while the statement writes, as a traversal of a graph without RLS does. Pinning a new copy for every row would keep them all until the query ends. - Policies and privileges hold for the whole statement: PostgreSQL applies the policies when it plans the statement and checks the privileges when it starts it, so a change made while the statement runs does not end the reuse. - When the RLS quals call only IMMUTABLE functions, besides values such as current_user that cannot change unnoticed, any traversal in the query may share the graph. Otherwise only the traversal that loaded it reuses it, as when PostgreSQL rescans the result of a scan, and another traversal loads again: a STABLE function such as current_setting() can give another result once the statement changes the setting. A traversal is identified by its FmgrInfo, which stays the same when it runs again for another row. Policies from the row-security hooks become the same security quals as catalog policies, so they are inspected as well. RLS-filtered contexts are still never reused across statements. An RLS traversal in a DETACH DELETE query loaded its graph again for every row. A reload saw a deleted vertex whose edge was still visible, and the traversal failed with "add_valid_vertex_edges: no vertex found", while the same query without RLS succeeded. Keeping the graph makes the two return the same rows. Tests cover interleaved traversals, two RLS graphs materialized in one statement, policy evaluation counts for one and two traversals with immutable, stable and volatile policies across traversal shapes, a policy reading a setting the statement changes, the contexts left after each query, a statement's own write, a policy change and a role switch made while a statement runs, and traversals before and after a DETACH DELETE with and without RLS, including the graph copies held. They also build stored paths after the caches are deleted and for a role whose context lacks their edges, run queries that fail after their traversal, and delete the caches while a query holds its graph. Co-authored-by: Copilot <[email protected]> --- regress/expected/rls_vle.out | 895 +++++++++++++++++++++++++++++++ regress/sql/rls_vle.sql | 528 ++++++++++++++++++ src/backend/utils/adt/age_global_graph.c | 580 ++++++++++++++++++-- src/backend/utils/adt/age_vle.c | 49 +- src/include/utils/age_global_graph.h | 9 + 5 files changed, 1995 insertions(+), 66 deletions(-) diff --git a/regress/expected/rls_vle.out b/regress/expected/rls_vle.out index a3f16cae..ec7220f9 100644 --- a/regress/expected/rls_vle.out +++ b/regress/expected/rls_vle.out @@ -350,6 +350,901 @@ NOTICE: graph "rls_acl" has been dropped (1 row) RESET age.enforce_rls_in_traversal; +-- ============================================================================ +-- PART 8: Traversal lifetime -- a traversal's graph context stays alive until +-- its query ends, even when another traversal in the same query must load its +-- own RLS-filtered context. Within one query an RLS-filtered context is reused +-- under the rules PostgreSQL gives a statement's scans (see 8.3 to 8.7). +-- ============================================================================ +SELECT create_graph('rls_vle_chain'); +NOTICE: graph "rls_vle_chain" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('rls_vle_chain', $$ + CREATE (:Node {n: 1})-[:LINK]->(:Node {n: 2})-[:LINK]->(:Node {n: 3})-[:LINK]->(:Node {n: 4}) +$$) AS (x agtype); + x +--- +(0 rows) + +SELECT create_graph('rls_vle_chain2'); +NOTICE: graph "rls_vle_chain2" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('rls_vle_chain2', $$ + CREATE (:Node {n: 1})-[:LINK]->(:Node {n: 2})-[:LINK]->(:Node {n: 3}) +$$) AS (x agtype); + x +--- +(0 rows) + +SELECT create_graph('rls_vle_many'); +NOTICE: graph "rls_vle_many" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('rls_vle_many', $$ + UNWIND range(1, 20) AS i + CREATE (:Source {i: i})-[:LINK]->(:Target {i: i}) +$$) AS (x agtype); + x +--- +(0 rows) + +GRANT USAGE ON SCHEMA rls_vle_chain, rls_vle_chain2, rls_vle_many TO rls_vle_user; +GRANT SELECT ON ALL TABLES IN SCHEMA rls_vle_chain, rls_vle_chain2, rls_vle_many + TO rls_vle_user; +ALTER TABLE rls_vle_chain."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_all ON rls_vle_chain."LINK" FOR SELECT USING (true); +ALTER TABLE rls_vle_chain2."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_all ON rls_vle_chain2."LINK" FOR SELECT USING (true); +ALTER TABLE rls_vle_many."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_all ON rls_vle_many."LINK" FOR SELECT USING (true); +SET ROLE rls_vle_user; +-- 8.1: two traversals advanced row by row in one target list: 6 and 20 paths +SELECT count(a) AS chain_paths, count(b) AS independent_paths +FROM ( + SELECT + (age_vle('"rls_vle_chain"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '3', '1')).start_id AS a, + (age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '2', '1')).end_id AS b +) AS interleaved; + chain_paths | independent_paths +-------------+------------------- + 6 | 20 +(1 row) + +-- 8.2: paths of the first graph are materialized after the second graph's +-- traversal loaded its own context: 6 x 3 pairs of distinct paths +SELECT count(*) AS pairs, count(DISTINCT p) AS chain_paths, + count(DISTINCT q) AS chain2_paths +FROM cypher('rls_vle_chain', $$ + MATCH p = (a)-[:LINK*1..3]->(b) RETURN p + $$) AS one(p agtype), + cypher('rls_vle_chain2', $$ + MATCH q = (a)-[:LINK*1..2]->(b) RETURN q + $$) AS two(q agtype); + pairs | chain_paths | chain2_paths +-------+-------------+-------------- + 18 | 6 | 3 +(1 row) + +RESET ROLE; +-- 8.3: policy evaluations per query. Each traversal loads the 20 visible +-- edges once and reuses its load when it is evaluated again for another row, +-- as PostgreSQL rescans a materialized scan. Another traversal in the query +-- shares that load only when the policy gives the same result throughout the +-- statement: when it calls only IMMUTABLE functions, besides values such as +-- current_user that cannot change unnoticed. Otherwise, as with the STABLE +-- and volatile counters below, it loads the graph again, as a separate scan +-- would evaluate the policy again. +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + graph_caches_cleared +---------------------- + t +(1 row) + +CREATE SEQUENCE public.rls_vle_policy_calls; +GRANT USAGE, SELECT ON SEQUENCE public.rls_vle_policy_calls TO rls_vle_user; +CREATE FUNCTION public.rls_vle_count_stable() RETURNS boolean + LANGUAGE plpgsql STABLE +AS $f$ +BEGIN + PERFORM nextval('public.rls_vle_policy_calls'); + RETURN true; +END +$f$; +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_stable()); +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); + n +---- + 20 +(1 row) + +RESET ROLE; +SELECT last_value AS stable_one_hop_evaluations FROM public.rls_vle_policy_calls; + stable_one_hop_evaluations +---------------------------- + 20 +(1 row) + +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..2]->(b:Target) RETURN count(*) +$$) AS (n agtype); + n +---- + 20 +(1 row) + +RESET ROLE; +SELECT last_value AS stable_one_to_two_hop_evaluations +FROM public.rls_vle_policy_calls; + stable_one_to_two_hop_evaluations +----------------------------------- + 20 +(1 row) + +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH p = (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(p) +$$) AS (n agtype); + n +---- + 20 +(1 row) + +RESET ROLE; +SELECT last_value AS stable_named_path_evaluations +FROM public.rls_vle_policy_calls; + stable_named_path_evaluations +------------------------------- + 20 +(1 row) + +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); + first | second +-------+-------- + 20 | 20 +(1 row) + +RESET ROLE; +SELECT last_value AS stable_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + stable_two_call_site_evaluations +---------------------------------- + 40 +(1 row) + +-- each traversal loaded its own context: the first one's was freed when the +-- query ended, and the second one's stays in the backend until the next +-- graph load frees it +SELECT count(*) AS contexts_after_stable_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + contexts_after_stable_query +----------------------------- + 1 +(1 row) + +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + graph_caches_cleared +---------------------- + t +(1 row) + +-- policies that give the same result throughout the statement: the two +-- traversals share one load. The counter is declared IMMUTABLE so that the +-- policy counts as fixed, and its argument keeps it from being evaluated +-- once at planning. +CREATE FUNCTION public.rls_vle_count_fixed(id graphid) RETURNS boolean + LANGUAGE plpgsql IMMUTABLE +AS $f$ +BEGIN + PERFORM nextval('public.rls_vle_policy_calls'); + RETURN true; +END +$f$; +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_fixed(id)); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); + first | second +-------+-------- + 20 | 20 +(1 row) + +RESET ROLE; +SELECT last_value AS immutable_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + immutable_two_call_site_evaluations +------------------------------------- + 20 +(1 row) + +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_fixed(id) AND current_user = 'rls_vle_user'); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); + first | second +-------+-------- + 20 | 20 +(1 row) + +RESET ROLE; +SELECT last_value AS current_user_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + current_user_two_call_site_evaluations +---------------------------------------- + 20 +(1 row) + +-- the search path can change during the statement, and a STABLE function +-- in a subquery of the policy counts as in the policy itself: each +-- traversal loads the graph +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_fixed(id) AND current_schema IS NOT NULL); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); + first | second +-------+-------- + 20 | 20 +(1 row) + +RESET ROLE; +SELECT last_value AS current_schema_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + current_schema_two_call_site_evaluations +------------------------------------------ + 40 +(1 row) + +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (EXISTS (SELECT 1 WHERE id IS NOT NULL + AND public.rls_vle_count_stable())); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); + first | second +-------+-------- + 20 | 20 +(1 row) + +RESET ROLE; +SELECT last_value AS stable_subquery_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + stable_subquery_two_call_site_evaluations +------------------------------------------- + 40 +(1 row) + +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + graph_caches_cleared +---------------------- + t +(1 row) + +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (nextval('public.rls_vle_policy_calls') > 0); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); + n +---- + 20 +(1 row) + +RESET ROLE; +SELECT last_value AS volatile_one_hop_evaluations +FROM public.rls_vle_policy_calls; + volatile_one_hop_evaluations +------------------------------ + 20 +(1 row) + +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); + first | second +-------+-------- + 20 | 20 +(1 row) + +RESET ROLE; +SELECT last_value AS volatile_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + volatile_two_call_site_evaluations +------------------------------------ + 40 +(1 row) + +-- the first call site's context was freed when the query ended; the second +-- one is kept, as for the stable policy +SELECT count(*) AS contexts_after_volatile_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + contexts_after_volatile_query +------------------------------- + 1 +(1 row) + +ALTER POLICY link_all ON rls_vle_many."LINK" USING (true); +-- a policy that reads a setting with current_setting(), which the statement +-- itself changes: each traversal sets the limit through its graph argument +-- and sees its own Targets, the first those up to 5 and the second those up +-- to 10, as two scans of the label in PostgreSQL would +ALTER TABLE rls_vle_many."Target" ENABLE ROW LEVEL SECURITY; +CREATE POLICY target_limit ON rls_vle_many."Target" FOR SELECT + USING ((properties->>'"i"')::int <= + coalesce(nullif(current_setting('rls_vle.max_target', true), ''), + '0')::int); +CREATE FUNCTION public.rls_vle_limit_targets(n int) RETURNS agtype + LANGUAGE plpgsql VOLATILE +AS $f$ +BEGIN + PERFORM set_config('rls_vle.max_target', n::text, false); + RETURN '"rls_vle_many"'::agtype; +END +$f$; +SET ROLE rls_vle_user; +SELECT count(DISTINCT first.end_id) AS first_targets, + count(DISTINCT second.end_id) AS second_targets +FROM age_vle(public.rls_vle_limit_targets(5), NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS first, + LATERAL age_vle(public.rls_vle_limit_targets( + CASE WHEN first.start_id IS NOT NULL THEN 10 END), + NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS second; + first_targets | second_targets +---------------+---------------- + 5 | 10 +(1 row) + +RESET ROLE; +RESET rls_vle.max_target; +DROP POLICY target_limit ON rls_vle_many."Target"; +ALTER TABLE rls_vle_many."Target" DISABLE ROW LEVEL SECURITY; +DROP FUNCTION public.rls_vle_limit_targets(int); +-- 8.4: a statement's own write is visible to its later traversal. The write +-- advances the command id, so the later traversal does not reuse the graph +-- loaded before it. +BEGIN; +GRANT INSERT ON rls_vle_many."LINK" TO rls_vle_user; +GRANT USAGE ON SEQUENCE rls_vle_many."LINK_id_seq" TO rls_vle_user; +CREATE POLICY link_insert ON rls_vle_many."LINK" FOR INSERT WITH CHECK (true); +SET LOCAL ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source {i: 1})-[:LINK*1..1]->(b:Target) + WITH a, count(b) AS before + MATCH (t:Target {i: 2}) + CREATE (a)-[:LINK]->(t) + WITH a, before, count(*) AS created + MATCH (a)-[:LINK*1..1]->(c:Target) + RETURN before, created, count(c) +$$) AS (before agtype, created agtype, after agtype); + before | created | after +--------+---------+------- + 1 | 1 | 2 +(1 row) + +ROLLBACK; +-- 8.5: a policy change made while a statement runs does not change what that +-- statement's traversal sees, as for a PostgreSQL scan; the next statement +-- sees it. The start vertex argument hides every edge when the traversal is +-- evaluated for the second row, after the first row loaded the graph. +CREATE FUNCTION public.rls_vle_start_hiding_links(id graphid, rn bigint) + RETURNS ag_catalog.agtype + LANGUAGE plpgsql SECURITY DEFINER +AS $f$ +BEGIN + IF rn = 2 THEN + ALTER POLICY link_all ON rls_vle_many."LINK" USING (false); + END IF; + RETURN (id::text)::ag_catalog.agtype; +END +$f$; +SET ROLE rls_vle_user; +SELECT count(*) AS paths_with_policy_changed_midway +FROM (SELECT id, row_number() OVER (ORDER BY id) AS rn + FROM rls_vle_many."Source") AS s, + LATERAL age_vle('"rls_vle_many"', + public.rls_vle_start_hiding_links(s.id, s.rn), NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS v; + paths_with_policy_changed_midway +---------------------------------- + 20 +(1 row) + +SELECT count(*) AS paths_in_next_statement +FROM rls_vle_many."Source" AS s, + LATERAL age_vle('"rls_vle_many"', (s.id::text)::agtype, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS v; + paths_in_next_statement +------------------------- + 0 +(1 row) + +RESET ROLE; +-- 8.6: a graph loaded for one role is not reused for another, even within a +-- statement. The start vertex argument switches to rls_vle_user2, from whom +-- every edge is hidden, when the traversal is evaluated for the second row. +ALTER POLICY link_all ON rls_vle_many."LINK" USING (true); +GRANT USAGE ON SCHEMA rls_vle_many TO rls_vle_user2; +GRANT SELECT ON ALL TABLES IN SCHEMA rls_vle_many TO rls_vle_user2; +CREATE POLICY link_hidden_from_user2 ON rls_vle_many."LINK" AS RESTRICTIVE + FOR SELECT TO rls_vle_user2 USING (false); +CREATE FUNCTION public.rls_vle_start_as_user2(id graphid, rn bigint) + RETURNS ag_catalog.agtype + LANGUAGE plpgsql +AS $f$ +BEGIN + IF rn = 2 THEN + PERFORM set_config('role', 'rls_vle_user2', true); + END IF; + RETURN (id::text)::ag_catalog.agtype; +END +$f$; +SET ROLE rls_vle_user; +SELECT count(*) AS paths_with_role_changed_midway +FROM (SELECT id, row_number() OVER (ORDER BY id) AS rn + FROM rls_vle_many."Source") AS s, + LATERAL age_vle('"rls_vle_many"', + public.rls_vle_start_as_user2(s.id, s.rn), NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS v; + paths_with_role_changed_midway +-------------------------------- + 1 +(1 row) + +RESET ROLE; +-- 8.7: a statement's own deletes do not make a traversal load its graph +-- again. The traversal keeps the graph it loaded, as it does without RLS, so +-- a policy that allows every row returns what no policy returns, and the +-- query holds one copy of the graph instead of one for every row. +SELECT create_graph('rls_vle_before_plain'); +NOTICE: graph "rls_vle_before_plain" has been created + create_graph +-------------- + +(1 row) + +SELECT create_graph('rls_vle_before_rls'); +NOTICE: graph "rls_vle_before_rls" has been created + create_graph +-------------- + +(1 row) + +SELECT create_graph('rls_vle_after_plain'); +NOTICE: graph "rls_vle_after_plain" has been created + create_graph +-------------- + +(1 row) + +SELECT create_graph('rls_vle_after_rls'); +NOTICE: graph "rls_vle_after_rls" has been created + create_graph +-------------- + +(1 row) + +SELECT * FROM cypher('rls_vle_before_plain', $$ + CREATE (:Source {i: 1})-[:LINK]->(m:Mid {i: 0})-[:LINK]->(:Target {i: 9}), + (:Source {i: 2})-[:LINK]->(m) +$$) AS (x agtype); + x +--- +(0 rows) + +SELECT * FROM cypher('rls_vle_before_rls', $$ + CREATE (:Source {i: 1})-[:LINK]->(m:Mid {i: 0})-[:LINK]->(:Target {i: 9}), + (:Source {i: 2})-[:LINK]->(m) +$$) AS (x agtype); + x +--- +(0 rows) + +SELECT * FROM cypher('rls_vle_after_plain', $$ + CREATE (:Node {i: 1, ord: 0})-[:LINK]->(:Node {i: 2, ord: 2}) + -[:LINK]->(:Node {i: 3, ord: 3}), + (:Node {i: 9, ord: 1}) +$$) AS (x agtype); + x +--- +(0 rows) + +SELECT * FROM cypher('rls_vle_after_rls', $$ + CREATE (:Node {i: 1, ord: 0})-[:LINK]->(:Node {i: 2, ord: 2}) + -[:LINK]->(:Node {i: 3, ord: 3}), + (:Node {i: 9, ord: 1}) +$$) AS (x agtype); + x +--- +(0 rows) + +GRANT USAGE ON SCHEMA rls_vle_before_plain, rls_vle_before_rls, + rls_vle_after_plain, rls_vle_after_rls TO rls_vle_user; +GRANT SELECT, DELETE ON ALL TABLES IN SCHEMA rls_vle_before_plain, + rls_vle_before_rls, rls_vle_after_plain, rls_vle_after_rls + TO rls_vle_user; +ALTER TABLE rls_vle_before_rls."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_read ON rls_vle_before_rls."LINK" FOR SELECT USING (true); +CREATE POLICY link_delete ON rls_vle_before_rls."LINK" FOR DELETE USING (true); +ALTER TABLE rls_vle_after_rls."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_read ON rls_vle_after_rls."LINK" FOR SELECT USING (true); +CREATE POLICY link_delete ON rls_vle_after_rls."LINK" FOR DELETE USING (true); +SET ROLE rls_vle_user; +-- a traversal before the delete, for each Source: s1 -> m <- s2, m -> t +SELECT * FROM cypher('rls_vle_before_plain', $$ + MATCH (a:Source)-[:LINK*1..2]->(b) + DETACH DELETE b + RETURN a.i, b.i +$$) AS (source agtype, deleted agtype) ORDER BY source, deleted; + source | deleted +--------+--------- + 1 | 0 + 1 | 9 + 2 | 9 +(3 rows) + +SELECT * FROM cypher('rls_vle_before_rls', $$ + MATCH (a:Source)-[:LINK*1..2]->(b) + DETACH DELETE b + RETURN a.i, b.i +$$) AS (source agtype, deleted agtype) ORDER BY source, deleted; + source | deleted +--------+--------- + 1 | 0 + 1 | 9 + 2 | 9 +(3 rows) + +-- a traversal after the delete of each node but n1, in the order of ord: +-- n1 -> n2 -> n3 and an unconnected n9, which is deleted first +SELECT * FROM cypher('rls_vle_after_plain', $$ + MATCH (d:Node) WHERE d.i > 1 + WITH d ORDER BY d.ord + DETACH DELETE d + WITH d + MATCH p = (n:Node {i: 1})-[:LINK*1..3]->() + RETURN d.i, length(p) +$$) AS (deleted agtype, path_length agtype) ORDER BY deleted, path_length; + deleted | path_length +---------+------------- + 2 | 2 + 9 | 1 + 9 | 2 +(3 rows) + +SELECT * FROM cypher('rls_vle_after_rls', $$ + MATCH (d:Node) WHERE d.i > 1 + WITH d ORDER BY d.ord + DETACH DELETE d + WITH d + MATCH p = (n:Node {i: 1})-[:LINK*1..3]->() + RETURN d.i, length(p) +$$) AS (deleted agtype, path_length agtype) ORDER BY deleted, path_length; + deleted | path_length +---------+------------- + 2 | 2 + 9 | 1 + 9 | 2 +(3 rows) + +RESET ROLE; +-- the copies of the graph held while a traversal for each Source is +-- followed by a delete, counted after every row +CREATE FUNCTION public.rls_vle_graph_copies(anyelement) RETURNS bigint + LANGUAGE plpgsql VOLATILE SECURITY DEFINER +AS $f$ +BEGIN + RETURN (SELECT count(*) FROM pg_backend_memory_contexts + WHERE name = 'AGE edge_table'); +END +$f$; +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + graph_caches_cleared +---------------------- + t +(1 row) + +BEGIN; +GRANT DELETE ON ALL TABLES IN SCHEMA rls_vle_many TO rls_vle_user; +CREATE POLICY link_delete ON rls_vle_many."LINK" FOR DELETE USING (true); +SET LOCAL ROLE rls_vle_user; +SELECT count(*) AS deleted, max(copies) AS most_graph_copies_held +FROM (SELECT public.rls_vle_graph_copies(i) AS copies + FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + DETACH DELETE b + RETURN a.i + $$) AS (i agtype)) AS per_row; + deleted | most_graph_copies_held +---------+------------------------ + 20 | 1 +(1 row) + +ROLLBACK; +-- 8.8: a path value kept after its query is built from a graph context that +-- holds all of its vertices and edges. When no context does, building it +-- raises an error: after the caches are deleted, and for a role whose +-- context lacks the path's edges. +CREATE TABLE public.rls_vle_stored_paths AS +SELECT edges FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1'); +SELECT count(age_materialize_vle_path(edges)) AS stored_paths_built +FROM public.rls_vle_stored_paths; + stored_paths_built +-------------------- + 20 +(1 row) + +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + graph_caches_cleared +---------------------- + t +(1 row) + +SELECT count(age_materialize_vle_path(edges)) AS stored_paths_built +FROM public.rls_vle_stored_paths; +ERROR: graph data for the referenced VLE path is not available +SELECT count(age_materialize_vle_edges(edges)) AS stored_edge_lists_built +FROM public.rls_vle_stored_paths; +ERROR: graph data for the referenced VLE path is not available +GRANT SELECT ON public.rls_vle_stored_paths TO rls_vle_user2; +SET ROLE rls_vle_user2; +SELECT count(*) AS paths_visible_to_user2 +FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), '1', '1', '1'); + paths_visible_to_user2 +------------------------ + 0 +(1 row) + +SELECT count(age_materialize_vle_path(edges)) AS stored_paths_built +FROM public.rls_vle_stored_paths; +ERROR: graph data for the referenced VLE path is not available +SELECT count(age_materialize_vle_edges(edges)) AS stored_edge_lists_built +FROM public.rls_vle_stored_paths; +ERROR: graph data for the referenced VLE path is not available +RESET ROLE; +DROP TABLE public.rls_vle_stored_paths; +-- 8.9: a query that fails after its traversal releases its graph context +-- during the error cleanup, at the top level and in PL/pgSQL, so failed +-- queries leave no contexts behind +SET ROLE rls_vle_user; +SELECT 1 / ((n::text)::int - 20) AS fails_after_traversal +FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); +ERROR: division by zero +SELECT 1 / ((n::text)::int - 20) AS fails_after_traversal +FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); +ERROR: division by zero +DO $d$ +BEGIN + PERFORM 1 / ((n::text)::int - 20) + FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) + $$) AS (n agtype); +EXCEPTION WHEN division_by_zero THEN + NULL; +END +$d$; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (paths agtype); + paths +------- + 20 +(1 row) + +RESET ROLE; +SELECT count(*) AS contexts_after_failed_queries +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + contexts_after_failed_queries +------------------------------- + 1 +(1 row) + +-- 8.10: deleting the graph caches while a query holds its graph retires the +-- context instead of freeing it. The query still builds its paths from it, +-- and it is freed when the query ends, also when the query then fails. +SET ROLE rls_vle_user; +SELECT count(age_materialize_vle_path(v.edges)) AS paths_built_after_delete +FROM (SELECT edges, row_number() OVER () AS rn + FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1')) AS v +WHERE CASE WHEN v.rn = 1 THEN age_delete_global_graphs(NULL) IS NOT NULL + ELSE true END; + paths_built_after_delete +-------------------------- + 20 +(1 row) + +RESET ROLE; +SELECT count(*) AS contexts_after_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + contexts_after_query +---------------------- + 0 +(1 row) + +SET ROLE rls_vle_user; +SELECT 1 / (count(age_materialize_vle_path(v.edges)) - 20) AS fails_after_paths +FROM (SELECT edges, row_number() OVER () AS rn + FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1')) AS v +WHERE CASE WHEN v.rn = 1 + THEN age_delete_global_graphs('"rls_vle_many"') IS NOT NULL + ELSE true END; +ERROR: division by zero +RESET ROLE; +SELECT count(*) AS contexts_after_failed_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + contexts_after_failed_query +----------------------------- + 0 +(1 row) + +DROP FUNCTION public.rls_vle_graph_copies(anyelement); +DROP FUNCTION public.rls_vle_start_as_user2(graphid, bigint); +DROP FUNCTION public.rls_vle_start_hiding_links(graphid, bigint); +DROP FUNCTION public.rls_vle_count_fixed(graphid); +DROP FUNCTION public.rls_vle_count_stable(); +DROP SEQUENCE public.rls_vle_policy_calls; +SELECT drop_graph('rls_vle_chain', true); +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table rls_vle_chain._ag_label_vertex +drop cascades to table rls_vle_chain._ag_label_edge +drop cascades to table rls_vle_chain."Node" +drop cascades to table rls_vle_chain."LINK" +NOTICE: graph "rls_vle_chain" has been dropped + drop_graph +------------ + +(1 row) + +SELECT drop_graph('rls_vle_chain2', true); +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table rls_vle_chain2._ag_label_vertex +drop cascades to table rls_vle_chain2._ag_label_edge +drop cascades to table rls_vle_chain2."Node" +drop cascades to table rls_vle_chain2."LINK" +NOTICE: graph "rls_vle_chain2" has been dropped + drop_graph +------------ + +(1 row) + +SELECT drop_graph('rls_vle_many', true); +NOTICE: drop cascades to 5 other objects +DETAIL: drop cascades to table rls_vle_many._ag_label_vertex +drop cascades to table rls_vle_many._ag_label_edge +drop cascades to table rls_vle_many."Source" +drop cascades to table rls_vle_many."LINK" +drop cascades to table rls_vle_many."Target" +NOTICE: graph "rls_vle_many" has been dropped + drop_graph +------------ + +(1 row) + +SELECT drop_graph('rls_vle_before_plain', true); +NOTICE: drop cascades to 6 other objects +DETAIL: drop cascades to table rls_vle_before_plain._ag_label_vertex +drop cascades to table rls_vle_before_plain._ag_label_edge +drop cascades to table rls_vle_before_plain."Source" +drop cascades to table rls_vle_before_plain."LINK" +drop cascades to table rls_vle_before_plain."Mid" +drop cascades to table rls_vle_before_plain."Target" +NOTICE: graph "rls_vle_before_plain" has been dropped + drop_graph +------------ + +(1 row) + +SELECT drop_graph('rls_vle_before_rls', true); +NOTICE: drop cascades to 6 other objects +DETAIL: drop cascades to table rls_vle_before_rls._ag_label_vertex +drop cascades to table rls_vle_before_rls._ag_label_edge +drop cascades to table rls_vle_before_rls."Source" +drop cascades to table rls_vle_before_rls."LINK" +drop cascades to table rls_vle_before_rls."Mid" +drop cascades to table rls_vle_before_rls."Target" +NOTICE: graph "rls_vle_before_rls" has been dropped + drop_graph +------------ + +(1 row) + +SELECT drop_graph('rls_vle_after_plain', true); +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table rls_vle_after_plain._ag_label_vertex +drop cascades to table rls_vle_after_plain._ag_label_edge +drop cascades to table rls_vle_after_plain."Node" +drop cascades to table rls_vle_after_plain."LINK" +NOTICE: graph "rls_vle_after_plain" has been dropped + drop_graph +------------ + +(1 row) + +SELECT drop_graph('rls_vle_after_rls', true); +NOTICE: drop cascades to 4 other objects +DETAIL: drop cascades to table rls_vle_after_rls._ag_label_vertex +drop cascades to table rls_vle_after_rls._ag_label_edge +drop cascades to table rls_vle_after_rls."Node" +drop cascades to table rls_vle_after_rls."LINK" +NOTICE: graph "rls_vle_after_rls" has been dropped + drop_graph +------------ + +(1 row) + -- ============================================================================ -- cleanup -- ============================================================================ diff --git a/regress/sql/rls_vle.sql b/regress/sql/rls_vle.sql index d171acd9..b12df277 100644 --- a/regress/sql/rls_vle.sql +++ b/regress/sql/rls_vle.sql @@ -256,6 +256,534 @@ RESET ROLE; SELECT drop_graph('rls_acl', true); RESET age.enforce_rls_in_traversal; +-- ============================================================================ +-- PART 8: Traversal lifetime -- a traversal's graph context stays alive until +-- its query ends, even when another traversal in the same query must load its +-- own RLS-filtered context. Within one query an RLS-filtered context is reused +-- under the rules PostgreSQL gives a statement's scans (see 8.3 to 8.7). +-- ============================================================================ +SELECT create_graph('rls_vle_chain'); +SELECT * FROM cypher('rls_vle_chain', $$ + CREATE (:Node {n: 1})-[:LINK]->(:Node {n: 2})-[:LINK]->(:Node {n: 3})-[:LINK]->(:Node {n: 4}) +$$) AS (x agtype); +SELECT create_graph('rls_vle_chain2'); +SELECT * FROM cypher('rls_vle_chain2', $$ + CREATE (:Node {n: 1})-[:LINK]->(:Node {n: 2})-[:LINK]->(:Node {n: 3}) +$$) AS (x agtype); +SELECT create_graph('rls_vle_many'); +SELECT * FROM cypher('rls_vle_many', $$ + UNWIND range(1, 20) AS i + CREATE (:Source {i: i})-[:LINK]->(:Target {i: i}) +$$) AS (x agtype); +GRANT USAGE ON SCHEMA rls_vle_chain, rls_vle_chain2, rls_vle_many TO rls_vle_user; +GRANT SELECT ON ALL TABLES IN SCHEMA rls_vle_chain, rls_vle_chain2, rls_vle_many + TO rls_vle_user; +ALTER TABLE rls_vle_chain."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_all ON rls_vle_chain."LINK" FOR SELECT USING (true); +ALTER TABLE rls_vle_chain2."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_all ON rls_vle_chain2."LINK" FOR SELECT USING (true); +ALTER TABLE rls_vle_many."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_all ON rls_vle_many."LINK" FOR SELECT USING (true); + +SET ROLE rls_vle_user; +-- 8.1: two traversals advanced row by row in one target list: 6 and 20 paths +SELECT count(a) AS chain_paths, count(b) AS independent_paths +FROM ( + SELECT + (age_vle('"rls_vle_chain"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '3', '1')).start_id AS a, + (age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '2', '1')).end_id AS b +) AS interleaved; + +-- 8.2: paths of the first graph are materialized after the second graph's +-- traversal loaded its own context: 6 x 3 pairs of distinct paths +SELECT count(*) AS pairs, count(DISTINCT p) AS chain_paths, + count(DISTINCT q) AS chain2_paths +FROM cypher('rls_vle_chain', $$ + MATCH p = (a)-[:LINK*1..3]->(b) RETURN p + $$) AS one(p agtype), + cypher('rls_vle_chain2', $$ + MATCH q = (a)-[:LINK*1..2]->(b) RETURN q + $$) AS two(q agtype); +RESET ROLE; + +-- 8.3: policy evaluations per query. Each traversal loads the 20 visible +-- edges once and reuses its load when it is evaluated again for another row, +-- as PostgreSQL rescans a materialized scan. Another traversal in the query +-- shares that load only when the policy gives the same result throughout the +-- statement: when it calls only IMMUTABLE functions, besides values such as +-- current_user that cannot change unnoticed. Otherwise, as with the STABLE +-- and volatile counters below, it loads the graph again, as a separate scan +-- would evaluate the policy again. +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; +CREATE SEQUENCE public.rls_vle_policy_calls; +GRANT USAGE, SELECT ON SEQUENCE public.rls_vle_policy_calls TO rls_vle_user; +CREATE FUNCTION public.rls_vle_count_stable() RETURNS boolean + LANGUAGE plpgsql STABLE +AS $f$ +BEGIN + PERFORM nextval('public.rls_vle_policy_calls'); + RETURN true; +END +$f$; +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_stable()); + +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); +RESET ROLE; +SELECT last_value AS stable_one_hop_evaluations FROM public.rls_vle_policy_calls; +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; + +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..2]->(b:Target) RETURN count(*) +$$) AS (n agtype); +RESET ROLE; +SELECT last_value AS stable_one_to_two_hop_evaluations +FROM public.rls_vle_policy_calls; +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; + +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH p = (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(p) +$$) AS (n agtype); +RESET ROLE; +SELECT last_value AS stable_named_path_evaluations +FROM public.rls_vle_policy_calls; +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; + +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); +RESET ROLE; +SELECT last_value AS stable_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + +-- each traversal loaded its own context: the first one's was freed when the +-- query ended, and the second one's stays in the backend until the next +-- graph load frees it +SELECT count(*) AS contexts_after_stable_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + +-- policies that give the same result throughout the statement: the two +-- traversals share one load. The counter is declared IMMUTABLE so that the +-- policy counts as fixed, and its argument keeps it from being evaluated +-- once at planning. +CREATE FUNCTION public.rls_vle_count_fixed(id graphid) RETURNS boolean + LANGUAGE plpgsql IMMUTABLE +AS $f$ +BEGIN + PERFORM nextval('public.rls_vle_policy_calls'); + RETURN true; +END +$f$; +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_fixed(id)); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); +RESET ROLE; +SELECT last_value AS immutable_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_fixed(id) AND current_user = 'rls_vle_user'); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); +RESET ROLE; +SELECT last_value AS current_user_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + +-- the search path can change during the statement, and a STABLE function +-- in a subquery of the policy counts as in the policy itself: each +-- traversal loads the graph +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (public.rls_vle_count_fixed(id) AND current_schema IS NOT NULL); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); +RESET ROLE; +SELECT last_value AS current_schema_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (EXISTS (SELECT 1 WHERE id IS NOT NULL + AND public.rls_vle_count_stable())); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); +RESET ROLE; +SELECT last_value AS stable_subquery_two_call_site_evaluations +FROM public.rls_vle_policy_calls; +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; + +ALTER POLICY link_all ON rls_vle_many."LINK" + USING (nextval('public.rls_vle_policy_calls') > 0); +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); +RESET ROLE; +SELECT last_value AS volatile_one_hop_evaluations +FROM public.rls_vle_policy_calls; +ALTER SEQUENCE public.rls_vle_policy_calls RESTART; + +SET ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + WITH count(*) AS first + MATCH (c:Source)-[:LINK*1..1]->(d:Target) + RETURN first, count(*) +$$) AS (first agtype, second agtype); +RESET ROLE; +SELECT last_value AS volatile_two_call_site_evaluations +FROM public.rls_vle_policy_calls; + +-- the first call site's context was freed when the query ended; the second +-- one is kept, as for the stable policy +SELECT count(*) AS contexts_after_volatile_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; +ALTER POLICY link_all ON rls_vle_many."LINK" USING (true); + +-- a policy that reads a setting with current_setting(), which the statement +-- itself changes: each traversal sets the limit through its graph argument +-- and sees its own Targets, the first those up to 5 and the second those up +-- to 10, as two scans of the label in PostgreSQL would +ALTER TABLE rls_vle_many."Target" ENABLE ROW LEVEL SECURITY; +CREATE POLICY target_limit ON rls_vle_many."Target" FOR SELECT + USING ((properties->>'"i"')::int <= + coalesce(nullif(current_setting('rls_vle.max_target', true), ''), + '0')::int); +CREATE FUNCTION public.rls_vle_limit_targets(n int) RETURNS agtype + LANGUAGE plpgsql VOLATILE +AS $f$ +BEGIN + PERFORM set_config('rls_vle.max_target', n::text, false); + RETURN '"rls_vle_many"'::agtype; +END +$f$; +SET ROLE rls_vle_user; +SELECT count(DISTINCT first.end_id) AS first_targets, + count(DISTINCT second.end_id) AS second_targets +FROM age_vle(public.rls_vle_limit_targets(5), NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS first, + LATERAL age_vle(public.rls_vle_limit_targets( + CASE WHEN first.start_id IS NOT NULL THEN 10 END), + NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS second; +RESET ROLE; +RESET rls_vle.max_target; +DROP POLICY target_limit ON rls_vle_many."Target"; +ALTER TABLE rls_vle_many."Target" DISABLE ROW LEVEL SECURITY; +DROP FUNCTION public.rls_vle_limit_targets(int); + +-- 8.4: a statement's own write is visible to its later traversal. The write +-- advances the command id, so the later traversal does not reuse the graph +-- loaded before it. +BEGIN; +GRANT INSERT ON rls_vle_many."LINK" TO rls_vle_user; +GRANT USAGE ON SEQUENCE rls_vle_many."LINK_id_seq" TO rls_vle_user; +CREATE POLICY link_insert ON rls_vle_many."LINK" FOR INSERT WITH CHECK (true); +SET LOCAL ROLE rls_vle_user; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source {i: 1})-[:LINK*1..1]->(b:Target) + WITH a, count(b) AS before + MATCH (t:Target {i: 2}) + CREATE (a)-[:LINK]->(t) + WITH a, before, count(*) AS created + MATCH (a)-[:LINK*1..1]->(c:Target) + RETURN before, created, count(c) +$$) AS (before agtype, created agtype, after agtype); +ROLLBACK; + +-- 8.5: a policy change made while a statement runs does not change what that +-- statement's traversal sees, as for a PostgreSQL scan; the next statement +-- sees it. The start vertex argument hides every edge when the traversal is +-- evaluated for the second row, after the first row loaded the graph. +CREATE FUNCTION public.rls_vle_start_hiding_links(id graphid, rn bigint) + RETURNS ag_catalog.agtype + LANGUAGE plpgsql SECURITY DEFINER +AS $f$ +BEGIN + IF rn = 2 THEN + ALTER POLICY link_all ON rls_vle_many."LINK" USING (false); + END IF; + RETURN (id::text)::ag_catalog.agtype; +END +$f$; + +SET ROLE rls_vle_user; +SELECT count(*) AS paths_with_policy_changed_midway +FROM (SELECT id, row_number() OVER (ORDER BY id) AS rn + FROM rls_vle_many."Source") AS s, + LATERAL age_vle('"rls_vle_many"', + public.rls_vle_start_hiding_links(s.id, s.rn), NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS v; +SELECT count(*) AS paths_in_next_statement +FROM rls_vle_many."Source" AS s, + LATERAL age_vle('"rls_vle_many"', (s.id::text)::agtype, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS v; +RESET ROLE; + +-- 8.6: a graph loaded for one role is not reused for another, even within a +-- statement. The start vertex argument switches to rls_vle_user2, from whom +-- every edge is hidden, when the traversal is evaluated for the second row. +ALTER POLICY link_all ON rls_vle_many."LINK" USING (true); +GRANT USAGE ON SCHEMA rls_vle_many TO rls_vle_user2; +GRANT SELECT ON ALL TABLES IN SCHEMA rls_vle_many TO rls_vle_user2; +CREATE POLICY link_hidden_from_user2 ON rls_vle_many."LINK" AS RESTRICTIVE + FOR SELECT TO rls_vle_user2 USING (false); +CREATE FUNCTION public.rls_vle_start_as_user2(id graphid, rn bigint) + RETURNS ag_catalog.agtype + LANGUAGE plpgsql +AS $f$ +BEGIN + IF rn = 2 THEN + PERFORM set_config('role', 'rls_vle_user2', true); + END IF; + RETURN (id::text)::ag_catalog.agtype; +END +$f$; + +SET ROLE rls_vle_user; +SELECT count(*) AS paths_with_role_changed_midway +FROM (SELECT id, row_number() OVER (ORDER BY id) AS rn + FROM rls_vle_many."Source") AS s, + LATERAL age_vle('"rls_vle_many"', + public.rls_vle_start_as_user2(s.id, s.rn), NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1') AS v; +RESET ROLE; + +-- 8.7: a statement's own deletes do not make a traversal load its graph +-- again. The traversal keeps the graph it loaded, as it does without RLS, so +-- a policy that allows every row returns what no policy returns, and the +-- query holds one copy of the graph instead of one for every row. +SELECT create_graph('rls_vle_before_plain'); +SELECT create_graph('rls_vle_before_rls'); +SELECT create_graph('rls_vle_after_plain'); +SELECT create_graph('rls_vle_after_rls'); +SELECT * FROM cypher('rls_vle_before_plain', $$ + CREATE (:Source {i: 1})-[:LINK]->(m:Mid {i: 0})-[:LINK]->(:Target {i: 9}), + (:Source {i: 2})-[:LINK]->(m) +$$) AS (x agtype); +SELECT * FROM cypher('rls_vle_before_rls', $$ + CREATE (:Source {i: 1})-[:LINK]->(m:Mid {i: 0})-[:LINK]->(:Target {i: 9}), + (:Source {i: 2})-[:LINK]->(m) +$$) AS (x agtype); +SELECT * FROM cypher('rls_vle_after_plain', $$ + CREATE (:Node {i: 1, ord: 0})-[:LINK]->(:Node {i: 2, ord: 2}) + -[:LINK]->(:Node {i: 3, ord: 3}), + (:Node {i: 9, ord: 1}) +$$) AS (x agtype); +SELECT * FROM cypher('rls_vle_after_rls', $$ + CREATE (:Node {i: 1, ord: 0})-[:LINK]->(:Node {i: 2, ord: 2}) + -[:LINK]->(:Node {i: 3, ord: 3}), + (:Node {i: 9, ord: 1}) +$$) AS (x agtype); +GRANT USAGE ON SCHEMA rls_vle_before_plain, rls_vle_before_rls, + rls_vle_after_plain, rls_vle_after_rls TO rls_vle_user; +GRANT SELECT, DELETE ON ALL TABLES IN SCHEMA rls_vle_before_plain, + rls_vle_before_rls, rls_vle_after_plain, rls_vle_after_rls + TO rls_vle_user; +ALTER TABLE rls_vle_before_rls."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_read ON rls_vle_before_rls."LINK" FOR SELECT USING (true); +CREATE POLICY link_delete ON rls_vle_before_rls."LINK" FOR DELETE USING (true); +ALTER TABLE rls_vle_after_rls."LINK" ENABLE ROW LEVEL SECURITY; +CREATE POLICY link_read ON rls_vle_after_rls."LINK" FOR SELECT USING (true); +CREATE POLICY link_delete ON rls_vle_after_rls."LINK" FOR DELETE USING (true); + +SET ROLE rls_vle_user; +-- a traversal before the delete, for each Source: s1 -> m <- s2, m -> t +SELECT * FROM cypher('rls_vle_before_plain', $$ + MATCH (a:Source)-[:LINK*1..2]->(b) + DETACH DELETE b + RETURN a.i, b.i +$$) AS (source agtype, deleted agtype) ORDER BY source, deleted; +SELECT * FROM cypher('rls_vle_before_rls', $$ + MATCH (a:Source)-[:LINK*1..2]->(b) + DETACH DELETE b + RETURN a.i, b.i +$$) AS (source agtype, deleted agtype) ORDER BY source, deleted; + +-- a traversal after the delete of each node but n1, in the order of ord: +-- n1 -> n2 -> n3 and an unconnected n9, which is deleted first +SELECT * FROM cypher('rls_vle_after_plain', $$ + MATCH (d:Node) WHERE d.i > 1 + WITH d ORDER BY d.ord + DETACH DELETE d + WITH d + MATCH p = (n:Node {i: 1})-[:LINK*1..3]->() + RETURN d.i, length(p) +$$) AS (deleted agtype, path_length agtype) ORDER BY deleted, path_length; +SELECT * FROM cypher('rls_vle_after_rls', $$ + MATCH (d:Node) WHERE d.i > 1 + WITH d ORDER BY d.ord + DETACH DELETE d + WITH d + MATCH p = (n:Node {i: 1})-[:LINK*1..3]->() + RETURN d.i, length(p) +$$) AS (deleted agtype, path_length agtype) ORDER BY deleted, path_length; +RESET ROLE; + +-- the copies of the graph held while a traversal for each Source is +-- followed by a delete, counted after every row +CREATE FUNCTION public.rls_vle_graph_copies(anyelement) RETURNS bigint + LANGUAGE plpgsql VOLATILE SECURITY DEFINER +AS $f$ +BEGIN + RETURN (SELECT count(*) FROM pg_backend_memory_contexts + WHERE name = 'AGE edge_table'); +END +$f$; +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; +BEGIN; +GRANT DELETE ON ALL TABLES IN SCHEMA rls_vle_many TO rls_vle_user; +CREATE POLICY link_delete ON rls_vle_many."LINK" FOR DELETE USING (true); +SET LOCAL ROLE rls_vle_user; +SELECT count(*) AS deleted, max(copies) AS most_graph_copies_held +FROM (SELECT public.rls_vle_graph_copies(i) AS copies + FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) + DETACH DELETE b + RETURN a.i + $$) AS (i agtype)) AS per_row; +ROLLBACK; + +-- 8.8: a path value kept after its query is built from a graph context that +-- holds all of its vertices and edges. When no context does, building it +-- raises an error: after the caches are deleted, and for a role whose +-- context lacks the path's edges. +CREATE TABLE public.rls_vle_stored_paths AS +SELECT edges FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1'); +SELECT count(age_materialize_vle_path(edges)) AS stored_paths_built +FROM public.rls_vle_stored_paths; +SELECT age_delete_global_graphs(NULL) IS NOT NULL AS graph_caches_cleared; +SELECT count(age_materialize_vle_path(edges)) AS stored_paths_built +FROM public.rls_vle_stored_paths; +SELECT count(age_materialize_vle_edges(edges)) AS stored_edge_lists_built +FROM public.rls_vle_stored_paths; +GRANT SELECT ON public.rls_vle_stored_paths TO rls_vle_user2; +SET ROLE rls_vle_user2; +SELECT count(*) AS paths_visible_to_user2 +FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), '1', '1', '1'); +SELECT count(age_materialize_vle_path(edges)) AS stored_paths_built +FROM public.rls_vle_stored_paths; +SELECT count(age_materialize_vle_edges(edges)) AS stored_edge_lists_built +FROM public.rls_vle_stored_paths; +RESET ROLE; +DROP TABLE public.rls_vle_stored_paths; + +-- 8.9: a query that fails after its traversal releases its graph context +-- during the error cleanup, at the top level and in PL/pgSQL, so failed +-- queries leave no contexts behind +SET ROLE rls_vle_user; +SELECT 1 / ((n::text)::int - 20) AS fails_after_traversal +FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); +SELECT 1 / ((n::text)::int - 20) AS fails_after_traversal +FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (n agtype); +DO $d$ +BEGIN + PERFORM 1 / ((n::text)::int - 20) + FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) + $$) AS (n agtype); +EXCEPTION WHEN division_by_zero THEN + NULL; +END +$d$; +SELECT * FROM cypher('rls_vle_many', $$ + MATCH (a:Source)-[:LINK*1..1]->(b:Target) RETURN count(*) +$$) AS (paths agtype); +RESET ROLE; +SELECT count(*) AS contexts_after_failed_queries +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + +-- 8.10: deleting the graph caches while a query holds its graph retires the +-- context instead of freeing it. The query still builds its paths from it, +-- and it is freed when the query ends, also when the query then fails. +SET ROLE rls_vle_user; +SELECT count(age_materialize_vle_path(v.edges)) AS paths_built_after_delete +FROM (SELECT edges, row_number() OVER () AS rn + FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1')) AS v +WHERE CASE WHEN v.rn = 1 THEN age_delete_global_graphs(NULL) IS NOT NULL + ELSE true END; +RESET ROLE; +SELECT count(*) AS contexts_after_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; +SET ROLE rls_vle_user; +SELECT 1 / (count(age_materialize_vle_path(v.edges)) - 20) AS fails_after_paths +FROM (SELECT edges, row_number() OVER () AS rn + FROM age_vle('"rls_vle_many"', NULL, NULL, + age_build_vle_match_edge('"LINK"', NULL), + '1', '1', '1')) AS v +WHERE CASE WHEN v.rn = 1 + THEN age_delete_global_graphs('"rls_vle_many"') IS NOT NULL + ELSE true END; +RESET ROLE; +SELECT count(*) AS contexts_after_failed_query +FROM pg_backend_memory_contexts WHERE name = 'AGE edge_table'; + +DROP FUNCTION public.rls_vle_graph_copies(anyelement); +DROP FUNCTION public.rls_vle_start_as_user2(graphid, bigint); +DROP FUNCTION public.rls_vle_start_hiding_links(graphid, bigint); +DROP FUNCTION public.rls_vle_count_fixed(graphid); +DROP FUNCTION public.rls_vle_count_stable(); +DROP SEQUENCE public.rls_vle_policy_calls; +SELECT drop_graph('rls_vle_chain', true); +SELECT drop_graph('rls_vle_chain2', true); +SELECT drop_graph('rls_vle_many', true); +SELECT drop_graph('rls_vle_before_plain', true); +SELECT drop_graph('rls_vle_before_rls', true); +SELECT drop_graph('rls_vle_after_plain', true); +SELECT drop_graph('rls_vle_after_rls', true); + -- ============================================================================ -- cleanup -- ============================================================================ diff --git a/src/backend/utils/adt/age_global_graph.c b/src/backend/utils/adt/age_global_graph.c index 44163bfb..ce973075 100644 --- a/src/backend/utils/adt/age_global_graph.c +++ b/src/backend/utils/adt/age_global_graph.c @@ -38,6 +38,13 @@ #include "executor/spi.h" #include "lib/stringinfo.h" #include "miscadmin.h" +#include "optimizer/optimizer.h" +#include "catalog/pg_proc.h" +#include "nodes/nodeFuncs.h" +#include "utils/json.h" +#include "utils/jsonb.h" +#include "utils/jsonpath.h" +#include "utils/plancache.h" #include "utils/rls.h" #include "utils/acl.h" #include "utils/inval.h" @@ -170,7 +177,10 @@ typedef struct GRAPH_global_context bool loaded_rls_enforced; /* age.enforce_rls_in_traversal value at load */ bool vertices_rls_filtered; /* true if any vertex label was RLS-filtered at load */ bool loaded_with_rls; /* true if any label was loaded through RLS (SPI) */ + bool rls_quals_vary; /* RLS quals can vary in a statement */ bool security_invalidated; /* set by ACL/RLS catalog inval callbacks */ + bool retired; /* not offered to new acquisitions */ + int pin_count; /* queries still using this context */ List *label_table_oids; /* label table OIDs, for targeted security inval */ int64 num_loaded_vertices; /* number of loaded vertices in this graph */ int64 num_loaded_edges; /* number of loaded edges in this graph */ @@ -181,6 +191,24 @@ typedef struct GRAPH_global_context /* global variable to hold the per process GRAPH global contexts */ static GRAPH_global_context *global_graph_contexts = NULL; +/* + * A query that traverses a graph pins its context until the query's memory + * context is reset or deleted. Results of the traversal can be materialized + * after the traversal itself has finished, so a pinned context is retired + * instead of freed when it becomes invalid. + */ +typedef struct GRAPH_context_pin +{ + GRAPH_global_context *ggctx; /* pinned graph context */ + MemoryContext owner; /* per-query memory context holding it */ + const void *call_site; /* traversal call site that pinned it */ + MemoryContextCallback release; /* unpins when owner goes away */ + struct GRAPH_context_pin *next; +} GRAPH_context_pin; + +/* backend-local list of live pins */ +static GRAPH_context_pin *graph_context_pins = NULL; + /* * VertexEdgeArray helpers — flat-array adjacency container used by * vertex_entry's edges_in / edges_out / edges_self. @@ -257,6 +285,13 @@ static void load_vertex_label_rls(GRAPH_global_context *ggctx, static void load_edge_label_rls(GRAPH_global_context *ggctx, Oid edge_label_table_oid, char *edge_label_name); +static bool rls_function_not_immutable(Oid func_id, void *context); +static bool rls_quals_vary_walker(Node *node, void *context); +static bool rls_plan_quals_vary(SPIPlanPtr plan); +static void unlink_GRAPH_global_context(GRAPH_global_context *ggctx); +static void release_GRAPH_context_pin(void *arg); +static bool graph_context_reusable(GRAPH_context_pin *pin, + const void *call_site); /* definitions */ /* @@ -272,6 +307,12 @@ static void load_edge_label_rls(GRAPH_global_context *ggctx, */ bool is_ggctx_invalid(GRAPH_global_context *ggctx) { + /* a retired context is only kept for the queries still pinning it */ + if (ggctx->retired) + { + return true; + } + /* * Security-relevant cache keys, checked before the version-counter * fast-path below. A cache that applied row-level security is never safe @@ -786,6 +827,157 @@ static bool insert_vertex_edge(GRAPH_global_context *ggctx, return false; } +/* checker for check_functions_in_node: a function that is not IMMUTABLE */ +static bool rls_function_not_immutable(Oid func_id, void *context) +{ + return (func_volatile(func_id) != PROVOLATILE_IMMUTABLE); +} + +/* + * PostgreSQL's contain_mutable_functions_walker, except for the SQL value + * functions whose result cannot change during a statement without the reuse + * check noticing: the date and time ones are fixed for the transaction, + * CURRENT_ROLE, CURRENT_USER and USER are the current role, which the reuse + * check compares, and CURRENT_CATALOG is fixed for the session. SESSION_USER + * and CURRENT_SCHEMA can change within a statement. + */ +static bool rls_quals_vary_walker(Node *node, void *context) +{ + if (node == NULL) + { + return false; + } + + if (check_functions_in_node(node, rls_function_not_immutable, context)) + { + return true; + } + + if (IsA(node, JsonConstructorExpr)) + { + JsonConstructorExpr *ctor = (JsonConstructorExpr *) node; + bool is_jsonb = (ctor->returning->format->format_type == + JS_FORMAT_JSONB); + ListCell *lc; + + foreach (lc, ctor->args) + { + Oid typid = exprType(lfirst(lc)); + + if (is_jsonb ? !to_jsonb_is_immutable(typid) : + !to_json_is_immutable(typid)) + { + return true; + } + } + } + + if (IsA(node, JsonExpr)) + { + JsonExpr *jexpr = (JsonExpr *) node; + Const *cnst; + + if (!IsA(jexpr->path_spec, Const)) + { + return true; + } + + cnst = (Const *) jexpr->path_spec; + if (cnst->constisnull) + { + return false; + } + + if (jspIsMutable(DatumGetJsonPathP(cnst->constvalue), + jexpr->passing_names, jexpr->passing_values)) + { + return true; + } + } + + if (IsA(node, SQLValueFunction)) + { + switch (((SQLValueFunction *) node)->op) + { + case SVFOP_CURRENT_DATE: + case SVFOP_CURRENT_TIME: + case SVFOP_CURRENT_TIME_N: + case SVFOP_CURRENT_TIMESTAMP: + case SVFOP_CURRENT_TIMESTAMP_N: + case SVFOP_LOCALTIME: + case SVFOP_LOCALTIME_N: + case SVFOP_LOCALTIMESTAMP: + case SVFOP_LOCALTIMESTAMP_N: + case SVFOP_CURRENT_ROLE: + case SVFOP_CURRENT_USER: + case SVFOP_USER: + case SVFOP_CURRENT_CATALOG: + return false; + default: + return true; + } + } + + if (IsA(node, NextValueExpr)) + { + return true; + } + + if (IsA(node, Query)) + { + return query_tree_walker((Query *) node, rls_quals_vary_walker, + context, 0); + } + + return expression_tree_walker(node, rls_quals_vary_walker, context); +} + +/* + * Whether the row-security quals the planner applied to an RLS load can give + * another result later in the same statement, so that another scan of the + * label could return other rows. They can when they call a function that is + * not IMMUTABLE: a volatile one, or a STABLE one such as current_setting(), + * whose input the statement itself can change with set_config(). Policies + * from row_security_policy_hook_permissive and _restrictive are turned into + * the same securityQuals as catalog policies, so they are inspected too. A + * plan whose queries cannot be inspected is reported as varying; reuse is + * then limited to the call site that loaded the graph, which is valid for any + * policy. + */ +static bool rls_plan_quals_vary(SPIPlanPtr plan) +{ + List *sources; + ListCell *source_cell; + + sources = SPI_plan_get_plan_sources(plan); + if (sources == NIL) + { + return true; + } + + foreach (source_cell, sources) + { + CachedPlanSource *source = lfirst(source_cell); + ListCell *query_cell; + + if (source->query_list == NIL) + { + return true; + } + + foreach (query_cell, source->query_list) + { + /* walks the range table's securityQuals and any sublinks */ + if (rls_quals_vary_walker(lfirst(query_cell), NULL)) + { + return true; + } + } + } + + return false; +} + /* * RLS-aware loader for a single vertex label table. * @@ -837,6 +1029,11 @@ static void load_vertex_label_rls(GRAPH_global_context *ggctx, SPI_result_code_string(SPI_result)))); } + if (rls_plan_quals_vary(plan)) + { + ggctx->rls_quals_vary = true; + } + /* read-only cursor: reuses the active snapshot; RLS applied by planner */ portal = SPI_cursor_open(NULL, plan, NULL, NULL, true); @@ -926,6 +1123,11 @@ static void load_edge_label_rls(GRAPH_global_context *ggctx, SPI_result_code_string(SPI_result)))); } + if (rls_plan_quals_vary(plan)) + { + ggctx->rls_quals_vary = true; + } + /* read-only cursor: reuses the active snapshot; RLS applied by planner */ portal = SPI_cursor_open(NULL, plan, NULL, NULL, true); @@ -1608,6 +1810,15 @@ GRAPH_global_context *manage_GRAPH_global_contexts(char *graph_name, { bool success = false; + /* a query still using it keeps it; it is freed when unpinned */ + if (curr_ggctx->pin_count > 0) + { + curr_ggctx->retired = true; + prev_ggctx = curr_ggctx; + curr_ggctx = next_ggctx; + continue; + } + /* * If prev_ggctx is NULL then we are freeing the top of the * contexts. So, we need to point the contexts variable to the @@ -1646,7 +1857,7 @@ GRAPH_global_context *manage_GRAPH_global_contexts(char *graph_name, curr_ggctx = global_graph_contexts; while (curr_ggctx != NULL) { - if (curr_ggctx->graph_oid == graph_oid) + if (curr_ggctx->graph_oid == graph_oid && !curr_ggctx->retired) { /* switch our context back */ MemoryContextSwitchTo(oldctx); @@ -1736,47 +1947,60 @@ GRAPH_global_context *manage_GRAPH_global_contexts(char *graph_name, * through a SECURITY DEFINER RLS policy can attach a context loaded as a * different role; returning it here would expose rows the current role may * not be allowed to see. If the existing context is not reusable, detach - * and discard it so the list keeps a single, correct context per graph. + * and discard it so the list keeps a single, correct context per graph; + * a context that a query still pins is retired instead. */ prev_ggctx = NULL; curr_ggctx = global_graph_contexts; while (curr_ggctx != NULL) { - if (curr_ggctx->graph_oid == graph_oid) + GRAPH_global_context *next_ggctx = curr_ggctx->next; + + if (curr_ggctx->graph_oid != graph_oid || curr_ggctx->retired) { - if (!is_ggctx_invalid(curr_ggctx)) - { - (void) free_specific_GRAPH_global_context(new_ggctx); - MemoryContextSwitchTo(oldctx); - return curr_ggctx; - } + prev_ggctx = curr_ggctx; + curr_ggctx = next_ggctx; + continue; + } - /* detach the incompatible context and discard it */ - if (prev_ggctx == NULL) - { - global_graph_contexts = curr_ggctx->next; - } - else - { - prev_ggctx->next = curr_ggctx->next; - } - if (!free_specific_GRAPH_global_context(curr_ggctx)) - { - /* - * The detached context was internally inconsistent (a - * vertex or edge entry went missing). Free our freshly- - * built context first so it does not leak, then surface - * the corruption as the other cleanup paths in this file - * do. - */ - (void) free_specific_GRAPH_global_context(new_ggctx); - ereport(ERROR, (errcode(ERRCODE_DATA_EXCEPTION), - errmsg("missing vertex or edge entry during free"))); - } - break; + if (!is_ggctx_invalid(curr_ggctx)) + { + (void) free_specific_GRAPH_global_context(new_ggctx); + MemoryContextSwitchTo(oldctx); + return curr_ggctx; } - prev_ggctx = curr_ggctx; - curr_ggctx = curr_ggctx->next; + + if (curr_ggctx->pin_count > 0) + { + curr_ggctx->retired = true; + prev_ggctx = curr_ggctx; + curr_ggctx = next_ggctx; + continue; + } + + /* detach the incompatible context and discard it */ + if (prev_ggctx == NULL) + { + global_graph_contexts = next_ggctx; + } + else + { + prev_ggctx->next = next_ggctx; + } + if (!free_specific_GRAPH_global_context(curr_ggctx)) + { + /* + * The detached context was internally inconsistent (a + * vertex or edge entry went missing). Free our freshly- + * built context first so it does not leak, then surface + * the corruption as the other cleanup paths in this file + * do. + */ + (void) free_specific_GRAPH_global_context(new_ggctx); + ereport(ERROR, (errcode(ERRCODE_DATA_EXCEPTION), + errmsg("missing vertex or edge entry during free"))); + } + curr_ggctx = next_ggctx; } /* prepend our freshly-built context to the shared list */ @@ -1791,12 +2015,12 @@ GRAPH_global_context *manage_GRAPH_global_contexts(char *graph_name, /* * Helper function to delete all of the global graph contexts used by the - * process. When done the global global_graph_contexts will be NULL. - * - * + * process. Contexts that a query still pins are retired instead and freed + * when the last pin is released. */ static bool delete_GRAPH_global_contexts(void) { + GRAPH_global_context *prev_ggctx = NULL; GRAPH_global_context *curr_ggctx = NULL; bool retval = false; @@ -1810,6 +2034,25 @@ static bool delete_GRAPH_global_contexts(void) GRAPH_global_context *next_ggctx = curr_ggctx->next; bool success = false; + retval = true; + + if (curr_ggctx->pin_count > 0) + { + curr_ggctx->retired = true; + prev_ggctx = curr_ggctx; + curr_ggctx = next_ggctx; + continue; + } + + if (prev_ggctx == NULL) + { + global_graph_contexts = next_ggctx; + } + else + { + prev_ggctx->next = next_ggctx; + } + /* free the current graph context */ success = free_specific_GRAPH_global_context(curr_ggctx); @@ -1823,26 +2066,21 @@ static bool delete_GRAPH_global_contexts(void) /* advance to the next context */ curr_ggctx = next_ggctx; - - retval = true; } - /* reset the head of the contexts to NULL */ - global_graph_contexts = NULL; - - return retval; } /* - * Helper function to delete a specific global graph context used by the - * process. + * Helper function to delete the global graph contexts of a specific graph. + * Contexts that a query still pins are retired instead. */ static bool delete_specific_GRAPH_global_contexts(char *graph_name) { GRAPH_global_context *prev_ggctx = NULL; GRAPH_global_context *curr_ggctx = NULL; Oid graph_oid = InvalidOid; + bool found = false; if (graph_name == NULL) { @@ -1856,7 +2094,7 @@ static bool delete_specific_GRAPH_global_contexts(char *graph_name) /* get the first context, if any */ curr_ggctx = global_graph_contexts; - /* find the specified GRAPH global context */ + /* find the specified GRAPH global contexts */ while (curr_ggctx != NULL) { GRAPH_global_context *next_ggctx = curr_ggctx->next; @@ -1864,6 +2102,17 @@ static bool delete_specific_GRAPH_global_contexts(char *graph_name) if (curr_ggctx->graph_oid == graph_oid) { bool success = false; + + found = true; + + if (curr_ggctx->pin_count > 0) + { + curr_ggctx->retired = true; + prev_ggctx = curr_ggctx; + curr_ggctx = next_ggctx; + continue; + } + /* * If prev_ggctx is NULL then we are freeing the top of the * contexts. So, we need to point the global variable to the @@ -1875,7 +2124,7 @@ static bool delete_specific_GRAPH_global_contexts(char *graph_name) } else { - prev_ggctx->next = curr_ggctx->next; + prev_ggctx->next = next_ggctx; } /* free the current graph context */ @@ -1889,8 +2138,8 @@ static bool delete_specific_GRAPH_global_contexts(char *graph_name) errmsg("missing vertex_entry during free"))); } - /* we found and freed it, return true */ - return true; + curr_ggctx = next_ggctx; + continue; } /* save the current as previous and advance to the next one */ @@ -1898,9 +2147,7 @@ static bool delete_specific_GRAPH_global_contexts(char *graph_name) curr_ggctx = next_ggctx; } - - /* we didn't find it, return false */ - return false; + return found; } /* @@ -1965,7 +2212,7 @@ GRAPH_global_context *find_GRAPH_global_context(Oid graph_oid) while(ggctx != NULL) { /* if we found it return it */ - if (ggctx->graph_oid == graph_oid) + if (ggctx->graph_oid == graph_oid && !ggctx->retired) { return ggctx; @@ -1980,6 +2227,233 @@ GRAPH_global_context *find_GRAPH_global_context(Oid graph_oid) return NULL; } +/* + * Find a graph context for materializing a VLE path. The path may come from a + * traversal whose context has since been retired, so retired contexts that a + * query still pins are considered too. The first context that holds every + * vertex and edge of the path is returned, or NULL if there is none. + */ +GRAPH_global_context *find_GRAPH_global_context_for_path(Oid graph_oid, + const graphid *ids, + int64 count) +{ + GRAPH_global_context *ggctx = NULL; + + for (ggctx = global_graph_contexts; ggctx != NULL; ggctx = ggctx->next) + { + bool complete = true; + int64 i; + + if (ggctx->graph_oid != graph_oid) + { + continue; + } + + for (i = 0; i < count; i++) + { + /* + * get_edge_entry() asserts that the edge exists; a context may + * lack it here, so look the edge up directly. + */ + if ((i % 2 == 0 && get_vertex_entry(ggctx, ids[i]) == NULL) || + (i % 2 == 1 && + agehash_lookup(ggctx->edge_table, &ids[i]) == NULL)) + { + complete = false; + break; + } + } + + if (complete) + { + return ggctx; + } + } + + return NULL; +} + +/* remove a context from the backend-local list without freeing it */ +static void unlink_GRAPH_global_context(GRAPH_global_context *ggctx) +{ + GRAPH_global_context **link = &global_graph_contexts; + + while (*link != NULL && *link != ggctx) + { + link = &(*link)->next; + } + + if (*link == ggctx) + { + *link = ggctx->next; + } + + ggctx->next = NULL; +} + +/* + * Memory-context callback for a pin. Runs when the query's memory context is + * reset or deleted, including during error cleanup. A retired context is + * freed once no query pins it. It only uses pfree, hash_destroy and + * MemoryContextDelete, so it is safe in an aborted transaction. + */ +static void release_GRAPH_context_pin(void *arg) +{ + GRAPH_context_pin *pin = (GRAPH_context_pin *) arg; + GRAPH_context_pin **link = &graph_context_pins; + GRAPH_global_context *ggctx = NULL; + + while (*link != NULL && *link != pin) + { + link = &(*link)->next; + } + + if (*link == NULL) + { + return; + } + + *link = pin->next; + ggctx = pin->ggctx; + + if (ggctx->pin_count > 0) + { + ggctx->pin_count--; + } + + if (ggctx->pin_count == 0 && ggctx->retired) + { + unlink_GRAPH_global_context(ggctx); + (void) free_specific_GRAPH_global_context(ggctx); + } +} + +/* + * Keep a graph context alive until the given per-query memory context goes + * away, recording the traversal call site that pinned it. Pinning the same + * context twice for one query is a no-op. + */ +void pin_GRAPH_global_context(GRAPH_global_context *ggctx, MemoryContext owner, + const void *call_site) +{ + GRAPH_context_pin *pin = NULL; + + if (ggctx == NULL || owner == NULL) + { + return; + } + + for (pin = graph_context_pins; pin != NULL; pin = pin->next) + { + if (pin->ggctx == ggctx && pin->owner == owner) + { + return; + } + } + + pin = MemoryContextAllocZero(owner, sizeof(GRAPH_context_pin)); + pin->ggctx = ggctx; + pin->owner = owner; + pin->call_site = call_site; + pin->release.func = release_GRAPH_context_pin; + pin->release.arg = pin; + MemoryContextRegisterResetCallback(owner, &pin->release); + + pin->next = graph_context_pins; + graph_context_pins = pin; + ggctx->pin_count++; +} + +/* + * Whether a graph context this query already pins can serve another + * acquisition in the same query, under the semantics PostgreSQL gives a + * statement's scans: + * + * - A context loaded with another role, enforcement setting or snapshot is a + * different scan. + * - A later command id means that a traversal which has not loaded the graph + * yet must see the statement's own writes. The traversal that loaded it + * keeps it while the statement writes, as a traversal of a graph without + * RLS does, instead of loading the graph again for every row. + * - Policies and privileges hold for the whole statement: PostgreSQL applies + * the policies when it plans the statement and checks the privileges when + * it starts it, so an authorization change made while the statement runs + * does not end the reuse. + * - When the RLS quals give the same result throughout the statement, every + * scan of the labels in this statement returns the same rows, so any + * traversal may share the graph (see rls_plan_quals_vary). + * - Otherwise only the traversal call site that loaded the graph shares it, + * as when PostgreSQL scans a relation once and rescans the result. Another + * call site is a separate scan and evaluates the policies itself. + */ +static bool graph_context_reusable(GRAPH_context_pin *pin, + const void *call_site) +{ + GRAPH_global_context *ggctx = pin->ggctx; + Snapshot snapshot = GetActiveSnapshot(); + + if (ggctx->load_as_role != GetUserId() || + ggctx->loaded_rls_enforced != age_enforce_rls_in_traversal || + ggctx->xmin != snapshot->xmin || + ggctx->xmax != snapshot->xmax) + { + return false; + } + + if (ggctx->curcid != snapshot->curcid && + (call_site == NULL || pin->call_site != call_site)) + { + return false; + } + + if (ggctx->rls_quals_vary && + (call_site == NULL || pin->call_site != call_site)) + { + return false; + } + + return true; +} + +/* + * Acquire a graph context for a traversal and pin it for the rest of the + * query that owns query_owner, so a later acquisition cannot free it while + * the traversal or the materialization of its results still needs it. + * call_site identifies the traversal within the query; it stays the same + * when the traversal is evaluated again for another row. A context the query + * already pins is reused when graph_context_reusable() allows it, so the + * policies run once for each row a load scans rather than again for every + * traversal. + */ +GRAPH_global_context *acquire_GRAPH_global_context(char *graph_name, + Oid graph_oid, + MemoryContext query_owner, + const void *call_site) +{ + GRAPH_global_context *ggctx = NULL; + GRAPH_context_pin *pin = NULL; + + if (query_owner == NULL) + { + return manage_GRAPH_global_contexts(graph_name, graph_oid); + } + + for (pin = graph_context_pins; pin != NULL; pin = pin->next) + { + if (pin->owner == query_owner && + pin->ggctx->graph_oid == graph_oid && + graph_context_reusable(pin, call_site)) + { + return pin->ggctx; + } + } + + ggctx = manage_GRAPH_global_contexts(graph_name, graph_oid); + pin_GRAPH_global_context(ggctx, query_owner, call_site); + + return ggctx; +} + /* graph vertices accessor */ ListGraphId *get_graph_vertices(GRAPH_global_context *ggctx) { diff --git a/src/backend/utils/adt/age_vle.c b/src/backend/utils/adt/age_vle.c index cb036b15..d3f22de5 100644 --- a/src/backend/utils/adt/age_vle.c +++ b/src/backend/utils/adt/age_vle.c @@ -639,6 +639,14 @@ static VLE_local_context *build_local_vle_context(FunctionCallInfo fcinfo, Oid graph_oid = InvalidOid; int64 vle_grammar_node_id = 0; bool use_cache = false; + ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; + MemoryContext query_owner = NULL; + + /* the graph context stays pinned for the rest of this query */ + if (rsinfo != NULL && IsA(rsinfo, ReturnSetInfo) && rsinfo->econtext != NULL) + { + query_owner = rsinfo->econtext->ecxt_per_query_memory; + } /* * Get the VLE grammar node id, if it exists. Remember, we overload the @@ -668,6 +676,8 @@ static VLE_local_context *build_local_vle_context(FunctionCallInfo fcinfo, * be stored in that context since the memory is allocated there. */ + pin_GRAPH_global_context(vlelctx->ggctx, query_owner, fcinfo->flinfo); + /* get and update the start vertex id */ if (PG_ARGISNULL(1) || is_agtype_null(AG_GET_ARG_AGTYPE_P(1))) { @@ -758,10 +768,13 @@ static VLE_local_context *build_local_vle_context(FunctionCallInfo fcinfo, graph_oid = get_graph_oid(graph_name); /* - * Create or retrieve the GRAPH global context for this graph. This function - * will also purge off invalidated contexts. + * Create or retrieve the GRAPH global context for this graph and pin it + * for this query. This function will also purge off invalidated contexts. + * The function's FmgrInfo identifies this traversal within the query; it + * stays the same when the traversal is evaluated again for another row. */ - ggctx = manage_GRAPH_global_contexts(graph_name, graph_oid); + ggctx = acquire_GRAPH_global_context(graph_name, graph_oid, query_owner, + fcinfo->flinfo); /* allocate and initialize local VLE context */ vlelctx = palloc0(sizeof(VLE_local_context)); @@ -1754,15 +1767,20 @@ static agtype_value *build_edge_list(VLE_path_container *vpc) /* get the graph_oid */ graph_oid = vpc->graph_oid; - /* get the GRAPH global context for this graph */ - ggctx = find_GRAPH_global_context(graph_oid); - /* verify we got a global context */ - Assert(ggctx != NULL); - /* get the graphid_array and size */ graphid_array = GET_GRAPHID_ARRAY_FROM_CONTAINER(vpc); graphid_array_size = vpc->graphid_array_size; + /* get a GRAPH global context that holds every entity of this path */ + ggctx = find_GRAPH_global_context_for_path(graph_oid, graphid_array, + graphid_array_size); + if (ggctx == NULL) + { + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("graph data for the referenced VLE path is not available"))); + } + /* initialize our agtype array */ MemSet(&edges_result, 0, sizeof(agtype_in_state)); edges_result.res = push_agtype_value(&edges_result.parse_state, @@ -1818,15 +1836,20 @@ static agtype_value *build_path(VLE_path_container *vpc) /* get the graph_oid */ graph_oid = vpc->graph_oid; - /* get the GRAPH global context for this graph */ - ggctx = find_GRAPH_global_context(graph_oid); - /* verify we got a global context */ - Assert(ggctx != NULL); - /* get the graphid_array and size */ graphid_array = GET_GRAPHID_ARRAY_FROM_CONTAINER(vpc); graphid_array_size = vpc->graphid_array_size; + /* get a GRAPH global context that holds every entity of this path */ + ggctx = find_GRAPH_global_context_for_path(graph_oid, graphid_array, + graphid_array_size); + if (ggctx == NULL) + { + ereport(ERROR, + (errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("graph data for the referenced VLE path is not available"))); + } + /* initialize our agtype array */ MemSet(&path_result, 0, sizeof(agtype_in_state)); path_result.res = push_agtype_value(&path_result.parse_state, diff --git a/src/include/utils/age_global_graph.h b/src/include/utils/age_global_graph.h index b08b0d18..c3d355ed 100644 --- a/src/include/utils/age_global_graph.h +++ b/src/include/utils/age_global_graph.h @@ -58,7 +58,16 @@ typedef struct GRAPH_global_context GRAPH_global_context; /* GRAPH global context functions */ GRAPH_global_context *manage_GRAPH_global_contexts(char *graph_name, Oid graph_oid); +GRAPH_global_context *acquire_GRAPH_global_context(char *graph_name, + Oid graph_oid, + MemoryContext query_owner, + const void *call_site); +void pin_GRAPH_global_context(GRAPH_global_context *ggctx, + MemoryContext owner, const void *call_site); GRAPH_global_context *find_GRAPH_global_context(Oid graph_oid); +GRAPH_global_context *find_GRAPH_global_context_for_path(Oid graph_oid, + const graphid *ids, + int64 count); bool is_ggctx_invalid(GRAPH_global_context *ggctx); /* GRAPH retrieval functions */ ListGraphId *get_graph_vertices(GRAPH_global_context *ggctx);
