My nbtree patch [1] needs to call index_getprocinfo() with an exclusive buffer lock held during a leaf page split. This has an undetectable self-deadlock (LWLock self-deadlock) risk: a syscache lookup against pg_proc might have a catcache miss, ending with an index scan that needs to access the very same buffer. That's not acceptable.
There is very similar code to this in SP-GiST: there is a index_getprocinfo() call within doPickSplit(), to get the user-defined method for a split (SPGIST_PICKSPLIT_PROC). My nbtree patch builds a new insertion scankey to determine how many attributes we can safely truncate away in new pivot tuples -- it would be tricky to do this lookup outside of the split function. I suppose that it's okay to do this in SP-GiST without special care because there cannot be an SP-GiST index on a system catalog. I'll need to do something else about it given that I'm doing this within nbtree, though -- I don't want to complicate the code that deals with insertion scankeys to make this work. Here is a strategy that fixes the problem without complicating matters for nbtree: It should be safe if I make a point of using a special comparator (the bitwise one that we already use in other contexts in the patch) with system catalog indexes. We know that they cannot be of types that have a varlena header + typstorage != 'p', which ensures that there are no cases where bitwise equality fails to be a reliable indicator of opclass equality (e.g. there are no cases like numeric display scale). We could make sure that this requirement isn't violated in the future by adding a pg_index test to opr_sanity.sql, limiting system catalog indexes to opclasses that are known-safe for the bitwise comparator. Does that seem sane? [1] https://commitfest.postgresql.org/21/1787/ -- Peter Geoghegan
