Hi Whenever I'm debugging some kind of corruption incident, possible visibility bug, etc, I always land up staring at integer infomasks or using a SQL helper function to decode them.
That's silly, so here's a patch to teach pageinspect how to decode infomasks to a human readable array of flag names. Example: SELECT t_infomask, t_infomask2, flags FROM heap_page_items(get_raw_page('test1', 0)), LATERAL heap_infomask_flags(t_infomask, t_infomask2, true) m(flags); t_infomask | t_infomask2 | flags ------------+-------------+---------------------------------------------------------------------------- 2816 | 2 | {HEAP_XMIN_COMMITTED,HEAP_XMIN_INVALID,HEAP_XMAX_INVALID,HEAP_XMIN_FROZEN} (1 row) To decode individual mask integers you can just call it directly. It's strict, so pass 0 for the other mask if you don't have both, e.g. SELECT heap_infomask_flags(2816, 0); The patch backports easily to older pageinspect versions for when you're debugging something old. BTW, I used text[] not enums. That costs a fair bit of memory, but it doesn't seem worth worrying too much about in this context. For convenience it also tests and reports HEAP_LOCKED_UPGRADED and HEAP_XMAX_IS_LOCKED_ONLY as pseudo-flags. I decided not to filter out HEAP_XMIN_COMMITTED,HEAP_XMIN_INVALID,HEAP_XMAX_INVALID when HEAP_XMIN_FROZEN is set; that doesn't make sense when we examine HEAP_XMAX_IS_LOCKED_ONLY or HEAP_LOCKED_UPGRADED, and filtering them out could be just as confusing as leaving them in. The infomask2 natts mask is ignored. You can bitwise-and it out in SQL pretty easily if needed. I could output it here as a constructed text datum, but it seems mostly pointless. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
From 488a1f69b8082258d508ba681a4f4a5f6fce2267 Mon Sep 17 00:00:00 2001 From: Craig Ringer <cr...@2ndquadrant.com> Date: Thu, 20 Jul 2017 11:20:21 +0800 Subject: [PATCH v1] Introduce heap_infomask_flags to decode infomask and infomask2 --- contrib/pageinspect/Makefile | 3 +- contrib/pageinspect/expected/page.out | 25 ++++++ contrib/pageinspect/heapfuncs.c | 120 ++++++++++++++++++++++++++ contrib/pageinspect/pageinspect--1.6--1.7.sql | 9 ++ contrib/pageinspect/pageinspect.control | 2 +- contrib/pageinspect/sql/page.sql | 14 +++ doc/src/sgml/pageinspect.sgml | 32 +++++++ 7 files changed, 203 insertions(+), 2 deletions(-) create mode 100644 contrib/pageinspect/pageinspect--1.6--1.7.sql diff --git a/contrib/pageinspect/Makefile b/contrib/pageinspect/Makefile index 0a3cbee..de114c7 100644 --- a/contrib/pageinspect/Makefile +++ b/contrib/pageinspect/Makefile @@ -5,7 +5,8 @@ OBJS = rawpage.o heapfuncs.o btreefuncs.o fsmfuncs.o \ brinfuncs.o ginfuncs.o hashfuncs.o $(WIN32RES) EXTENSION = pageinspect -DATA = pageinspect--1.5.sql pageinspect--1.5--1.6.sql \ +DATA = pageinspect--1.6--1.7.sql \ + pageinspect--1.5.sql pageinspect--1.5--1.6.sql \ pageinspect--1.4--1.5.sql pageinspect--1.3--1.4.sql \ pageinspect--1.2--1.3.sql pageinspect--1.1--1.2.sql \ pageinspect--1.0--1.1.sql pageinspect--unpackaged--1.0.sql diff --git a/contrib/pageinspect/expected/page.out b/contrib/pageinspect/expected/page.out index 8e15947..054c69d 100644 --- a/contrib/pageinspect/expected/page.out +++ b/contrib/pageinspect/expected/page.out @@ -82,6 +82,31 @@ SELECT * FROM fsm_page_contents(get_raw_page('test1', 'fsm', 0)); (1 row) +-- If we freeze the only tuple on test1, the infomask should +-- always be the same in all test runs. +VACUUM FREEZE test1; +SELECT t_infomask, t_infomask2, flags +FROM heap_page_items(get_raw_page('test1', 0)), + LATERAL heap_infomask_flags(t_infomask, t_infomask2, true) m(flags); + t_infomask | t_infomask2 | flags +------------+-------------+---------------------------------------------------------------------------- + 2816 | 2 | {HEAP_XMIN_COMMITTED,HEAP_XMIN_INVALID,HEAP_XMAX_INVALID,HEAP_XMIN_FROZEN} +(1 row) + +SELECT t_infomask, t_infomask2, flags +FROM heap_page_items(get_raw_page('test1', 0)), + LATERAL heap_infomask_flags(t_infomask, t_infomask2, false) m(flags); + t_infomask | t_infomask2 | flags +------------+-------------+----------------------------------------------------------- + 2816 | 2 | {HEAP_XMIN_COMMITTED,HEAP_XMIN_INVALID,HEAP_XMAX_INVALID} +(1 row) + +SELECT heap_infomask_flags(2816, 0); + heap_infomask_flags +----------------------------------------------------------- + {HEAP_XMIN_COMMITTED,HEAP_XMIN_INVALID,HEAP_XMAX_INVALID} +(1 row) + DROP TABLE test1; -- check that using any of these functions with a partitioned table would fail create table test_partitioned (a int) partition by range (a); diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c index 72d1776..17bea2a 100644 --- a/contrib/pageinspect/heapfuncs.c +++ b/contrib/pageinspect/heapfuncs.c @@ -478,3 +478,123 @@ tuple_data_split(PG_FUNCTION_ARGS) PG_RETURN_ARRAYTYPE_P(res); } + +/* + * Brian Kernighan's popcount algorithm for counting number of set bits in a + * mask. The faster methods aren't worth the complexity here. + * + * See e.g. http://graphics.stanford.edu/~seander/bithacks.html#CountBitsSetKernighan + * + * gcc has __builtin_popcount but there's no standard alternative. + */ +static inline int +pg_popcount32(uint32 mask) +{ + unsigned int c; // c accumulates the total bits set in v + for (c = 0; mask; c++) + { + mask &= mask - 1; // clear the least significant bit set + } + return c; +} + +/* + * Infomask flag names. The infomask2 values are shifted into the + * high 16 bits. + */ +struct infomask_details +{ + uint32 flag_value; + const char flag_name[24]; +}; + +#define MASKINFO_LENGTH 19 +static struct infomask_details maskinfo[MASKINFO_LENGTH] = + { + /* infomask1 values */ + {(uint32)HEAP_HASNULL, "HEAP_HASNULL"}, + {(uint32)HEAP_HASVARWIDTH, "HEAP_HASVARWIDTH"}, + {(uint32)HEAP_HASEXTERNAL, "HEAP_HASEXTERNAL"}, + {(uint32)HEAP_HASOID, "HEAP_HASOID"}, + {(uint32)HEAP_XMAX_KEYSHR_LOCK, "HEAP_XMAX_KEYSHR_LOCK"}, + {(uint32)HEAP_COMBOCID, "HEAP_COMBOCID"}, + {(uint32)HEAP_XMAX_EXCL_LOCK, "HEAP_XMAX_EXCL_LOCK"}, + {(uint32)HEAP_XMAX_LOCK_ONLY, "HEAP_XMAX_LOCK_ONLY"}, + {(uint32)HEAP_XMIN_COMMITTED, "HEAP_XMIN_COMMITTED"}, + {(uint32)HEAP_XMIN_INVALID, "HEAP_XMIN_INVALID"}, + {(uint32)HEAP_XMAX_COMMITTED, "HEAP_XMAX_COMMITTED"}, + {(uint32)HEAP_XMAX_INVALID, "HEAP_XMAX_INVALID"}, + {(uint32)HEAP_XMAX_IS_MULTI, "HEAP_XMAX_IS_MULTI"}, + {(uint32)HEAP_UPDATED, "HEAP_UPDATED"}, + {(uint32)HEAP_MOVED_OFF, "HEAP_MOVED_OFF"}, + {(uint32)HEAP_MOVED_IN, "HEAP_MOVED_IN"}, + /* infomask2 values */ + {((uint32)HEAP_KEYS_UPDATED)<<16, "HEAP_KEYS_UPDATED"}, + {((uint32)HEAP_HOT_UPDATED)<<16, "HEAP_HOT_UPDATED"}, + {((uint32)HEAP_ONLY_TUPLE)<<16, "HEAP_ONLY_TUPLE"}, + }; + +/* + * Decode an infomask, per htup_details.c, into human readable + * form. + */ +PG_FUNCTION_INFO_V1(heap_infomask_flags); + +Datum +heap_infomask_flags(PG_FUNCTION_ARGS) +{ + uint16 t_infomask = PG_GETARG_INT16(0); + uint16 t_infomask2 = PG_GETARG_INT16(1); + bool include_combined = PG_GETARG_BOOL(2); + uint32 combomask = (((uint32)t_infomask2) << 16) + (uint32)t_infomask; + unsigned int maxarray = pg_popcount32(combomask); + Datum *d; + ArrayType *a; + int i; + int insertpos; + + Assert((t_infomask == 0 && t_infomask2 == 0) == (maxarray == 0)); + + if (maxarray == 0) + { + a = construct_empty_array(TEXTOID); + PG_RETURN_POINTER(a); + } + + if (include_combined) + { + maxarray += (t_infomask & HEAP_XMIN_FROZEN) == HEAP_XMIN_FROZEN; + maxarray += HEAP_XMAX_IS_LOCKED_ONLY(t_infomask); + maxarray += HEAP_LOCKED_UPGRADED(t_infomask); + } + + d = (Datum *) palloc(sizeof(Datum) * maxarray); + + insertpos = 0; + for (i = 0; i < MASKINFO_LENGTH; ++i) + { + if ((combomask & maskinfo[i].flag_value) == maskinfo[i].flag_value) + d[insertpos++] = CStringGetTextDatum(maskinfo[i].flag_name); + } + + /* + * These tests are useful to report in the mask we output, since they're + * much more simply done here than in SQL, and here they won't get out of + * sync with what Pg does if we change it later. + */ + if (include_combined) + { + if ((t_infomask & HEAP_XMIN_FROZEN) == HEAP_XMIN_FROZEN) + d[insertpos++] = CStringGetTextDatum("HEAP_XMIN_FROZEN"); + + if (HEAP_XMAX_IS_LOCKED_ONLY(t_infomask)) + d[insertpos++] = CStringGetTextDatum("HEAP_XMAX_IS_LOCKED_ONLY"); + + if (HEAP_LOCKED_UPGRADED(t_infomask)) + d[insertpos++] = CStringGetTextDatum("HEAP_LOCKED_UPGRADED"); + } + + a = construct_array(d, maxarray - 1, TEXTOID, -1, false, 'i'); + + PG_RETURN_POINTER(a); +} diff --git a/contrib/pageinspect/pageinspect--1.6--1.7.sql b/contrib/pageinspect/pageinspect--1.6--1.7.sql new file mode 100644 index 0000000..b3d1fe4 --- /dev/null +++ b/contrib/pageinspect/pageinspect--1.6--1.7.sql @@ -0,0 +1,9 @@ +/* contrib/pageinspect/pageinspect--1.6--1.7.sql */ + +-- complain if script is sourced in psql, rather than via ALTER EXTENSION +\echo Use "ALTER EXTENSION pageinspect UPDATE TO '1.7'" to load this file. \quit + +-- decode infomask flags as human readable flag names +CREATE FUNCTION heap_infomask_flags(infomask1 integer, infomask2 integer, + include_combined boolean DEFAULT true) +RETURNS text[] STRICT LANGUAGE 'c' AS 'MODULE_PATHNAME','heap_infomask_flags'; diff --git a/contrib/pageinspect/pageinspect.control b/contrib/pageinspect/pageinspect.control index 1a61c9f..dcfc61f 100644 --- a/contrib/pageinspect/pageinspect.control +++ b/contrib/pageinspect/pageinspect.control @@ -1,5 +1,5 @@ # pageinspect extension comment = 'inspect the contents of database pages at a low level' -default_version = '1.6' +default_version = '1.7' module_pathname = '$libdir/pageinspect' relocatable = true diff --git a/contrib/pageinspect/sql/page.sql b/contrib/pageinspect/sql/page.sql index 493ca9b..b7a1f0b 100644 --- a/contrib/pageinspect/sql/page.sql +++ b/contrib/pageinspect/sql/page.sql @@ -31,6 +31,20 @@ SELECT tuple_data_split('test1'::regclass, t_data, t_infomask, t_infomask2, t_bi SELECT * FROM fsm_page_contents(get_raw_page('test1', 'fsm', 0)); +-- If we freeze the only tuple on test1, the infomask should +-- always be the same in all test runs. +VACUUM FREEZE test1; + +SELECT t_infomask, t_infomask2, flags +FROM heap_page_items(get_raw_page('test1', 0)), + LATERAL heap_infomask_flags(t_infomask, t_infomask2, true) m(flags); + +SELECT t_infomask, t_infomask2, flags +FROM heap_page_items(get_raw_page('test1', 0)), + LATERAL heap_infomask_flags(t_infomask, t_infomask2, false) m(flags); + +SELECT heap_infomask_flags(2816, 0); + DROP TABLE test1; -- check that using any of these functions with a partitioned table would fail diff --git a/doc/src/sgml/pageinspect.sgml b/doc/src/sgml/pageinspect.sgml index ccdaf3e..24925a0 100644 --- a/doc/src/sgml/pageinspect.sgml +++ b/doc/src/sgml/pageinspect.sgml @@ -151,6 +151,10 @@ test=# SELECT * FROM heap_page_items(get_raw_page('pg_class', 0)); <filename>src/include/access/htup_details.h</> for explanations of the fields returned. </para> + <para> + The <function>heap_infomask</function> function can be used to unpack the + recognised bits of the infomasks of heap tuples. + </para> </listitem> </varlistentry> @@ -206,6 +210,34 @@ test=# SELECT * FROM heap_page_item_attrs(get_raw_page('pg_class', 0), 'pg_class <varlistentry> <term> + <function>heap_infomask_flags(infomask1 integer, infomask2 integer, show_combined bool) returns text[]</function> + <indexterm> + <primary>heap_infomask_flags</primary> + </indexterm> + </term> + <listitem> + <para> + <function>heap_infomask_flags</function> decodes the + <structfield>t_infomask1</structfield> and + <structfield>t_infomask2</structfield> returned by + <function>heap_page_items</function> into a human-readable array of flag + names. This can be used to see the tuple hint bits etc. + </para> + <para> + If show_combined is set (the default), combination flags like + <literal>HEAP_XMIN_FROZEN</literal> are also output. The original fields + are not filtered out, so e.g. a frozen tuple will have + <literal>HEAP_XMIN_FROZEN, HEAP_XMIN_COMMITTED, HEAP_XMIN_INVALID</literal>. + </para> + <para> + For the meaning of these flags see + <filename>src/include/access/htup_details.h</> + </para> + </listitem> + </varlistentry> + + <varlistentry> + <term> <function>fsm_page_contents(page bytea) returns text</function> <indexterm> <primary>fsm_page_contents</primary> -- 2.9.4
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers