pgsql: Re-disallow Memoize for parameterized nested loops with join fil
Re-disallow Memoize for parameterized nested loops with join filters This was previously fixed in 9e215378d but got broken again as a result of 2489d76c4. It seems that commit causes ppi_clauses to contain duplicate clauses and it's no longer safe to check the list_length of that list to determine if there are join conditions other than what's mentioned in ppi_clauses. Here we adjust the check to count the distinct rinfo_serial mentioned in ppi_clauses. We expect that extra->restrictlist won't have duplicate rinfo_serials. Reported-by: Amadeo Gallardo Author: Richard Guo Discussion: https://postgr.es/m/CADFREbW-BLJd7-a5J%2B5wjVumeFG1ByXiSOFzMtkmY_SDWckTxw%40mail.gmail.com Backpatch-through: 16, where 2489d76c4 was introduced. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/2bcf0785cd1080b85f0d2ccd595b195970730491 Modified Files -- src/backend/optimizer/path/joinpath.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Re-disallow Memoize for parameterized nested loops with join fil
Re-disallow Memoize for parameterized nested loops with join filters This was previously fixed in 9e215378d but got broken again as a result of 2489d76c4. It seems that commit causes ppi_clauses to contain duplicate clauses and it's no longer safe to check the list_length of that list to determine if there are join conditions other than what's mentioned in ppi_clauses. Here we adjust the check to count the distinct rinfo_serial mentioned in ppi_clauses. We expect that extra->restrictlist won't have duplicate rinfo_serials. Reported-by: Amadeo Gallardo Author: Richard Guo Discussion: https://postgr.es/m/CADFREbW-BLJd7-a5J%2B5wjVumeFG1ByXiSOFzMtkmY_SDWckTxw%40mail.gmail.com Backpatch-through: 16, where 2489d76c4 was introduced. Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/74f770ef202fd3f1f1d1f0889f04c16440d23bd0 Modified Files -- src/backend/optimizer/path/joinpath.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
pgsql: Make documentation builds reproducible
Make documentation builds reproducible Currently, the documentation builds are not fully reproducible (in the sense of https://reproducible-builds.org/). A fix is available upstream (https://github.com/docbook/xslt10-stylesheets/issues/54) but not released. This commit patches the upstream fix into our customization layer. This patch addresses both the HTML and the FO output. The man output is already reproducible. Discussion: https://www.postgresql.org/message-id/flat/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b0f0a9432d0b6f53634a96715f2666f6d4ea25a1 Modified Files -- doc/src/sgml/stylesheet-fo.xsl | 259 doc/src/sgml/stylesheet-html-common.xsl | 234 + 2 files changed, 493 insertions(+)
pgsql: Test EXPLAIN (FORMAT JSON) ... XMLTABLE
Test EXPLAIN (FORMAT JSON) ... XMLTABLE Also, add an alias to the XMLTABLE expression in an existing test. This covers some code in explain.c that wasn't previously covered. I patched xml_2.out blindly :-( Discussion: https://postgr.es/m/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/752533d40fd50de0b09d4b956cc32c38f5df2f05 Modified Files -- src/test/regress/expected/xml.out | 51 + src/test/regress/expected/xml_1.out | 51 + src/test/regress/expected/xml_2.out | 51 + src/test/regress/sql/xml.sql| 5 +++- 4 files changed, 142 insertions(+), 16 deletions(-)
pgsql: Abort pgbench if script end is reached with an open pipeline
Abort pgbench if script end is reached with an open pipeline When a pipeline is opened with \startpipeline and not closed, pgbench will either error on the next transaction with a "already in pipeline mode" error or successfully end if this was the last transaction -- despite not sending anything that was piped in the pipeline. Make it an error to reach end of script is reached while there's an open pipeline. Backpatch to 14, where pgbench got support for pipelines. Author: Anthonin Bonnefoy Reported-by: Michael Paquier Discussion: https://postgr.es/m/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/49f7c6c44a5f6a7f7b39d140e490c9c627511893 Modified Files -- src/bin/pgbench/pgbench.c| 15 +-- src/bin/pgbench/t/001_pgbench_with_server.pl | 28 2 files changed, 41 insertions(+), 2 deletions(-)
pgsql: Abort pgbench if script end is reached with an open pipeline
Abort pgbench if script end is reached with an open pipeline When a pipeline is opened with \startpipeline and not closed, pgbench will either error on the next transaction with a "already in pipeline mode" error or successfully end if this was the last transaction -- despite not sending anything that was piped in the pipeline. Make it an error to reach end of script is reached while there's an open pipeline. Backpatch to 14, where pgbench got support for pipelines. Author: Anthonin Bonnefoy Reported-by: Michael Paquier Discussion: https://postgr.es/m/[email protected] Branch -- REL_14_STABLE Details --- https://git.postgresql.org/pg/commitdiff/85ecff891525d61bc975584e2dc2d3c01e5a48ee Modified Files -- src/bin/pgbench/pgbench.c| 15 +-- src/bin/pgbench/t/001_pgbench_with_server.pl | 28 2 files changed, 41 insertions(+), 2 deletions(-)
pgsql: Abort pgbench if script end is reached with an open pipeline
Abort pgbench if script end is reached with an open pipeline When a pipeline is opened with \startpipeline and not closed, pgbench will either error on the next transaction with a "already in pipeline mode" error or successfully end if this was the last transaction -- despite not sending anything that was piped in the pipeline. Make it an error to reach end of script is reached while there's an open pipeline. Backpatch to 14, where pgbench got support for pipelines. Author: Anthonin Bonnefoy Reported-by: Michael Paquier Discussion: https://postgr.es/m/[email protected] Branch -- REL_15_STABLE Details --- https://git.postgresql.org/pg/commitdiff/3fd36be52b70b95eea6fe7d296ce871f49eedb50 Modified Files -- src/bin/pgbench/pgbench.c| 15 +-- src/bin/pgbench/t/001_pgbench_with_server.pl | 28 2 files changed, 41 insertions(+), 2 deletions(-)
pgsql: Abort pgbench if script end is reached with an open pipeline
Abort pgbench if script end is reached with an open pipeline When a pipeline is opened with \startpipeline and not closed, pgbench will either error on the next transaction with a "already in pipeline mode" error or successfully end if this was the last transaction -- despite not sending anything that was piped in the pipeline. Make it an error to reach end of script is reached while there's an open pipeline. Backpatch to 14, where pgbench got support for pipelines. Author: Anthonin Bonnefoy Reported-by: Michael Paquier Discussion: https://postgr.es/m/[email protected] Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/07b53de700390c8bdee99ce373d84f0cdcad74d6 Modified Files -- src/bin/pgbench/pgbench.c| 15 +-- src/bin/pgbench/t/001_pgbench_with_server.pl | 28 2 files changed, 41 insertions(+), 2 deletions(-)
pgsql: Fix two memcpy() bugs in the new injection point code
Fix two memcpy() bugs in the new injection point code 1. The memcpy()s in InjectionPointAttach() would copy garbage from beyond the end of input string to the buffer in shared memory. You won't usually notice, but if there is not enough valid mapped memory beyond the end of the string, the read of unmapped memory will segfault. This was flagged by the Cirrus CI build with address sanitizer enabled. 2. The memcpy() in injection_point_cache_add() failed to copy the NULL terminator. Discussion: https://www.postgresql.org/message-id/0615a424-b726-4157-afa7-4245629f9512%40iki.fi Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/0eb23285a2579591c09a591e5a52829f65665341 Modified Files -- src/backend/utils/misc/injection_point.c | 8 1 file changed, 4 insertions(+), 4 deletions(-)
pgsql: Fix ERROR message in injection_point.c
Fix ERROR message in injection_point.c This commit fixes an error message that failed to show the correct function and library names when a function cannot be loaded. While on it, adjust the call to load_external_function() so as this ERROR can be reached, by making load_external_function() return NULL rather than fail if a function cannot be found for a given injection point. Thinkos in d86d20f0ba79. Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/cdd863480c27b406ac14e94f1fe122b98ea562b4 Modified Files -- src/backend/utils/misc/injection_point.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
pgsql: Fix possible NULL pointer dereference in GetNamedDSMSegment().
Fix possible NULL pointer dereference in GetNamedDSMSegment(). GetNamedDSMSegment() doesn't check whether dsm_attach() returns NULL, which creates the possibility of a NULL pointer dereference soon after. To fix, emit an ERROR if dsm_attach() returns NULL. This shouldn't happen, but it would be nice to avoid a segfault if it does. In passing, tidy up the surrounding code. Reported-by: Tom Lane Reviewed-by: Michael Paquier, Bharath Rupireddy Discussion: https://postgr.es/m/3348869.1705854106%40sss.pgh.pa.us Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/4372adfa24f2f5ddc587317d634b5389bd764106 Modified Files -- src/backend/storage/ipc/dsm_registry.c | 21 - 1 file changed, 12 insertions(+), 9 deletions(-)
pgsql: ci: Enable injection points in builds
ci: Enable injection points in builds --enable-injection-points and -Dinjection_points=true are now given to the commands triggered by the tasks where assertions are enabled, providing coverage for injection points or any test using them. Author: Heikki Linnakangas Discussion: https://postgr.es/m/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/183b6f73b04e41cd8468eab74ab191602c66617b Modified Files -- .cirrus.tasks.yml | 17 + 1 file changed, 9 insertions(+), 8 deletions(-)
pgsql: Add better handling of redundant IS [NOT] NULL quals
Add better handling of redundant IS [NOT] NULL quals Until now PostgreSQL has not been very smart about optimizing away IS NOT NULL base quals on columns defined as NOT NULL. The evaluation of these needless quals adds overhead. Ordinarily, anyone who came complaining about that would likely just have been told to not include the qual in their query if it's not required. However, a recent bug report indicates this might not always be possible. Bug 17540 highlighted that when we optimize Min/Max aggregates the IS NOT NULL qual that the planner adds to make the rewritten plan ignore NULLs can cause issues with poor index choice. That particular case demonstrated that other quals, especially ones where no statistics are available to allow the planner a chance at estimating an approximate selectivity for can result in poor index choice due to cheap startup paths being prefered with LIMIT 1. Here we take generic approach to fixing this by having the planner check for NOT NULL columns and just have the planner remove these quals (when they're not needed) for all queries, not just when optimizing Min/Max aggregates. Additionally, here we also detect IS NULL quals on a NOT NULL column and transform that into a gating qual so that we don't have to perform the scan at all. This also works for join relations when the Var is not nullable by any outer join. This also helps with the self-join removal work as it must replace strict join quals with IS NOT NULL quals to ensure equivalence with the original query. Author: David Rowley, Richard Guo, Andy Fan Reviewed-by: Richard Guo, David Rowley Discussion: https://postgr.es/m/caaphdvqg6xzdhyrpz0zgocevsmo0d3vxa9dvhrztkfqo30w...@mail.gmail.com Discussion: https://postgr.es/m/17540-7aa1855ad5ec18b4%40postgresql.org Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/b262ad440edecda0b1aba81d967ab560a83acb8a Modified Files -- contrib/postgres_fdw/expected/postgres_fdw.out | 16 +- contrib/postgres_fdw/sql/postgres_fdw.sql | 4 +- src/backend/optimizer/plan/initsplan.c | 197 +++- src/backend/optimizer/util/joininfo.c | 28 +++ src/backend/optimizer/util/plancat.c | 19 ++ src/backend/optimizer/util/relnode.c | 3 + src/include/nodes/pathnodes.h | 7 +- src/include/optimizer/planmain.h | 4 + src/test/regress/expected/equivclass.out | 18 +- src/test/regress/expected/join.out | 67 --- src/test/regress/expected/predicate.out| 244 + src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/predicate.sql | 122 + 13 files changed, 664 insertions(+), 67 deletions(-)
pgsql: Improve stability of recovery test 035_standby_logical_decoding
Improve stability of recovery test 035_standby_logical_decoding This commit tweaks a couple of things in 035_standby_logical_decoding to hopefully stabilize it: - Autovacuum is now disabled, as it could hold a global xmin with a transaction. - Conflicts are generated with command sequences that removed rows (on catalogs, shared or non-shared, or just plain relations) followed by a VACUUM. This was unstable because these did not check that the horizon moved between the SQL commands and the VACUUM. The logic is refactored as follows, to ensure that VACUUM removes dead rows before testing for slot invalidation on a standby (idea suggested by Andres Freund): -- Grab the current horizon. -- Launch SQL commands removing rows. -- Check that the snapshot horizon has been updated. -- Launch VACUUM on the relation whose rows have been removed by the first step. Note that there are still some issues because of standby snapshot WAL records generated by the bgwriter, but this makes the test much more stable. Per reports from buildfarm members dikkop and skink, with analysis and tests from Alexander Lakhin. While on it, fix a couple of incorrect comments. Author: Bertrand Drouvot Reviewed-by: Alexander Lakhin, Michael Paquier Discussion: https://postgr.es/m/oszpr01mb6310ed3cedb531bcedbc6af2fd...@oszpr01mb6310.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/[email protected] Backpatch-through: 16 Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/46d8587b504170ca14f064890bc7ccbbd7523f81 Modified Files -- .../recovery/t/035_standby_logical_decoding.pl | 90 ++ 1 file changed, 56 insertions(+), 34 deletions(-)
pgsql: Improve stability of recovery test 035_standby_logical_decoding
Improve stability of recovery test 035_standby_logical_decoding This commit tweaks a couple of things in 035_standby_logical_decoding to hopefully stabilize it: - Autovacuum is now disabled, as it could hold a global xmin with a transaction. - Conflicts are generated with command sequences that removed rows (on catalogs, shared or non-shared, or just plain relations) followed by a VACUUM. This was unstable because these did not check that the horizon moved between the SQL commands and the VACUUM. The logic is refactored as follows, to ensure that VACUUM removes dead rows before testing for slot invalidation on a standby (idea suggested by Andres Freund): -- Grab the current horizon. -- Launch SQL commands removing rows. -- Check that the snapshot horizon has been updated. -- Launch VACUUM on the relation whose rows have been removed by the first step. Note that there are still some issues because of standby snapshot WAL records generated by the bgwriter, but this makes the test much more stable. Per reports from buildfarm members dikkop and skink, with analysis and tests from Alexander Lakhin. While on it, fix a couple of incorrect comments. Author: Bertrand Drouvot Reviewed-by: Alexander Lakhin, Michael Paquier Discussion: https://postgr.es/m/oszpr01mb6310ed3cedb531bcedbc6af2fd...@oszpr01mb6310.jpnprd01.prod.outlook.com Discussion: https://postgr.es/m/[email protected] Backpatch-through: 16 Branch -- REL_16_STABLE Details --- https://git.postgresql.org/pg/commitdiff/cae25d110e0898df2df4fe31fdc45208554ac908 Modified Files -- .../recovery/t/035_standby_logical_decoding.pl | 90 ++ 1 file changed, 56 insertions(+), 34 deletions(-)
pgsql: Generate syscache info from catalog files
Generate syscache info from catalog files Add a new genbki macros MAKE_SYSCACHE that specifies the syscache ID macro, the underlying index, and the number of buckets. From that, we can generate the existing tables in syscache.h and syscache.c via genbki.pl. Reviewed-by: John Naylor Discussion: https://www.postgresql.org/message-id/flat/[email protected] Branch -- master Details --- https://git.postgresql.org/pg/commitdiff/9b1a6f50b91dca6610932650c8c81a3c924259f9 Modified Files -- src/backend/catalog/.gitignore | 2 + src/backend/catalog/Catalog.pm | 10 + src/backend/catalog/Makefile | 2 +- src/backend/catalog/genbki.pl | 75 +++ src/backend/utils/cache/syscache.c | 632 + src/include/catalog/.gitignore | 2 + src/include/catalog/genbki.h | 6 + src/include/catalog/meson.build| 18 +- src/include/catalog/pg_aggregate.h | 2 + src/include/catalog/pg_am.h| 3 + src/include/catalog/pg_amop.h | 3 + src/include/catalog/pg_amproc.h| 2 + src/include/catalog/pg_attribute.h | 3 + src/include/catalog/pg_auth_members.h | 3 + src/include/catalog/pg_authid.h| 3 + src/include/catalog/pg_cast.h | 2 + src/include/catalog/pg_class.h | 3 + src/include/catalog/pg_collation.h | 3 + src/include/catalog/pg_constraint.h| 2 + src/include/catalog/pg_conversion.h| 4 + src/include/catalog/pg_database.h | 2 + src/include/catalog/pg_default_acl.h | 2 + src/include/catalog/pg_enum.h | 3 + src/include/catalog/pg_event_trigger.h | 3 + src/include/catalog/pg_foreign_data_wrapper.h | 3 + src/include/catalog/pg_foreign_server.h| 3 + src/include/catalog/pg_foreign_table.h | 2 + src/include/catalog/pg_index.h | 2 + src/include/catalog/pg_language.h | 3 + src/include/catalog/pg_namespace.h | 3 + src/include/catalog/pg_opclass.h | 3 + src/include/catalog/pg_operator.h | 2 + src/include/catalog/pg_opfamily.h | 3 + src/include/catalog/pg_parameter_acl.h | 2 + src/include/catalog/pg_partitioned_table.h | 2 + src/include/catalog/pg_proc.h | 3 + src/include/catalog/pg_publication.h | 3 + src/include/catalog/pg_publication_namespace.h | 3 + src/include/catalog/pg_publication_rel.h | 3 + src/include/catalog/pg_range.h | 3 + src/include/catalog/pg_replication_origin.h| 3 + src/include/catalog/pg_rewrite.h | 2 + src/include/catalog/pg_sequence.h | 2 + src/include/catalog/pg_statistic.h | 2 + src/include/catalog/pg_statistic_ext.h | 3 + src/include/catalog/pg_statistic_ext_data.h| 1 + src/include/catalog/pg_subscription.h | 3 + src/include/catalog/pg_subscription_rel.h | 2 + src/include/catalog/pg_tablespace.h| 2 + src/include/catalog/pg_transform.h | 3 + src/include/catalog/pg_ts_config.h | 3 + src/include/catalog/pg_ts_config_map.h | 2 + src/include/catalog/pg_ts_dict.h | 3 + src/include/catalog/pg_ts_parser.h | 3 + src/include/catalog/pg_ts_template.h | 3 + src/include/catalog/pg_type.h | 3 + src/include/catalog/pg_user_mapping.h | 3 + src/include/utils/syscache.h | 98 +--- src/tools/pginclude/cpluspluscheck | 5 + src/tools/pginclude/headerscheck | 5 + 60 files changed, 267 insertions(+), 717 deletions(-)
