pgsql: Update outdated comment for TransactionIdSetTreeStatus

2022-10-25 Thread Heikki Linnakangas
Update outdated comment for TransactionIdSetTreeStatus

Commit 06da3c570f changed the way subtransactions are marked as
SUBCOMMITTED, but the example it included actually documented the old
way. Update it.

Author: Japin Li
Discussion: 
https://www.postgresql.org/message-id/MEYP282MB16690BC96DFBE08CC857E1E3B6319%40MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/0e972f50fdc9f33b01d02bbcc2f26aa32f1c58ab

Modified Files
--
src/backend/access/transam/clog.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)



Re: pgsql: Fix unlink() for STATUS_DELETE_PENDING on Windows.

2022-10-25 Thread Justin Pryzby
On Tue, Oct 25, 2022 at 03:29:42AM +, Thomas Munro wrote:
> Fix unlink() for STATUS_DELETE_PENDING on Windows.
> 
> Commit f357233c assumed that it was OK to return ENOENT directly if
> lstat() failed that way.  If we got STATUS_DELETE_PENDING while trying
> to unlink a file that we had already unlinked successfully once before
> but someone else still had open (on a kernel version that has "pending"
> unlinks by default), then we would no longer reach the retry loop in
> pgunlink().  That loop claims to be only for handling sharing violations
> (a different phenomenon), but the errno is the same.
> 
> Restore that behavior with an explicit check, to see if it fixes the
> occasional 'directory not empty' failures seen in the pg_upgrade tests
> on CI.  Further improvements are possible with proposed upgrades to
> modern Windows APIs that would replace this convoluted code.
> 
> Reported-by: Justin Pryzby 
> Reviewed-by: Michael Paquier 
> Discussion: https://postgr.es/m/20220920013122.GA31833%40telsasoft.com
> Discussion: 
> https://postgr.es/m/CA%2BhUKG%2BajSQ_8eu2AogTncOnZ5me2D-Cn66iN_-wZnRjLN%2Bicg%40mail.gmail.com
...
> Details
> ---
> https://git.postgresql.org/pg/commitdiff/e109e43921d21d069c03f18d7c9d8f4e5cb6a0c3

If I'm not wrong, this didn't fix the issue you said it fixed.

cfbot says this ran 1h ago, and its HEAD^3 is e109e43.
https://cirrus-ci.com/task/5939314583404544




Re: pgsql: Fix unlink() for STATUS_DELETE_PENDING on Windows.

2022-10-25 Thread Justin Pryzby
On Tue, Oct 25, 2022 at 04:21:02PM -0500, Justin Pryzby wrote:
> > Restore that behavior with an explicit check, to see if it fixes the
> > occasional 'directory not empty' failures seen in the pg_upgrade tests
> > on CI.  Further improvements are possible with proposed upgrades to
> > modern Windows APIs that would replace this convoluted code.
> > 
> > Reported-by: Justin Pryzby 
> > Reviewed-by: Michael Paquier 
> > Discussion: https://postgr.es/m/20220920013122.GA31833%40telsasoft.com
> > Discussion: 
> > https://postgr.es/m/CA%2BhUKG%2BajSQ_8eu2AogTncOnZ5me2D-Cn66iN_-wZnRjLN%2Bicg%40mail.gmail.com
> ...
> > Details
> > ---
> > https://git.postgresql.org/pg/commitdiff/e109e43921d21d069c03f18d7c9d8f4e5cb6a0c3
> 
> If I'm not wrong, this didn't fix the issue you said it fixed.

s/said/hoped/sorry




pgsql: Doc/improve confusing, inefficient tests to locate CTID variable

2022-10-25 Thread Tom Lane
Doc/improve confusing, inefficient tests to locate CTID variable.

The IsCTIDVar() tests in nodeTidscan.c and nodeTidrangescan.c
look buggy at first sight: they aren't checking that the varno
matches the table to be scanned.  Actually they're safe because
any Var in a scan-level qual must be for the correct table ...
but if we're depending on that, it's pretty pointless to verify
varlevelsup.  (Besides which, varlevelsup is *always* zero at
execution, since we've flattened the rangetable long since.)

Remove the useless varlevelsup check, and instead add some
commentary explaining why we don't need to check varno.

Noted while fooling with a planner change that causes the order
of "t1.ctid = t2.ctid" to change in some tidscan.sql tests;
I was briefly fooled into thinking there was a live bug here.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/13d53aa7a83383c9d1343e7d725e615f8678aea8

Modified Files
--
src/backend/executor/nodeTidrangescan.c | 9 +++--
src/backend/executor/nodeTidscan.c  | 9 +++--
2 files changed, 14 insertions(+), 4 deletions(-)



Re: pgsql: Fix unlink() for STATUS_DELETE_PENDING on Windows.

2022-10-25 Thread Thomas Munro
On Wed, Oct 26, 2022 at 10:31 AM Justin Pryzby  wrote:
> On Tue, Oct 25, 2022 at 04:21:02PM -0500, Justin Pryzby wrote:
> > > Restore that behavior with an explicit check, to see if it fixes the
> > > occasional 'directory not empty' failures seen in the pg_upgrade tests
> > > on CI.  Further improvements are possible with proposed upgrades to
> > > modern Windows APIs that would replace this convoluted code.

> > If I'm not wrong, this didn't fix the issue you said it fixed.
>
> s/said/hoped/sorry

Drat.  More theories needed then.  Perhaps it has nothing to do with
my recent changes.  I am starting to wonder if we should have an
rmdir() wrapper that waits for zombie files to go away, and spits out
the name of the file that's in the way, and also the name/pid of the
process that has it open[1] if it times out, so we have a fighting
chance of debugging this type of stuff.

[1] https://devblogs.microsoft.com/oldnewthing/20120217-00/?p=8283




Re: pgsql: Fix unlink() for STATUS_DELETE_PENDING on Windows.

2022-10-25 Thread Justin Pryzby
On Wed, Oct 26, 2022 at 11:15:16AM +1300, Thomas Munro wrote:
> On Wed, Oct 26, 2022 at 10:31 AM Justin Pryzby  wrote:
> > On Tue, Oct 25, 2022 at 04:21:02PM -0500, Justin Pryzby wrote:
> > > > Restore that behavior with an explicit check, to see if it fixes the
> > > > occasional 'directory not empty' failures seen in the pg_upgrade tests
> > > > on CI.  Further improvements are possible with proposed upgrades to
> > > > modern Windows APIs that would replace this convoluted code.
> 
> > > If I'm not wrong, this didn't fix the issue you said it fixed.
> >
> > s/said/hoped/sorry
> 
> Drat.  More theories needed then.  Perhaps it has nothing to do with
> my recent changes.  I am starting to wonder if we should have an
> rmdir() wrapper that waits for zombie files to go away, and spits out
> the name of the file that's in the way, and also the name/pid of the
> process that has it open[1] if it times out, so we have a fighting
> chance of debugging this type of stuff.
> 
> [1] https://devblogs.microsoft.com/oldnewthing/20120217-00/?p=8283

FWIW, your description of the problem sounded a bit off to me.

You seemed to say that the problem is with rmdir() on a directory, which
contains a file which was "recently removed", but still opened by
something.

But as I recall, at the point that pg_upgrade is running rmdir(), the
contained files have been unlinked, and the postgres process has ended:

| # Running: pg_upgrade --no-sync -d 
C:\cirrus\build/testrun/pg_upgrade/002_pg_upgrade\data/t_002_pg_upgrade_old_node_data/pgdata
 -D 
C:\cirrus\build/testrun/pg_upgrade/002_pg_upgrade\data/t_002_pg_upgrade_new_node_data/pgdata
 -b C:/cirrus/build/tmp_install/bin -B C:/cirrus/build/tmp_install/bin -s 
C:/Users/ContainerAdministrator/AppData/Local/Temp/f67bWmckRH -p 57611 -P 57612 
--check
| Performing Consistency Checks
| -
| Checking cluster versions   ok
| Checking database user is the install user  ok
| Checking database connection settings   ok
| Checking for prepared transactions  ok
| Checking for system-defined composite types in user tables  ok
| Checking for reg* data types in user tables ok
| Checking for contrib/isn with bigint-passing mismatch   ok
| Checking for presence of required libraries ok
| Checking database user is the install user  ok
| Checking for prepared transactions  ok
| Checking for new cluster tablespace directories ok
| 
| *Clusters are compatible*
| pg_upgrade: warning: could not remove file or directory 
"C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/t_002_pg_upgrade_new_node_data/pgdata/pg_upgrade_output.d/20220925T193346.158/log":
 Directory not empty
| pg_upgrade: warning: could not remove file or directory 
"C:/cirrus/build/testrun/pg_upgrade/002_pg_upgrade/data/t_002_pg_upgrade_new_node_data/pgdata/pg_upgrade_output.d/20220925T193346.158":
 Directory not empty

My thinking has always been that this is essentially a bug/deficiency in
the windows FS.  It seems absurd that unlink/rmdir would be
asynchronous.

It'd be swell if we could use a separate device in CI, to be used for
running tests.  Bonus points if it supports COW.

-- 
Justin




pgsql: Fix ordering issue with WAL operations in GIN fast insert path

2022-10-25 Thread Michael Paquier
Fix ordering issue with WAL operations in GIN fast insert path

Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: 
https://postgr.es/m/caeze2whl8ulmqynnncu1lapwxd5rkeo0nhv+exgg_n6elu8...@mail.gmail.com
Backpatch-through: 10

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b02fc7df193b2c71f359cc2b42e21515311ab188

Modified Files
--
src/backend/access/gin/ginfast.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)



pgsql: Fix ordering issue with WAL operations in GIN fast insert path

2022-10-25 Thread Michael Paquier
Fix ordering issue with WAL operations in GIN fast insert path

Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: 
https://postgr.es/m/caeze2whl8ulmqynnncu1lapwxd5rkeo0nhv+exgg_n6elu8...@mail.gmail.com
Backpatch-through: 10

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5a30d43fa9869adc5a6f708dd0994a8a4e53d905

Modified Files
--
src/backend/access/gin/ginfast.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)



pgsql: Fix ordering issue with WAL operations in GIN fast insert path

2022-10-25 Thread Michael Paquier
Fix ordering issue with WAL operations in GIN fast insert path

Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: 
https://postgr.es/m/caeze2whl8ulmqynnncu1lapwxd5rkeo0nhv+exgg_n6elu8...@mail.gmail.com
Backpatch-through: 10

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/341fba2a6291f6f1c5f801c308b72fa62e46e5c8

Modified Files
--
src/backend/access/gin/ginfast.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)



pgsql: Fix ordering issue with WAL operations in GIN fast insert path

2022-10-25 Thread Michael Paquier
Fix ordering issue with WAL operations in GIN fast insert path

Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: 
https://postgr.es/m/caeze2whl8ulmqynnncu1lapwxd5rkeo0nhv+exgg_n6elu8...@mail.gmail.com
Backpatch-through: 10

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/594b97509e49e30a9f6ada09110405663a2e1d6f

Modified Files
--
src/backend/access/gin/ginfast.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)



pgsql: Fix ordering issue with WAL operations in GIN fast insert path

2022-10-25 Thread Michael Paquier
Fix ordering issue with WAL operations in GIN fast insert path

Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: 
https://postgr.es/m/caeze2whl8ulmqynnncu1lapwxd5rkeo0nhv+exgg_n6elu8...@mail.gmail.com
Backpatch-through: 10

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/51c24d9e21d7c36cbb6890655fd8f57270cdb15a

Modified Files
--
src/backend/access/gin/ginfast.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)



pgsql: Fix ordering issue with WAL operations in GIN fast insert path

2022-10-25 Thread Michael Paquier
Fix ordering issue with WAL operations in GIN fast insert path

Contrary to what is documented in src/backend/access/transam/README,
ginHeapTupleFastInsert() had a few ordering issues with the way it does
its WAL operations when inserting items in its fast path.

First, when using a separate list, XLogBeginInsert() was being always
called before START_CRIT_SECTION(), and in this case a second thing was
wrong when merging lists, as an exclusive lock was taken on the tail
page *before* calling XLogBeginInsert().  Finally, when inserting items
into a tail page, the order of XLogBeginInsert() and
START_CRIT_SECTION() was reversed.  This commit addresses all these
issues by moving the calls of XLogBeginInsert() after all the pages
logged are locked and pinned, within a critical section.

This has been applied first only on HEAD as of 56b6625, but as per
discussion with Tom Lane and Álvaro Herrera, a backpatch is preferred to
keep all the branches consistent and to respect the transam's README
where we can.

Author:  Matthias van de Meent, Zhang Mingli
Discussion: 
https://postgr.es/m/caeze2whl8ulmqynnncu1lapwxd5rkeo0nhv+exgg_n6elu8...@mail.gmail.com
Backpatch-through: 10

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/ca4070f2b4b1ed62e3d333a51a2f412b1c841ba4

Modified Files
--
src/backend/access/gin/ginfast.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)



Re: pgsql: Fix unlink() for STATUS_DELETE_PENDING on Windows.

2022-10-25 Thread Thomas Munro
On Wed, Oct 26, 2022 at 12:57 PM Justin Pryzby  wrote:
> On Wed, Oct 26, 2022 at 11:15:16AM +1300, Thomas Munro wrote:
> > On Wed, Oct 26, 2022 at 10:31 AM Justin Pryzby  wrote:
> > > > If I'm not wrong, this didn't fix the issue you said it fixed.
> > >
> > > s/said/hoped/sorry
> >
> > Drat.  More theories needed then.  Perhaps it has nothing to do with
> > my recent changes.  I am starting to wonder if we should have an
> > rmdir() wrapper that waits for zombie files to go away, and spits out
> > the name of the file that's in the way, and also the name/pid of the
> > process that has it open[1] if it times out, so we have a fighting
> > chance of debugging this type of stuff.
> >
> > [1] https://devblogs.microsoft.com/oldnewthing/20120217-00/?p=8283
>
> FWIW, your description of the problem sounded a bit off to me.
>
> You seemed to say that the problem is with rmdir() on a directory, which
> contains a file which was "recently removed", but still opened by
> something.

Right, that's what I was guessing because that is a known phenomenon,
and we can see that the error is "Directory not empty".  If, on the
other hand, that's caused by a file that's never been unlinked, we
might expect to see occasional failures on non-Windows systems too.

Wait a minute.  Even if that hunch were correct, what I changed would
not be enough to fix it.  Looking at the code that presumably performs
the recursive unlink and emits the warning, namely
src/common/rmtree.c, I see that it calls lstat() itself before calling
unlink().  So a zombie pending-deleted file would be skipped (because
ENOENT) before even reaching the code in question, and that was
already the case before anything I changed in the past few months in
this area AFAIK.

Which is a clue that we've been looking in the wrong place.  Perhaps
it has to do with file handles opened by pg_upgrade itself, which did
indeed recently change to a new path, though you'd think that'd be
more deterministic.  Now I'm tempted to write the patch that would
tell us the names of the files that are in the way to get more
visibility here.

> But as I recall, at the point that pg_upgrade is running rmdir(), the
> contained files have been unlinked, and the postgres process has ended:

Oh.  Hmm.  I wondered if it might be a logger process (they shut down
slightly after the postmaster IIRC), but it doesn't look like we start
one.

> My thinking has always been that this is essentially a bug/deficiency in
> the windows FS.  It seems absurd that unlink/rmdir would be
> asynchronous.

It certainly doesn't make life easy for open source projects from
planet Unix.  I wonder if those semantics came from the VMS orbit.

> It'd be swell if we could use a separate device in CI, to be used for
> running tests.  Bonus points if it supports COW.

I think it might be possible to create a ReFS filesystem inside a
loopback file.  On that filesystem, the tests I propose in CF #3951
take the !have_posix_unlink_semantics path, even if we commit the
patch to turn on POSIX semantics (which has useful effect only on
NTFS, according to experiments so far).  If we decided to commit that
without also setting up coverage for non-POSIX-semantics filesystems,
I predict that our support for non-POSIX unlink semantics would very
soon decay and become unsalvageable, which is why it amounts to a
policy decision about future support...




pgsql: Refactor code handling the names of files loaded in hba.c

2022-10-25 Thread Michael Paquier
Refactor code handling the names of files loaded in hba.c

This has the advantage to limit the presence of the GUC values
hba_file and ident_file to the code paths where these files are loaded,
easing the introduction of an upcoming feature aimed at adding inclusion
logic for files and directories in HBA and ident files.

Note that this needs the addition of the source file name to HbaLine, in
addition to the line number, which is something needed by the backend in
two places of auth.c (authentication failure details and auth_id log
when log_connections is enabled).

While on it, adjust a log generated on authentication failure to report
the name of the actual HBA file on which the connection attempt matched,
where the line number and the raw line written in the HBA file were
already included.  This was previously hardcoded as pg_hba.conf, which
would be incorrect when a custom value is used at postmaster startup for
the GUC hba_file.

Extracted from a larger patch by the same author.

Author: Julien Rouhaud
Discussion: https://postgr.es/m/20220223045959.35ipdsvbxcstrhya@jrouhaud

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/1b73d0b1c3934f703d68031957d37c2a9765e798

Modified Files
--
src/backend/libpq/auth.c |   7 +--
src/backend/libpq/hba.c  | 113 +--
src/include/libpq/hba.h  |   2 +
3 files changed, 65 insertions(+), 57 deletions(-)



Re: pgsql: Refactor code handling the names of files loaded in hba.c

2022-10-25 Thread Julien Rouhaud
Hi,

On Wed, Oct 26, 2022 at 02:53:08AM +, Michael Paquier wrote:
> Refactor code handling the names of files loaded in hba.c

For the record there is a couple of thinko in the extracted patch:

@@ -1063,6 +1064,7 @@ HbaLine *
 parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
 {
int line_num = tok_line->line_num;
+   char   *file_name = tok_line->file_name;
char  **err_msg = &tok_line->err_msg;
char   *str;
struct addrinfo *gai_result;
@@ -1077,6 +1079,7 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
HbaLine*parsedline;

parsedline = palloc0(sizeof(HbaLine));
+   parsedline->sourcefile = pstrdup(tok_line->file_name);

This should have used the "file_name" variable rather than accessing it via
tok_line, same for parse_ident_line.

I fixed it in the v14 sent on the main thread, unfortunately sent a bit too
late.




Re: pgsql: Refactor code handling the names of files loaded in hba.c

2022-10-25 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:27:30AM +0800, Julien Rouhaud wrote:
> I fixed it in the v14 sent on the main thread, unfortunately sent a bit too
> late.

Nya, yeah.  It looks like I fat-fingered a copy-paste here.  The
result is the same, still your suggested change is more consistent
with the rest of the file.  Will adjust in a minute..
--
Michael


signature.asc
Description: PGP signature


pgsql: Fix variable assignment thinko in hba.c

2022-10-25 Thread Michael Paquier
Fix variable assignment thinko in hba.c

The intention behind 1b73d0b was to limit the use of TokenizedAuthLine,
but I have fat-fingered one location in parse_hba_line() when creating
the HbaLine, where this should use the local variable and not the value
coming from TokenizedAuthLine.  This logic is the exactly the same, but
let's be clean about all that on consistency grounds.

Reported-by: Julien Rouhaud
Discussion: https://postgr.es/m/20221026032730.k3sib5krgm7l6njk@jrouhaud

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/37d264478ab47e8fa03751c39ba2c5dd447b89c8

Modified Files
--
src/backend/libpq/hba.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



Re: pgsql: Refactor code handling the names of files loaded in hba.c

2022-10-25 Thread Julien Rouhaud
On Wed, Oct 26, 2022 at 12:54:24PM +0900, Michael Paquier wrote:
> On Wed, Oct 26, 2022 at 11:27:30AM +0800, Julien Rouhaud wrote:
> > I fixed it in the v14 sent on the main thread, unfortunately sent a bit too
> > late.
> 
> Nya, yeah.  It looks like I fat-fingered a copy-paste here.  The
> result is the same, still your suggested change is more consistent
> with the rest of the file.  Will adjust in a minute..

Yeah, no errors in the code just lack of consistency.  Thanks for pushing the
refactor and the fix!




pgsql: Add rule_number to pg_hba_file_rules and map_number to pg_ident_

2022-10-25 Thread Michael Paquier
Add rule_number to pg_hba_file_rules and map_number to pg_ident_file_mappings

These numbers are strictly-monotone identifiers assigned to each rule
of pg_hba_file_rules and each map of pg_ident_file_mappings when loading
the HBA and ident configuration files, indicating the order in which
they are checked at authentication time, until a match is found.

With only one file loaded currently, this is equivalent to the line
numbers assigned to the entries loaded if one wants to know their order,
but this becomes mandatory once the inclusion of external files is
added to the HBA and ident files to be able to know in which order the
rules and/or maps are applied at authentication.  Note that NULL is used
when a HBA or ident entry cannot be parsed or validated, aka when an
error exists, contrary to the line number.

Bump catalog version.

Author: Julien Rouhaud
Discussion: https://postgr.es/m/20220223045959.35ipdsvbxcstrhya@jrouhaud

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c591300a8f54d9711157d9a8866f022a257ec4ee

Modified Files
--
doc/src/sgml/system-views.sgml  | 21 +++
src/backend/utils/adt/hbafuncs.c| 51 +
src/include/catalog/catversion.h|  2 +-
src/include/catalog/pg_proc.dat | 11 
src/test/regress/expected/rules.out | 10 +---
5 files changed, 74 insertions(+), 21 deletions(-)