Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-28 Thread Michael Paquier
On Mon, Nov 28, 2022 at 04:12:40PM +0900, Michael Paquier wrote: > Attached is a rebased patch of the rest. With everything we have > dealt with in this CF, perhaps it would be better to mark this entry > as committed and switch to a new thread where the negative TAP tests > could be discussed?

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Michael Paquier
On Sun, Nov 27, 2022 at 07:04:46PM +0800, Julien Rouhaud wrote: > And here's the rebased patch for the TAP tests. I will switch the CF entry to > Needs Review. I have been looking at that, and applied the part 1 of the test for the positive tests to cover the basic ground I'd like to see covered

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Michael Paquier
On Sun, Nov 27, 2022 at 09:49:31PM +0900, Ian Lawrence Barwick wrote: > Thanks for the quick update! FWIW, I do intend to tackle this last part ASAP, as the last piece to commit to get the full picture in the tree. -- Michael signature.asc Description: PGP signature

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Ian Lawrence Barwick
2022年11月27日(日) 20:04 Julien Rouhaud : > > On Sun, Nov 27, 2022 at 03:39:44PM +0800, Julien Rouhaud wrote: > > Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > > > > > > > I'm trying to reconcile open CommitFest entries with their actual > > > status; the entry for this: > > > > > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-27 Thread Julien Rouhaud
On Sun, Nov 27, 2022 at 03:39:44PM +0800, Julien Rouhaud wrote: > Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > > > > I'm trying to reconcile open CommitFest entries with their actual > > status; the entry for this: > > > > https://commitfest.postgresql.org/40/3558/ > > > > shows as

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-26 Thread Julien Rouhaud
Le dim. 27 nov. 2022 à 15:31, Ian Lawrence Barwick > > I'm trying to reconcile open CommitFest entries with their actual > status; the entry for this: > > https://commitfest.postgresql.org/40/3558/ > > shows as "Waiting on Author", but looks like it's all been committed; > is there anything >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-26 Thread Ian Lawrence Barwick
2022年11月25日(金) 11:25 Julien Rouhaud : > > On Fri, Nov 25, 2022 at 07:41:59AM +0900, Michael Paquier wrote: > > On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > > > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached > > > v22 > > > that fixes it in all the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Julien Rouhaud
On Fri, Nov 25, 2022 at 07:41:59AM +0900, Michael Paquier wrote: > On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached v22 > > that fixes it in all the places I found. > > Sounds fine. Added one comment,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Michael Paquier
On Thu, Nov 24, 2022 at 05:07:24PM +0800, Julien Rouhaud wrote: > So I went with CONF_FILE_START_DEPTH and CONF_FILE_MAX_DEPTH. Attached v22 > that fixes it in all the places I found. Sounds fine. Added one comment, fixed one comment, and applied. Thanks! -- Michael signature.asc Description:

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-24 Thread Julien Rouhaud
Hi, On Thu, Nov 24, 2022 at 02:37:23PM +0800, Julien Rouhaud wrote: > On Thu, Nov 24, 2022 at 02:07:21PM +0900, Michael Paquier wrote: > > On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > > > The depth 0 is getting used quite a lot now, maybe we should have a > > > define for >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-23 Thread Julien Rouhaud
On Thu, Nov 24, 2022 at 02:07:21PM +0900, Michael Paquier wrote: > On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > > The depth 0 is getting used quite a lot now, maybe we should have a define > > for > > it to make it easier to grep, like TOP_LEVEL_AUTH_FILE or something like >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-23 Thread Michael Paquier
On Wed, Nov 23, 2022 at 03:56:50PM +0800, Julien Rouhaud wrote: > The depth 0 is getting used quite a lot now, maybe we should have a define for > it to make it easier to grep, like TOP_LEVEL_AUTH_FILE or something like that? > And also add a define for the magical 10 for the max inclusion depth,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Julien Rouhaud
Hi, Sorry for the very late answer, I had quite a lot of other things going on recently. And thanks for taking care of the patchset! On Wed, Nov 23, 2022 at 03:05:18PM +0900, Michael Paquier wrote: > On Tue, Nov 22, 2022 at 05:20:01PM +0900, Michael Paquier wrote: > > + /* XXX: this should

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
e parse_ident_line memory */ MemoryContextSwitchTo(oldcxt); MemoryContextDelete(identcxt); -- 2.38.1 From 1173e0badf1572816b30a251106a6d4feb8c070c Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Wed, 23 Nov 2022 14:47:17 +0900 Subject: [PATCH v21 2/2] Allow file inclusion in pg_hba and pg_id

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-22 Thread Michael Paquier
be used only when we have no err_msg, meaning that these have been consumed in some TokenizedAuthLines already. -- Michael From c585c8d4dffb99f6abb4129cd9adf27849fa0ce0 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 14 Nov 2022 13:47:15 +0900 Subject: [PATCH v20 1/2] Allow file inclusi

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-16 Thread Michael Paquier
On Wed, Nov 16, 2022 at 10:53:02AM +0800, Julien Rouhaud wrote: > While being the same inclusion infrastructure, it's likely that people will > have different usage. I'm assuming that for GUCs the main usage is to have > your automation tool put one of your template conf for instance >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Julien Rouhaud
Le mer. 16 nov. 2022 à 13:01, Ian Lawrence Barwick a écrit : > 2022年11月14日(月) 14:41 Michael Paquier : > > > > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > > It's looks good to me. I agree that file name and line number should > be enough > > > to diagnose any unexpected

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Ian Lawrence Barwick
2022年11月14日(月) 14:41 Michael Paquier : > > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > It's looks good to me. I agree that file name and line number should be > > enough > > to diagnose any unexpected error. > > Thanks for checking. I have looked at 0001 and 0002 again

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-15 Thread Julien Rouhaud
On Tue, Nov 15, 2022 at 08:46:55AM +0900, Michael Paquier wrote: > On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > > > > If you have an include_dir directive and multiple files have wrong > > permission > > (or maybe broken symlink or something like that), you will get multiple

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-14 Thread Michael Paquier
On Mon, Nov 14, 2022 at 03:47:27PM +0800, Julien Rouhaud wrote: > On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: >> If the caller passes NULL for *linectx as the initial line context, >> just create it as we do now. If *linectx is not NULL, just reuse it. >> That may be cleaner

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-13 Thread Julien Rouhaud
Hi, On Mon, Nov 14, 2022 at 02:40:37PM +0900, Michael Paquier wrote: > On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > > It's looks good to me. I agree that file name and line number should be > > enough > > to diagnose any unexpected error. > > Thanks for checking. I have

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-13 Thread Michael Paquier
On Sat, Nov 12, 2022 at 04:13:53PM +0800, Julien Rouhaud wrote: > It's looks good to me. I agree that file name and line number should be > enough > to diagnose any unexpected error. Thanks for checking. I have looked at 0001 and 0002 again with a fresh mind, and applied both of them this

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-12 Thread Julien Rouhaud
On Thu, Nov 10, 2022 at 10:29:40AM +0900, Michael Paquier wrote: > > FWIW, I have been playing with the addition of a ErrorContextCallback > in tokenize_auth_file(), and this addition leads to a really nice > result. With this method, it is possible to know the full chain of > events leading to a

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-09 Thread Michael Paquier
ror_context_stack; + error_context_stack = linecxt = AllocSetContextCreate(CurrentMemoryContext, "tokenize_auth_file", @@ -686,10 +714,13 @@ tokenize_auth_file(const char *filename, FILE *file, List **tok_lines, } line_number += continuations + 1; + callback_arg.linenum =

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-08 Thread Julien Rouhaud
Hi, On Wed, Nov 09, 2022 at 09:51:17AM +0900, Michael Paquier wrote: > On Tue, Nov 08, 2022 at 10:04:16AM +0900, Michael Paquier wrote: > > CF bot unhappy as I have messed up with rules.out. Rebased. I have > > removed the restriction on MAXPGPATH in AbsoluteConfigLocation() in > > 0001, while

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-08 Thread Michael Paquier
cxt = tokenize_auth_file(IdentFileName, file, _lines, DEBUG3); + linecxt = tokenize_auth_file(IdentFileName, file, _lines, DEBUG3, 0); FreeFile(file); /* Now parse all the lines */ -- 2.38.1 From ab0d1d13edacfaa951b834ce71573671f05be41e Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Tue,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-07 Thread Michael Paquier
rcode_for_file_access(), - errmsg("could not open usermap file \"%s\": %m", - IdentFileName))); + file = open_auth_file(IdentFileName, ERROR, 0, NULL); - linecxt = tokenize_auth_file(IdentFileName, file, _lines, DEBUG3); + linecxt = tokenize_auth_file(IdentFileN

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-06 Thread Michael Paquier
aFileName, file, _lines, DEBUG3, 0); FreeFile(file); /* Now parse all the lines */ @@ -529,14 +524,9 @@ fill_ident_view(Tuplestorestate *tuple_store, TupleDesc tupdesc) * (Most other error conditions should result in a message in a view * entry.) */ - file = AllocateFile(IdentFileName, &

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-03 Thread Michael Paquier
On Wed, Nov 02, 2022 at 09:06:02PM +0800, Julien Rouhaud wrote: > Maybe one alternative approach would be to keep modifying the current test, > and > only do the split when committing the first part? The split itself should be > easy and mechanical: just remove everything unneeded (different

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-02 Thread Julien Rouhaud
On Wed, Nov 02, 2022 at 04:46:48PM +0900, Michael Paquier wrote: > On Fri, Oct 28, 2022 at 11:49:54AM +0800, Julien Rouhaud wrote: > > To be honest I'd rather not to. It's excessively annoying to work on those > > tests (I spent multiple days trying to make it as clean and readable as > >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-11-02 Thread Michael Paquier
uier Date: Wed, 2 Nov 2022 16:42:53 +0900 Subject: [PATCH v15 1/2] Allow file inclusion in pg_hba and pg_ident files. pg_hba.conf file now has support for "include", "include_dir" and "include_if_exists" directives, which work similarly to the same directives

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Julien Rouhaud
On Fri, Oct 28, 2022 at 10:24:23AM +0900, Michael Paquier wrote: > On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > > I am still not completely sure what's the best way to do things here, > so let's do the following: let's keep the patch the way you think is > better for now (I

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-27 Thread Michael Paquier
On Thu, Oct 27, 2022 at 12:26:25PM +0800, Julien Rouhaud wrote: > On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: >> >> Putting things afresh, there are two different things here (sorry I >> need to see that typed ;p): >> 1) How do we want to check reliably the loading of the HBA

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Julien Rouhaud
Hi, On Thu, Oct 27, 2022 at 12:08:31PM +0900, Michael Paquier wrote: > > Putting things afresh, there are two different things here (sorry I > need to see that typed ;p): > 1) How do we want to check reliably the loading of the HBA and ident > files on errors? I guess you meant the failure to

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:32:14PM +0800, Julien Rouhaud wrote: > Have you already done a rebase while working on the patch or are you intending > to take care of it, or should I? Let's no both do the work :) Spoiler alert: I have not done a rebase yet ;) -- Michael signature.asc Description:

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:32:14PM +0800, Julien Rouhaud wrote: > I don't mind taking care of that, but before doing so I'd like to have some > feedback on whether you're ok with my approach (per my initial email about it > at [1]) or if you had some different > ideas on how to do it. Putting

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Julien Rouhaud
On Wed, Oct 26, 2022 at 03:56:07PM +0900, Michael Paquier wrote: > > So, I have spent a good portion of today looking at what you have > here, applying 0001 and 0003 while fixing, rebasing and testing the > whole, discarding 0002 (we could do more for the line number and > source file in terms of

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-26 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:19:48AM +0800, Julien Rouhaud wrote: > That wouldn't be overdoing anymore if we remove the line number / filename > from > the fill_*_line prototypes right? So, I have spent a good portion of today looking at what you have here, applying 0001 and 0003 while fixing,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Wed, Oct 26, 2022 at 11:19:48AM +0800, Julien Rouhaud wrote: > That wouldn't be overdoing anymore if we remove the line number / filename > from > the fill_*_line prototypes right? Yeah, but there is a twist: HbaLine or IdentLine can be passed as NULL when entering in fill_hba_line() or

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Julien Rouhaud
On Tue, Oct 25, 2022 at 08:59:57PM +0900, Michael Paquier wrote: > > Hmm. I would be tempted to keep track of the file name and the line > number as well in IdentLine. One reason is that this can become > useful for debugging. A second is that this can reduce a bit the > arguments of

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
On Tue, Oct 25, 2022 at 03:08:59PM +0800, Julien Rouhaud wrote: > On Tue, Oct 25, 2022 at 03:43:21PM +0900, Michael Paquier wrote: >> Another advantage is that it minimizes the presence of the hardcoded >> HbaFileName and IdentFileName in hba.c, which is one thing we are >> trying to achieve here

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Julien Rouhaud
On Tue, Oct 25, 2022 at 03:43:21PM +0900, Michael Paquier wrote: > > Another advantage is that it minimizes the presence of the hardcoded > HbaFileName and IdentFileName in hba.c, which is one thing we are > trying to achieve here for the inclusion of more files. I found a bit > strange that

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-25 Thread Michael Paquier
mapping_number = 0; MemoryContext linecxt; MemoryContext identcxt; MemoryContext oldcxt; @@ -529,8 +551,12 @@ fill_ident_view(Tuplestorestate *tuple_store, TupleDesc tupdesc) if (tok_line->err_msg == NULL) identline = parse_ident_line(tok_line, DEBUG3); - fill_ident_line(tuple_store, tupdes

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Julien Rouhaud
Hi, On Mon, Oct 24, 2022 at 04:13:51PM +0900, Michael Paquier wrote: > On Mon, Oct 24, 2022 at 01:33:12PM +0800, Julien Rouhaud wrote: > > v12 attached, fixing multiple conflicts with recent activity. > > typedef struct TokenizedAuthLine > { > List *fields; /* List of lists of

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-24 Thread Michael Paquier
On Mon, Oct 24, 2022 at 01:33:12PM +0800, Julien Rouhaud wrote: > v12 attached, fixing multiple conflicts with recent activity. typedef struct TokenizedAuthLine { List *fields; /* List of lists of AuthTokens */ + char *file_name; /* File name * Hmm. While

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-23 Thread Julien Rouhaud
r, map_name, sys_name, pg_username, error); pg_indexes| SELECT n.nspname AS schemaname, c.relname AS tablename, i.relname AS indexname, -- 2.37.0 >From 6c4827443c4ec4f386f4f3519c52cebf3b57d6b9 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 30 May 2022 11:15:06 +0800 Subject:

Re: Allow file inclusion in pg_hba and pg_ident files

2022-10-09 Thread Julien Rouhaud
r - FROM pg_ident_file_mappings() a(line_number, map_name, sys_name, pg_username, error); + FROM pg_ident_file_mappings() a(mapping_number, line_number, map_name, sys_name, pg_username, error); pg_indexes| SELECT n.nspname AS schemaname, c.relname AS tablename, i.relname AS indexname

Re: Allow file inclusion in pg_hba and pg_ident files

2022-09-17 Thread Julien Rouhaud
M pg_ident_file_mappings() a(mapping_number, line_number, map_name, sys_name, pg_username, error); pg_indexes| SELECT n.nspname AS schemaname, c.relname AS tablename, i.relname AS indexname, -- 2.37.0 >From e7cdde8d69bb2dfb9d338615baa687cc5321dc22 Mon Sep 17 00:00:0

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-16 Thread Julien Rouhaud
AS schemaname, c.relname AS tablename, i.relname AS indexname, -- 2.37.0 >From f50ef9722e38e2fd1f52cf8e6b31a126ecd811c7 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 30 May 2022 11:15:06 +0800 Subject: [PATCH v9 2/3] Allow file inclusion in pg_hba and pg_ident files. pg_h

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-06 Thread Julien Rouhaud
Hi, On Fri, Aug 05, 2022 at 09:56:29AM +0900, Michael Paquier wrote: > On Tue, Aug 02, 2022 at 07:32:54PM +0900, Michael Paquier wrote: > > As a quick update from my side, I intend to look and apply 0001~0003 > > (not double-checked yet) shortly. > > And a couple of days later, these look fine so

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-04 Thread Michael Paquier
On Tue, Aug 02, 2022 at 07:32:54PM +0900, Michael Paquier wrote: > As a quick update from my side, I intend to look and apply 0001~0003 > (not double-checked yet) shortly. And a couple of days later, these look fine so done as of 47ab1ac and 718fe0a. 0002 and 0003 have been merged together. --

Re: Allow file inclusion in pg_hba and pg_ident files

2022-08-02 Thread Michael Paquier
On Sat, Jul 30, 2022 at 04:09:36PM +0800, Julien Rouhaud wrote: > I've been working on all of that and came up with the attached v8. > > - 0001: I modified things as discussed previously to report the real auth file > names rather than the hardcoded "pg_ident.conf" and "pg_hba.conf" in the >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-31 Thread Julien Rouhaud
On Tue, Jul 26, 2022 at 1:14 PM Michael Paquier wrote: > > On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > > It doesn't have much impact most of the time. The filename is reported if > > there's an IO error while reading the already opened correct file. The real > > problem is

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-30 Thread Julien Rouhaud
ule_number, line_number, type, database, user_name, address, netmask, auth_method, options, error); +pg_ident_file_mappings| SELECT a.mapping_number, +a.line_number, a.map_name, a.sys_name, a.pg_username, a.error - FROM pg_ident_file_mappings() a(line_num

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Michael Paquier
On Tue, Jul 26, 2022 at 01:04:02PM +0800, Julien Rouhaud wrote: > It doesn't have much impact most of the time. The filename is reported if > there's an IO error while reading the already opened correct file. The real > problem is if the hba_file and ident_file are stored in different directory,

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-25 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:22:32PM +0900, Michael Paquier wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > See the attached, for reference, but it would fail with EXEC_BACKEND > > on WIN32. > > Ditto. While working on the full regression test coverage for the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-19 Thread Michael Paquier
On Mon, Jul 18, 2022 at 03:11:51PM +0800, Julien Rouhaud wrote: > So first, even if we can test 99% of the features with just testing the views > output, I think it's should use the TAP framework since the tests will have to > mess with the pg_ident/pg_hba files. It's way easier to modify the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-18 Thread Julien Rouhaud
Hi, On Mon, Jul 11, 2022 at 10:16:44AM +0900, Michael Paquier wrote: > On Fri, Jul 08, 2022 at 02:57:21PM +0800, Julien Rouhaud wrote: > > My apologies for the late reply. > > > I don't have an extensive knowledge of all the user facing error messages, > > but > > after a quick grep I see

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-10 Thread Michael Paquier
On Fri, Jul 08, 2022 at 02:57:21PM +0800, Julien Rouhaud wrote: My apologies for the late reply. > I don't have an extensive knowledge of all the user facing error messages, but > after a quick grep I see multiple usage of OID, PID, GIN and other defined > acronyms. I also see multiple

Re: Allow file inclusion in pg_hba and pg_ident files

2022-07-08 Thread Julien Rouhaud
Hi, On Thu, Jun 02, 2022 at 10:08:15AM +0900, Michael Paquier wrote: > On Thu, May 26, 2022 at 03:26:57PM +0800, Julien Rouhaud wrote: > > > After a bit more digging, I think that this comes from the fact that > > there's no > > "official" name for this file. Even the documentation just says

Re: Allow file inclusion in pg_hba and pg_ident files

2022-06-01 Thread Michael Paquier
On Thu, May 26, 2022 at 03:26:57PM +0800, Julien Rouhaud wrote: > So you mean having an error message like that (having an "include myconf" > in the HBA file): > > LOG: could not open authentication file "myconf" as "/path/to/myconf": No > such file or directory > LOG: pg_hba.conf was not

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-26 Thread Julien Rouhaud
On Tue, May 24, 2022 at 10:44:05AM +0900, Michael Paquier wrote: > On Mon, May 23, 2022 at 07:43:08PM +0800, Julien Rouhaud wrote: > > That being said, I'd gladly drop that enum and only handle a single error > > message, as the rest of the error context (including the owning file name > > and >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Michael Paquier
On Mon, May 23, 2022 at 07:43:08PM +0800, Julien Rouhaud wrote: > That being said, I'd gladly drop that enum and only handle a single error > message, as the rest of the error context (including the owning file name and > line) should provide enough information to users. > > If so, should I use

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Julien Rouhaud
Hi, On Mon, May 23, 2022 at 03:53:06PM +0900, Michael Paquier wrote: > > + switch (kind) > + { > + case SecondaryAuthFile: > + msglog = "could not open secondary authentication file > \"@%s\" as \"%s\": %m"; > + msgview = "could not open

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-23 Thread Michael Paquier
On Wed, May 18, 2022 at 03:12:45PM +0800, Julien Rouhaud wrote: > The cfbot reports that the patch doesn't apply anymore, rebased v7 attached. + switch (kind) + { + case SecondaryAuthFile: + msglog = "could not open secondary authentication file \"@%s\" as

Re: Allow file inclusion in pg_hba and pg_ident files

2022-05-18 Thread Julien Rouhaud
use case). The cfbot reports that the patch doesn't apply anymore, rebased v7 attached. >From 04943f3ff8dfb8efa5e538e0d9524fb041c3b39b Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 21 Feb 2022 15:45:26 +0800 Subject: [PATCH v7 1/2] Allow file inclusion in pg_hba and pg_ident fil

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-29 Thread Julien Rouhaud
>From e5dc352cd2eb8cff24714930338adb0b2d20ff94 Mon Sep 17 00:00:00 2001 From: Julien Rouhaud Date: Mon, 21 Feb 2022 15:45:26 +0800 Subject: [PATCH v6 1/2] Allow file inclusion in pg_hba and pg_ident files. Catversion is bumped. Author: Julien Rouhaud Reviewed-by: FIXME Discussion: https://postg

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 04:33:30PM +0800, Julien Rouhaud wrote: > Ok, v5 attached without the TAP tests and updated sysviews tests. The update of the query related to pg_hba_file_rules in the regression tests was independant, so I have split and applied that first, as of 091a971. Now, for the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Julien Rouhaud
= 0 as ok, count(*) FILTER (WHERE error IS NOT NULL) = 0 AS no_err + from pg_ident_file_mappings; -- There will surely be at least one active lock select count(*) > 0 as ok from pg_locks; -- 2.33.1 >From 84759803f64d3532341d0bf877e2a3a1e31554f2 Mon Sep 17 00:00:00 2001 From: Juli

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 03:43:41PM +0800, Julien Rouhaud wrote: > On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: >> We could use a failure path for each psql command rather than a SKIP >> block, as you told me, if the psql fails and check that we get some >> error strings related

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > > > Note that those tests fail on Windows (and I'm assuming on EXEC_BACKEND > > builds), as they're testing invalid files which by definition prevent any > > further connection attempt. I'm not sure what would be best to do

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Mon, Mar 28, 2022 at 04:20:07PM +0900, Michael Paquier wrote: > See the attached, for reference, but it would fail with EXEC_BACKEND > on WIN32. Ditto. -- Michael From 69e02734fd0199ba02cc34bc468b04584bdf0efd Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 28 Mar 2022 16:20:40 +0900

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-28 Thread Michael Paquier
On Sun, Mar 27, 2022 at 05:52:22PM +0800, Julien Rouhaud wrote: > I didn't like the various suggestions, as it would mean to scatter the tests > all over the place. The whole point of those views is indeed to check the > current content of a file without applying the configuration change (not on

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-27 Thread Julien Rouhaud
ok diff --git a/src/test/regress/sql/sysviews.sql b/src/test/regress/sql/sysviews.sql index 4980f07be2..1148014e47 100644 --- a/src/test/regress/sql/sysviews.sql +++ b/src/test/regress/sql/sysviews.sql @@ -28,6 +28,8 @@ select count(*) >= 0 as ok from pg_file_settings; -- There will surely be at least one rule select count

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-25 Thread Michael Paquier
On Thu, Mar 24, 2022 at 04:08:38PM +0800, Julien Rouhaud wrote: > On Thu, Mar 24, 2022 at 04:50:31PM +0900, Michael Paquier wrote: >> And so, this first part has been applied as of d4781d8. I'll try to >> look at 0002 shortly. > > Thanks! Now looking at 0002. The changes in hba.c are

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-24 Thread Julien Rouhaud
On Thu, Mar 24, 2022 at 04:50:31PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 01:14:02PM +0900, Michael Paquier wrote: > > On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > >> Yeah, I thought about it but didn't rename it given your concerns about git > >> history.

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-24 Thread Michael Paquier
On Wed, Mar 23, 2022 at 01:14:02PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: >> Yeah, I thought about it but didn't rename it given your concerns about git >> history. I'm fine either way. > > Oh, OK. The amount of diffs created by 0001 is

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Wed, Mar 23, 2022 at 10:16:34AM +0800, Julien Rouhaud wrote: > Yeah, I thought about it but didn't rename it given your concerns about git > history. I'm fine either way. Oh, OK. The amount of diffs created by 0001 is still fine to grab even with the struct rename, so let's stick with it. --

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Wed, Mar 23, 2022 at 11:03:46AM +0900, Michael Paquier wrote: > > Pushing forward with 0001 by the end of the CF is the part that has no > controversy IMO, and I have no objections to it. Now, after looking > at this part, I found a few things, as of: > - HbaToken, the set of elements in

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Michael Paquier
On Tue, Mar 22, 2022 at 09:38:00PM +0800, Julien Rouhaud wrote: > On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: >> Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also >> increase CATALOG_VERSION_NO? Not sure if we generally do this in the >> patches or

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Julien Rouhaud
Hi, On Tue, Mar 22, 2022 at 03:21:20PM +0300, Aleksander Alekseev wrote: > > The v3-0001 patch LGTM. > > Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also > increase CATALOG_VERSION_NO? Not sure if we generally do this in the > patches or expect the committer to make the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Aleksander Alekseev
Hi hackers, > It passes `make installcheck` ... `make installcheck-world`, of course. Sorry for the confusion. -- Best regards, Aleksander Alekseev

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-22 Thread Aleksander Alekseev
Hi hackers, > The cfbot says that the patch doesn't apply anymore, so here's a v3 with the > changes mentioned below. I came across this thread while looking for the patches that need review. The v3-0001 patch LGTM. Since v3-0002 adds a new view and alters pg_proc.dat shouldn't it also

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-15 Thread Julien Rouhaud
Hi, The cfbot says that the patch doesn't apply anymore, so here's a v3 with the changes mentioned below. On Tue, Mar 01, 2022 at 05:19:50PM +0800, Julien Rouhaud wrote: > > If you prefer to interleave static and non static function I can change it. Change the split to not reorder functions. >

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-01 Thread Michael Paquier
On Tue, Mar 01, 2022 at 05:19:50PM +0800, Julien Rouhaud wrote: > On Tue, Mar 01, 2022 at 04:45:48PM +0900, Michael Paquier wrote: >> Hmm. The diffs of 0001 are really hard to read. Do you know why this >> is happening? Is that because some code has been moved around? > > Yes, I followed the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-03-01 Thread Julien Rouhaud
Hi, On Tue, Mar 01, 2022 at 04:45:48PM +0900, Michael Paquier wrote: > > Hmm. The diffs of 0001 are really hard to read. Do you know why this > is happening? Is that because some code has been moved around? Yes, I followed the file convention to put the static functions first and then the

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Michael Paquier
On Mon, Feb 28, 2022 at 07:42:17PM +0800, Julien Rouhaud wrote: > Done in attached v2. I did the split in a separate commit, as the diff is > otherwise unreadable. While at it I also fixed a few minor issues (I missed a > MemoryContextDelete, and now avoid relying on inet_net_pton which

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Julien Rouhaud
Hi, On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > > > Finally I also added 0003, which is a POC for a new pg_hba_matches() > > function, > > that can help DBA to understand why their configuration isn't working as > > they > > expect. This only to start the discussion on

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-28 Thread Michael Paquier
On Sat, Feb 26, 2022 at 02:50:33PM +0800, Julien Rouhaud wrote: > Of course. I was thinking using "auth" for something that's common to pg_hba > and pg_ident (like e.g. TokenizeAuthFile()), and otherwise keep the current > hba/ident prefix. Okay, thanks. > Unless someone object or suggest

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Julien Rouhaud
On Sat, Feb 26, 2022 at 03:36:19PM +0900, Michael Paquier wrote: > On Sat, Feb 26, 2022 at 02:27:15PM +0800, Julien Rouhaud wrote: > > > Note that in order to do so we would need to expose quite a lot more about > > hba > > internals, like tokenize_file() and parse_hba_line(), along with structs

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Michael Paquier
On Sat, Feb 26, 2022 at 02:27:15PM +0800, Julien Rouhaud wrote: > I'm fine with it. Assuming that you meant to move also the underlying > functions that goes with it (fill_hba_line and such), that would end up > removing about 15% of hba.c (after applying 0001, 0002 and 0003). Cool. Thanks for

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Julien Rouhaud
Hi, On Sat, Feb 26, 2022 at 03:04:43PM +0900, Michael Paquier wrote: > On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > > On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > >> 0001 adds a new pg_ident_file_mappings view, which is basically the same as > >>

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-25 Thread Michael Paquier
On Wed, Feb 23, 2022 at 09:44:58AM -0800, Nathan Bossart wrote: > On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: >> 0001 adds a new pg_ident_file_mappings view, which is basically the same as >> pg_hba_file_rules view but for mappings. It's probably already useful, for >>

Re: Allow file inclusion in pg_hba and pg_ident files

2022-02-23 Thread Nathan Bossart
On Wed, Feb 23, 2022 at 12:59:59PM +0800, Julien Rouhaud wrote: > To address that, I'd like to propose the possibility to include files in hba > and ident configuration files. This was already discussed in the past, and in > my understanding this is mostly wanted, while some people expressed

Allow file inclusion in pg_hba and pg_ident files

2022-02-22 Thread Julien Rouhaud
a/src/test/regress/sql/sysviews.sql +++ b/src/test/regress/sql/sysviews.sql @@ -28,6 +28,9 @@ select count(*) >= 0 as ok from pg_file_settings; -- There will surely be at least one rule select count(*) > 0 as ok from pg_hba_file_rules; +-- We expect no user mapping in this test +selec