Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 9/29/16 10:06 AM, Tom Lane wrote:
> Personally I'm also on board with using this for regression testing:
> 
>   log_line_prefix = '%t [%p] %q%a '

Committed that way, but with %m instead of %t, as discussed earlier.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Alvaro Herrera]

Tom Lane wrote:
> Robert Haas  writes:
> > As long as we get %t and %p in there we're going to be way ahead, really.
> 
> Could we get consensus on just changing the default to
> 
>   log_line_prefix = '%t [%p] '
> 
> and leaving the rest out of it?

+1 from me.

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Christoph Berg  writes:
> Re: Tom Lane 2016-09-29 <16946.1475157...@sss.pgh.pa.us>
>> Personally I'm also on board with using this for regression testing:
>> log_line_prefix = '%t [%p] %q%a '
>> but I doubt it can be sold as a general-purpose default.

> I don't think it makes much sense to log only %a unconditionally,

Right; this is helpful for the regression tests, now that Peter has set
up pg_regress to set the application name, but I can't see trying to
push it on the rest of the world.

> Possibly the longer version could be added as an example in the
> documentation.

I suspect that simply having a nonempty default in the first place
is going to do more to raise peoples' awareness than anything we
could do in the documentation.  But perhaps an example along these
lines would be useful for showing proper use of %q.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Christoph Berg]

Re: Tom Lane 2016-09-29 <16946.1475157...@sss.pgh.pa.us>
> Robert Haas  writes:
> > As long as we get %t and %p in there we're going to be way ahead, really.
> 
> Could we get consensus on just changing the default to
> 
>   log_line_prefix = '%t [%p] '
> 
> and leaving the rest out of it?  I think pretty much everybody agrees
> that those fields are useful, whereas the rest of it is a lot more
> context-dependent.  (For example, there are a whole lot of real
> installations where neither %u nor %d would be worth the log space.)

Nod. In many installations %u and %d will be effectively constants.

> Personally I'm also on board with using this for regression testing:
> 
>   log_line_prefix = '%t [%p] %q%a '
> 
> but I doubt it can be sold as a general-purpose default.

I don't think it makes much sense to log only %a unconditionally,
except maybe for making more applications actually set it.

If we were to add more fields I'd go for

log_line_prefix = '%t [%p] %q%u@%d/%a '

but the above-proposed '%t [%p] ' is already fixing 95% of the problem
(and the remaining 5% are unclear).

Possibly the longer version could be added as an example in the
documentation.

So, in short, +1.

Christoph


signature.asc
Description: PGP signature

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Christoph Berg]

Re: Peter Eisentraut 2016-09-29 
<21d2719f-36ff-06d2-5856-25ed48b96...@2ndquadrant.com>
> > Christoph/Debian:
> > log_line_prefix = '%t [%p-%l] %q%u@%d '
> > Peter:
> > log_line_prefix = '%t [%p]: [%l] %qapp=%a '
> 
> I'm aware of two existing guidelines on log line formats: syslog and
> pgbadger.  Syslog output looks like this:
> 
> Sep 28 00:58:56 hostname syslogd[46]: some text here
> 
> pgbadger by default asks for this:
> 
> log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
> 
> I don't know why it wants that "-1" there, and I'm actually not sure
> what the point of %l is in practice.  Those are separate issues that are
> having their own lively discussions at times.  I could drop the [%l]
> from my proposal if that causes concerns.

[%l-1] is originally from pgfouine, and I vaguely remember that it
used to be something like [%l-%c] where %c was some extra line
numbering removed in later (7.something?) PG versions. In any case,
the -1 isn't useful.

I'm happy to remove %l as well. Log lines won't be out of order
anyway, and one needs to look at %p anyway to correlate them. %l
doesn't help there.

Christoph


signature.asc
Description: PGP signature

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Robert Haas  writes:
> As long as we get %t and %p in there we're going to be way ahead, really.

Could we get consensus on just changing the default to

log_line_prefix = '%t [%p] '

and leaving the rest out of it?  I think pretty much everybody agrees
that those fields are useful, whereas the rest of it is a lot more
context-dependent.  (For example, there are a whole lot of real
installations where neither %u nor %d would be worth the log space.)

Personally I'm also on board with using this for regression testing:

log_line_prefix = '%t [%p] %q%a '

but I doubt it can be sold as a general-purpose default.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Robert Haas]

On Wed, Sep 28, 2016 at 10:30 PM, Peter Eisentraut
 wrote:
> On 9/28/16 6:13 PM, Robert Haas wrote:
>> Christoph/Debian:
>> log_line_prefix = '%t [%p-%l] %q%u@%d '
>> Peter:
>> log_line_prefix = '%t [%p]: [%l] %qapp=%a '
>
> I'm aware of two existing guidelines on log line formats: syslog and
> pgbadger.  Syslog output looks like this:
>
> Sep 28 00:58:56 hostname syslogd[46]: some text here
>
> pgbadger by default asks for this:
>
> log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
>
> I don't know why it wants that "-1" there, and I'm actually not sure
> what the point of %l is in practice.  Those are separate issues that are
> having their own lively discussions at times.  I could drop the [%l]
> from my proposal if that causes concerns.
>
> On balance, I think my proposal is more in line with existing
> wide-spread conventions.

As long as we get %t and %p in there we're going to be way ahead, really.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Peter Eisentraut  writes:
> On 9/28/16 6:13 PM, Robert Haas wrote:
>> Christoph/Debian:
>> log_line_prefix = '%t [%p-%l] %q%u@%d '
>> Peter:
>> log_line_prefix = '%t [%p]: [%l] %qapp=%a '

> ...
> I don't know why it wants that "-1" there, and I'm actually not sure
> what the point of %l is in practice.  Those are separate issues that are
> having their own lively discussions at times.  I could drop the [%l]
> from my proposal if that causes concerns.

+1 for dropping %l --- seems to me that its main result is to add useless
bytes to the log.  Surely if you need to match up lines from the same
process, that's not that hard as long as %p is in there.

I'd also vote for dropping "app=" out of the regression test version;
again, that seems like basically dead weight.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 9/28/16 6:13 PM, Robert Haas wrote:
> Christoph/Debian:
> log_line_prefix = '%t [%p-%l] %q%u@%d '
> Peter:
> log_line_prefix = '%t [%p]: [%l] %qapp=%a '

I'm aware of two existing guidelines on log line formats: syslog and
pgbadger.  Syslog output looks like this:

Sep 28 00:58:56 hostname syslogd[46]: some text here

pgbadger by default asks for this:

log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '

I don't know why it wants that "-1" there, and I'm actually not sure
what the point of %l is in practice.  Those are separate issues that are
having their own lively discussions at times.  I could drop the [%l]
from my proposal if that causes concerns.

On balance, I think my proposal is more in line with existing
wide-spread conventions.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 9/28/16 6:07 PM, Alvaro Herrera wrote:
> Adopting a default prefix is a different question.

A default prefix would require different settings for syslog, plain
text, and possibly some of the other variants.  I'm all in favor of
figuring that out, but it needs more work.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Alvaro Herrera  writes:
> Tom Lane wrote:
>> Perhaps we should first try to get a consensus on the regression test
>> use-case.

> I thought Peter's suggestion for regression test drivers was a good one
> and I see no reason to block that.  Why do you (Tom) object so strongly
> against having a different one on buildfarm than elsewhere?  I'd rather
> have buildfarm adopt the new suggestion than having buildfarm drive the
> new stuff.

Well, my point is only that if you can't convince Andrew to sync the
buildfarm's choices with whatever your proposal is, then you haven't
got consensus.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Robert Haas]

On Wed, Sep 28, 2016 at 6:07 PM, Alvaro Herrera
 wrote:
> I thought Peter's suggestion for regression test drivers was a good one
> and I see no reason to block that.  Why do you (Tom) object so strongly
> against having a different one on buildfarm than elsewhere?  I'd rather
> have buildfarm adopt the new suggestion than having buildfarm drive the
> new stuff.
>
> Adopting a default prefix is a different question.  For one thing IMHO
> it should not have %a (application name).  Christoph's suggestion
> (Debian's default) seemed good.

Yeah, I like Cristoph's suggestion fine.  It meets my criteria of
"includes timestamp and PID" and overall seems reasonable.   If we
adopted that across the board, it wouldn't be too much different from
what Peter proposed for the regression test.  Just to compare.

Christoph/Debian:
log_line_prefix = '%t [%p-%l] %q%u@%d '
Peter:
log_line_prefix = '%t [%p]: [%l] %qapp=%a '

So Peter's got %p and %l separated by "]: [" whereas Christoph has
them separated only by a dash.  Presumably that's minor.  Then they've
both got %q.  After that, Christoph has %u@%d, which seems reasonable
for an actual system, and Peter's got app=%a, which is better for the
regression tests because the user name will depend on the UNIX
username of the person running the tests.

So how about we adopt both suggestions, except changing Peter's to '%t
[%p-%l] %qapp=%a ' so that they are a bit more similar?  I bet that
would make more people happier than it would make less happy.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Alvaro Herrera]

Tom Lane wrote:
> Robert Haas  writes:
> > On Thu, Sep 15, 2016 at 5:18 PM, Robert Haas  wrote:
> >> On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane  wrote:
> >>> I think the odds of getting to something that everyone would agree on
> >>> are nil, so I'm not excited about getting into that particular
> >>> bikeshed-painting discussion.  Look at the amount of trouble we're
> >>> having converging on a default for the regression tests, which are
> >>> a far narrower use-case than "everybody".
> 
> >> Well, practically anything that includes a PID and the timestamp is
> >> going to be an improvement over the status quo.  Just because we can't
> >> all agree on what would be perfect does not mean that we can't do
> >> better than what we've got now.  +1 for trying.
> 
> > Is there any chance we can move forward here, or is this effort doomed for 
> > now?
> 
> It seemed like nobody wanted to try to push this forward, and it will take
> somebody actively pushing, IMO, for something to happen.
> 
> Perhaps we should first try to get a consensus on the regression test
> use-case.

I thought Peter's suggestion for regression test drivers was a good one
and I see no reason to block that.  Why do you (Tom) object so strongly
against having a different one on buildfarm than elsewhere?  I'd rather
have buildfarm adopt the new suggestion than having buildfarm drive the
new stuff.

Adopting a default prefix is a different question.  For one thing IMHO
it should not have %a (application name).  Christoph's suggestion
(Debian's default) seemed good.

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Robert Haas  writes:
> On Thu, Sep 15, 2016 at 5:18 PM, Robert Haas  wrote:
>> On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane  wrote:
>>> I think the odds of getting to something that everyone would agree on
>>> are nil, so I'm not excited about getting into that particular
>>> bikeshed-painting discussion.  Look at the amount of trouble we're
>>> having converging on a default for the regression tests, which are
>>> a far narrower use-case than "everybody".

>> Well, practically anything that includes a PID and the timestamp is
>> going to be an improvement over the status quo.  Just because we can't
>> all agree on what would be perfect does not mean that we can't do
>> better than what we've got now.  +1 for trying.

> Is there any chance we can move forward here, or is this effort doomed for 
> now?

It seemed like nobody wanted to try to push this forward, and it will take
somebody actively pushing, IMO, for something to happen.

Perhaps we should first try to get a consensus on the regression test
use-case.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Christoph Berg]

Re: Robert Haas 2016-09-28 

> > Well, practically anything that includes a PID and the timestamp is
> > going to be an improvement over the status quo.  Just because we can't
> > all agree on what would be perfect does not mean that we can't do
> > better than what we've got now.  +1 for trying.
> 
> Is there any chance we can move forward here, or is this effort doomed for 
> now?

IMHO it would make sense. Maybe we should collect a few suggestions,
and then take a poll?

Christoph


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Robert Haas]

On Thu, Sep 15, 2016 at 5:18 PM, Robert Haas  wrote:
> On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane  wrote:
>> Christoph Berg  writes:
>>> I've always been wondering why we don't set a log_line_prefix by
>>> default.
>>
>> I think the odds of getting to something that everyone would agree on
>> are nil, so I'm not excited about getting into that particular
>> bikeshed-painting discussion.  Look at the amount of trouble we're
>> having converging on a default for the regression tests, which are
>> a far narrower use-case than "everybody".
>
> Well, practically anything that includes a PID and the timestamp is
> going to be an improvement over the status quo.  Just because we can't
> all agree on what would be perfect does not mean that we can't do
> better than what we've got now.  +1 for trying.

Is there any chance we can move forward here, or is this effort doomed for now?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Robert Haas]

On Sat, Aug 27, 2016 at 3:59 PM, Tom Lane  wrote:
> Christoph Berg  writes:
>> I've always been wondering why we don't set a log_line_prefix by
>> default.
>
> I think the odds of getting to something that everyone would agree on
> are nil, so I'm not excited about getting into that particular
> bikeshed-painting discussion.  Look at the amount of trouble we're
> having converging on a default for the regression tests, which are
> a far narrower use-case than "everybody".

Well, practically anything that includes a PID and the timestamp is
going to be an improvement over the status quo.  Just because we can't
all agree on what would be perfect does not mean that we can't do
better than what we've got now.  +1 for trying.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Christoph Berg  writes:
> I've always been wondering why we don't set a log_line_prefix by
> default.

I think the odds of getting to something that everyone would agree on
are nil, so I'm not excited about getting into that particular
bikeshed-painting discussion.  Look at the amount of trouble we're
having converging on a default for the regression tests, which are
a far narrower use-case than "everybody".

> The above looks quite similar to what the Debian packages have been
> setting as their default for some time, with standard stderr logging:

I think Debian's choice was probably made by fiat, not by consensus.
Packagers seem to be able to get away with quite a lot in that regard.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Christoph Berg]

Re: Fabien COELHO 2016-08-26 
> So I would suggest something like the following, which is also a little bit
> more compact:
> 
>   log_line_prefix = '%m [%p:%l] %q%a '
> 
> If you want to keep something with %a, maybe parentheses?
> 
> Finally I'm wondering also whether a timestamp since the server has started
> (which does not exists) would be more useful for a "make check", or at
> default maybe %n?

I've always been wondering why we don't set a log_line_prefix by
default. Logs without timestamps and (pid or session id or equivalent)
are useless. Of course in practise the log_line_prefix needs to be
different depending on the log_destination (syslog adds its own
timestamps, ...), but the current default of '' doesn't help anyone.

The above looks quite similar to what the Debian packages have been
setting as their default for some time, with standard stderr logging:

log_line_prefix = '%t [%p-%l] %q%u@%d '

People who want a different log channel need to touch the config
anyway and can update log_line_prefix as they go.

The concrete value to be used needs to be discussed, but I think we'd
end up with something like '%m [%p:%l] ' plus maybe some suffix.

Christoph


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Fabien COELHO]

Hello Peter,


log_line_prefix = '%t [%p]: [%l] %qapp=%a '

which is modeled after the pgfouine recommendation, which is I believe a
wide-spread convention, and it also vaguely follows syslog customs.

The build farm client has

log_line_prefix = '%m [%c:%l] '

which is very similar, but the lack of the PID makes it unsuitable for
the purposes that I have set out, and there is no obvious place to put
additional information such as %a.

%m vs %t is obviously a minor issue that I will gladly adjust, but
besides that I prefer to stick with my version.


v2 patch looks ok, applies without trouble and works as intended:

  2016-08-26 09:19:31.191 CEST [7571]: [58] app=pg_regress/event_trigger 
STATEMENT:  alter type rewritetype alter attribute a type varchar cascade;

About the format: '[\d+]' pattern is used twice, which makes the output 
less easily grep-able.


Also, the ':' is used as a separator in the remainder of the message, so 
maybe once is enough at this level.


I'm not sure about the "app=" is really necessary, given its very explicit 
definition as can be seen above above.


So I would suggest something like the following, which is also a little 
bit more compact:


  log_line_prefix = '%m [%p:%l] %q%a '

If you want to keep something with %a, maybe parentheses?

Finally I'm wondering also whether a timestamp since the server has 
started (which does not exists) would be more useful for a "make check", 
or at default maybe %n?


--
Fabien


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 8/16/16 2:23 PM, Peter Eisentraut wrote:
> On 8/10/16 9:36 PM, Peter Eisentraut wrote:
>> %m vs %t is obviously a minor issue that I will gladly adjust, but
>> besides that I prefer to stick with my version.
> 
> Updated patch with %m instead of %t.  Will submit to CF.

attached

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>From 1fed06603c0c0cacfb78ccc2985d77bee527fad7 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 16 Aug 2016 12:00:00 -0400
Subject: [PATCH v2] Set log_line_prefix and application name in test drivers

Before pg_regress runs psql, set the application name to the test name.
Similarly, set the application name to the test file name in the TAP
tests.  Also, set a default log_line_prefix that show the application
name, as well as the PID and a time stamp.

That way, the server log output can be correlated to the test input
files, making debugging a bit easier.
---
 src/test/perl/PostgresNode.pm  | 1 +
 src/test/perl/TestLib.pm   | 2 ++
 src/test/regress/pg_regress.c  | 1 +
 src/test/regress/pg_regress_main.c | 7 +++
 4 files changed, 11 insertions(+)

diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index fede1e6..b3a5457 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -402,6 +402,7 @@ sub init
 	open my $conf, ">>$pgdata/postgresql.conf";
 	print $conf "\n# Added by PostgresNode.pm\n";
 	print $conf "fsync = off\n";
+	print $conf "log_line_prefix = '%m [%p]: [%l] %qapp=%a '\n";
 	print $conf "log_statement = all\n";
 	print $conf "port = $port\n";
 
diff --git a/src/test/perl/TestLib.pm b/src/test/perl/TestLib.pm
index 649fd82..27fcc78 100644
--- a/src/test/perl/TestLib.pm
+++ b/src/test/perl/TestLib.pm
@@ -60,6 +60,8 @@ BEGIN
 	delete $ENV{PGPORT};
 	delete $ENV{PGHOST};
 
+	$ENV{PGAPPNAME} = $0;
+
 	# Must be set early
 	$windows_os = $Config{osname} eq 'MSWin32' || $Config{osname} eq 'msys';
 }
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 574f5b8..1d6e1d8 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -2247,6 +2247,7 @@ regression_main(int argc, char *argv[], init_function ifunc, test_function tfunc
 		fputs("\n# Configuration added by pg_regress\n\n", pg_conf);
 		fputs("log_autovacuum_min_duration = 0\n", pg_conf);
 		fputs("log_checkpoints = on\n", pg_conf);
+		fputs("log_line_prefix = '%m [%p]: [%l] %qapp=%a '\n", pg_conf);
 		fputs("log_lock_waits = on\n", pg_conf);
 		fputs("log_temp_files = 128kB\n", pg_conf);
 		fputs("max_prepared_transactions = 2\n", pg_conf);
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index d9591c0..2733635 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -34,6 +34,7 @@ psql_start_test(const char *testname,
 	char		expectfile[MAXPGPATH];
 	char		psql_cmd[MAXPGPATH * 3];
 	size_t		offset = 0;
+	char	   *appnameenv;
 
 	/*
 	 * Look for files in the output dir first, consistent with a vpath search.
@@ -63,6 +64,9 @@ psql_start_test(const char *testname,
 		offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
 		   "%s ", launcher);
 
+	appnameenv = psprintf("PGAPPNAME=pg_regress/%s", testname);
+	putenv(appnameenv);
+
 	snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
 			 "\"%s%spsql\" -X -a -q -d \"%s\" < \"%s\" > \"%s\" 2>&1",
 			 bindir ? bindir : "",
@@ -80,6 +84,9 @@ psql_start_test(const char *testname,
 		exit(2);
 	}
 
+	unsetenv("PGAPPNAME");
+	free(appnameenv);
+
 	return pid;
 }
 
-- 
2.9.3


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 8/10/16 9:36 PM, Peter Eisentraut wrote:
> %m vs %t is obviously a minor issue that I will gladly adjust, but
> besides that I prefer to stick with my version.

Updated patch with %m instead of %t.  Will submit to CF.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 8/10/16 5:18 PM, Tom Lane wrote:
> Or in short: I don't want to be seeing one prefix format in some buildfarm
> logs and a different format in others.

Sure.  My patch has

log_line_prefix = '%t [%p]: [%l] %qapp=%a '

which is modeled after the pgfouine recommendation, which is I believe a
wide-spread convention, and it also vaguely follows syslog customs.

The build farm client has

log_line_prefix = '%m [%c:%l] '

which is very similar, but the lack of the PID makes it unsuitable for
the purposes that I have set out, and there is no obvious place to put
additional information such as %a.

%m vs %t is obviously a minor issue that I will gladly adjust, but
besides that I prefer to stick with my version.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Peter Eisentraut  writes:
> On 8/9/16 12:16 PM, Tom Lane wrote:
>> Peter Eisentraut  writes:
>>> Here is a small patch that sets log_line_prefix and application name in
>>> pg_regress and the TAP tests, to make analyzing the server log output
>>> easier.

>> How would this interact with the buildfarm's existing policies
>> on setting log_line_prefix?

> AFAICT, that only applies if the build farm client runs initdb itself,
> that is, for the installcheck parts.

Well, I guess the subtext of my question was whether we shouldn't try
to align this with the buildfarm's choices, or vice versa.  Andrew made
some different choices than you have done here, and it seems like we
ought to strive for a meeting of the minds on what's appropriate.

Or in short: I don't want to be seeing one prefix format in some buildfarm
logs and a different format in others.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

On 8/9/16 12:16 PM, Tom Lane wrote:
> Peter Eisentraut  writes:
>> > Here is a small patch that sets log_line_prefix and application name in
>> > pg_regress and the TAP tests, to make analyzing the server log output
>> > easier.
> How would this interact with the buildfarm's existing policies
> on setting log_line_prefix?

AFAICT, that only applies if the build farm client runs initdb itself,
that is, for the installcheck parts.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Set log_line_prefix and application name in test drivers

[Tom Lane]

Peter Eisentraut  writes:
> Here is a small patch that sets log_line_prefix and application name in
> pg_regress and the TAP tests, to make analyzing the server log output
> easier.

How would this interact with the buildfarm's existing policies
on setting log_line_prefix?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Set log_line_prefix and application name in test drivers

[Peter Eisentraut]

Here is a small patch that sets log_line_prefix and application name in
pg_regress and the TAP tests, to make analyzing the server log output
easier.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>From 0595672e9272a53162eda315c15835c26fdb6d10 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 9 Aug 2016 11:56:49 -0400
Subject: [PATCH] Set log_line_prefix and application name in test drivers

Before pg_regress runs psql, set the application name to the test name.
Similarly, set the application name to the test file name in the TAP
tests.  Also, set a default log_line_prefix that show the application
name, as well as the PID and a time stamp.

That way, the server log output can be correlated to the test input
files, making debugging a bit easier.
---
 src/test/perl/PostgresNode.pm  | 1 +
 src/test/perl/TestLib.pm   | 2 ++
 src/test/regress/pg_regress.c  | 1 +
 src/test/regress/pg_regress_main.c | 7 +++
 4 files changed, 11 insertions(+)

diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index e629373..3f2ca91 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -402,6 +402,7 @@ sub init
 	open my $conf, ">>$pgdata/postgresql.conf";
 	print $conf "\n# Added by PostgresNode.pm\n";
 	print $conf "fsync = off\n";
+	print $conf "log_line_prefix = '%t [%p]: [%l] %qapp=%a '\n";
 	print $conf "log_statement = all\n";
 	print $conf "port = $port\n";
 
diff --git a/src/test/perl/TestLib.pm b/src/test/perl/TestLib.pm
index 649fd82..27fcc78 100644
--- a/src/test/perl/TestLib.pm
+++ b/src/test/perl/TestLib.pm
@@ -60,6 +60,8 @@ BEGIN
 	delete $ENV{PGPORT};
 	delete $ENV{PGHOST};
 
+	$ENV{PGAPPNAME} = $0;
+
 	# Must be set early
 	$windows_os = $Config{osname} eq 'MSWin32' || $Config{osname} eq 'msys';
 }
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 574f5b8..d5a8e16 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -2247,6 +2247,7 @@ regression_main(int argc, char *argv[], init_function ifunc, test_function tfunc
 		fputs("\n# Configuration added by pg_regress\n\n", pg_conf);
 		fputs("log_autovacuum_min_duration = 0\n", pg_conf);
 		fputs("log_checkpoints = on\n", pg_conf);
+		fputs("log_line_prefix = '%t [%p]: [%l] %qapp=%a '\n", pg_conf);
 		fputs("log_lock_waits = on\n", pg_conf);
 		fputs("log_temp_files = 128kB\n", pg_conf);
 		fputs("max_prepared_transactions = 2\n", pg_conf);
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index d9591c0..2733635 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -34,6 +34,7 @@ psql_start_test(const char *testname,
 	char		expectfile[MAXPGPATH];
 	char		psql_cmd[MAXPGPATH * 3];
 	size_t		offset = 0;
+	char	   *appnameenv;
 
 	/*
 	 * Look for files in the output dir first, consistent with a vpath search.
@@ -63,6 +64,9 @@ psql_start_test(const char *testname,
 		offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
 		   "%s ", launcher);
 
+	appnameenv = psprintf("PGAPPNAME=pg_regress/%s", testname);
+	putenv(appnameenv);
+
 	snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
 			 "\"%s%spsql\" -X -a -q -d \"%s\" < \"%s\" > \"%s\" 2>&1",
 			 bindir ? bindir : "",
@@ -80,6 +84,9 @@ psql_start_test(const char *testname,
 		exit(2);
 	}
 
+	unsetenv("PGAPPNAME");
+	free(appnameenv);
+
 	return pid;
 }
 
-- 
2.9.2


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers