Re: [HACKERS] Non-empty default log_line_prefix

[Robert Haas]

On Fri, Oct 14, 2016 at 7:04 AM, Christoph Berg  wrote:
> Re: Stephen Frost 2016-10-12 <20161012190732.gj13...@tamriel.snowman.net>
>> For my 2c, I'd rather have %m, but I definitely agree with Robert that
>> we need to do *something* here and if the only thing holding us back is
>> %t vs. %m, then let's just pick one and move on.  I'll just hold my nose
>> when I see the default and change it to %m.
>
> Here's the very same patch with %m instead of %t. Pick one :)
>
> (Fwiw, I'm still leaning towards %t, but my eyes are becoming more and
> more accustomed to %m as well. I'd be fine with it as well. (I'd
> rather want to try to get rid of the timezone identifier there...))

So, surveying this thread, what I see is:

People who prefer %m: Jeff Janes, Devrim Gündüz, Stephen Frost
People who prefer %t: Christoph Berg, Tom Lane (I think)

We can kibitz this more later, but for now I've committed it with %m.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Christoph Berg]

Re: Stephen Frost 2016-10-12 <20161012190732.gj13...@tamriel.snowman.net>
> For my 2c, I'd rather have %m, but I definitely agree with Robert that
> we need to do *something* here and if the only thing holding us back is
> %t vs. %m, then let's just pick one and move on.  I'll just hold my nose
> when I see the default and change it to %m.

Here's the very same patch with %m instead of %t. Pick one :)

(Fwiw, I'm still leaning towards %t, but my eyes are becoming more and
more accustomed to %m as well. I'd be fine with it as well. (I'd
rather want to try to get rid of the timezone identifier there...))

Christoph
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
new file mode 100644
index e826c19..bec9483
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*** local0.*/var/log/postgresql
*** 5004,5010 
   value will pad on the left. Padding can be useful to aid human
   readability in log files.
   This parameter can only be set in the postgresql.conf
!  file or on the server command line. The default is an empty string.
  
   

--- 5004,5011 
   value will pad on the left. Padding can be useful to aid human
   readability in log files.
   This parameter can only be set in the postgresql.conf
!  file or on the server command line. The default is
!  %m [%p]  which logs a time stamp and the process ID.
  
   

*** FROM pg_stat_activity;
*** 5142,5147 
--- 5143,5159 
   include those escapes if you are logging to syslog.
  
 
+ 
+
+ 
+  The %q escape is useful when including information that 
is
+  only available in session (backend) context like user or database
+  name. An example would be:
+ 
+ log_line_prefix = '%m [%p] %q%u@%d/%a '
+ 
+ 
+

   
  
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index 622279b..65660c1
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*** static struct config_string ConfigureNam
*** 3014,3020 
gettext_noop("If blank, no prefix is used.")
},
_line_prefix,
!   "",
NULL, NULL, NULL
},
  
--- 3014,3020 
gettext_noop("If blank, no prefix is used.")
},
_line_prefix,
!   "%m [%p] ",
NULL, NULL, NULL
},
  
diff --git a/src/backend/utils/misc/postgresql.conf.sample 
b/src/backend/utils/misc/postgresql.conf.sample
new file mode 100644
index 05b1373..159ada3
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***
*** 430,436 
  #log_duration = off
  #log_error_verbosity = default# terse, default, or verbose 
messages
  #log_hostname = off
! #log_line_prefix = '' # special values:
#   %a = application name
#   %u = user name
#   %d = database name
--- 430,436 
  #log_duration = off
  #log_error_verbosity = default# terse, default, or verbose 
messages
  #log_hostname = off
! #log_line_prefix = '%m [%p] ' # special values:
#   %a = application name
#   %u = user name
#   %d = database name

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Alvaro Herrera]

Peter Eisentraut wrote:
> On 10/12/16 11:58 AM, Christoph Berg wrote:
> > (Yes, the '' default might be fine for syslog, but I don't think
> > that's a good argument for sticking with it for default installs. I've
> > seen way too many useless log files out there, and at worst we'll have
> > syslogs with two timestamps.)
> 
> We'd have to think of a way to sort this out during upgrades.

Sure, but we don't have to do that in this patch, do we?

Some systems such as Debian copy the postgresql.conf file from the old
installation to the new one, and then have specific code to adjust
settings that no longer exist in the new version.  I'm not a fan of that
approach myself precisely because if PGDG installs new defaults (such as
in this case), the sites using that upgrade tool don't benefit.  A
better approach might be to copy over the non-default settings from the
old install to the new.  If Debian were to do that then we wouldn't need
to do anything here.

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Peter Eisentraut]

On 10/12/16 11:58 AM, Christoph Berg wrote:
> (Yes, the '' default might be fine for syslog, but I don't think
> that's a good argument for sticking with it for default installs. I've
> seen way too many useless log files out there, and at worst we'll have
> syslogs with two timestamps.)

We'd have to think of a way to sort this out during upgrades.
Documentation might be enough.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Peter Eisentraut]

On 10/12/16 11:40 AM, Tom Lane wrote:
> There would be some value in the complexity you're thinking of for
> installations that log to multiple targets concurrently, but really,
> who does that?

I see that a lot.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Stephen Frost]

* Robert Haas (robertmh...@gmail.com) wrote:
> On Wed, Oct 12, 2016 at 11:20 AM, Christoph Berg  wrote:
> > Re: Jeff Janes 2016-10-12 
> > 

Re: [HACKERS] Non-empty default log_line_prefix

[Devrim Gündüz]

Hi,

On Wed, 2016-10-12 at 13:11 -0400, Tom Lane wrote:
> > What is the cost of using %m, other than 4 (rather compressible) bytes per
> > log entry?
> 
> More log I/O, which is not free ...

FWIW, we've been setting log_line_prefix to '< %m > ' for quite a long time in
PGDG RPMs, and did not get any complaints. I'd vote for %m for default.

Regards,
-- 
Devrim GÜNDÜZ
EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Twitter: @DevrimGunduz , @DevrimGunduzTR


signature.asc
Description: This is a digitally signed message part

Re: [HACKERS] Non-empty default log_line_prefix

[Robert Haas]

On Wed, Oct 12, 2016 at 11:20 AM, Christoph Berg  wrote:
> Re: Jeff Janes 2016-10-12 
> 

Re: [HACKERS] Non-empty default log_line_prefix

[Christoph Berg]

Re: Jeff Janes 2016-10-12 

Re: [HACKERS] Non-empty default log_line_prefix

[Jeff Janes]

On Wed, Oct 12, 2016 at 10:11 AM, Tom Lane  wrote:

> Jeff Janes  writes:
> > On Sun, Oct 2, 2016 at 1:20 PM, Christoph Berg  wrote:
> >> Patch attached. (Still using %t, I don't think %m makes sense for the
> >> default.)
>
> > What is the cost of using %m, other than 4 (rather compressible) bytes
> per
> > log entry?
>
> More log I/O, which is not free ... and that remark about compressibility
> is bogus for anyone who doesn't pipe their postmaster stderr into gzip.
> I'm already afraid that adding the timestamps will get us some pushback
> about log volume.
>

I don't pipe them into gzip, but every few months I go and pxz any of them
more than few months old.

Do you think the pushback will come from people who just accept the
defaults?

Cheers,

Jeff

Re: [HACKERS] Non-empty default log_line_prefix

[Tom Lane]

Jeff Janes  writes:
> On Sun, Oct 2, 2016 at 1:20 PM, Christoph Berg  wrote:
>> Patch attached. (Still using %t, I don't think %m makes sense for the
>> default.)

> What is the cost of using %m, other than 4 (rather compressible) bytes per
> log entry?

More log I/O, which is not free ... and that remark about compressibility
is bogus for anyone who doesn't pipe their postmaster stderr into gzip.
I'm already afraid that adding the timestamps will get us some pushback
about log volume.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Jeff Janes]

On Sun, Oct 2, 2016 at 1:20 PM, Christoph Berg  wrote:

> Re: Tom Lane 2016-09-29 <18642.1475159...@sss.pgh.pa.us>
> > > Possibly the longer version could be added as an example in the
> > > documentation.
> >
> > I suspect that simply having a nonempty default in the first place
> > is going to do more to raise peoples' awareness than anything we
> > could do in the documentation.  But perhaps an example along these
> > lines would be useful for showing proper use of %q.
>
> Patch attached. (Still using %t, I don't think %m makes sense for the
> default.)
>

I don't agree with that part.  When looking at sections of log files that
people post on help forums, I've often wished people had shared
milliseconds, and I've never wished they had truncated them off.

If two messages are separated by 0.950 seconds, it can have entirely
different implications than if they are separated by 0.002 seconds.

What is the cost of using %m, other than 4 (rather compressible) bytes per
log entry?

Cheers,

Jeff

Re: [HACKERS] Non-empty default log_line_prefix

[Christoph Berg]

Re: Peter Eisentraut 2016-10-12 
<0caa6d7f-deb6-9a43-2b38-60e63af93...@2ndquadrant.com>
> >> > is going to do more to raise peoples' awareness than anything we
> >> > could do in the documentation.  But perhaps an example along these
> >> > lines would be useful for showing proper use of %q.
> > Patch attached. (Still using %t, I don't think %m makes sense for the
> > default.)
> 
> That still doesn't address what to do about syslog and eventlog users.
> We would need either a separate prefix setting for those, or maybe
> something like %q that says, skip to here if using syslog.  (I don't
> know eventlog, so I don't know if a common setting for syslog and
> eventlog would work.)

This patch simply tries to fix the default (stderr + '') which wasn't
useful for anyone. Note that there is already a hint in the
documentation that says timestamps and PID are not useful for syslog.

(Yes, the '' default might be fine for syslog, but I don't think
that's a good argument for sticking with it for default installs. I've
seen way too many useless log files out there, and at worst we'll have
syslogs with two timestamps.)

Christoph


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Tom Lane]

Peter Eisentraut  writes:
> That still doesn't address what to do about syslog and eventlog users.
> We would need either a separate prefix setting for those, or maybe
> something like %q that says, skip to here if using syslog.  (I don't
> know eventlog, so I don't know if a common setting for syslog and
> eventlog would work.)

Meh.  Those aren't default log output targets, so I don't think the
default prefix needs to cater for them.  People who adjust one setting
can adjust the other too.

There would be some value in the complexity you're thinking of for
installations that log to multiple targets concurrently, but really,
who does that?  Most production installations already begrudge
the I/O volume for a single log target.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Non-empty default log_line_prefix

[Peter Eisentraut]

On 10/2/16 4:20 PM, Christoph Berg wrote:
>> I suspect that simply having a nonempty default in the first place
>> > is going to do more to raise peoples' awareness than anything we
>> > could do in the documentation.  But perhaps an example along these
>> > lines would be useful for showing proper use of %q.
> Patch attached. (Still using %t, I don't think %m makes sense for the
> default.)

That still doesn't address what to do about syslog and eventlog users.
We would need either a separate prefix setting for those, or maybe
something like %q that says, skip to here if using syslog.  (I don't
know eventlog, so I don't know if a common setting for syslog and
eventlog would work.)

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Non-empty default log_line_prefix

[Christoph Berg]

Re: Tom Lane 2016-09-29 <18642.1475159...@sss.pgh.pa.us>
> > Possibly the longer version could be added as an example in the
> > documentation.
> 
> I suspect that simply having a nonempty default in the first place
> is going to do more to raise peoples' awareness than anything we
> could do in the documentation.  But perhaps an example along these
> lines would be useful for showing proper use of %q.

Patch attached. (Still using %t, I don't think %m makes sense for the
default.)

Christoph
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
new file mode 100644
index e826c19..a4d8b74
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*** local0.*/var/log/postgresql
*** 5004,5010 
   value will pad on the left. Padding can be useful to aid human
   readability in log files.
   This parameter can only be set in the postgresql.conf
!  file or on the server command line. The default is an empty string.
  
   

--- 5004,5011 
   value will pad on the left. Padding can be useful to aid human
   readability in log files.
   This parameter can only be set in the postgresql.conf
!  file or on the server command line. The default is
!  %t [%p]  which logs a time stamp and the process ID.
  
   

*** FROM pg_stat_activity;
*** 5142,5147 
--- 5143,5159 
   include those escapes if you are logging to syslog.
  
 
+ 
+
+ 
+  The %q escape is useful when including information that 
is
+  only available in session (backend) context like user or database
+  name. An example would be:
+ 
+ log_line_prefix = '%t [%p] %q%u@%d/%a '
+ 
+ 
+

   
  
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index cced814..b71fa93
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
*** static struct config_string ConfigureNam
*** 3014,3020 
gettext_noop("If blank, no prefix is used.")
},
_line_prefix,
!   "",
NULL, NULL, NULL
},
  
--- 3014,3020 
gettext_noop("If blank, no prefix is used.")
},
_line_prefix,
!   "%t [%p] ",
NULL, NULL, NULL
},
  
diff --git a/src/backend/utils/misc/postgresql.conf.sample 
b/src/backend/utils/misc/postgresql.conf.sample
new file mode 100644
index 05b1373..9eaa23e
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***
*** 430,436 
  #log_duration = off
  #log_error_verbosity = default# terse, default, or verbose 
messages
  #log_hostname = off
! #log_line_prefix = '' # special values:
#   %a = application name
#   %u = user name
#   %d = database name
--- 430,436 
  #log_duration = off
  #log_error_verbosity = default# terse, default, or verbose 
messages
  #log_hostname = off
! #log_line_prefix = '%t [%p] ' # special values:
#   %a = application name
#   %u = user name
#   %d = database name

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers