Re: [HACKERS] syslog configurable line splitting behavior

2016-03-16 Thread Andreas Karlsson

On 03/16/2016 03:50 AM, Peter Eisentraut wrote:

On 3/8/16 9:12 PM, Andreas Karlsson wrote:

I have one nitpick: why is one of the variables "true" while the other
is "on" in the example? I think both should be "on".

#syslog_sequence_numbers = true
#syslog_split_lines = on

Another possible improvement would be to change "Split messages sent to
syslog." to something more verbose like "Split messages sent to syslog,
by lines and to fit in 1024 bytes.".


Updated patches with your suggestions.  I also renamed
syslog_split_lines to syslog_split_messages, which I think is more accurate.


I think the patch looks good now. I will change the status to "Ready for 
committer".


Andreas



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] syslog configurable line splitting behavior

2016-03-15 Thread Peter Eisentraut
On 3/8/16 9:12 PM, Andreas Karlsson wrote:
> I have one nitpick: why is one of the variables "true" while the other
> is "on" in the example? I think both should be "on".
> 
> #syslog_sequence_numbers = true
> #syslog_split_lines = on
> 
> Another possible improvement would be to change "Split messages sent to
> syslog." to something more verbose like "Split messages sent to syslog,
> by lines and to fit in 1024 bytes.".

Updated patches with your suggestions.  I also renamed
syslog_split_lines to syslog_split_messages, which I think is more accurate.


From 70bacecba46eb38c02c43957c2f1812faf5684df Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Fri, 26 Feb 2016 22:34:30 -0500
Subject: [PATCH 1/2] Add syslog_sequence_numbers parameter

---
 doc/src/sgml/config.sgml  | 28 +++
 src/backend/utils/error/elog.c| 12 ++--
 src/backend/utils/misc/guc.c  | 10 ++
 src/backend/utils/misc/postgresql.conf.sample |  1 +
 src/include/utils/elog.h  |  1 +
 5 files changed, 50 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6c73fb4..bbe87ce 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -4305,6 +4305,34 @@ Where To Log

   
 
+  
+   syslog_sequence_numbers (boolean)
+
+ syslog_sequence_numbers configuration parameter
+
+   
+
+   
+
+ When logging to syslog and this is on (the
+ default), then each message will be prefixed by an increasing
+ sequence number (such as [2]).  This circumvents
+ the --- last message repeated N times --- suppression
+ that many syslog implementations perform by default.  In more modern
+ syslog implementations, repeat message suppression can be configured
+ (for example, $RepeatedMsgReduction
+ in rsyslog), so this might not be
+ necessary.  Also, you could turn this off if you actually want to
+ suppress repeated messages.
+
+
+
+ This parameter can only be set in the postgresql.conf
+ file or on the server command line.
+
+  
+ 
+
  
   event_source (string)
   
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index 5b7554b..88421c7 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -106,6 +106,7 @@ int			Log_error_verbosity = PGERROR_VERBOSE;
 char	   *Log_line_prefix = NULL;		/* format for extra log line info */
 int			Log_destination = LOG_DESTINATION_STDERR;
 char	   *Log_destination_string = NULL;
+bool		syslog_sequence_numbers = true;
 
 #ifdef HAVE_SYSLOG
 
@@ -2018,7 +2019,11 @@ write_syslog(int level, const char *line)
 
 			chunk_nr++;
 
-			syslog(level, "[%lu-%d] %s", seq, chunk_nr, buf);
+			if (syslog_sequence_numbers)
+syslog(level, "[%lu-%d] %s", seq, chunk_nr, buf);
+			else
+syslog(level, "[%d] %s", chunk_nr, buf);
+
 			line += buflen;
 			len -= buflen;
 		}
@@ -2026,7 +2031,10 @@ write_syslog(int level, const char *line)
 	else
 	{
 		/* message short enough */
-		syslog(level, "[%lu] %s", seq, line);
+		if (syslog_sequence_numbers)
+			syslog(level, "[%lu] %s", seq, line);
+		else
+			syslog(level, "%s", line);
 	}
 }
 #endif   /* HAVE_SYSLOG */
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index f0d4ec1..3ef432a 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1632,6 +1632,16 @@ static struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"syslog_sequence_numbers", PGC_SIGHUP, LOGGING_WHERE,
+			gettext_noop("Add sequence number to syslog messags to avoid duplicate suppression."),
+			NULL
+		},
+		_sequence_numbers,
+		true,
+		NULL, NULL, NULL
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee3d378..b72ea6d 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -358,6 +358,7 @@
 # These are relevant when logging to syslog:
 #syslog_facility = 'LOCAL0'
 #syslog_ident = 'postgres'
+#syslog_sequence_numbers = on
 
 # This is only relevant when logging to eventlog (win32):
 #event_source = 'PostgreSQL'
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 7d338dd..e245b2e 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -397,6 +397,7 @@ extern int	Log_error_verbosity;
 extern char *Log_line_prefix;
 extern int	Log_destination;
 extern char *Log_destination_string;
+extern bool syslog_sequence_numbers;
 
 /* Log destination bitmap */
 #define LOG_DESTINATION_STDERR	 1
-- 
2.7.3

From 66f367f37d3e3d67977ccace5aa3bb9bb8f947f6 Mon Sep 17 00:00:00 2001
From: Peter 

Re: [HACKERS] syslog configurable line splitting behavior

2016-03-08 Thread Andreas Karlsson

On 03/09/2016 03:20 AM, Peter Eisentraut wrote:

On 3/8/16 9:12 PM, Andreas Karlsson wrote:

As someone who uses syslog for my servers I find both of these GUCs
useful, especially when used in combination, and I do not think a
compile time option like suggest by Alexander would be suitable
substitute because then I would need a custom build of PostgreSQL just
to change this which seems too much effort just for this.


I think he was suggesting to take the existing compile-time constant and
make a run-time setting out of it.


Ah, that makes more sense than my reading of his message. Still I do not 
think there would be much gain from doing so. I do not see much benefit 
from making it configurable. In the cases I envision you would either 
want to split with a log file in mind or not split at all.


Andreas



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] syslog configurable line splitting behavior

2016-03-08 Thread Peter Eisentraut
On 3/8/16 9:12 PM, Andreas Karlsson wrote:
> As someone who uses syslog for my servers I find both of these GUCs
> useful, especially when used in combination, and I do not think a
> compile time option like suggest by Alexander would be suitable
> substitute because then I would need a custom build of PostgreSQL just
> to change this which seems too much effort just for this.

I think he was suggesting to take the existing compile-time constant and
make a run-time setting out of it.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] syslog configurable line splitting behavior

2016-03-08 Thread Peter Eisentraut
On 3/4/16 11:01 AM, Alexander Korotkov wrote:
> On Sat, Feb 27, 2016 at 6:49 AM, Peter Eisentraut  > wrote:
> 
> Writing log messages to syslog caters to ancient syslog implementations
> in two ways:
> 
> - sequence numbers
> - line splitting
> 
> While these are arguably reasonable defaults, I would like a way to turn
> them off, because they get in the way of doing more interesting things
> with syslog (e.g., logging somewhere that is not just a text file).
> 
> So I propose the two attached patches that introduce new configuration
> Boolean parameters syslog_sequence_numbers and syslog_split_lines that
> can toggle these behaviors.
> 
> 
> Would it have any usage if we make PG_SYSLOG_LIMIT configurable (-1 for
> disable) instead of introducing boolean?

That would work, too.  But then we'd need another setting to disable
splitting on newlines.  That way we'd have more settings, but they
actually mirror the corresponding settings on the rsyslogd side better.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] syslog configurable line splitting behavior

2016-03-08 Thread Andreas Karlsson

On 02/27/2016 04:49 AM, Peter Eisentraut wrote:

So I propose the two attached patches that introduce new configuration
Boolean parameters syslog_sequence_numbers and syslog_split_lines that
can toggle these behaviors.


As someone who uses syslog for my servers I find both of these GUCs 
useful, especially when used in combination, and I do not think a 
compile time option like suggest by Alexander would be suitable 
substitute because then I would need a custom build of PostgreSQL just 
to change this which seems too much effort just for this.


The code itself is clean and I find the documentation to be easy to 
understand.


I have one nitpick: why is one of the variables "true" while the other 
is "on" in the example? I think both should be "on".


#syslog_sequence_numbers = true
#syslog_split_lines = on

Another possible improvement would be to change "Split messages sent to 
syslog." to something more verbose like "Split messages sent to syslog, 
by lines and to fit in 1024 bytes.".


I tested the code and it worked as advertised and also passed the test 
suite. My local rsyslogd cut the message at about 8 kB, but I do not 
think it is worth it to expose PG_SYSLOG_LIMIT as a GUC.


Andreas


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] syslog configurable line splitting behavior

2016-03-04 Thread Alexander Korotkov
On Sat, Feb 27, 2016 at 6:49 AM, Peter Eisentraut  wrote:

> Writing log messages to syslog caters to ancient syslog implementations
> in two ways:
>
> - sequence numbers
> - line splitting
>
> While these are arguably reasonable defaults, I would like a way to turn
> them off, because they get in the way of doing more interesting things
> with syslog (e.g., logging somewhere that is not just a text file).
>
> So I propose the two attached patches that introduce new configuration
> Boolean parameters syslog_sequence_numbers and syslog_split_lines that
> can toggle these behaviors.
>

Would it have any usage if we make PG_SYSLOG_LIMIT configurable (-1 for
disable) instead of introducing boolean?

--
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company