Hello community, here is the log from the commit of package rsyslog.14302 for openSUSE:Leap:15.1:Update checked in at 2020-12-10 14:23:23 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Update/rsyslog.14302 (Old) and /work/SRC/openSUSE:Leap:15.1:Update/.rsyslog.14302.new.2328 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "rsyslog.14302" Thu Dec 10 14:23:23 2020 rev:1 rq:852588 version:8.33.1 Changes: -------- New Changes file: --- /dev/null 2020-12-09 01:05:43.965003977 +0100 +++ /work/SRC/openSUSE:Leap:15.1:Update/.rsyslog.14302.new.2328/rsyslog.changes 2020-12-10 14:23:24.634041716 +0100 @@ -0,0 +1,2631 @@ +------------------------------------------------------------------- +Tue Sep 8 07:21:46 UTC 2020 - Thomas Blume <[email protected]> + +- rsyslog.conf.in: Fix the URL for bug reporting (bsc#1173433) + +------------------------------------------------------------------- +Tue Jul 28 15:35:17 UTC 2020 - Thomas Blume <[email protected]> + +- fix coredump after logrotate (bsc#1173338) + * add 0001-Fixed-race-in-asyncWriter.patch + +------------------------------------------------------------------- +Thu Oct 10 08:16:28 UTC 2019 - Thomas Blume <[email protected]> + +- avoid SEGFAULT due to a mutex double-unlock in iminternal module + (bsc#1141063) + * add 0001-iminternal-suppress-mutex-double-unlock.patch +- fix potential misaddressing in pmaixforwardedfrom (CVE-2019-17041, + bsc#1153451) + * add 0001-pmaixforwardedfrom-bugfix-potential-misadressing.patch +- fix potential misaddressing in pmcisconames (CVE-2019-17042, + bsc#1153459) + * add 0001-pmcisconames-bugfix-potential-misadressing.patch + +------------------------------------------------------------------- +Mon Jul 15 08:15:07 UTC 2019 - Thomas Blume <[email protected]> + +- support mmkubernetes module and bug fix for imfile module + (ECO#327800/bsc#1146872) + * backport https://github.com/rsyslog/rsyslog/pull/3051 + * backport patch that added mmkubernetes module inclusive a + bugfix to mmkubernetes + * added patches: + 0001-imfile-large-refactoring-of-complete-module.patch + 0002-bugfix-imfile-file-change-was-not-reliably-detected.patch + 0003-imfile-improve-truncation-detection.patch + 0004-imfile-add-declaration-for-restart.patch + 0005-WIP-Add-the-mmkubernetes-plugin.patch + 0006-Kubernetes-Metadata-plugin-mmkubernetes.patch + 0007-mmkubernetes-fix-lnrules-add-defaults-add-test.patch + 0008-default-rules-use-container_name_and_id-include-rule.patch + +------------------------------------------------------------------- +Tue Jul 2 12:38:21 UTC 2019 - Thomas Blume <[email protected]> + +- suppress error message about missing environment variable TZ (boo#1137681) + * add 0001-core-emit-TZ-warning-on-startup-not-on-Linux-non-con.patch + +------------------------------------------------------------------- +Fri Feb 22 09:30:37 UTC 2019 - [email protected] + +- set default permissions before include directives (bsc#1126233) + * Logfile open modes depend on position in the config file, see + https://www.rsyslog.com/doc/rsconf1_filecreatemode.html + +------------------------------------------------------------------- +Wed Jul 18 14:09:26 UTC 2018 - [email protected] + +- remove references to obsolete SYSLOG_REQUIRES_NETWORK + variable (bsc#1101642) + +------------------------------------------------------------------- +Thu Jun 21 13:37:09 UTC 2018 - [email protected] + +- fix CVE-2015-3243 rsyslog: some log files are created world-readable + (bsc#935393) + +------------------------------------------------------------------- +Sat Mar 24 18:47:34 UTC 2018 - [email protected] + +- rsyslog 8.33.1: + * devcontainer: use some more sensible defaults + * auto-detect if running inside a container (as pid 1) + * config: add include() script object + * template: add option to generate json "container" + * core/template: add format jsonf to constant template entries + * config: add ability to disable config parameter ("config.enable") + * script: permit to use environment variables during configuration + * new global config parameter "shutdown.enable.ctlc" + * config optimizer: detect totally empty "if" statements and optimize them out + * template: constant entry can now also be formatted as json field + * omstdout: support for new-style configuration parameters added + * core: set TZ on startup if not already set + * imjournal bugfix: file handle leak during journal rotation + * lmsig_ksils12 bugfix: dirOwner and dirGroup config was not respected + * script bugfix: replace() function worked incorrectly in some cases + * core bugfix: misadressing in external command parser + * core bugfix: small memory leak in external command parser + * core bugfix: string not properly terminated when RFC5424 MSGID is used + * bugfix: strndup() compatibility layer func copies too much +- the upstream systemd unit file was changed to no longer write the + rsyslog pid, as it is no longer required for tracking under + systemd (-iNONE). Adjust rsyslog-unit.patch to match. + +------------------------------------------------------------------- +Mon Mar 19 11:43:55 CET 2018 - [email protected] + +- Use %license instead of %doc [bsc#1082318] + +------------------------------------------------------------------- +Mon Feb 19 09:16:20 UTC 2018 - [email protected] + +- fix includes for apparmor profile (bsc#1080238) + +------------------------------------------------------------------- +Fri Jan 26 14:47:16 UTC 2018 - [email protected] + +- rsyslog 8.32.0 + * libfastjson 0.99.8 required + * libczmq >= 3.0.2 is now required for omczmq + * libcurl is now needed for rsyslog core + * rsyslogd: add capability to specify that no pid file shall be written + * core improvements and bug fixes + * RainerScript improvements and bug fixes + * build fixes, including gcc7 fixes + drop 0001-imgssapi-fix-compiler-warnings.patch + * various bug fixes in multiple modules + +------------------------------------------------------------------- +Fri Jan 19 08:48:59 UTC 2018 - [email protected] + +- remove build dependency on libee + +------------------------------------------------------------------- +Thu Dec 7 14:56:03 CET 2017 - [email protected] + +- Disable news by default, we don't need to clobber all systems + with this for the very few remaining news servers + +------------------------------------------------------------------- +Thu Nov 23 13:41:39 UTC 2017 - [email protected] + +- Replace references to /var/adm/fillup-templates with new + %_fillupdir macro (boo#1069468) + +------------------------------------------------------------------- +Fri Nov 17 15:34:42 UTC 2017 - [email protected] + +- Ensure user "news" exists - bsc#1068678 + +------------------------------------------------------------------- +Tue Nov 14 15:33:34 UTC 2017 - [email protected] + +- rsyslog 8.30.0 + * changed behaviour: all variables are now case-insensitive by default + * core: handle (JSON) variables in case-insensitive way + * imjournal: made switching to persistent journal in runtime possible + * mmanon: complete refactor and enhancements + * imfile: add "fileoffset" metadata + * RainerScript: add ltrim and rtrim functions + * core: report module name when suspending action + * core: add ability to limit number of error messages going to stderr + * tcpsrv subsystem: improvate clarity of some error messages + * imptcp: include module name in error msg + * imtcp: include module name in error msg + * tls improvement: better error message if certificate file cannot be read + * omfwd: slightly improved error messages during config parsing + * ommysql improvements + * ommysql bugfix: do not duplicate entries on failed transaction + * imtcp bugfix: parameter priorityString was ignored + * template/bugfix: invalid template option conflict detection + * core/actions: fix handling of data-induced errors + * core/action bugfix: no "action suspended" message during retry processing + * core/ratelimit bugfix: race can lead to segfault + * core bugfix: rsyslog aborts if errmsg is generated in early startup + * core bugfix: informational messages was logged with error severity + * core bugfix: --enable-debugless build was broken + * queue bugfix: file write error message was incorrect + * omrelp bugfix: segfault when rebindinterval parameter is used + * omkafka bugfix: invalid load of failedmsg file on startup if disabled + * kafka bugfix: problem on invalid kafka configuration values + * imudp bugfix: UDP oversize message not properly handled + * core bugfix: memory corruption during configuration parsing + * core bugfix: race on worker thread termination during shutdown + * omelasticsearch: avoid ES5 warnings while sending json in bulkmode + * omelasticsearch bugfix: incompatibility with newer ElasticSearch version + * imptcp bugfix: invalid mutex addressing on some platforms + * imptcp bugfix: do not accept missing port in legacy listener definition +- build requirements: + * libfastjson 0.99.7 is now mandatory + * libsystemd-journal >= 234 is now recommended +- packaging: + * add upstream build fix 0001-imgssapi-fix-compiler-warnings.patch + +------------------------------------------------------------------- +Tue Aug 8 20:39:00 UTC 2017 - [email protected] + +- rsyslog 8.29.0: + * imptcp: add experimental parameter "multiline" + * imptcp: framing-related error messages now also indicate remote peer + * imtcp: framing-related error messages now also indicate remote peer + * imptcp: add session statistics conunter + * imtcp: add ability to specify GnuTLS priority string + * impstats: add new ressoure counter "openfiles" + * pmnormalize: new parser module + * core/queue: provide informational messages on thread startup and shutdown + * omfwd/udp: improve error reporting, depricate maxerrormessages parameter ++++ 2434 more lines (skipped) ++++ between /dev/null ++++ and /work/SRC/openSUSE:Leap:15.1:Update/.rsyslog.14302.new.2328/rsyslog.changes New: ---- 0001-Fixed-race-in-asyncWriter.patch 0001-core-emit-TZ-warning-on-startup-not-on-Linux-non-con.patch 0001-imfile-large-refactoring-of-complete-module.patch 0001-iminternal-suppress-mutex-double-unlock.patch 0001-pmaixforwardedfrom-bugfix-potential-misadressing.patch 0001-pmcisconames-bugfix-potential-misadressing.patch 0002-bugfix-imfile-file-change-was-not-reliably-detected.patch 0003-imfile-improve-truncation-detection.patch 0004-imfile-add-declaration-for-restart.patch 0005-WIP-Add-the-mmkubernetes-plugin.patch 0006-Kubernetes-Metadata-plugin-mmkubernetes.patch 0007-mmkubernetes-fix-lnrules-add-defaults-add-test.patch 0008-default-rules-use-container_name_and_id-include-rule.patch module-mysql module-snmp module-udpspoof rsyslog-8.33.1.tar.gz rsyslog-doc-8.33.1.tar.gz rsyslog-service-prepare.in rsyslog-unit.patch rsyslog.changes rsyslog.conf.in rsyslog.d.remote.conf.in rsyslog.firewall rsyslog.spec rsyslog.sysconfig usr.sbin.rsyslogd ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ rsyslog.spec ++++++ ++++ 1195 lines (skipped) ++++++ 0001-Fixed-race-in-asyncWriter.patch ++++++ From 7e88634f3dc0fb059d4f16b217b4b1502521c3a3 Mon Sep 17 00:00:00 2001 From: Radovan Sroka <[email protected]> Date: Tue, 26 Jun 2018 12:50:02 +0200 Subject: [PATCH] Fixed race in asyncWriter strmFlushInternal needs to be called with locked mutex --- runtime/stream.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/runtime/stream.c b/runtime/stream.c index f93b4042e..d84dd3642 100644 --- a/runtime/stream.c +++ b/runtime/stream.c @@ -1499,10 +1499,8 @@ asyncWriterThread(void *pPtr) } if(bTimedOut && pThis->iBufPtr > 0) { /* if we timed out, we need to flush pending data */ - d_pthread_mutex_unlock(&pThis->mut); strmFlushInternal(pThis, 1); bTimedOut = 0; - d_pthread_mutex_lock(&pThis->mut); continue; } bTimedOut = 0; -- 2.26.2 ++++++ 0001-core-emit-TZ-warning-on-startup-not-on-Linux-non-con.patch ++++++ From d26343dc2fcfa4773da4e97e178ea3d94c3da120 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards <[email protected]> Date: Mon, 17 Sep 2018 08:56:09 +0200 Subject: [PATCH] core: emit TZ warning on startup not on Linux non-container On Linux it seems common that the TZ variable is NOT properly set. There are some concerns that the warning related to rsyslog correcting this confuses users. It also seems that the corrective action rsyslog takes is right, and so there is no hard need to inform users on that. In Linux containers, however, the warning seems to be useful as the timezone setup there seems to be frequently-enough different and rsyslog's corrective action may not be correct. So we now check if we are running under Linux and not within a container. If so, we do not emit the warning. In all other case, we do. This is based on the assumption that other unixoid systems still should have TZ properly set. closes https://github.com/rsyslog/rsyslog/issues/2994 (cherry picked from commit 502759aea05abaf9a405278e4fe3c41a9c0edaeb) --- tools/rsyslogd.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/tools/rsyslogd.c b/tools/rsyslogd.c index 4fad3ad5c..93308ac9b 100644 --- a/tools/rsyslogd.c +++ b/tools/rsyslogd.c @@ -61,6 +61,17 @@ #include "dirty.h" #include "janitor.h" +/* some global vars we need to differentiate between environments, + * for TZ-related things see + * https://github.com/rsyslog/rsyslog/issues/2994 + */ +static int runningInContainer = 0; +#ifdef OS_LINUX +static int emitTZWarning = 0; +#else +static int emitTZWarning = 1; +#endif + #if defined(_AIX) /* AIXPORT : start * The following includes and declarations are for support of the System @@ -1307,8 +1318,12 @@ initAll(int argc, char **argv) const char *const tz = (access("/etc/localtime", R_OK) == 0) ? "TZ=/etc/localtime" : "TZ=UTC"; putenv((char*)tz); - LogMsg(0, RS_RET_NO_TZ_SET, LOG_WARNING, "environment variable TZ is not " - "set, auto correcting this to %s\n", tz); + if(emitTZWarning) { + LogMsg(0, RS_RET_NO_TZ_SET, LOG_WARNING, "environment variable TZ is not " + "set, auto correcting this to %s", tz); + } else { + dbgprintf("environment variable TZ is not set, auto correcting this to %s\n", tz); + } } /* END core initializations - we now come back to carrying out command line options*/ @@ -1960,6 +1975,8 @@ main(int argc, char **argv) "terminate rsyslog\n", VERSION); PidFile = strdup("NONE"); /* disables pid file writing */ glblPermitCtlC = 1; + runningInContainer = 1; + emitTZWarning = 1; } else { /* "dynamic defaults" - non-container case */ PidFile = strdup(PATH_PIDFILE); -- 2.16.4 ++++++ 0001-imfile-large-refactoring-of-complete-module.patch ++++++ ++++ 4730 lines (skipped) ++++++ 0001-iminternal-suppress-mutex-double-unlock.patch ++++++ From 2e9d389419a42b63c8d6815302ec0807045207ee Mon Sep 17 00:00:00 2001 From: Thomas Blume <[email protected]> Date: Thu, 11 Jul 2019 12:58:10 +0200 Subject: [PATCH] iminternal: suppress mutex double-unlock If there is a burst of log messages during a time when rsyslog is unable to output (either during log rotation, an out-of-space condition, or some other similar condition), rsyslog can SEGFAULT due to a mutex double-unlock. --- tools/iminternal.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/iminternal.c b/tools/iminternal.c index 66b419a6b..4983aa177 100644 --- a/tools/iminternal.c +++ b/tools/iminternal.c @@ -93,7 +93,6 @@ rsRetVal iminternalAddMsg(smsg_t *pMsg) #else r = pthread_mutex_trylock(&mutList); // must check #endif - is_locked = 1; if(r != 0) { dbgprintf("iminternalAddMsg: timedlock for mutex failed with %d, msg %s\n", r, getMSG(pMsg)); @@ -101,6 +100,7 @@ rsRetVal iminternalAddMsg(smsg_t *pMsg) msgDestruct(&pMsg); ABORT_FINALIZE(RS_RET_ERR); } + is_locked = 1; CHKiRet(iminternalConstruct(&pThis)); pThis->pMsg = pMsg; CHKiRet(llAppend(&llMsgs, NULL, (void*) pThis)); -- 2.16.4 ++++++ 0001-pmaixforwardedfrom-bugfix-potential-misadressing.patch ++++++ From 10549ba915556c557b22b3dac7e4cb73ad22d3d8 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards <[email protected]> Date: Fri, 27 Sep 2019 13:36:02 +0200 Subject: [PATCH] pmaixforwardedfrom bugfix: potential misadressing --- contrib/pmaixforwardedfrom/pmaixforwardedfrom.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/contrib/pmaixforwardedfrom/pmaixforwardedfrom.c b/contrib/pmaixforwardedfrom/pmaixforwardedfrom.c index 37157c7d4..ebf12ebbe 100644 --- a/contrib/pmaixforwardedfrom/pmaixforwardedfrom.c +++ b/contrib/pmaixforwardedfrom/pmaixforwardedfrom.c @@ -109,6 +109,10 @@ CODESTARTparse /* bump the message portion up by skipLen(23 or 5) characters to overwrite the "Message forwarded from " or "From " with the hostname */ lenMsg -=skipLen; + if(lenMsg < 2) { + dbgprintf("not a AIX message forwarded from message has nothing after header\n"); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } memmove(p2parse, p2parse + skipLen, lenMsg); *(p2parse + lenMsg) = '\n'; *(p2parse + lenMsg + 1) = '\0'; @@ -120,6 +124,11 @@ really an AIX log, but has a similar preamble */ --lenMsg; ++p2parse; } + if (lenMsg < 1) { + dbgprintf("not a AIX message forwarded from message has nothing after colon " + "or no colon at all\n"); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } if (lenMsg && *p2parse != ':') { DBGPRINTF("not a AIX message forwarded from mangled log but similar enough that the preamble has " "been removed\n"); -- 2.16.4 ++++++ 0001-pmcisconames-bugfix-potential-misadressing.patch ++++++ From abc0960a7561e18944a0e08d48f4eb570ea7435a Mon Sep 17 00:00:00 2001 From: Rainer Gerhards <[email protected]> Date: Fri, 27 Sep 2019 15:02:52 +0200 Subject: [PATCH] pmcisconames bugfix: potential misadressing --- contrib/pmcisconames/pmcisconames.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/contrib/pmcisconames/pmcisconames.c b/contrib/pmcisconames/pmcisconames.c index 7f376ad17..39506ce59 100644 --- a/contrib/pmcisconames/pmcisconames.c +++ b/contrib/pmcisconames/pmcisconames.c @@ -119,6 +119,11 @@ CODESTARTparse --lenMsg; ++p2parse; } + /* Note: we deliberately count the 0-byte below because we need to go chars+1! */ + if(lenMsg < (int) sizeof(OpeningText)) { + dbgprintf("pmcisconames: too short for being cisco messages\n"); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } /* skip the space after the hostname */ lenMsg -=1; p2parse +=1; @@ -126,7 +131,7 @@ CODESTARTparse log and fix it */ if(strncasecmp((char*) p2parse, OpeningText, sizeof(OpeningText)-1) != 0) { /* wrong opening text */ - DBGPRINTF("not a cisco name mangled log!\n"); + DBGPRINTF("not a cisco name mangled log!\n"); ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); } /* bump the message portion up by two characters to overwrite the extra : */ -- 2.16.4 ++++++ 0002-bugfix-imfile-file-change-was-not-reliably-detected.patch ++++++ From 158edac32e36e17625833a74e0c7746585a8c543 Mon Sep 17 00:00:00 2001 From: Rainer Gerhards <[email protected]> Date: Sun, 23 Sep 2018 13:19:31 +0200 Subject: [PATCH 2/7] bugfix imfile: file change was not reliably detected A change in the inode was not detected under all circumstances, most importantly not in some logrotate cases. Includes new tests made by Andre Lorbach. They now use the logrotate tool natively to reproduce the issue. (cherry picked from commit f85ef7aabcec84497a5eaf9670616b3402c79d9c) --- plugins/imfile/imfile.c | 27 ++++-- runtime/stream.c | 120 ++++++++++++++++++++++---- runtime/stream.h | 9 ++ tests/Makefile.am | 17 +++- tests/imfile-logrotate.sh | 78 +++++++++++++++++ tests/imfile-wildcards-dirs-multi5-polling.sh | 28 +++--- 6 files changed, 238 insertions(+), 41 deletions(-) create mode 100755 tests/imfile-logrotate.sh diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index a5d20823f..a13687df8 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -621,13 +621,22 @@ detect_updates(fs_edge_t *const edge) const int r = stat(act->name, &fileInfo); if(r == -1) { /* object gone away? */ DBGPRINTF("object gone away, unlinking: '%s'\n", act->name); - act_obj_t *toDel = act; - act = act->next; - DBGPRINTF("new next act %p\n", act); - act_obj_unlink(toDel); - continue; + act_obj_unlink(act); + restart = 1; + break; + } else if(fileInfo.st_ino != act->ino) { + DBGPRINTF("file '%s' inode changed from %llu to %llu, unlinking from " + "internal lists\n", act->name, (long long unsigned) act->ino, + (long long unsigned) fileInfo.st_ino); + if(act->pStrm != NULL) { + /* we do no need to re-set later, as act_obj_unlink + * will destroy the strm obj */ + strmSet_checkRotation(act->pStrm, STRM_ROTATION_DO_NOT_CHECK); + } + act_obj_unlink(act); + restart = 1; + break; } - // TODO: add inode check for change notification! /* Note: active nodes may get deleted, so we need to do the * pointer advancement at the end of the for loop! @@ -816,10 +825,10 @@ chk_active(const act_obj_t *act, const act_obj_t *const deleted) /* unlink act object from linked list and then * destruct it. */ -static void //ATTR_NONNULL() -act_obj_unlink(act_obj_t *const act) +static void ATTR_NONNULL() +act_obj_unlink(act_obj_t *act) { - DBGPRINTF("act_obj_unlink %p: %s\n", act, act->name); + DBGPRINTF("act_obj_unlink %p: %s, pStrm %p\n", act, act->name, act->pStrm); if(act->prev == NULL) { act->edge->active = act->next; } else { diff --git a/runtime/stream.c b/runtime/stream.c index 7286e819e..028a4652c 100644 --- a/runtime/stream.c +++ b/runtime/stream.c @@ -325,8 +325,8 @@ CheckFileChange(strm_t *pThis) CHKiRet(strmSetCurrFName(pThis)); if(stat((char*) pThis->pszCurrFName, &statName) == -1) ABORT_FINALIZE(RS_RET_IO_ERROR); - DBGPRINTF("stream/after deserialize checking for file change on '%s', " - "inode %u/%u, size/currOffs %llu/%llu\n", + DBGPRINTF("CheckFileChange: stream/after deserialize checking for file change " + "on '%s', inode %u/%u, size/currOffs %llu/%llu\n", pThis->pszCurrFName, (unsigned) pThis->inode, (unsigned) statName.st_ino, (long long unsigned) statName.st_size, @@ -536,8 +536,8 @@ finalize_it: * circumstances). So starting as of now, we only check the inode number and * a file change is detected only if the inode changes. -- rgerhards, 2011-01-10 */ -static rsRetVal -strmHandleEOFMonitor(strm_t *pThis) +static rsRetVal ATTR_NONNULL() +strmHandleEOFMonitor(strm_t *const pThis) { DEFiRet; struct stat statName; @@ -545,9 +545,9 @@ strmHandleEOFMonitor(strm_t *pThis) ISOBJ_TYPE_assert(pThis, strm); if(stat((char*) pThis->pszCurrFName, &statName) == -1) ABORT_FINALIZE(RS_RET_IO_ERROR); - DBGPRINTF("stream checking for file change on '%s', inode %u/%u\n", - pThis->pszCurrFName, (unsigned) pThis->inode, - (unsigned) statName.st_ino); + DBGPRINTF("strmHandleEOFMonitor: stream checking for file change on '%s', inode %u/%u size %llu/%llu\n", + pThis->pszCurrFName, (unsigned) pThis->inode, (unsigned) statName.st_ino, + (long long unsigned) pThis->iCurrOffs, (long long unsigned) statName.st_size); /* Inode unchanged but file size on disk is less than current offset * means file was truncated, we also reopen if 'reopenOnTruncate' is on @@ -573,8 +573,8 @@ finalize_it: * try to open the next one. * rgerhards, 2008-02-13 */ -static rsRetVal -strmHandleEOF(strm_t *pThis) +static rsRetVal ATTR_NONNULL() +strmHandleEOF(strm_t *const pThis) { DEFiRet; @@ -591,7 +591,13 @@ strmHandleEOF(strm_t *pThis) CHKiRet(strmNextFile(pThis)); break; case STREAMTYPE_FILE_MONITOR: - CHKiRet(strmHandleEOFMonitor(pThis)); + DBGOPRINT((obj_t*) pThis, "file '%s' (%d) EOF, rotationCheck %d\n", + pThis->pszCurrFName, pThis->fd, pThis->rotationCheck); + if(pThis->rotationCheck == STRM_ROTATION_DO_CHECK) { + CHKiRet(strmHandleEOFMonitor(pThis)); + } else { + ABORT_FINALIZE(RS_RET_EOF); + } break; } @@ -599,6 +605,75 @@ finalize_it: RETiRet; } + +/* helper to checkTruncation */ +static rsRetVal ATTR_NONNULL() +rereadTruncated(strm_t *const pThis, const char *const reason) +{ + DEFiRet; + + LogMsg(errno, RS_RET_FILE_TRUNCATED, LOG_WARNING, "file '%s': truncation detected, " + "(%s) - re-start reading from beginning", + pThis->pszCurrFName, reason); + DBGPRINTF("checkTruncation, file %s last buffer CHANGED\n", pThis->pszCurrFName); + CHKiRet(strmCloseFile(pThis)); + CHKiRet(strmOpenFile(pThis)); + iRet = RS_RET_FILE_TRUNCATED; + +finalize_it: + RETiRet; +} +/* helper to read: + * Check if file has been truncated since last read and, if so, re-set reading + * to begin of file. To detect truncation, we try to re-read the last block. + * If that does not succeed or different data than from the original read is + * returned, truncation is assumed. + * NOTE: this function must be called only if truncation is enabled AND + * when the previous read buffer still is valid (aka "before the next read"). + * It is ok to call with a 0-size buffer, which we than assume as begin of + * reading. In that case, no truncation will be detected. + * rgerhards, 2018-09-20 + */ +static rsRetVal ATTR_NONNULL() +checkTruncation(strm_t *const pThis) +{ + DEFiRet; + int ret; + off64_t backseek; + assert(pThis->bReopenOnTruncate); + + DBGPRINTF("checkTruncation, file %s, iBufPtrMax %zd\n", pThis->pszCurrFName, pThis->iBufPtrMax); + if(pThis->iBufPtrMax == 0) { + FINALIZE; + } + + int currpos = lseek64(pThis->fd, 0, SEEK_CUR); + backseek = -1 * (off64_t) pThis->iBufPtrMax; + dbgprintf("checkTruncation in actual processing, currpos %d, backseek is %d\n", (int)currpos, (int) backseek); + ret = lseek64(pThis->fd, backseek, SEEK_CUR); + if(ret < 0) { + iRet = rereadTruncated(pThis, "cannot seek backward to begin of last block"); + FINALIZE; + } + + const ssize_t lenRead = read(pThis->fd, pThis->pIOBuf_truncation, pThis->iBufPtrMax); + dbgprintf("checkTruncation proof-read: %d bytes\n", (int) lenRead); + if(lenRead < 0) { + iRet = rereadTruncated(pThis, "last block could not be re-read"); + FINALIZE; + } + + if(!memcmp(pThis->pIOBuf_truncation, pThis->pIOBuf, pThis->iBufPtrMax)) { + DBGPRINTF("checkTruncation, file %s last buffer unchanged\n", pThis->pszCurrFName); + } else { + iRet = rereadTruncated(pThis, "last block data different"); + } + +finalize_it: + RETiRet; +} + + /* read the next buffer from disk * rgerhards, 2008-02-13 */ @@ -744,7 +819,7 @@ static rsRetVal strmUnreadChar(strm_t *pThis, uchar c) * a line, but following lines that are indented are part of the same log entry */ static rsRetVal -strmReadLine(strm_t *pThis, cstr_t **ppCStr, uint8_t mode, sbool bEscapeLF, +strmReadLine(strm_t *const pThis, cstr_t **ppCStr, uint8_t mode, sbool bEscapeLF, uint32_t trimLineOverBytes, int64 *const strtOffs) { uchar c; @@ -2018,14 +2093,25 @@ DEFpropSetMeth(strm, cryprov, cryprov_if_t*) DEFpropSetMeth(strm, cryprovData, void*) /* sets timeout in seconds */ -void +void ATTR_NONNULL() strmSetReadTimeout(strm_t *const __restrict__ pThis, const int val) { + ISOBJ_TYPE_assert(pThis, strm); pThis->readTimeout = val; } -static rsRetVal strmSetbDeleteOnClose(strm_t *pThis, int val) +void ATTR_NONNULL() +strmSet_checkRotation(strm_t *const pThis, const int val) { + ISOBJ_TYPE_assert(pThis, strm); + assert(val == STRM_ROTATION_DO_CHECK || val == STRM_ROTATION_DO_NOT_CHECK); + pThis->rotationCheck = val; +} + + +static rsRetVal ATTR_NONNULL() +strmSetbDeleteOnClose(strm_t *const pThis, const int val) { + ISOBJ_TYPE_assert(pThis, strm); pThis->bDeleteOnClose = val; if(pThis->cryprov != NULL) { pThis->cryprov->SetDeleteOnClose(pThis->cryprovFileData, pThis->bDeleteOnClose); @@ -2033,15 +2119,19 @@ static rsRetVal strmSetbDeleteOnClose(strm_t *pThis, int val) return RS_RET_OK; } -static rsRetVal strmSetiMaxFiles(strm_t *pThis, int iNewVal) +static rsRetVal ATTR_NONNULL() +strmSetiMaxFiles(strm_t *const pThis, const int iNewVal) { + ISOBJ_TYPE_assert(pThis, strm); pThis->iMaxFiles = iNewVal; pThis->iFileNumDigits = getNumberDigits(iNewVal); return RS_RET_OK; } -static rsRetVal strmSetFileNotFoundError(strm_t *pThis, int pFileNotFoundError) +static rsRetVal ATTR_NONNULL() +strmSetFileNotFoundError(strm_t *const pThis, const int pFileNotFoundError) { + ISOBJ_TYPE_assert(pThis, strm); pThis->fileNotFoundError = pFileNotFoundError; return RS_RET_OK; } diff --git a/runtime/stream.h b/runtime/stream.h index bafa5fbd3..57d8c68b0 100644 --- a/runtime/stream.h +++ b/runtime/stream.h @@ -91,6 +91,10 @@ typedef enum { /* when extending, do NOT change existing modes! */ STREAMMODE_WRITE_APPEND = 4 } strmMode_t; +/* settings for stream rotation (applies not to all processing modes!) */ +#define STRM_ROTATION_DO_CHECK 0 +#define STRM_ROTATION_DO_NOT_CHECK 1 + #define STREAM_ASYNC_NUMBUFS 2 /* must be a power of 2 -- TODO: make configurable */ /* The strm_t data structure */ typedef struct strm_s { @@ -114,6 +118,7 @@ typedef struct strm_s { sbool bDisabled; /* should file no longer be written to? (currently set only if omfile file size limit fails) */ sbool bSync; /* sync this file after every write? */ sbool bReopenOnTruncate; + int rotationCheck; /* rotation check mode */ size_t sIOBufSize;/* size of IO buffer */ uchar *pszDir; /* Directory */ int lenDir; @@ -230,5 +235,9 @@ rsRetVal strmReadMultiLine(strm_t *pThis, cstr_t **ppCStr, regex_t *preg, int strmReadMultiLine_isTimedOut(const strm_t *const __restrict__ pThis); void strmDebugOutBuf(const strm_t *const pThis); void strmSetReadTimeout(strm_t *const __restrict__ pThis, const int val); +const uchar * ATTR_NONNULL() strmGetPrevLineSegment(strm_t *const pThis); +const uchar * ATTR_NONNULL() strmGetPrevMsgSegment(strm_t *const pThis); +int ATTR_NONNULL() strmGetPrevWasNL(const strm_t *const pThis); +void ATTR_NONNULL() strmSet_checkRotation(strm_t *const pThis, const int val); #endif /* #ifndef STREAM_H_INCLUDED */ diff --git a/tests/Makefile.am b/tests/Makefile.am index 2283066ac..4386c626d 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -784,7 +784,17 @@ TESTS += \ imfile-wildcards-dirs-multi5.sh \ imfile-wildcards-dirs-multi5-polling.sh \ imfile-rename-while-stopped.sh \ - imfile-rename.sh + imfile-rename.sh \ + imfile-symlink.sh \ + imfile-symlink-multi.sh \ + imfile-logrotate.sh \ + imfile-growing-file-id.sh \ + glbl-oversizeMsg-truncate-imfile.sh + +if ENABLE_MMNORMALIZE +TESTS += \ + imfile-endmsg.regex-with-example.sh +endif if HAVE_VALGRIND TESTS += \ @@ -1485,6 +1495,11 @@ EXTRA_DIST= \ testsuites/imfile-wildcards-dirs-multi2.conf \ testsuites/imfile-wildcards-dirs-multi3.conf \ testsuites/imfile-wildcards-dirs-multi4.conf \ + imfile-symlink.sh \ + imfile-symlink-multi.sh \ + imfile-logrotate.sh \ + imfile-growing-file-id.sh \ + glbl-oversizeMsg-truncate-imfile.sh \ dynfile_invld_async.sh \ dynfile_invld_sync.sh \ dynfile_cachemiss.sh \ diff --git a/tests/imfile-logrotate.sh b/tests/imfile-logrotate.sh new file mode 100755 index 000000000..9e5476105 --- /dev/null +++ b/tests/imfile-logrotate.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# This is part of the rsyslog testbench, licensed under ASL 2.0 +. $srcdir/diag.sh check-inotify-only +. $srcdir/diag.sh init +check_command_available logrotate + +export TESTMESSAGES=10000 +export RETRIES=50 +export TESTMESSAGESFULL=19999 + +generate_conf +add_conf ' +$WorkDirectory '$RSYSLOG_DYNNAME'.spool + +/* Filter out busy debug output */ +global( + debug.whitelist="off" + debug.files=["rainerscript.c", "ratelimit.c", "ruleset.c", "main Q", "msg.c", "../action.c"] + ) + +module( load="../plugins/imfile/.libs/imfile" + mode="inotify" + PollingInterval="1") + +input(type="imfile" + File="./'$RSYSLOG_DYNNAME'.input.*.log" + Tag="file:" + Severity="error" + Facility="local7" + addMetadata="on" +) + +$template outfmt,"%msg:F,58:2%\n" +if $msg contains "msgnum:" then + action( + type="omfile" + file=`echo $RSYSLOG_OUT_LOG` + template="outfmt" + ) +' + +# Write logrotate config file +echo '"./'$RSYSLOG_DYNNAME'.input.*.log" +{ + rotate 7 + create + daily + missingok + notifempty + compress +}' > $RSYSLOG_DYNNAME.logrotate + +# generate input file first. +./inputfilegen -m $TESTMESSAGES > $RSYSLOG_DYNNAME.input.1.log +ls -l $RSYSLOG_DYNNAME.input* + +startup + +# Wait until testmessages are processed by imfile! +. $srcdir/diag.sh wait-file-lines $RSYSLOG_OUT_LOG $TESTMESSAGES $RETRIES + +# Logrotate on logfile +logrotate -f $RSYSLOG_DYNNAME.logrotate +./msleep 1000 + +# generate more input after logrotate into new logfile +./inputfilegen -m $TESTMESSAGES -i $TESTMESSAGES >> $RSYSLOG_DYNNAME.input.1.log +ls -l $RSYSLOG_DYNNAME.input* +echo ls ${RSYSLOG_DYNNAME}.spool: +ls -l ${RSYSLOG_DYNNAME}.spool + +let msgcount="2* $TESTMESSAGES" +. $srcdir/diag.sh wait-file-lines $RSYSLOG_OUT_LOG $msgcount $RETRIES + +shutdown_when_empty # shut down rsyslogd when done processing messages +wait_shutdown # we need to wait until rsyslogd is finished! +seq_check 0 $TESTMESSAGESFULL +exit_test diff --git a/tests/imfile-wildcards-dirs-multi5-polling.sh b/tests/imfile-wildcards-dirs-multi5-polling.sh index 0689bd087..7d8fda5a9 100755 --- a/tests/imfile-wildcards-dirs-multi5-polling.sh +++ b/tests/imfile-wildcards-dirs-multi5-polling.sh @@ -1,11 +1,9 @@ #!/bin/bash -# This is part of the rsyslog testbench, licensed under GPLv3 -export IMFILEINPUTFILES="8" #"8" -export IMFILEINPUTFILESSTEPS="5" #"5" -#export IMFILEINPUTFILESALL=$(($IMFILEINPUTFILES * $IMFILEINPUTFILESSTEPS)) -export IMFILECHECKTIMEOUT="5" +# This is part of the rsyslog testbench, licensed under ASL 2.0 . $srcdir/diag.sh init -#. $srcdir/diag.sh check-inotify-only +export IMFILEINPUTFILES="8" +export IMFILEINPUTFILESSTEPS="5" +export IMFILECHECKTIMEOUT="5" # generate input files first. Note that rsyslog processes it as # soon as it start up (so the file should exist at that point). @@ -13,7 +11,7 @@ export IMFILECHECKTIMEOUT="5" . $srcdir/diag.sh generate-conf . $srcdir/diag.sh add-conf ' global( debug.whitelist="on" - debug.files=["imfile.c"]) + debug.files=["imfile.c", "stream.c"]) # debug.files=["rainerscript.c", "ratelimit.c", "ruleset.c", "main Q", # "msg.c", "../action.c", "imdiag.c"]) @@ -46,30 +44,28 @@ if $msg contains "msgnum:" then . $srcdir/diag.sh startup -for j in `seq 1 $IMFILEINPUTFILESSTEPS`; -do +for j in $(seq 1 $IMFILEINPUTFILESSTEPS); do + cp log log.$j echo "Loop Num $j" for i in `seq 1 $IMFILEINPUTFILES`; do - mkdir rsyslog.input.dir$i - mkdir rsyslog.input.dir$i/dir$i - ./inputfilegen -m 1 > rsyslog.input.dir$i/dir$i/file.logfile + mkdir $RSYSLOG_DYNNAME.input.dir$i + mkdir $RSYSLOG_DYNNAME.input.dir$i/dir$i + ./inputfilegen -i $j -m 1 > $RSYSLOG_DYNNAME.input.dir$i/dir$i/file.logfile done ls -d rsyslog.input.* # Check correct amount of input files each time - let IMFILEINPUTFILESALL=$(($IMFILEINPUTFILES * $j)) - . $srcdir/diag.sh content-check-with-count "HEADER msgnum:00000000:" $IMFILEINPUTFILESALL $IMFILECHECKTIMEOUT - + let IMFILEINPUTFILESALL=$((IMFILEINPUTFILES * j)) + content_check_with_count "HEADER msgnum:000000" $IMFILEINPUTFILESALL $IMFILECHECKTIMEOUT # Delete all but first! for i in `seq 1 $IMFILEINPUTFILES`; do rm -rf rsyslog.input.dir$i/dir$i/file.logfile rm -rf rsyslog.input.dir$i done - done . $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages -- 2.16.4 ++++++ 0003-imfile-improve-truncation-detection.patch ++++++ From e84113816d69d428c96924647a0167be4f6a330a Mon Sep 17 00:00:00 2001 From: Rainer Gerhards <[email protected]> Date: Thu, 20 Sep 2018 10:39:47 +0200 Subject: [PATCH 3/7] imfile: improve truncation detection previously, truncation was only detected at end of file. Especially with busy files that could cause loss of data and possibly also stall imfile reading. The new code now also checks during each read. Obviously, there is some additional overhead associated with that, but this is unavoidable. It still is highly recommended NOT to turn on "reopenOnTruncate" in imfile. Note that there are also inherant reliability issues. There is no way to "fix" these, as they are caused by races between the process(es) who truncate and rsyslog reading the file. But with the new code, the "problem window" should be much smaller and, more importantly, imfile should not stall. see also https://github.com/rsyslog/rsyslog/issues/2659 see also https://github.com/rsyslog/rsyslog/issues/1605 (cherry picked from commit 2d15cbc8221e385c5aa821e4a851d7498ed81850) --- runtime/rsyslog.h | 6 +++--- runtime/stream.c | 13 +++++++++++++ runtime/stream.h | 1 + 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index f7107097f..a43149501 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -196,9 +196,9 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth /* begin regular error codes */ RS_RET_NOT_IMPLEMENTED = -7, /**< implementation is missing (probably internal error or lazyness ;)) */ RS_RET_OUT_OF_MEMORY = -6, /**< memory allocation failed */ - RS_RET_PROVIDED_BUFFER_TOO_SMALL = -50, -/*< the caller provided a buffer, but the called function sees the size of this buffer is too small - -operation not carried out */ + RS_RET_PROVIDED_BUFFER_TOO_SMALL = -50, /*< the caller provided a buffer, but the called function sees + the size of this buffer is too small - operation not carried out */ + RS_RET_FILE_TRUNCATED = -51, /**< (input) file was truncated, not an error but a status */ RS_RET_TRUE = -3, /**< to indicate a true state (can be used as TRUE, legacy) */ RS_RET_FALSE = -2, /**< to indicate a false state (can be used as FALSE, legacy) */ RS_RET_NO_IRET = -8, /**< This is a trick for the debuging system - it means no iRet is provided */ diff --git a/runtime/stream.c b/runtime/stream.c index 028a4652c..cbe618cdd 100644 --- a/runtime/stream.c +++ b/runtime/stream.c @@ -363,6 +363,7 @@ static rsRetVal strmOpenFile(strm_t *pThis) CHKiRet(doPhysOpen(pThis)); pThis->iCurrOffs = 0; + pThis->iBufPtrMax = 0; CHKiRet(getFileSize(pThis->pszCurrFName, &offset)); if(pThis->tOperationsMode == STREAMMODE_WRITE_APPEND) { pThis->iCurrOffs = offset; @@ -655,6 +656,9 @@ checkTruncation(strm_t *const pThis) iRet = rereadTruncated(pThis, "cannot seek backward to begin of last block"); FINALIZE; } +dbgprintf("checkTruncation seek backwrds: %d\n", ret); +currpos = lseek64(pThis->fd, 0, SEEK_CUR); +dbgprintf("checkTruncation in actual processing, NEW currpos %d, backseek is %d\n", (int)currpos, (int) backseek); const ssize_t lenRead = read(pThis->fd, pThis->pIOBuf_truncation, pThis->iBufPtrMax); dbgprintf("checkTruncation proof-read: %d bytes\n", (int) lenRead); @@ -706,6 +710,13 @@ strmReadBuf(strm_t *pThis, int *padBytes) toRead = (size_t) bytesLeft; } } + if(pThis->bReopenOnTruncate) { + rsRetVal localRet = checkTruncation(pThis); + if(localRet == RS_RET_FILE_TRUNCATED) { + continue; + } + CHKiRet(localRet); + } iLenRead = read(pThis->fd, pThis->pIOBuf, toRead); DBGOPRINT((obj_t*) pThis, "file %d read %ld bytes\n", pThis->fd, iLenRead); /* end crypto */ @@ -1210,6 +1221,7 @@ static rsRetVal strmConstructFinalize(strm_t *pThis) } else { /* we work synchronously, so we need to alloc a fixed pIOBuf */ CHKmalloc(pThis->pIOBuf = (uchar*) MALLOC(pThis->sIOBufSize)); + CHKmalloc(pThis->pIOBuf_truncation = (char*) MALLOC(pThis->sIOBufSize)); } finalize_it: @@ -1257,6 +1269,7 @@ CODESTARTobjDestruct(strm) } } else { free(pThis->pIOBuf); + free(pThis->pIOBuf_truncation); } /* Finally, we can free the resources. diff --git a/runtime/stream.h b/runtime/stream.h index 57d8c68b0..f395bc123 100644 --- a/runtime/stream.h +++ b/runtime/stream.h @@ -129,6 +129,7 @@ typedef struct strm_s { ino_t inode; /* current inode for files being monitored (undefined else) */ uchar *pszCurrFName; /* name of current file (if open) */ uchar *pIOBuf; /* the iobuffer currently in use to gather data */ + char *pIOBuf_truncation; /* iobuffer used during trucation detection block re-reads */ size_t iBufPtrMax; /* current max Ptr in Buffer (if partial read!) */ size_t iBufPtr; /* pointer into current buffer */ int iUngetC; /* char set via UngetChar() call or -1 if none set */ -- 2.16.4 ++++++ 0004-imfile-add-declaration-for-restart.patch ++++++ From 84cf08c376d7e1941a0a8fe4022f397211d6c0dd Mon Sep 17 00:00:00 2001 From: Thomas Blume <[email protected]> Date: Mon, 15 Jul 2019 09:47:50 +0200 Subject: [PATCH 4/7] imfile add declaration for restart taken from 3822da837e4d531e8a9cd78ae76359a410f8d98d (Symlink support for imfile) --- plugins/imfile/imfile.c | 1 + 1 file changed, 1 insertion(+) diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index a13687df8..a09ff4c9f 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -615,6 +615,7 @@ detect_updates(fs_edge_t *const edge) { act_obj_t *act; struct stat fileInfo; + int restart = 0; for(act = edge->active ; act != NULL ; ) { DBGPRINTF("detect_updates checking active obj '%s'\n", act->name); -- 2.16.4 ++++++ 0005-WIP-Add-the-mmkubernetes-plugin.patch ++++++ ++++ 791 lines (skipped) ++++++ 0006-Kubernetes-Metadata-plugin-mmkubernetes.patch ++++++ ++++ 1539 lines (skipped) ++++++ 0007-mmkubernetes-fix-lnrules-add-defaults-add-test.patch ++++++ ++++ 690 lines (skipped) ++++++ 0008-default-rules-use-container_name_and_id-include-rule.patch ++++++ From 32e3332cd37f5ba167007c5d27ce1a57f7b6fbdc Mon Sep 17 00:00:00 2001 From: Rich Megginson <[email protected]> Date: Mon, 2 Jul 2018 08:00:11 -0600 Subject: [PATCH] default rules use container_name_and_id; include rulebase files in dist --- contrib/mmkubernetes/Makefile.am | 2 ++ contrib/mmkubernetes/mmkubernetes.c | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/contrib/mmkubernetes/Makefile.am b/contrib/mmkubernetes/Makefile.am index 3dcc235a6..88dca48ec 100644 --- a/contrib/mmkubernetes/Makefile.am +++ b/contrib/mmkubernetes/Makefile.am @@ -4,3 +4,5 @@ mmkubernetes_la_SOURCES = mmkubernetes.c mmkubernetes_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(CURL_CFLAGS) $(LIBLOGNORM_CFLAGS) mmkubernetes_la_LDFLAGS = -module -avoid-version mmkubernetes_la_LIBADD = $(CURL_LIBS) $(LIBLOGNORM_LIBS) + +EXTRA_DIST = k8s_filename.rulebase k8s_container_name.rulebase diff --git a/contrib/mmkubernetes/mmkubernetes.c b/contrib/mmkubernetes/mmkubernetes.c index 1892cc10c..e2c83e8f8 100644 --- a/contrib/mmkubernetes/mmkubernetes.c +++ b/contrib/mmkubernetes/mmkubernetes.c @@ -77,9 +77,9 @@ DEFobjCurrIf(regexp) */ #define DFLT_FILENAME_LNRULES "rule=:/var/log/containers/%pod_name:char-to:.%."\ "%container_hash:char-to:_%_"\ - "%namespace_name:char-to:_%_%container_name:char-to:-%-%container_id:char-to:.%.log\n"\ + "%namespace_name:char-to:_%_%container_name_and_id:char-to:.%.log\n"\ "rule=:/var/log/containers/%pod_name:char-to:_%_"\ - "%namespace_name:char-to:_%_%container_name:char-to:-%-%container_id:char-to:.%.log" + "%namespace_name:char-to:_%_%container_name_and_id:char-to:.%.log" #define DFLT_FILENAME_RULEBASE "/etc/rsyslog.d/k8s_filename.rulebase" /* original from fluentd plugin: * '^(?<name_prefix>[^_]+)_(?<container_name>[^\._]+)\ -- 2.16.4 ++++++ module-mysql ++++++ # for logging to mysql DB (rsyslog-module-mysql) #include <abstractions/mysql> #include <abstractions/p11-kit> /etc/my.cnf r, /etc/my.cnf.d/ r, /etc/my.cnf.d/* r, ++++++ module-snmp ++++++ # for logging to (rsyslog-module-snmp) #include <abstractions/wutmp> /proc/uptime r, /usr/share/snmp/mibs/ r, /usr/share/snmp/mibs/*.txt r, /var/lib/net-snmp/mib_indexes/ rw, /var/lib/net-snmp/mib_indexes/* rw, ++++++ module-udpspoof ++++++ # for logging with omudpspoof (rsyslog-module-udpspoof) capability net_raw, network inet raw, ++++++ rsyslog-service-prepare.in ++++++ #!/bin/bash test -s "/etc/sysconfig/syslog" && \ . "/etc/sysconfig/syslog" run_dir="RUN_DIR" cfg_file="ADDITIONAL_SOCKETS" umask 0022 /bin/mkdir -p -m 0755 "${run_dir}" # # Prepare include with sockets in chroot's # > "${cfg_file}" for variable in ${!SYSLOGD_ADDITIONAL_SOCKET*}; do eval value=\$$variable test -z "$value" && continue test -d "${value%/*}" || continue echo "\$AddUnixListenSocket $value" done >> "${cfg_file}" # # make sure xconsole exists and is a pipe # if test -e /dev/xconsole -a ! -p /dev/xconsole ; then /bin/rm -f /dev/xconsole fi if test ! -e /dev/xconsole ; then /bin/mknod -m 0600 /dev/xconsole p /bin/chown root:tty /dev/xconsole restorecon /dev/xconsole 2> /dev/null fi exit 0 ++++++ rsyslog-unit.patch ++++++ Index: rsyslog-8.33.1/rsyslog.service.in =================================================================== --- rsyslog-8.33.1.orig/rsyslog.service.in +++ rsyslog-8.33.1/rsyslog.service.in @@ -1,14 +1,21 @@ [Unit] Description=System Logging Service Requires=syslog.socket +Requires=var-run.mount +After=var-run.mount +Conflicts=syslog-ng.service syslogd.service Documentation=man:rsyslogd(8) Documentation=http://www.rsyslog.com/doc/ [Service] Type=notify -ExecStart=@sbindir@/rsyslogd -n -iNONE +Environment=RSYSLOGD_PARAMS= +EnvironmentFile=-/etc/sysconfig/syslog +ExecStartPre=@sbindir@/rsyslog-service-prepare +ExecStart=@sbindir@/rsyslogd -n -iNONE $RSYSLOGD_PARAMS +ExecReload=/bin/kill -HUP $MAINPID StandardOutput=null -Restart=on-failure +Restart=on-abort [Install] WantedBy=multi-user.target ++++++ rsyslog.conf.in ++++++ ## ## === When you're using remote logging, enable on-disk queues === ## === in rsyslog.d/remote.conf. === ## ## Note, that when the MYSQL, PGSQL, GSSAPI, GnuTLS or SNMP modules ## (provided in separate rsyslog-module-* packages) are enabled, the ## configuration can't be used on a system with /usr on a remote ## filesystem, except on newer systems where initrd mounts /usr. ## [The modules are linked against libraries installed bellow of ## /usr thus also installed in /usr/lib*/rsyslog because of this.] ## # # if you experience problems, check # http://www.rsyslog.com/troubleshoot for assistance # and report them at https://bugzilla.suse.com/ for SUSE Linux Enterprise # or https://bugzilla.opensuse.org/ for openSUSE # # since rsyslog v3: load input modules # If you do not load inputs, nothing happens! # provides --MARK-- message capability (every 1 hour) $ModLoad immark.so $MarkMessagePeriod 3600 # provides support for local system logging (e.g. via logger command) $ModLoad imuxsock.so # reduce dupplicate log messages (last message repeated n times) $RepeatedMsgReduction on # kernel logging (may be also provided by /sbin/klogd) # see also http://www.rsyslog.com/doc-imklog.html. $ModLoad imklog.so # set log level 1 (same as in /etc/sysconfig/syslog). $klogConsoleLogLevel 1 # # Set the default permissions for all log files. # $FileOwner root $FileGroup root $FileCreateMode 0640 $DirCreateMode 0750 $Umask 0022 # Use rsyslog native, rfc5424 conform log format as default # ($ActionFileDefaultTemplate RSYSLOG_FileFormat). # # To change a single file to use obsolete BSD syslog format # (rfc 3164, no high-precision timestamps), set the variable # bellow or append ";RSYSLOG_FileFormat" to the filename. # See # http://www.rsyslog.com/doc/rsyslog_conf_templates.html # for more informations. # #$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # # Include config generated by /etc/init.d/syslog script # using the SYSLOGD_ADDITIONAL_SOCKET* variables in the # /etc/sysconfig/syslog file. # $IncludeConfig ADDITIONAL_SOCKETS # # Include config files, that the admin provided? : # $IncludeConfig ETC_RSYSLOG_D_GLOB ### # print most important on tty10 and on the xconsole pipe # if ( \ /* kernel up to warning except of firewall */ \ ($syslogfacility-text == 'kern') and \ ($syslogseverity <= 4 /* warning */ ) and not \ ($msg contains 'IN=' and $msg contains 'OUT=') \ ) or ( \ /* up to errors except of facility authpriv */ \ ($syslogseverity <= 3 /* errors */ ) and not \ ($syslogfacility-text == 'authpriv') \ ) \ then { /dev/tty10 |/dev/xconsole } # Emergency messages to everyone logged on (wall) *.emerg :omusrmsg:* # enable this, if you want that root is informed # immediately, e.g. of logins #*.alert root # # firewall messages into separate file and stop their further processing # if ($syslogfacility-text == 'kern') and \ ($msg contains 'IN=' and $msg contains 'OUT=') \ then { -/var/log/firewall stop } # # acpid messages into separate file and stop their further processing # # => all acpid messages for debuging (uncomment if needed): #if ($programname == 'acpid' or $syslogtag == '[acpid]:') then \ # -/var/log/acpid # # => up to notice (skip info and debug) if ($programname == 'acpid' or $syslogtag == '[acpid]:') and \ ($syslogseverity <= 5 /* notice */) \ then { -/var/log/acpid stop } # # NetworkManager into separate file and stop their further processing # if ($programname == 'NetworkManager') or \ ($programname startswith 'nm-') \ then { -/var/log/NetworkManager stop } # # email-messages # mail.* -/var/log/mail mail.info -/var/log/mail.info mail.warning -/var/log/mail.warn mail.err /var/log/mail.err # # news-messages # #news.crit -/var/log/news/news.crit #news.err -/var/log/news/news.err #news.notice -/var/log/news/news.notice # enable this, if you want to keep all news messages # in one file #news.* -/var/log/news.all # # Warnings in one file # *.=warning;*.=err -/var/log/warn *.crit /var/log/warn # # the rest in one file # *.*;mail.none;news.none -/var/log/messages # # enable this, if you want to keep all messages # in one file #*.* -/var/log/allmessages # # Some foreign boot scripts require local7 # local0.*;local1.* -/var/log/localmessages local2.*;local3.* -/var/log/localmessages local4.*;local5.* -/var/log/localmessages local6.*;local7.* -/var/log/localmessages ### ++++++ rsyslog.d.remote.conf.in ++++++ ## ## === When you're using remote logging, enable on-disk queues === ## === in rsyslog.d/remote.conf. === ## ## Note, that when the MYSQL, PGSQL, GSSAPI, GnuTLS or SNMP modules ## (provided in separate rsyslog-module-* packages) are enabled, the ## configuration can't be used on a system with /usr on a remote ## filesystem, except on newer systems where initrd mounts /usr. ## [The modules are linked against libraries installed bellow of ## /usr thus also installed in /usr/lib*/rsyslog because of this.] ## # ######### Enable On-Disk queues for remote logging ########## # # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. # #$WorkDirectory RSYSLOG_SPOOL_DIR # where to place spool files #$ActionQueueFileName uniqName # unique name prefix for spool files #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown #$ActionQueueType LinkedList # run asynchronously #$ActionResumeRetryCount -1 # infinite retries if host is down # ######### Sending Messages to Remote Hosts ########## # Remote Logging using TCP for reliable delivery # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional #*.* @@remote-host # Remote Logging using UDP # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional #*.* @remote-host # ######### Receiving Messages from Remote Hosts ########## # TCP Syslog Server: # provides TCP syslog reception and GSS-API (if compiled to support it) #$ModLoad imtcp.so # load module ##$UDPServerAddress 10.10.0.1 # force to listen on this IP only #$InputTCPServerRun <port> # Starts a TCP server on selected port # UDP Syslog Server: #$ModLoad imudp.so # provides UDP syslog reception ##$UDPServerAddress 10.10.0.1 # force to listen on this IP only #$UDPServerRun 514 # start a UDP syslog server at standard port 514 ########### Encrypting Syslog Traffic with TLS ########## # -- TLS Syslog Server: ## make gtls driver the default #$DefaultNetstreamDriver gtls # ## certificate files #$DefaultNetstreamDriverCAFile ETC_RSYSLOG_D_DIR/ca.pem #$DefaultNetstreamDriverCertFile ETC_RSYSLOG_D_DIR/server_cert.pem #$DefaultNetstreamDriverKeyFile ETC_RSYSLOG_D_DIR/server_key.pem # #$ModLoad imtcp # load TCP listener # #$InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode #$InputTCPServerStreamDriverAuthMode anon # client is NOT authenticated #$InputTCPServerRun 10514 # start up listener at port 10514 # # -- TLS Syslog Client: ## certificate files - just CA for a client #$DefaultNetstreamDriverCAFile ETC_RSYSLOG_D_DIR/ca.pem # ## set up the action #$DefaultNetstreamDriver gtls # use gtls netstream driver #$ActionSendStreamDriverMode 1 # require TLS for the connection #$ActionSendStreamDriverAuthMode anon # server is NOT authenticated #*.* @@(o)server.example.net:10514 # send (all) messages ++++++ rsyslog.firewall ++++++ ## Name: Syslog Server ## Description: Opens ports to accept remote syslog entries. # space separated list of allowed TCP ports TCP="" # space separated list of allowed UDP ports UDP="syslog" # space separated list of allowed RPC services RPC="" # space separated list of allowed IP protocols IP="" # space separated list of allowed UDP ports that accept broadcasts BROADCAST="" ### variables below are only needed in very special cases # space separated list of net,protocol[,sport[,dport]] # see FW_SERVICES_ACCEPT_RELATED_EXT # net 0/0 means IPv4 and IPv6. If this sevice should only work for # IPv4 use 0.0.0.0/0 RELATED="" # additional kernel modules needed for this service # see FW_LOAD_MODULES MODULES="" ++++++ rsyslog.sysconfig ++++++ ## Type: string ## Default: "" ## Config: "" ## ServiceRestart: syslog # # Parameters for rsyslogd, except of the version compatibility (-c) # and the config file (-f), because they're used by sysconfig and # earlysysconfig init scripts. # # See also the RSYSLOGD_COMPAT_VERSION variable in this file, the # documentation provided in /usr/share/doc/packages/rsyslog/doc by # the rsyslog-doc package and the rsyslogd(8) and rsyslog.conf(5) # manual pages. # RSYSLOGD_PARAMS="" ++++++ usr.sbin.rsyslogd ++++++ # ------------------------------------------------------------------ # # Copyright (C) 2014 Novell/SUSE # # This program is free software; you can redistribute it and/or # modify it under the terms of version 2 of the GNU General Public # License published by the Free Software Foundation. # # ------------------------------------------------------------------ #include <tunables/global> /usr/sbin/rsyslogd { #include <abstractions/base> #include <abstractions/consoles> # general networking is allowed here #include <abstractions/nameservice> capability dac_override, capability sys_nice, capability sys_tty_config, capability syslog, deny capability block_suspend, /dev/tty* w, /dev/xconsole rw, /etc/rsyslog.conf r, /etc/rsyslog.d/ r, /etc/rsyslog.d/* r, /usr/lib{,32,64}/rsyslog/* mr, /usr/sbin/rsyslogd mr, /var/log/** rw, /var/lib/*/dev/log w, /proc/kmsg r, /{var/,}run/rsyslog/* r, /{var/,}run/rsyslogd.pid rwk, /{var/,}run/systemd/journal/syslog w, # include rules for rsyslog-module-* packages #include "/usr/share/apparmor/extra-profiles/rsyslog.d" # for logging via TLS (rsyslog-module-gtls) # keys/certificates need to be located under /etc/rsyslog.d or permissions need to be adjusted here # rsyslog tries to write to the certificates for no reason, so deny this quietly deny /etc/rsyslog.d/* w, } _______________________________________________ openSUSE Commits mailing list -- [email protected] To unsubscribe, email [email protected] List Netiquette: https://en.opensuse.org/openSUSE:Mailing_list_netiquette List Archives: https://lists.opensuse.org/archives/list/[email protected]
