Hi David, fortunately I had already done this. I'm including an actual log entry but have anonymized the data to keep the actual user and email address confidential:
Debug line with all properties: FROMHOST: 'ause1oagatst02.aws.mycompany.com' , fromhost-ip: '10.41.102.143' , HOSTNAME: 'ause1oagatst02.aws.mycompany.com' , PRI: 13, syslogtag '03-23T16:' , programname: '03-23T16' , APP-NAME: '03-23T16' , PROCID: '-' , MSGID: '-' , TIMNESTAMP: 'Mar 23 21:47:13' , STRUCTURED-DATA: '-' , *msg*: '47:20.708-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ SESSION INFO USER_SESSION [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT=" [email protected]" APP="Ignio Uat OAG" APP_TYPE="HEADERWEB2015_APP" APP_DOMAIN="apigniodashboard-uat.mycompany.com" RESULT="ALLOW" REASON="SESSION_INTEGRITY_VERIFIED" REMOTE_IP="10.44.65.38" USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"] SRF Request RemoteIP: 10.44.65.38 verified session RemoteIP: 10.44.65.38#0122021-03-23T16:47:20.708-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ POLICY INFO USER_AUTHZ [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT="[email protected]" RESOURCE="/_dash-dependencies" METHOD="GET" POLICY="root" POLICY_TYPE="PROTECTED" DURATION="0" APP="Ignio Uat OAG" APP_TYPE="HEADERWEB2015_APP" APP_DOMAIN=" apigniodashboard-uat.mycompany.com" RESULT="ALLOW" REASON="N/A - SESSIONID=_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0 oag_username= [email protected] RelayDomain=apigniodashboard-uat.mycompany.com [email protected] SourceAuthNType=urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport RemoteIP=10.44.65.38 USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36 creationTime=1616536016811 maxInactiveInterval=3600000 maxActiveInterval=28800000 lastAccessedTime=1616536029221 " REMOTE_IP="10.44.65.38" USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"] allow access to resource#0122021-03-23T16:47:20.711-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ SESSION INFO USER_SESSION [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT="Joe.User@mycompan' *escaped msg*: '47:20.708-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ SESSION INFO USER_SESSION [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT=" [email protected]" APP="Ignio Uat OAG" APP_TYPE="HEADERWEB2015_APP" APP_DOMAIN="apigniodashboard-uat.mycompany.com" RESULT="ALLOW" REASON="SESSION_INTEGRITY_VERIFIED" REMOTE_IP="10.44.65.38" USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"] SRF Request RemoteIP: 10.44.65.38 verified session RemoteIP: 10.44.65.38#0122021-03-23T16:47:20.708-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ POLICY INFO USER_AUTHZ [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT="[email protected]" RESOURCE="/_dash-dependencies" METHOD="GET" POLICY="root" POLICY_TYPE="PROTECTED" DURATION="0" APP="Ignio Uat OAG" APP_TYPE="HEADERWEB2015_APP" APP_DOMAIN=" apigniodashboard-uat.mycompany.com" RESULT="ALLOW" REASON="N/A - SESSIONID=_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0 oag_username= [email protected] RelayDomain=apigniodashboard-uat.mycompany.com [email protected] SourceAuthNType=urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport RemoteIP=10.44.65.38 USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36 creationTime=1616536016811 maxInactiveInterval=3600000 maxActiveInterval=28800000 lastAccessedTime=1616536029221 " REMOTE_IP="10.44.65.38" USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"] allow access to resource#0122021-03-23T16:47:20.711-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ SESSION INFO USER_SESSION [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT="Joe.User@mycompan' *rawmsg*: '03-23T16:47:20.708-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY *ACCESS* AUTHZ SESSION INFO USER_SESSION [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT=" [email protected]" APP="Ignio Uat OAG" APP_TYPE="HEADERWEB2015_APP" APP_DOMAIN="apigniodashboard-uat.mycompany.com" RESULT="ALLOW" REASON="SESSION_INTEGRITY_VERIFIED" REMOTE_IP="10.44.65.38" USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"] SRF Request RemoteIP: 10.44.65.38 verified session RemoteIP: 10.44.65.38#0122021-03-23T16:47:20.708-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ POLICY INFO USER_AUTHZ [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT="[email protected]" RESOURCE="/_dash-dependencies" METHOD="GET" POLICY="root" POLICY_TYPE="PROTECTED" DURATION="0" APP="Ignio Uat OAG" APP_TYPE="HEADERWEB2015_APP" APP_DOMAIN=" apigniodashboard-uat.mycompany.com" RESULT="ALLOW" REASON="N/A - SESSIONID=_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0 oag_username= [email protected] RelayDomain=apigniodashboard-uat.mycompany.com [email protected] SourceAuthNType=urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport RemoteIP=10.44.65.38 USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36 creationTime=1616536016811 maxInactiveInterval=3600000 maxActiveInterval=28800000 lastAccessedTime=1616536029221 " REMOTE_IP="10.44.65.38" USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36"] allow access to resource#0122021-03-23T16:47:20.711-05:00 AUSE1OAGATST02.aws.mycompany.com ACCESS_GATEWAY ACCESS AUTHZ SESSION INFO USER_SESSION [SESSION_ID="_eddf580f5a5b9436ea27e5e6fa4515a2175ca69ac0" SUBJECT="Joe.User@mycompan' By inspecting the rawmsg, I can see that field four (space-delimited) indicates this is the ACCESS log. So if I were able to extract the log identifier from the msg, I could then write all access logs to the same daily file. There are other formats as well from the same device but the idea is the same. *Scott Slattery* *Sr. Systems & Cloud Architect* *Cloud, Compute, Information & Architecture Team* motorolasolutions.com *O: 602.529.8226* *E*: [email protected] On Tue, Mar 23, 2021 at 5:29 PM David Lang <[email protected]> wrote: > the source logfile name is not included in the payload by the syslog spec. > It > may be in the case of your appliance, but we would need to see a sample > log to > understand ho to parse it. > > based on your template, you are using app-name, which may be listed > separtely if > it's a RFC5424 format log, or may be part of the syslog tag if it's a > RFC3164 > format log over the wire (neither format has a way to specify a source log > file > by default) > > you can look at > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Chojins_LinuxCNC-2DPolargraph&d=DwIDaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=N7tTREp8_2WP8ZJPT8UpZsuFedG7Q4cil9vsQoiSiGM&s=XRQ9OP8K-KeJO3-s6-unMBIRqEZONzs6npmrQYaXnds&e= > and see the *-cc > options that you could apply to the app-name to eliminate control > characters. > > Again, we really need to see the original log message to understand what's > what. > Please log it with the templateRSYSLOG_DebugFormat so we can see exactly > what is > sent over the wire and how rsyslog has parsed it. > > David Lang > > On Tue, 23 Mar 2021, Scott Slattery via rsyslog > wrote: > > > Date: Tue, 23 Mar 2021 16:05:45 -0700 > > From: Scott Slattery via rsyslog <[email protected]> > > To: John Chivian <[email protected]> > > Cc: Scott Slattery <[email protected]>, > > rsyslog-users <[email protected]> > > Subject: Re: [rsyslog] Altering forwarded logfile names > > > > Thanks, John, let me try to clarify what I mean. > > > > Normally when I forward from a remote server to the central log server, I > > can include a tag that can then be used to determine the file name I want > > on the central server. Since I have no real way to include this tag from > > the appliance, this is not an option. > > > > I'm looking for a way of inspecting the incoming packets to determining > the > > source logfile name (which is included in the payload) and use that > > filename on the target central server. Since there are multiple logs > being > > sent (access, audit, monitor, etc.), I'd like to segregate these into > their > > own files. I'm already using a template with the host information to > > dynamically create the file names. I just don't know how I can go beyond > > this to also include the source logname. > > > > Here's the template I'm using. It works for all other hosts where I can > > configure the tag but I get garbage names from the appliance. I had hoped > > that the appliance included some standard syslog tags but it doesn't seem > > so. > > > > template(name="DynRemoteLogFile" type="string" > > > string="/remote/%FROMHOST%-%FROMHOST-IP%/%$year%-%$month%-%$day%-%app-name%.log") > > > > *Scott Slattery* > > > > *Sr. Systems & Cloud Architect* > > > > *Cloud, Compute, Information & Architecture Team* > > > > motorolasolutions.com > > > > *O: 602.529.8226* > > > > *E*: [email protected] > > > > > > > > > > On Tue, Mar 23, 2021 at 3:30 PM John Chivian <[email protected]> > wrote: > > > >> Your use of the term “file name” is confusing. When senders deliver to > >> rsyslog over the network there is no exchange of files or filenames, > only > >> packets of information. Those packets are expected to be in a format > that > >> syslog understands such that useful information (header elements and > >> message body) may be parsed from them. If you as the rsyslog admin > choose > >> to use some of that header information to compose filenames for output > >> files, then yes you are sort of at the mercy of the senders content > >> (especially if the sender doesn’t follow the syslog rules). However, > there > >> are functions in the advanced syntax that can be used to perform the > type > >> of character replacements you’re talking about. > >> > >> It is common practice to use the syslog header/rsyslog property element > >> called “hostname” for just such purposes. Is this what you’re talking > >> about? You’d have to provide your configuration for real analysis, at > >> least the part you perceive to be responsible for the problem. > >> > >> Regards, > >> > >> > >> > >> > On Mar 23, 2021, at 12:35, Scott Slattery via rsyslog < > >> [email protected]> wrote: > >> > > >> > I have a configured central log collector using rsyslog. A few of the > >> > devices forwarding their logs are appliances that have no > configuration > >> > options other than the IP forwarding address and protocol. I cannot > >> control > >> > what file names are being sent. > >> > > >> > Unfortunately, they are sending unintelligible file names with > characters > >> > that normally would be escaped. Is there any way I can control or > alter > >> the > >> > incoming file name to normalize it to avoid these odd characters? > >> > > >> > For example, could I establish a character map that maps the unallowed > >> > character to something acceptable? > >> > > >> > thanks, > >> > > >> > *Scott Slattery* > >> > > >> > *Sr. Systems & Cloud Architect* > >> > > >> > *Cloud, Compute, Information & Architecture Team* > >> > > >> > motorolasolutions.com > >> > > >> > *O: 602.529.822* > >> > > >> > *E*: [email protected] > >> > > >> > -- > >> > > >> > > >> > *For more information on how and why we collect your personal > >> > information, please visit our Privacy Policy > >> > < > >> > https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement > >> >.* > >> > _______________________________________________ > >> > rsyslog mailing list > >> > > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=F25vuEW_UOr4xhEXRHv4FYzBC10xi8a7L7cY9KDJz-E&s=O-radZKC6RhALSGrunmgfnDcUe0FBEzQXlwVMv4rwrk&e= > >> > > >> > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=F25vuEW_UOr4xhEXRHv4FYzBC10xi8a7L7cY9KDJz-E&s=Ujl6rNYsQwlkacdBkNSQI3_ugt9iTahsA2ALpSb1zWA&e= > >> > What's up with rsyslog? Follow > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIFaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=F25vuEW_UOr4xhEXRHv4FYzBC10xi8a7L7cY9KDJz-E&s=5gFALcKlKXLfCND69qR14lRU4iA42kMWjsC9PDoIb3Q&e= > >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >> DON'T LIKE THAT. > >> > >> > > > > -- > > > > > > *For more information on how and why we collect your personal > > information, please visit our Privacy Policy > > < > https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement > >.* > > _______________________________________________ > > rsyslog mailing list > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.adiscon.net_mailman_listinfo_rsyslog&d=DwIDaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=N7tTREp8_2WP8ZJPT8UpZsuFedG7Q4cil9vsQoiSiGM&s=4ENTgbqNRL4m9EpaPD487wHPCEOI1UMUrZ6zizJ25HE&e= > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.rsyslog.com_professional-2Dservices_&d=DwIDaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=N7tTREp8_2WP8ZJPT8UpZsuFedG7Q4cil9vsQoiSiGM&s=YboIrpBbwiXlhlR3JZnvNDi2QWxYQqNifb7d8JV6Xn0&e= > > What's up with rsyslog? Follow > https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_rgerhards&d=DwIDaQ&c=q3cDpHe1hF8lXU5EFjNM_A&r=9VZN8jOeh6Wq3zsBr6Mr_GSxmEpodGbXQ2UxP3oRpciBnWp1cJKyh3iyX6xKS_Zd&m=N7tTREp8_2WP8ZJPT8UpZsuFedG7Q4cil9vsQoiSiGM&s=gVD-Vwy9VAK7xAHPrmGhwhORXImwEoBcYZZVVG-KbZQ&e= > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. -- *For more information on how and why we collect your personal information, please visit our Privacy Policy <https://www.motorolasolutions.com/en_us/about/privacy-policy.html?elqTrackId=8980d888905940e39a2613a7a3dcb0a7&elqaid=2786&elqat=2#privacystatement>.* _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
