Glad it's working now!
There's one downside to consider about having impstats write directly to a file
like:
module (load="impstats" interval="60" log.file="/var/spool/rsyslog/stats.log")
I think this may cause issues with log file rotation (of the stats file). If
you give impstats a direct "out of band" file to write to, it is not handled
the same way (via main/ruleset queues) as other open files, and rsyslog will
not close and re-open it on a HUP signal.
Instead, have impstats call a ruleset to write to the file, like this (thanks
to David's feedback):
module (load="impstats" interval="60" ruleset="pstats")
ruleset(name="pstats"
queue.type="LinkedList") {
action(type="omfile"
name="pstats"
file="/var/spool/rsyslog/stats.log"
)
}
According to
http://www.rsyslog.com/doc/v8-stable/configuration/modules/impstats.html this
will also get your "rsyslogd-pstats:" messages out of the standard syslog
stream because using log.file="stats.log" without also setting log.syslog="off"
will send the pstats logs to *both* the local file and into the syslog stream.
One of the motivations for having impstats write directly to it's own file is
that you could be losing pstats messages due to some queue getting full and
discarding them (and when you have log delivery issues is exactly the time you
don't want to be dropping the stats output), but giving the pstats messages
their own queue (via dedicated ruleset) helps mitigate this. It's unlikely
that the stats queue will get
--
Dave Caplinger, Director, Technical Product Management
Solutionary — An NTT Group Security Company
> On Oct 8, 2015, at 7:19 PM, Randy Baca <[email protected]> wrote:
>
> Dang! Yup, it was SELinux. Got it working end to end now, just need to test
> failed connections and spooling. Many thanks for everything. I think we are
> golden now.
>
> And I agree with your arguments. Maybe I will use them on management some
> time real soon. ;-)
>
> ________________________________________
> From: [email protected] [[email protected]]
> on behalf of David Lang [[email protected]]
> Sent: Thursday, October 08, 2015 3:50 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>
> On Thu, 8 Oct 2015, Randy Baca wrote:
>
>> Regarding the tagging of messages, we can't really add anything due to the
>> way
>> the SEIM parses. If we change the format of the message from the default we
>> lose manageability. I was told there may be compliance issues with that,
>> also.
>
> the nice thing about adding metadata by encapsulating the message in JSON is
> that you have the ability to send the original format and still use the
> metadata
> in your decision making
>
> Compliance is worried about logs being modified in ways that makes them
> unreliable and untrustworthy. There is a myth that if the log is modified in
> any way it's not admissible in court, but just by sending logs they frequently
> get modified and there are no problems using the results.
>
> The key is that the changes to the logs MUST be automated, predictable, and
> not
> loose/modify the data in the log.
>
> They key worry here is that someone may modify the messages to hide what they
> are doing. Adding additional metadata to the log makes the logs more useful,
> not
> less.
>
> I've worked in Banking environments and gone through dozens of Government
> audits
> looking at our logging processes. I've also had numerous cases where the
> resulting logs have been used in legal cases (not all have ended up going to
> court, but lawyers and the FBI have requested logs and examined them and our
> logging process). There has never been a problem with automated, progromatic
> changes to the logs as they are processed.
>
> If you think about it. What gets displayed in the SIEM is not the exact,
> unmodified string that was sent to the SIEM anyway, it breaks out the
> timestamp,
> hostname, etc. That counts as "modifying the log" just as much as what I'm
> talking about.
>
>> Regarding impstat, I don't get any stats for either the port=514 action or
>> the
>> port=10000 action. In my test scenario I am receiving about 50-100 EPS so
>> there should be something there. The directory permissions are set to 755.
>> There is no stats.log file at all.
>
> ahh, I misunderstood what you were saying. IF you are not getting
> /var/spool/rsyslog/stats.log, check your SELinux permissions. Try changing the
> file to be under /var/log instead.
>
> Also, you should put an interval on the impstats line to tell it how
> frequently
> to dump the data.
>
> For our initial testing here, set it to something nice and short (say 10
> seconds) so that it takes less time to get output to work on.
>
> David Lang
>
>> ________________________________________
>> From: [email protected] [[email protected]]
>> on behalf of David Lang [[email protected]]
>> Sent: Thursday, October 08, 2015 3:13 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Complex forwarding and spoofing question
>>
>> On Thu, 8 Oct 2015, Randy Baca wrote:
>>
>>> That is a correct assessment of the flow. There is no impstats output.
>>> The line in the conf is:
>>> module (load="impstats" log.file="/var/spool/rsyslog/stats.log")
>>>
>>> There is no file created whether on 514 or 10000.
>>
>> so the impstats line for that action shows 0 messages processed?
>>
>> that means that nothing ended up calling that ruleset. Rsyslog won't connect
>> until it has the first message to deliver.
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com/professional-services/
>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
>> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
>> LIKE THAT.
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
