-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73875/
-----------------------------------------------------------
Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh,
Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan
Periasamy.
Bugs: RANGER-3603
https://issues.apache.org/jira/browse/RANGER-3603
Repository: ranger
Description
-------
RANGER-3603:HDFS audit files rollover improvement to trigger rollover in
monitoring thread
Diffs
-----
agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractRangerAuditWriter.java
2e8e71d68
agents-audit/src/main/java/org/apache/ranger/audit/utils/RangerJSONAuditWriter.java
eb3ea7766
Diff: https://reviews.apache.org/r/73875/diff/1/
Testing
-------
Problem Statement:
Current behavior of Ranger Audit Framework is closing of audit files created
in HDFS or other cloud storage done inline with audit event trigger i.e when
audit event occurs it checks for the configure rollout time and then closes the
file if the threshold has reached. Default audit rollout time is 24 hrs. In
certain scenarios there won’t be any audit event happening in that 24hrs window
and in that case files are kept open even beyond the 24hrs.
In a customer environment where they want to read those audit logs for
analytical purposes see that file is still opened beyond the current date and
it causes exceptions in their systems. Customer wants the files to be closed
every day, so that the audit log file will have only that day's log and the
next day’s log will be in the next day's file.
Proposed Solution:
To achieve this current Audit Framework has to be enhanced to have
configurations which enables the closing of audit files everyday or the
configured time. This configuration has to be set in ranger-<service>-audit.xml
for each of the services via safety value in CM. By default this feature is
disabled.
E.g hive plugin will have this configuration in ranger-hive.audit.xml
• Parameter to configure:
xasecure.audit.destination.hdfs.file.rollover.enable.periodic.rollover=true
• When this is enabled Ranger Audit Framework will spawn a
Scheduler thread which monitors the occurrence of closing threshold and closes
the file. By default every night the file gets closed.
xasecure.audit.destination.hdfs.file.rollover.sec
• This is the time in seconds when the file has to be closed. By
default it is 1 day (86400 sec) and this triggers the file to be closed at
midnight and opens a new audit log for the next day.
• Default value can be overridden by setting this parameter.
• E.g if the value “3600” (1 hr) is set, every hour the file gets
closed.
xasecure.audit.destination.hdfs.file.rollover.periodic.rollover.check.sec
• This is the time frequency of the check to be done whether the
threshold time for rollover has occurred.
• By default the check is done every 60 secs.
• This param can be configured to delay the check time.
- Verified in local vm:
Thanks,
Ramesh Mani
