----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73875/#review224136 -----------------------------------------------------------
Ship it! Ship It! - Abhay Kulkarni On Feb. 25, 2022, 8:15 p.m., Ramesh Mani wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73875/ > ----------------------------------------------------------- > > (Updated Feb. 25, 2022, 8:15 p.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, > Pradeep Agrawal, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan > Periasamy. > > > Bugs: RANGER-3603 > https://issues.apache.org/jira/browse/RANGER-3603 > > > Repository: ranger > > > Description > ------- > > RANGER-3603:HDFS audit files rollover improvement to trigger rollover in > monitoring thread > > > Diffs > ----- > > > agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractRangerAuditWriter.java > 2e8e71d68 > > agents-audit/src/main/java/org/apache/ranger/audit/utils/RangerJSONAuditWriter.java > eb3ea7766 > > > Diff: https://reviews.apache.org/r/73875/diff/1/ > > > Testing > ------- > > Problem Statement: > Current behavior of Ranger Audit Framework is closing of audit files created > in HDFS or other cloud storage done inline with audit event trigger i.e when > audit event occurs it checks for the configure rollout time and then closes > the file if the threshold has reached. Default audit rollout time is 24 hrs. > In certain scenarios there won’t be any audit event happening in that 24hrs > window and in that case files are kept open even beyond the 24hrs. > In a customer environment where they want to read those audit logs for > analytical purposes see that file is still opened beyond the current date and > it causes exceptions in their systems. Customer wants the files to be closed > every day, so that the audit log file will have only that day's log and the > next day’s log will be in the next day's file. > Proposed Solution: > To achieve this current Audit Framework has to be enhanced to have > configurations which enables the closing of audit files everyday or the > configured time. This configuration has to be set in > ranger-<service>-audit.xml for each of the services via safety value in CM. > By default this feature is disabled. > > E.g hive plugin will have this configuration in ranger-hive.audit.xml > • Parameter to configure: > xasecure.audit.destination.hdfs.file.rollover.enable.periodic.rollover=true > • When this is enabled Ranger Audit Framework will spawn a > Scheduler thread which monitors the occurrence of closing threshold and > closes the file. By default every night the file gets closed. > > xasecure.audit.destination.hdfs.file.rollover.sec > • This is the time in seconds when the file has to be closed. By > default it is 1 day (86400 sec) and this triggers the file to be closed at > midnight and opens a new audit log for the next day. > • Default value can be overridden by setting this parameter. > • E.g if the value “3600” (1 hr) is set, every hour the file gets > closed. > > xasecure.audit.destination.hdfs.file.rollover.periodic.rollover.check.sec > > • This is the time frequency of the check to be done whether the > threshold time for rollover has occurred. > • By default the check is done every 60 secs. > • This param can be configured to delay the check time. > > - Verified in local vm: > > > Thanks, > > Ramesh Mani > >
