[
https://issues.apache.org/jira/browse/RANGER-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983378#comment-15983378
]
Ramesh Mani commented on RANGER-1501:
-------------------------------------
[~coheig] What you seeing is the behaviour even without this patch also. When
the file handle on the hdfs is closed either by a rollover of logging to new
file ( by default its set to @12.00 am everyday) or in case if NN got restarted
for various reason. What we found what that during the abrupt restart of NN
there were some audit log loss and
https://issues.apache.org/jira/browse/RANGER-1310 toke care of it by
introducing a AuditFileSpool which logs into local FS before pushing into
destination. Still need to test this patch throughly to see that Audits are not
lost even without the RANGER-1310 fix.
> Audit Flush to HDFS does not actually cause the audit logs to be flushed to
> HDFS
> ---------------------------------------------------------------------------------
>
> Key: RANGER-1501
> URL: https://issues.apache.org/jira/browse/RANGER-1501
> Project: Ranger
> Issue Type: Bug
> Components: audit
> Affects Versions: 0.7.0
> Reporter: Yan
> Assignee: Yan
> Fix For: 1.0.0
>
> Attachments:
> 0001-RANGER-1501-Audit-Flush-to-HDFS-does-not-actually-ca.patch
>
>
> The reason is that HDFS file stream's flush() call does not really flush the
> data all the way to disk, nor even makes the data visible to HDFS users. See
> the HDFS semantics of the flush/sync at
> https://issues.apache.org/jira/browse/HADOOP-6313.
> Consequently the audit logs on HDFS won't be visible/durable from HDFS client
> until the log file is closed. This will, among other issues, boost chances of
> losing audit logs in case of system failure.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
