[
https://issues.apache.org/jira/browse/RANGER-397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Don Bosco Durai resolved RANGER-397.
------------------------------------
Resolution: Fixed
> Implement reliable streaming audits to configurable destinations
> ----------------------------------------------------------------
>
> Key: RANGER-397
> URL: https://issues.apache.org/jira/browse/RANGER-397
> Project: Ranger
> Issue Type: Improvement
> Components: audit
> Affects Versions: 0.5.0
> Reporter: Don Bosco Durai
> Assignee: Don Bosco Durai
> Fix For: 0.5.0
>
>
> Currently for audit to HDFS, Ranger writes to the file and then transfers the
> entire file to HDFS on regular interval. This adds additional write operation
> to the local disk.
> The proposal is to write a more intelligent audit writer, audits are sent to
> destination in real time (or batches) and if the destination is down, then
> write to local file. When the destination is available, then first send the
> audit logs from the file system and after it is caught up, resume real-time
> streaming.
> This design also need to address use cases where the destination is slower
> than the audit producer. In which case, if the internal queue reaches a
> certain threshold, then the audit will be written to local file till the
> destination till the in-memory queue is drained.
> The design should be generic enough to support any type of destination. By
> default, the implementation for the following destinations should be provided:
> 1. HDFS
> 2. Solr
> 3. Local File
> 4. Log4J (with any supported appender)
> Additional good to have destinations are :
> 1. RDBMS
> 2. Kafka
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
