fateh288 opened a new pull request, #279: URL: https://github.com/apache/ranger/pull/279
Other changes: Test cases for orc audits. Typo fix for zlib compression. ## What changes were proposed in this pull request? This is a follow up patch for RANGER-1837 and RANGER-4178 which enables audit logging in ORC format for HDFS audit destination. The pull request has the following 3 components: 1) Config consolidation: Currently there are 2 configs which will be required to enable orc i.e. `xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc `and config `xasecure.audit.destination.hdfs.filetype=orc` Ideally, we would want only 1 config to enable orc `xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc`. Also, orc can only be enabled from filequeue, so above config seems more reasonable. 2) Fixed a typo for zlib compression 3) Test cases to verify the ORC audit logs functionality and also verify the above config consolidation works as expected. The test cases also serves as configuration list requirement to enable orc audit logs ## How was this patch tested? Unit tests added to test the ORC feature including config consolidation proposed Manual testing on cluster also done to verify the functionality ## Additional documentation/configuration for reference to enable ORC audit logs (derived from one of the test cases): ``` xasecure.audit.destination.hdfs.orc.buffersize=100000000000000, xasecure.audit.destination.hdfs.batch.filequeue.filespool.dir=target/spool, xasecure.audit.destination.hdfs.batch.queuetype=filequeue, xasecure.audit.destination.hdfs.batch.filequeue.filespool.buffer.size=100000000000000, xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc, xasecure.audit.is.enabled=true, xasecure.audit.destination.hdfs.filename.format=%app-type%_ranger_audit.orc, xasecure.audit.destination.hdfs=enable, xasecure.audit.destination.hdfs.orc.stripesize=100000000000000, xasecure.audit.destination.hdfs.dir=target/testAuditFileQueueSpoolORC, xasecure.audit.destination.hdfs.orc.compression=snappy, xasecure.audit.destination.hdfs.batch.filequeue.filespool.file.rollover.sec=300 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
