[jira] [Commented] (RANGER-4384) ORC audit for hdfs destination is too slow
[ https://issues.apache.org/jira/browse/RANGER-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760232#comment-17760232 ] Bhavik Patel commented on RANGER-4384: -- we don't have any such script, we are actually running the hbase jobs. > ORC audit for hdfs destination is too slow > -- > > Key: RANGER-4384 > URL: https://issues.apache.org/jira/browse/RANGER-4384 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.3.0, 2.4.0 >Reporter: Bhavik Patel >Priority: Major > > ORC audit for hdfs destination is too slow when we generate 60GB audit in a > day and due to slowness it started writing to spool directory. Even audits > are not able flush from spool directories to HDFS due to slowness. > Below Configurations are set on the test cluster: > orc.buffersize=10mb > orc.stripesize=10mb > filespool.buffer.size=10k > file.rollover.sec=1day > xasecure.audit.provider.queue.size=1mb default > > CC: [~rmani] [~fateh288] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-4384) ORC audit for hdfs destination is too slow
[ https://issues.apache.org/jira/browse/RANGER-4384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760140#comment-17760140 ] Fateh Singh commented on RANGER-4384: - [~bpatel] Can you share any script you have to generate large amount of audit data > ORC audit for hdfs destination is too slow > -- > > Key: RANGER-4384 > URL: https://issues.apache.org/jira/browse/RANGER-4384 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.3.0, 2.4.0 >Reporter: Bhavik Patel >Priority: Major > > ORC audit for hdfs destination is too slow when we generate 60GB audit in a > day and due to slowness it started writing to spool directory. Even audits > are not able flush from spool directories to HDFS due to slowness. > Below Configurations are set on the test cluster: > orc.buffersize=10mb > orc.stripesize=10mb > filespool.buffer.size=10k > file.rollover.sec=1day > xasecure.audit.provider.queue.size=1mb default > > CC: [~rmani] [~fateh288] -- This message was sent by Atlassian Jira (v8.20.10#820010)
