[jira] [Commented] (RANGER-1501) Audit Flush to HDFS does not actually cause the audit logs to be flushed to HDFS

2017-04-08 Thread Yan (JIRA)

[ 
https://issues.apache.org/jira/browse/RANGER-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961924#comment-15961924
 ] 

Yan commented on RANGER-1501:
-

In summary, in a true asynchorous, non-blocking execution path, the performance 
difference for each indivual hflush() vs flush() has little impact; in 
stressed, blocking scenarios, hflush() has some advantages. From semantics 
point of view, hflush() clearly wins.

Finally I believe this jira is a small refinement to Ranger-1310.

> Audit Flush to HDFS does not actually cause the audit logs to be flushed to 
> HDFS 
> -
>
> Key: RANGER-1501
> URL: https://issues.apache.org/jira/browse/RANGER-1501
> Project: Ranger
>  Issue Type: Bug
>  Components: audit
>Affects Versions: 0.7.0
>Reporter: Yan
>Assignee: Yan
> Fix For: master
>
> Attachments: 
> 0001-RANGER-1501-Audit-Flush-to-HDFS-does-not-actually-ca.patch
>
>
> The reason is that HDFS file stream's flush() call does not really flush the 
> data all the way to disk, nor even makes the data visible to HDFS users. See 
> the HDFS semantics of the flush/sync at 
> https://issues.apache.org/jira/browse/HADOOP-6313.
> Consequently the audit logs on HDFS won't be visible/durable from HDFS client 
> until the log file is closed. This will, among other issues, boost chances of 
> losing audit logs in case of system failure.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)


Re: Review Request 56490: RANGER-1369:There is invalid group error when install Solr for Ranger Audits

2017-04-08 Thread Qiang Zhang


> On 四月 7, 2017, 1:42 p.m., Colm O hEigeartaigh wrote:
> > Did you verify it works also with the default (solr) value for the group?

Yes, $SOLR_USER and $SOLR_GROUP can be set to the same value(solr).
The main problem is the following script:
id $SOLR_USER &> /dev/null
if [ $? -ne 0 ]; then
echo "`date`|INFO|Creating user $SOLR_USER"
groupadd $SOLR_USER 2> /dev/null
useradd -g $SOLR_USER $SOLR_USER 2>/dev/null
fi
When $SOLR_USER does not exist, then create group $SOLR_USER and user 
$SOLR_USER whose group is $SOLR_USER.
But if user $SOLR_USER has already exist, then the group $SOLR_USER will not be 
created,
(Such as we create group hadoop instead of creating group solr,
and create user solr whose group is hadoop manually),
so the Linux system would not have the group which name is $SOLR_USER,
it wolud result in invalid group error when executing the following script:
sudo chown -R $SOLR_USER:$SOLR_USER $SOLR_INSTALL_FOLDER
So this patch fixed the above bug.
And it can support the case where the user name is different with the user 
group.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56490/#review171344
---


On 四月 7, 2017, 10:10 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56490/
> ---
> 
> (Updated 四月 7, 2017, 10:10 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Gautam 
> Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-1369
> https://issues.apache.org/jira/browse/RANGER-1369
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> #Description:
> There are some logic defects in the following 
> script(/security-admin/contrib/solr_for_audit_setup/setup.sh):
> sudo chown -R $SOLR_USER:$SOLR_USER $SOLR_INSTALL_FOLDER
> set_ownership $SOLR_USER $SOLR_USER $SOLR_INSTALL_FOLDER
> It will cause invalid group error if the group does not exist. 
> And these logics do not support the case where the user name is different 
> with the user group. 
> We should use SOLR_GROUP to resolve these logic defects. The new code is as 
> following:
> sudo chown -R $SOLR_USER:$SOLR_GROUP $SOLR_INSTALL_FOLDER
> set_ownership $SOLR_USER $SOLR_GROUP $SOLR_INSTALL_FOLDER
> And set a value for SOLR_GROUP alone.
> 
> 
> Diffs
> -
> 
>   security-admin/contrib/solr_for_audit_setup/install.properties fc9955d 
>   security-admin/contrib/solr_for_audit_setup/setup.sh 41a51d8 
> 
> 
> Diff: https://reviews.apache.org/r/56490/diff/2/
> 
> 
> Testing
> ---
> 
> #Test Result:
> 1.Configure install.properties of solr_for_audit_setup:
> SOLR_USER=solr
> SOLR_GROUP=hadoop
> SOLR_INSTALL=false
> SOLR_DEPLOYMENT=standalone
> 2.Execute the script setup.sh of solr_for_audit_setup as root success
> 3.Start Solr and Ranger-Admin(use slor for audit) success
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>