Hi Don Bosco Durai
I has the same scenario as amitsha but i can see the agent in the
rager console but when i try to put a file in HDFS using the created ranger
authorization permission
in denied for the user and when checked in namenode log it shows
2014-12-17 10:50:15,524 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 54310, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.create from 10.10.10.72:49897
Call#0 Retry#0: org.apache.hadoop.security.AccessControlException:
Permission denied: user=ami, access=WRITE,
inode="/ami.txt":hadoop:supergroup:-rw-r--r--
@Amitsha Pls check xasecure-hdfs-security.xml file at conf/ there the
property "xasecure.hdfs.policymgr.url" which has the url for ranger has
white space in URL, clear that and restart
hadoop Agent will be in Ranger Web console.
*RegardsMuthupandi.K*
Think before you print.
On Wed, Dec 17, 2014 at 2:50 AM, Don Bosco Durai <[email protected]> wrote:
>
> Hi Amithsha
>
> Seems one step was missing in the document. I have updated the Wiki, but
> here it is:
>
> - Create a repository in Ranger Policy Manager. E.g. "local_hdfs". The
> same name needs to be configured during plugin setup
>
>
> Please let me know whether this works?
>
> Thanks
>
> Bosco
>
> On Dec 16, 2014, at 4:06 AM, Amith sha <[email protected]> wrote:
>
> Hi Bosco,
>
> As you mentioned earlier to check the log for HDFS i Found this
>
> 2014-12-16 17:32:53,391 [http-bio-6080-exec-9] ERROR
> com.xasecure.biz.AssetMgr (AssetMgr.java:791) - Requested repository
> not found
> 2014-12-16 17:32:53,391 [http-bio-6080-exec-9] INFO
> com.xasecure.common.RESTErrorUtil (RESTErrorUtil.java:66) - Request
> failed. SessionId=null, loginId=null, logMessage=No Data Found.
> javax.ws.rs.WebApplicationException
> at
> com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:57)
> at
> com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:281)
> at com.xasecure.biz.AssetMgr.getLatestRepoPolicy(AssetMgr.java:792)
> at com.xasecure.rest.AssetREST.getResourceJSON(AssetREST.java:501)
> at
> com.xasecure.rest.AssetREST$$FastClassByCGLIB$$90363ab.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
> at
> org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
> at
> com.xasecure.rest.AssetREST$$EnhancerByCGLIB$$9f2d0d58.getResourceJSON(<generated>)
> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:168)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:70)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:279)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:86)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:74)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1357)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1289)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1239)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1229)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:420)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:497)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:684)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:186)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:744)
> 2014-12-16 17:32:53,392 [http-bio-6080-exec-9] INFO
> com.xasecure.common.RESTErrorUtil (RESTErrorUtil.java:282) - Operation
> error. response=VXResponse={com.xasecure.view.VXResponse@2ba07a78statusCode
> ={1}
> msgDesc={No Data Found.}
> messageList={[VXMessage={com.xasecure.view.VXMessage@34c872a8name
> ={DATA_NOT_FOUND}
> rbKey={xa.error.data_not_found} message={Data not found}
> objectId={null} fieldName={null} }]} }
> javax.ws.rs.WebApplicationException
> at
> com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:57)
> at
> com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:281)
> at com.xasecure.biz.AssetMgr.getLatestRepoPolicy(AssetMgr.java:792)
> at com.xasecure.rest.AssetREST.getResourceJSON(AssetREST.java:501)
> at
> com.xasecure.rest.AssetREST$$FastClassByCGLIB$$90363ab.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
> at
> org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
> at
> com.xasecure.rest.AssetREST$$EnhancerByCGLIB$$9f2d0d58.getResourceJSON(<generated>)
> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:168)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:70)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:279)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:86)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:74)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1357)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1289)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1239)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1229)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:420)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:497)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:684)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:186)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
> at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:744)
>
> On Tue, Dec 16, 2014 at 11:36 AM, Amith sha <[email protected]> wrote:
>
> Hi Bosco,
>
> Thanks for your Kind reply from which i understood the
> ranger role exactly,I have a one more doubt i made a users and
> policies in ranger but how can i check those policies either using
> back end or any 3rd party software
>
> ex:- i created a user called bigdata who is not a unix user in hadoop
> machine but here i set a policies for that user with all
> privileges.now how can i access the HDFS using bigdata user ? same
> thing for Hive
>
> Thanks and Regards
>
> Amithsha S
>
> On Tue, Dec 16, 2014 at 5:05 AM, Don Bosco Durai <[email protected]> wrote:
>
> Hi Amitsha
>
> My answers are embedded...
>
>
> On Dec 15, 2014, at 4:25 AM, Amith sha <[email protected]> wrote:
>
> Hi Bosco,
>
> As per the past instructions.I have installed the Apache Ranger
> successfully.By which i can access the Ranger web interface but i got
> some following Errors,
>
> 1.No Access Audit found!
> I installed Hdfs,Hive,Knox and etc by which
> i should get some Agent files in web Interface of Audit as you
> mentioned in the document ****** You can verify by logging into the
> Ranger Admin Web interface -> Audit -> Agents ****** But i got .No
> Access Audit found! in web Interface.i tried to find out the process
> by where i traced the username and db for the rangeraudit and i
> checked out the db ( ranger_audit) and table ( xa_access_audit ) in
> mysql where there is no records in the table.
>
> Common cause is mismatch in the repository name given in the PolicyAdmin
> and
> install.properties of the plugin.
>
> Let’s pick one component for testing the plugin connection and after
> restart
> of the component, check in the component logs (hivesever2.log or NameNode
> log) and see if you see any exceptions. FYI, the plugin connection logs are
> in x_policy_export_audit table.
>
>
> 2.knox.url and Common Name For Certificate
> Here i have configured knox successfully
> and able to acces the hdfs information using Knox gateway via knox
> Users,But i want to know the exact knox.url ex:- I used the following
> link to access my Hdfs Status
> curl -k -u guest:guest-password
> 'https://127.0.0.1:8443/gateway/knox_sample/webhdfs/v1?op=LISTSTATUS'
>
> Here which is my knox url and i have to
> provide the Certificate name so how can i?
> I provided while creating the repository for Knox
> https://127.0.0.1:8443/gateway/knox_sample as Knox url and so on but
> while testing i got Connection error.
>
> Not sure I understood your question here. Are you able to “telnet 127.0.0.1
> 8443” ?
>
> 3.As a Beginner For Apache knox and Ranger i want to Clarify Some Doubts
> *knox is also a security Agent to provide Security for
> hdfs,hive,hbase etc so why we need ApacheRanger
>
> Different purpose. Knox is service level coarse grain authorization. And
> more importantly, it is API gateway, which provides single URL (hostname)
> for access all the services and authentication mapping (e.g. your Hadoop
> could be Kerberoized, but you can still access it via Knox with LDAP
> authentication). Ranger provides more finer grain access control, central
> administration and centralized auditing.
>
> *In Hortonworks After Configuring Ranger they Checked using knox
>
> Knox is one of the component where you can use Ranger for managing policy
> administration and centralized auditing. So not sure what your question is.
>
> *So Ranger is only to see graphically the users Login and Logs
>
> Ranger does administration, policy enforcement and audit collection. The
> policies can be configured via UI or via REST APIs. So UI is just a tool
> over the core Ranger features.
>
> *Can u provide a examples to run using Ranger as examples
> Available Like sqoop2,hive etc
>
> Few examples are:
> 1. HDFS folder/file permission. Different users and groups can have
> different level of permission.
> 2. In HiveServer2, database, table and column level access control.
> 3. For scoop, you will setup the policies at the DB level. If it is
> HiveCLI,
> then at the HDFS level.
> 4. Centralized auditing of access to data
> 5. Auditing of admin actions.
>
>
>
> Since we are Planing to Secure the Hadoop process we are so interested
> in Ranger In-depth.but unfortunatly there is no examples around the
> search engines.kindly Provide a solution for us
>
> We are working on the documentation and providing more use cases. Let me
> see
> if there are better way in the meanwhile.
>
>
>
> Thank you,
> Amithsha
>
> On Thu, Dec 11, 2014 at 11:24 AM, Amith sha <[email protected]> wrote:
>
> Hi Bosco,
>
> Thanks for your reply, I have checked out the log files Actually i did the
> mistake where file named setup.sh i didnt set the mysql,rangeradmin and
> rangerlogger password.so finally have made a entry in that file and started
> the script have got the access for web console.
>
> Thanks for your guidance and will ping u after completing further
> installation.
>
> On Thu, Dec 11, 2014 at 11:16 AM, Amith sha <[email protected]> wrote:
>
>
> Hi Bosco,
>
>
> On Thu, Dec 11, 2014 at 12:21 AM, Don Bosco Durai <[email protected]>
> wrote:
>
>
> Hi Amith
>
> Seems MySQL is down or not reachable. Can you check the logs in:
>
> Logs are in ews/logs folder. The path is relative to where you have
> installed ranger-admin. Check xa_portal.log and catalina.out files for
> ERROR
> and WARN log messages
>
> I have updated the installation wiki with the above comment (for log
> location).
>
> Thanks
>
> Bosco
>
> On Dec 10, 2014, at 4:09 AM, Amith sha <[email protected]> wrote:
>
> Hi Bosco,
>
> Thanks for your update.So far it is fine to build and got the web
> console. But cannot login the web console using default authentication
> username and password admin,admin. Is there any File to edit or Login
> Information is Required.
>
> Thanks
>
> On Wed, Dec 10, 2014 at 3:23 PM, Amith sha <[email protected]> wrote:
>
>
> Hi bosco,
> Thanks for ur reply.Will check and Ping you soon.
>
> On Wed, Dec 10, 2014 at 1:17 PM, Don Bosco Durai <[email protected]>
> wrote:
>
>
> Hi Amith
>
> I was trying to find from where ranger-script-env.sh was getting
> invoked, but couldn’t.
>
> Below are the instructions to build and run. Happy to get your feedback
> based on this document.
>
>
>
> https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide
>
>
> Thanks
>
> Bosco
>
> On Dec 9, 2014, at 9:38 PM, Amith sha <[email protected]> wrote:
>
> Hi all,
> As advised by Madhan,I was able to build the Ranger
> Successfully.And got the tar.gz files and finally by unzipping
> it.Tried
> to install (ranger-admin)using the shell script setup.sh where it got
> some inputs and finally it shows *Installation of XASecure
> PolicyManager Web Application is completed.*
>
> But i cant access the service in the port 6080 have alse checked
> whether
> any service is running on that port
>
> finally goggled and got this file location
> incubator-ranger-master/
> embededwebserver/scripts
> Below files are found
> logs ranger-admin startcopy
> start-ranger-admin.sh stop-ranger-admin.sh
>
> tried ./start-ranger-admin.sh
> This script trying to find a file ranger-script-env.sh
> But it cannot found
>
> Can anyone help or suggest !!!!
> Is that possible to work before the new release .
> Thank u
>
>
>
>
>
>
>
>
>
>
