Steps i followed *unzip the ranger-0.4.0-hdfs *ln -s /opt/ranger/ranger-0.4.0-hdfs /opt/ranger/ranger-hdfs *ln -s /opt/hadoop/etc/hadoop /opt/ranger/hadoop/conf *cp /opt/hadoop/lib/* /opt/hadoop/share/hadoop/hdfs/lib/ *./enable-hdfs-plugin.sh *vi xasecure-audit.xml *restart the hdfs
Thank you On Tue, Dec 16, 2014 at 5:36 PM, Amith sha <[email protected]> wrote: > Hi Bosco, > > As you mentioned earlier to check the log for HDFS i Found this > > 2014-12-16 17:32:53,391 [http-bio-6080-exec-9] ERROR > com.xasecure.biz.AssetMgr (AssetMgr.java:791) - Requested repository > not found > 2014-12-16 17:32:53,391 [http-bio-6080-exec-9] INFO > com.xasecure.common.RESTErrorUtil (RESTErrorUtil.java:66) - Request > failed. SessionId=null, loginId=null, logMessage=No Data Found. > javax.ws.rs.WebApplicationException > at > com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:57) > at > com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:281) > at com.xasecure.biz.AssetMgr.getLatestRepoPolicy(AssetMgr.java:792) > at com.xasecure.rest.AssetREST.getResourceJSON(AssetREST.java:501) > at > com.xasecure.rest.AssetREST$$FastClassByCGLIB$$90363ab.invoke(<generated>) > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191) > at > org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622) > at > com.xasecure.rest.AssetREST$$EnhancerByCGLIB$$9f2d0d58.getResourceJSON(<generated>) > at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:168) > at > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:70) > at > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:279) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136) > at > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:86) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136) > at > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:74) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1357) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1289) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1239) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1229) > at > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:420) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:497) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:684) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:186) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:744) > 2014-12-16 17:32:53,392 [http-bio-6080-exec-9] INFO > com.xasecure.common.RESTErrorUtil (RESTErrorUtil.java:282) - Operation > error. > response=VXResponse={com.xasecure.view.VXResponse@2ba07a78statusCode={1} > msgDesc={No Data Found.} > messageList={[VXMessage={com.xasecure.view.VXMessage@34c872a8name={DATA_NOT_FOUND} > rbKey={xa.error.data_not_found} message={Data not found} > objectId={null} fieldName={null} }]} } > javax.ws.rs.WebApplicationException > at > com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:57) > at > com.xasecure.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:281) > at com.xasecure.biz.AssetMgr.getLatestRepoPolicy(AssetMgr.java:792) > at com.xasecure.rest.AssetREST.getResourceJSON(AssetREST.java:501) > at > com.xasecure.rest.AssetREST$$FastClassByCGLIB$$90363ab.invoke(<generated>) > at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191) > at > org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622) > at > com.xasecure.rest.AssetREST$$EnhancerByCGLIB$$9f2d0d58.getResourceJSON(<generated>) > at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:168) > at > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:70) > at > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:279) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136) > at > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:86) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:136) > at > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:74) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1357) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1289) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1239) > at > com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1229) > at > com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:420) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:497) > at > com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:684) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:186) > at > org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) > at > org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) > at > org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) > at > org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) > at > org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:744) > > On Tue, Dec 16, 2014 at 11:36 AM, Amith sha <[email protected]> wrote: >> Hi Bosco, >> >> Thanks for your Kind reply from which i understood the >> ranger role exactly,I have a one more doubt i made a users and >> policies in ranger but how can i check those policies either using >> back end or any 3rd party software >> >> ex:- i created a user called bigdata who is not a unix user in hadoop >> machine but here i set a policies for that user with all >> privileges.now how can i access the HDFS using bigdata user ? same >> thing for Hive >> >> Thanks and Regards >> >> Amithsha S >> >> On Tue, Dec 16, 2014 at 5:05 AM, Don Bosco Durai <[email protected]> wrote: >>> Hi Amitsha >>> >>> My answers are embedded... >>> >>> >>> On Dec 15, 2014, at 4:25 AM, Amith sha <[email protected]> wrote: >>> >>> Hi Bosco, >>> >>> As per the past instructions.I have installed the Apache Ranger >>> successfully.By which i can access the Ranger web interface but i got >>> some following Errors, >>> >>> 1.No Access Audit found! >>> I installed Hdfs,Hive,Knox and etc by which >>> i should get some Agent files in web Interface of Audit as you >>> mentioned in the document ****** You can verify by logging into the >>> Ranger Admin Web interface -> Audit -> Agents ****** But i got .No >>> Access Audit found! in web Interface.i tried to find out the process >>> by where i traced the username and db for the rangeraudit and i >>> checked out the db ( ranger_audit) and table ( xa_access_audit ) in >>> mysql where there is no records in the table. >>> >>> Common cause is mismatch in the repository name given in the PolicyAdmin and >>> install.properties of the plugin. >>> >>> Let’s pick one component for testing the plugin connection and after restart >>> of the component, check in the component logs (hivesever2.log or NameNode >>> log) and see if you see any exceptions. FYI, the plugin connection logs are >>> in x_policy_export_audit table. >>> >>> >>> 2.knox.url and Common Name For Certificate >>> Here i have configured knox successfully >>> and able to acces the hdfs information using Knox gateway via knox >>> Users,But i want to know the exact knox.url ex:- I used the following >>> link to access my Hdfs Status >>> curl -k -u guest:guest-password >>> 'https://127.0.0.1:8443/gateway/knox_sample/webhdfs/v1?op=LISTSTATUS' >>> >>> Here which is my knox url and i have to >>> provide the Certificate name so how can i? >>> I provided while creating the repository for Knox >>> https://127.0.0.1:8443/gateway/knox_sample as Knox url and so on but >>> while testing i got Connection error. >>> >>> Not sure I understood your question here. Are you able to “telnet 127.0.0.1 >>> 8443” ? >>> >>> 3.As a Beginner For Apache knox and Ranger i want to Clarify Some Doubts >>> *knox is also a security Agent to provide Security for >>> hdfs,hive,hbase etc so why we need ApacheRanger >>> >>> Different purpose. Knox is service level coarse grain authorization. And >>> more importantly, it is API gateway, which provides single URL (hostname) >>> for access all the services and authentication mapping (e.g. your Hadoop >>> could be Kerberoized, but you can still access it via Knox with LDAP >>> authentication). Ranger provides more finer grain access control, central >>> administration and centralized auditing. >>> >>> *In Hortonworks After Configuring Ranger they Checked using knox >>> >>> Knox is one of the component where you can use Ranger for managing policy >>> administration and centralized auditing. So not sure what your question is. >>> >>> *So Ranger is only to see graphically the users Login and Logs >>> >>> Ranger does administration, policy enforcement and audit collection. The >>> policies can be configured via UI or via REST APIs. So UI is just a tool >>> over the core Ranger features. >>> >>> *Can u provide a examples to run using Ranger as examples >>> Available Like sqoop2,hive etc >>> >>> Few examples are: >>> 1. HDFS folder/file permission. Different users and groups can have >>> different level of permission. >>> 2. In HiveServer2, database, table and column level access control. >>> 3. For scoop, you will setup the policies at the DB level. If it is HiveCLI, >>> then at the HDFS level. >>> 4. Centralized auditing of access to data >>> 5. Auditing of admin actions. >>> >>> >>> >>> Since we are Planing to Secure the Hadoop process we are so interested >>> in Ranger In-depth.but unfortunatly there is no examples around the >>> search engines.kindly Provide a solution for us >>> >>> We are working on the documentation and providing more use cases. Let me see >>> if there are better way in the meanwhile. >>> >>> >>> >>> Thank you, >>> Amithsha >>> >>> On Thu, Dec 11, 2014 at 11:24 AM, Amith sha <[email protected]> wrote: >>> >>> Hi Bosco, >>> >>> Thanks for your reply, I have checked out the log files Actually i did the >>> mistake where file named setup.sh i didnt set the mysql,rangeradmin and >>> rangerlogger password.so finally have made a entry in that file and started >>> the script have got the access for web console. >>> >>> Thanks for your guidance and will ping u after completing further >>> installation. >>> >>> On Thu, Dec 11, 2014 at 11:16 AM, Amith sha <[email protected]> wrote: >>> >>> >>> Hi Bosco, >>> >>> >>> On Thu, Dec 11, 2014 at 12:21 AM, Don Bosco Durai <[email protected]> >>> wrote: >>> >>> >>> Hi Amith >>> >>> Seems MySQL is down or not reachable. Can you check the logs in: >>> >>> Logs are in ews/logs folder. The path is relative to where you have >>> installed ranger-admin. Check xa_portal.log and catalina.out files for ERROR >>> and WARN log messages >>> >>> I have updated the installation wiki with the above comment (for log >>> location). >>> >>> Thanks >>> >>> Bosco >>> >>> On Dec 10, 2014, at 4:09 AM, Amith sha <[email protected]> wrote: >>> >>> Hi Bosco, >>> >>> Thanks for your update.So far it is fine to build and got the web >>> console. But cannot login the web console using default authentication >>> username and password admin,admin. Is there any File to edit or Login >>> Information is Required. >>> >>> Thanks >>> >>> On Wed, Dec 10, 2014 at 3:23 PM, Amith sha <[email protected]> wrote: >>> >>> >>> Hi bosco, >>> Thanks for ur reply.Will check and Ping you soon. >>> >>> On Wed, Dec 10, 2014 at 1:17 PM, Don Bosco Durai <[email protected]> >>> wrote: >>> >>> >>> Hi Amith >>> >>> I was trying to find from where ranger-script-env.sh was getting >>> invoked, but couldn’t. >>> >>> Below are the instructions to build and run. Happy to get your feedback >>> based on this document. >>> >>> >>> https://cwiki.apache.org/confluence/display/RANGER/Ranger+Installation+Guide >>> >>> >>> Thanks >>> >>> Bosco >>> >>> On Dec 9, 2014, at 9:38 PM, Amith sha <[email protected]> wrote: >>> >>> Hi all, >>> As advised by Madhan,I was able to build the Ranger >>> Successfully.And got the tar.gz files and finally by unzipping >>> it.Tried >>> to install (ranger-admin)using the shell script setup.sh where it got >>> some inputs and finally it shows *Installation of XASecure >>> PolicyManager Web Application is completed.* >>> >>> But i cant access the service in the port 6080 have alse checked >>> whether >>> any service is running on that port >>> >>> finally goggled and got this file location >>> incubator-ranger-master/ >>> embededwebserver/scripts >>> Below files are found >>> logs ranger-admin startcopy >>> start-ranger-admin.sh stop-ranger-admin.sh >>> >>> tried ./start-ranger-admin.sh >>> This script trying to find a file ranger-script-env.sh >>> But it cannot found >>> >>> Can anyone help or suggest !!!! >>> Is that possible to work before the new release . >>> Thank u >>> >>> >>> >>> >>> >>> >>> >>>
