Here is the attached logs from HDFS namenode: at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ... 24 more 2015-08-23 00:13:53,171 ERROR org.apache.ranger.audit.queue.AuditFileSpool: Error sending logs to consumer. provider=hdfs.async.batch, consumer=hdfs.async.batch.solr 2015-08-23 00:13:59,043 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user root 2015-08-23 00:14:23,173 ERROR org.apache.ranger.audit.provider.BaseAuditHandler: Error sending message to Solr org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:570) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:131) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:76) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:62) at org.apache.ranger.audit.destination.SolrAuditDestination.log(SolrAuditDestination.java:132) at org.apache.ranger.audit.provider.BaseAuditHandler.logJSON(BaseAuditHandler.java:161) at org.apache.ranger.audit.queue.AuditFileSpool.sendEvent(AuditFileSpool.java:882) at org.apache.ranger.audit.queue.AuditFileSpool.runDoAs(AuditFileSpool.java:830) at org.apache.ranger.audit.queue.AuditFileSpool$2.run(AuditFileSpool.java:759) at org.apache.ranger.audit.queue.AuditFileSpool$2.run(AuditFileSpool.java:757) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:356) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637) at org.apache.ranger.audit.queue.AuditFileSpool.run(AuditFileSpool.java:765) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://localhost:6083/solr/ranger_audits at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:565) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:214) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:210) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:546) ... 14 more Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:6083 refused at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:464) ... 17 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ... 24 more 2015-08-23 00:14:23,174 ERROR org.apache.ranger.audit.queue.AuditFileSpool: Error sending logs to consumer. provider=hdfs.async.batch, consumer=hdfs.async.batch.solr
On Sat, Aug 22, 2015 at 8:32 PM, larry mccay <[email protected]> wrote: > At this point, you need to determine what is going on with your > namenode/webhdfs. > > Try hitting webhdfs directly instead of going through Knox. > That should lead you to changes for your topology or something. > > The bottom line is that LDAPS is working for you now and you just need to > triage your HDFS related configuration. > > On Sat, Aug 22, 2015 at 10:50 AM, Aneela Saleem <[email protected]> > wrote: > >> Any response Kevin? >> >> Regards, >> Aneela Saleem >> On Aug 22, 2015 2:14 AM, "Aneela Saleem" <[email protected]> wrote: >> >>> Following is the response body: >>> >>> HTTP/1.1 302 Found >>> Location: https://localhost/gateway/cluster1/webhdfs/v1/?op=LISTSTATUS >>> Content-Length: 0 >>> Server: Jetty(8.1.14.v20131031) >>> >>> >>> And no i'm just using single node cluster for testing purpose. My HDFS >>> is not in HA mode. >>> >>> On Sat, Aug 22, 2015 at 1:50 AM, Kevin Minder < >>> [email protected]> wrote: >>> >>>> The last line in the audit log below indicates that the NameNode is >>>> returning a 302 to Knox. The implication here is that your authentication >>>> at Knox to LDAP via LDAPS was successful. >>>> >>>> Is your HDFS in HA mode? >>>> >>>> Can you provide the response body returned by the curl command? >>>> >>>> From: Aneela Saleem >>>> Reply-To: "[email protected]" >>>> Date: Friday, August 21, 2015 at 4:35 PM >>>> To: "[email protected]" >>>> Subject: Re: Working with ldaps ( LDAP over SSL ) >>>> >>>> gateway-audit.log has following content: >>>> >>>> 15/08/22 01:18:20 |||audit|||||redeploy|topology|sandbox|unavailable| >>>> 15/08/22 01:18:20 |||audit|||||redeploy|topology|admin|unavailable| >>>> 15/08/22 01:18:20 |||audit|||||deploy|topology|cluster1|unavailable| >>>> 15/08/22 01:18:29 >>>> ||1202b16f-be70-4cfe-aded-76e5f52dcc8e|audit|WEBHDFS||||access|uri|/gateway/cluster1/webhdfs/v1/?op=LISTSTATUS|unavailable| >>>> 15/08/22 01:18:29 >>>> ||1202b16f-be70-4cfe-aded-76e5f52dcc8e|audit|WEBHDFS||||access|uri|/gateway/cluster1/webhdfs/v1/?op=LISTSTATUS|success|Response >>>> status: 302 >>>> >>>> >>>> and attache is the gateway.log file. I have not made HDFS work with >>>> ldaps yet, im having so much trouble in this. >>>> >>>> >>>> >>>> On Sat, Aug 22, 2015 at 1:26 AM, Kevin Minder < >>>> [email protected]> wrote: >>>> >>>>> Can you provide your gateway.log and gateway-audit.log for a request >>>>> that returns a 302. From your topology file I’m assuming that this is a >>>>> WebHdfs request since the other service definitions seem invalid. Is HDFS >>>>> is safe or standby mode? To my knowledge Knox doesn’t return any 302 >>>>> status codes so I’m assuming this is coming from the NameNode. >>>>> >>>>> From: Aneela Saleem >>>>> Reply-To: "[email protected]" >>>>> Date: Friday, August 21, 2015 at 3:43 PM >>>>> To: "[email protected]" >>>>> Subject: Working with ldaps ( LDAP over SSL ) >>>>> >>>>> Hi all, >>>>> I have setup ldaps and want to make it work with knox. I have changed >>>>> topology file accordingly but when i run cURL command i get http:/1.1 >>>>> 302 Found error. Please look at the topology file and do tell me what >>>>> i'm missing >>>>> >>>> >>>> >>> >
