Curl against localhost should be fine without changing anything in the topology. It seems that something is wrong with WebHDFS and Knox can’t connect to it. Can you try to go directly against WebHDFS instead and check if this request works?
Best, Jan From: "McParland, John" <[email protected]> Reply-To: <[email protected]> Date: Tuesday, November 1, 2016 at 10:30 AM To: "[email protected]" <[email protected]> Subject: RE: Deploying/Running Knox in HDP 2.4 Sandbox Hi Jan, thanks for your quick response! In answer to your questions 1) I believe I do have a sandbox topology; [knox@sandbox topologies]$ pwd /usr/hdp/0.10.0-SNAPSHOT/knox-server/conf/topologies [knox@sandbox topologies]$ ls default.xml sandbox.xml 2) From my logs, this appears to be the most pertinent part 2016-11-01 08:26:07,553 WARN hadoop.gateway (DefaultDispatch.java:executeOutboundRequest(146)) - Connection exception dispatching request: http://MYHOST:50070/webhdfs/v1/?op=LISTSTATUS&user.name=guest org.apache.http.conn.ConnectTimeoutException: Connect to MYHOST:50070 [MYHOST/64.4.6.233, MYHOST/65.55.39.12] failed: Connection timed out org.apache.http.conn.ConnectTimeoutException: Connect to MYHOST.com:50070 [MYHOST/64.4.6.233, MYHOST/65.55.39.12] failed: Connection timed out ... Caused by: java.net.ConnectException: Connection timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) ... 89 more I've replaced the fully-qualified-domain-name with MYHOST. Upon further inspection, I do notice that I'm running curl from the host on which Knox/HDP 2.4 is deployed, and using localhost as the hostname, however in my sandbox.xml the URLs use the fully-qualified domain name. Do I need to change everything in the sandbox.xml to use localhost? Thanks, John McParland MIET CEng | System Architect, ODSC Health Local and Scotland | CGI CGI Ltd (UK) Second Floor, Inovo Building, 121 George St, Glasgow, UK, G1 1RD M: +44 7920 183 019 [email protected] | www.cgi-group.co.uk CGI IT UK Limited. A CGI Group Inc. Company Registered Office: 250 Brook Drive, Green Park, Reading RG2 6UA, United Kingdom. Registered in England & Wales - Number 947968 CONFIDENTIALITY NOTICE: Proprietary/Confidential Information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail. From: Jan Hentschel [[email protected]] Sent: 01 November 2016 09:11 To: [email protected] Subject: Re: Deploying/Running Knox in HDP 2.4 Sandbox Hi John, two questions: 1) Do you have a topology called „sandbox“ deployed? You can find the configuration of the topologies in the conf/topologies directory. 2) Can you post the appropriate part of your logs? Best, Jan From: "McParland, John" <[email protected]> Reply-To: <[email protected]> Date: Tuesday, November 1, 2016 at 9:36 AM To: "[email protected]" <[email protected]> Subject: Deploying/Running Knox in HDP 2.4 Sandbox Hi all, I'm attempting to run Knox (built from master) in an Hortonworks Data Platform (HDP) 2.4 Sandbox hosted on Azure. I've followed both the Knox Users Guide [1] and the Hortonworks instructions [2] but when I attempt to curl HDFS via Knox, I get the following when I curl the webhdfs LISTSTATUS command. $ curl -i -k -u guest:guest-password -X GET 'https://localhost:8443/gateway/sandbox/webhdfs/v1/?op=LISTSTATUS' HTTP/1.1 500 Server Error Date: Tue, 01 Nov 2016 08:23:58 GMT Set-Cookie: JSESSIONID=1q3kw9423ahmk4fwy2mvmxy2c;Path=/gateway/sandbox;Secure;HttpOnly Set-Cookie: rememberMe=deleteMe; Path=/gateway/sandbox; Max-Age=0; Expires=Mon, 31-Oct-2016 08:23:59 GMT Content-Type: text/html; charset=ISO-8859-1 Cache-Control: must-revalidate,no-cache,no-store Content-Length: 317 Connection: close Server: Jetty(9.2.15.v20160210) <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> <title>Error 500 Server Error</title> </head> <body><h2>HTTP ERROR 500</h2> <p>Problem accessing /gateway/sandbox/webhdfs/v1/. Reason: <pre> Server Error</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/> </body> </html> Is there additional configuration I need? [1] https://knox.apache.org/books/knox-0-9-1/user-guide.html [2] https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_upgrading_hdp_manually/content/access_subtab_2_3.html Thanks, John McParland MIET CEng | System Architect, ODSC Health Local and Scotland | CGI CGI Ltd (UK) Second Floor, Inovo Building, 121 George St, Glasgow, UK, G1 1RD M: +44 7920 183 019 [email protected] | www.cgi-group.co.uk CGI IT UK Limited. A CGI Group Inc. Company Registered Office: 250 Brook Drive, Green Park, Reading RG2 6UA, United Kingdom. Registered in England & Wales - Number 947968 CONFIDENTIALITY NOTICE: Proprietary/Confidential Information belonging to CGI Group Inc. and its affiliates may be contained in this message. If you are not a recipient indicated or intended in this message (or responsible for delivery of this message to such person), or you think for any reason that this message may have been addressed to you in error, you may not use or copy or deliver this message to anyone else. In such case, you should destroy this message and are asked to notify the sender by reply e-mail.
