You can try to workaround this by replacing the dep/httpclient-4.5.2.jar jar with the 4.5.1 version. 4.5.1 did not include the change that broke 4.5.2. Hopefully, there is no use of any incompatible changes in our dispatch classes.
Please let me know if you can get by this error with this workaround. On Fri, Oct 21, 2016 at 8:38 AM, larry mccay <[email protected]> wrote: > From what I can see, this problem is directly related to: > https://issues.apache.org/jira/browse/HTTPCLIENT-1712. > > I have asked them to provide a release that removes this incorrect patch > but we will likely have to deal with it in Knox - if at all possible. > I will look into overriding GGSSchemeBase in Knox and figure out how to > use the extension or forked version as a downloadable patch. > > Sorry for the inconvenience! > > On Fri, Oct 21, 2016 at 7:48 AM, larry mccay <[email protected]> wrote: > >> Hi Benjamin - >> >> I suspect, based on the error message, that you are right. >> The HTTP service name in the SPN is incorrectly set as HTTPS. >> >> Not sure why this would be. >> I will look into our kerberos dispatch code and see if we are explicitly >> setting this for some reason. >> We should be just letting HttpClient do it for us but I will check. >> >> thanks, >> >> --larry >> >> On Fri, Oct 21, 2016 at 4:59 AM, Ruland, Benjamin < >> [email protected]> wrote: >> >>> Hi everyone, >>> >>> >>> >>> I am experiencing problems with Knox while using WebHDFS in a cluster >>> with Kerberos and SSL. >>> >>> The KDC is an Microsoft AD 2012. Kerberos-Encryption is set to AES256. >>> Knox is connected to AD via LDAP sync (this is working fine for other Knox >>> services). >>> >>> I am running HDP 2.5 with Knox 0.9.0 >>> >>> >>> >>> In general, the cluster runs fine. WebHDFS using SPNEGO is working. >>> >>> >>> >>> But when accessing WebHDFS over Knox, I get an 401 error and some >>> strange logs. >>> >>> I suspect that Knox is trying to get a ticket for a HTTPS/namenode@REALM >>> principal, which does not exist. Although running SSL, all principals for >>> SPNEGO are HTTP/... >>> >>> >>> >>> I this a Knox Bug or is this a misconfiguration at some point? >>> >>> >>> >>> It would be great, if someone has advice. >>> >>> >>> >>> Best regards, >>> >>> Benjamin >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> The used command is: >>> >>> >>> >>> [root@utilitynode ~]# curl -ik -u validuser " >>> https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS"; >>> >>> Enter host password for user 'validuser': >>> >>> HTTP/1.1 401 Unauthorized >>> >>> Date: Wed, 12 Oct 2016 07:47:41 GMT >>> >>> Set-Cookie: rememberMe=deleteMe; Path=/gateway/default; Max-Age=0; >>> Expires=Tue,11-Oct-2016 07:47:41 GMT >>> >>> WWW-Authenticate: BASIC realm="application" >>> >>> Content-Length: 0 >>> >>> Server: Jetty(9.2.15.v20160210) >>> >>> >>> >>> >>> >>> Debug Log in knox gateway.log >>> >>> >>> >>> 2016-10-12 09:51:49,735 DEBUG hadoop.gateway >>> (GatewayFilter.java:doFilter(116)) - Received request: GET /webhdfs/v1/ >>> >>> 2016-10-12 09:51:49,740 DEBUG hadoop.gateway >>> (KnoxLdapRealm.java:getUserDn(673)) - Searching from >>> OU=someOU,DC=somedomain,DC=de where >>> (&(objectclass=person)(sAMAccountName=validuser)) >>> scope subtree >>> >>> 2016-10-12 09:51:49,745 INFO hadoop.gateway >>> (KnoxLdapRealm.java:getUserDn(679)) - Computed userDn: >>> CN=validuser,OU=Users,OU=someOU,DC=somedomain,DC=de using ldapSearch >>> for principal: validuser >>> >>> 2016-10-12 09:51:49,749 DEBUG hadoop.gateway >>> (UrlRewriteProcessor.java:rewrite(166)) - Rewrote URL: >>> https://utilitynode:8443/gateway/default/webhdfs/v1/?OP=LISTSTATUS, >>> direction: IN via explicit rule: WEBHDFS/webhdfs/inbound/namenode/root >>> to URL: https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTS >>> TATUS >>> >>> 2016-10-12 09:51:49,749 DEBUG hadoop.gateway >>> (DefaultDispatch.java:executeOutboundRequest(120)) - Dispatch request: >>> GET https://utilitynode.somedomain.de:50470/webhdfs/v1/?OP=LISTS >>> TATUS&doAs=validuser >>> >>> 2016-10-12 09:51:49,781 WARN auth.HttpAuthenticator >>> (HttpAuthenticator.java:generateAuthResponse(207)) - NEGOTIATE >>> authentication error: No valid credentials provided (Mechanism level: No >>> valid credentials provided (Mechanism level: Server not found in Kerberos >>> database (7))) >>> >>> 2016-10-12 09:51:49,782 DEBUG hadoop.gateway >>> (DefaultDispatch.java:executeOutboundRequest(133)) - Dispatch response >>> status: 401 >>> >>> 2016-10-12 09:51:49,783 DEBUG hadoop.gateway >>> (DefaultDispatch.java:getInboundResponseContentType(202)) - Using >>> explicit character set ISO-8859-1 for entity of type text/html >>> >>> 2016-10-12 09:51:49,783 DEBUG hadoop.gateway >>> (DefaultDispatch.java:getInboundResponseContentType(210)) - Inbound >>> response entity content type: text/html; charset=iso-8859-1 >>> >>> >>> >>> >>> >>> Log in knox gateway.out >>> >>> >>> >>> Found ticket for knox/[email protected] to go to >>> krbtgt/[email protected] expiring on Wed Oct 12 19:53:51 CEST >>> 2016 >>> >>> Entered Krb5Context.initSecContext with state=STATE_NEW >>> >>> Service ticket not found in the subject >>> >>> >>> Credentials acquireServiceCreds: same realm >>> >>> default etypes for default_tgs_enctypes: 18. >>> >>> >>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType >>> >>> >>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType >>> >>> getKDCFromDNS using UDP >>> >>> >>> KrbKdcReq send: kdc=domaincontroller.somedomain.de. TCP:88, >>> timeout=30000, number of retries =3, #bytes=1661 >>> >>> >>> KDCCommunication: kdc=domaincontroller.somedomain.de. TCP:88, >>> timeout=30000,Attempt =1, #bytes=1661 >>> >>> >>>DEBUG: TCPClient reading 127 bytes >>> >>> >>> KrbKdcReq send: #bytes read=127 >>> >>> >>> KdcAccessibility: remove domaincontroller.somedomain.de.:88 >>> >>> >>> KDCRep: init() encoding tag is 126 req type is 13 >>> >>> >>>KRBError: >>> >>> sTime is Wed Oct 12 09:53:51 CEST 2016 1476258831000 >>> >>> suSec is 8354 suSec is 8354 >>> >>> error code is 7 >>> >>> error Message is Server not found in Kerberos database >>> >>> sname is HTTPS/[email protected] >>> >>> msgType is 30 >>> >>> >>> >>> >>> >>> Extracts from topology config: >>> >>> >>> >>> <topology> >>> >>> >>> >>> <gateway> >>> >>> >>> >>> <provider> >>> >>> <role>authentication</role> >>> >>> <name>ShiroProvider</name> >>> >>> <enabled>true</enabled> >>> >>> >>> >>> <!-- LDAP Sync properties sit here --> >>> >>> >>> >>> <provider> >>> >>> <role>identity-assertion</role> >>> >>> <name>Default</name> >>> >>> <enabled>true</enabled> >>> >>> </provider> >>> >>> >>> >>> <provider> >>> >>> <role>authorization</role> >>> >>> <name>XASecurePDPKnox</name> >>> >>> <enabled>true</enabled> >>> >>> </provider> >>> >>> >>> >>> <provider> >>> >>> <role>ha</role> >>> >>> <name>HaProvider</name> >>> >>> <enabled>true</enabled> >>> >>> <param> >>> >>> <name>WEBHDFS</name> >>> >>> <value>maxFailoverAttempts=3;failoverSleep=1000;maxRe >>> tryAttempts=300;retrySleep=1000;enabled=true</value> >>> >>> </param> >>> >>> </provider> >>> >>> >>> >>> </gateway> >>> >>> >>> >>> <service> >>> >>> <role>NAMENODE</role> >>> >>> <url>hdfs://namenode.somedomain.de:8020</url> >>> >>> <url>hdfs://namenode2.somedomain.de:8020</url> >>> >>> </service> >>> >>> >>> >>> <service> >>> >>> <role>WEBHDFS</role> >>> >>> <url>https://namenode.somedomain.de:50470/webhdfs</url> >>> >>> <url>https://namenode2.somedomain.de:50470/webhdfs</url> >>> >>> </service> >>> >>> >>> >>> </topology> >>> >>> >>> ----------------------------------- >>> Computacenter AG & Co. oHG, mit Sitz in Kerpen >>> (Amtsgericht Köln HRA 18096) >>> Vertretungsberechtigte Gesellschafter: >>> Computacenter Aktiengesellschaft, mit Sitz in Köln (Amtsgericht Köln HRB >>> 28384) >>> Vorstand: Tony Conophy >>> Aufsichtsrat: Michael Norris (Vorsitzender) >>> Computacenter Management GmbH, mit Sitz in Köln (Amtsgericht Köln HRB 28284) >>> Geschäftsführer: Dr. Karsten Freihube, Dr. Thomas Kottmann, Reiner Louis, >>> Thomas Jescheck >>> Visit us on the Internet: http://www.computacenter.de >>> Visit our Online-Shop: https://shop.computacenter.de >>> >>> This email is confidential. If you are not the intended recipient, you must >>> not disclose or use the information contained in it. If you have received >>> this mail in error, please tell us immediately by return email and delete >>> the document. >>> ----------------------------------- >>> >>> >> >
