Typically the UI would hang if expected properties are missing, the causing a Javascript error. This would happen more often when enabling Kerberos via the REST API since users a more prone to leaving out required properties than Ambari’s UI.
Looking at your configs, thing look good though. So I am sort of stumped as to why you are having an issue with the Kerberos service config page. Regarding the missing keytabs issue... If you are having issues stopping services on a host, maybe there is an issue with the agent on that host. If this is the case, then Ambari may fail to distribute keytabs to the host since it does this using the normal ambari-server to ambari-agent communication mechanism. Do the hosts with missing keytab files have the a properly installed KERBEROS_CLIENT component? Rob From: Fay Wang <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Fay Wang <[email protected]<mailto:[email protected]>> Date: Wednesday, February 24, 2016 at 1:55 AM To: Robert Levas <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: another Kerberos issue hi Rob, # /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8443 -s get localhost MyCluster krb5-conf USERID=admin PASSWORD=admin PORT=:8443 SSL is enabled ########## Performing 'GET' on (Site:krb5-conf, Tag:version1) "properties" : { "conf_dir" : "/etc", "content" : "[libdefaults]\n renew_lifetime = 7d\nmax_renewable_life = 7d\n forwardable= true\n default_realm = {{realm|upper()}}\n ticket_lifetime = 24h\n dns_lookup_realm = false\n dns_lookup_kdc = false\n #default_tgs_enctypes = {{encryption_types}}\n #default_tkt_enctypes ={{encryption_types}}\n\n{% if domains %}\n[domain_realm]\n{% for domain in domains.split(',') %}\n {{domain}} = {{realm|upper()}}\n{% endfor %}\n{%endif %}\n\n[logging]\n default = FILE:/var/log/krb5kdc.log\nadmin_server = FILE:/var/log/kadmind.log\n kdc = FILE:/var/log/krb5kdc.log\n\n[realms]\n {{realm}} = {\n admin_server = {{admin_server_host|default(kdc_host, True)}}\n kdc = {{kdc_host}}\n }\n\n{# Append additional realm declarations below #}\n", "domains" : "", "manage_krb5_conf" : "true" } Not sure why UI hangs in loading the kerberos page. We double checked the browser and did not see anything special. Actually, to my knowledge, this UI hang issue always happens if kerberos is enabled using REST API. I don't see any loading problem if enabling kerberos via Ambari UI. Also, regarding the missing keytab issue, it seems that the keytab files are missing in the nodes where some of the services could not be stopped. This is just a speculation, I could be wrong... Please let me know if there is anything i should check for this UI hang problem. Many thanks, -f On Tuesday, February 23, 2016 11:58 AM, Robert Levas <[email protected]<mailto:[email protected]>> wrote: Fay… That looks good, but I may be overlooking a property. Can you check to see if krb5-conf is set? /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8443 -s get localhost MyCluster krb5-conf Rob From: Fay Wang <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Fay Wang <[email protected]<mailto:[email protected]>> Date: Tuesday, February 23, 2016 at 1:44 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: another Kerberos issue Hi Rob, Thanks for your prompt attention! Setting the flag -s works fine. Please see below. Please let me know if there is anything wrong in the output which may cause the page loading issue. At the same time, I will check the browser and javascript as you suggest.... /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8443 -s get localhost MyCluster kerberos-env USERID=admin PASSWORD=admin PORT=:8443 SSL is enabled ########## Performing 'GET' on (Site:kerberos-env, Tag:version1) "properties" : { "admin_server_host" : "faywang.com", "case_insensitive_username_rules" : "false", "encryption_types" : "aes des3-cbc-sha1 rc4 des-cbc-md5", "executable_search_paths" : "/usr/bin, /usr/kerberos/bin, /usr/sbin, /usr/lib/mit/bin, /usr/lib/mit/sbin", "install_packages" : "true", "kdc_host" : "faywang.com", "kdc_type" : "mit-kdc", "manage_identities" : "true", "password_length" : "20", "password_min_digits" : "1", "password_min_lowercase_letters" : "1", "password_min_punctuation" : "1", "password_min_uppercase_letters" : "1", "password_min_whitespace" : "0", "realm" : "EXAMPLE.COM", "service_check_principal_name" : "${cluster_name}-${short_date}" } -f On Tuesday, February 23, 2016 10:31 AM, Robert Levas <[email protected]<mailto:[email protected]>> wrote: Hi Fay… For the configs.sh script, -s will enable SSL support and change that http to https. That said, assuming the –s works, what do you get when you do /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8443 -s get localhost MyCluster kerberos–env kdc-type It should be either mit-kdc or active-directory. If not, this may be your issue. You can also check the Javascript console in your web browser to see if there are any interesting errors there, that might give a hit to what is going on. Rob From: Fay Wang <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Fay Wang <[email protected]<mailto:[email protected]>> Date: Tuesday, February 23, 2016 at 1:13 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: another Kerberos issue Hi Rob, I got confusing result: (1) curl -k -H "X-Requested-By:ambari" -u admin:admin -i -X GET https://localhost:8443/api/v1/clusters/MyCluster/configurations?type=kerberos-env HTTP/1.1 200 OK User: admin Set-Cookie: AMBARISESSIONID=2z7yntlt934j1gimc52473i30;Path=/;Secure;HttpOnly Expires: Thu, 01 Jan 1970 00:00:00 GMT Content-Type: text/plain Vary: Accept-Encoding, User-Agent Content-Length: 433 Server: Jetty(8.1.17.v20150415) { "href" : "https://localhost:8443/api/v1/clusters/MyCluster/configurations?type=kerberos-env";, "items" : [ { "href" : "https://localhost:8443/api/v1/clusters/MyCluster/configurations?type=kerberos-env&tag=version1";, "tag" : "version1", "type" : "kerberos-env", "version" : 1, "Config" : { "cluster_name" : "MyCluster", "stack_id" : "BigInsights-4.1" } } ] (2) /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8443 get localhost MyCluster kerberos-env USERID=admin PASSWORD=admin PORT=:8443 [ERROR] "kerberos-env" not found in server response. [ERROR] Output of `curl -s -u admin:admin "http://localhost:8443/api/v1/clusters/MyCluster?fields=Clusters/desired_configs"`<http://localhost:8443/api/v1/clusters/MyCluster?fields=Clusters/desired_configs> is: [ERROR] Please note that in the curl command in (2) is still using http, instead of https. This may explain the error. Please let me know how to set the flag to use https. (3) The result from find / -name kerberos-env.xml: /var/lib/ambari-agent/cache/stacks/.../services/KERBEROS/configuration/kerberos-env.xml /var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml /var/lib/ambari-server/resources/stacks/.../services/KERBEROS/configuration/kerberos-env.xml /var/lib/ambari-server/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml Please advise. -f On Tuesday, February 23, 2016 6:56 AM, Robert Levas <[email protected]<mailto:[email protected]>> wrote: Hi Fay… This can happen if the Kerberos service config data becomes corrupted. If you do the following API call, you do get any data back? GET /api/v1/clusters/MyCluster/configurations?type=kerberos-env In my cluster I don’t have Kerberos installed, so I get the following response: { "href" : "http://c6501:8080/api/v1/clusters/c1/configurations?type=kerberos-env";, "items" : [ ] } If you get this, this is the issue and you will need to recreate the kerberos-env config manually. If you get a set of configuration versions back, then we need to figure out what property of the current version is making the UI upset. A better way to get the configs is to use the configs.sh script on the Ambari server host: /var/lib/ambari-server/resources/scripts/configs.sh -u admin -p admin -port 8080 get localhost c1 kerberos-env If you get the following error, the kerberos-env config was lost somehow: [ERROR] "kerberos-env" not found in server response. Else if configuration data comes back, we can figure out what might be the offending property or properties. Rob From: Fay Wang <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Fay Wang <[email protected]<mailto:[email protected]>> Date: Tuesday, February 23, 2016 at 1:58 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Fay Wang <[email protected]<mailto:[email protected]>> Subject: Re: another Kerberos issue Kerberization is successful. All services are up and service checks pass. Ambari version is 2.0. (1) POST http://localhost:8080/api/v1/clusters/MyCluster/services/KERBEROS (2) POST http://localhost:8080/api/v1/clusters/MyCluster/services/KERBEROS/components/KERBEROS_CLIENT (3) PUT -d @/tmp/kerberos_config.json http://localhost:8080/api/v1/clusters/MyCluster (4) POST -d '{"host_components" : [{"HostRoles" : {"component_name":"KERBEROS_CLIENT"}}]}' http://localhost:8080/api/v1/clusters/MyCluster/hosts?Hosts/host_name=$host (5) PUT -d '{"ServiceInfo": {"state" : "INSTALLED"}}' http://localhost:8080/api/v1/clusters/MyCluster/services/KERBEROS (6) POST -d @/tmp/kerberos_descriptor.json http://localhost:8080/api/v1/clusters/MyCluster/artifacts/kerberos_descriptor (7) PUT -d @/tmp/kerberos_enable.json http://localhost:8080/api/v1/clusters/MyCluster Not sure why the page (Ambari UI -> Admin -> Kerberos) is not loaded. Please help... -f On Monday, February 22, 2016 10:26 PM, Fay Wang <[email protected]<mailto:[email protected]>> wrote: Hi, Another issue with automated kerberos via REST API is that after kerberization, on Ambari UI -> Admin -> Kerberos -> The page does not load. It must be something to do with the value not been set/ or not been retrieved correctly. There is nothing special in ambari-server.log. Any help is highly appreciated.... -fay
