[
https://issues.apache.org/jira/browse/AMBARI-19224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitaly Brodetskyi updated AMBARI-19224:
---------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk and branch-2.5
> Perf: Host Checks response is way too big (320MB for 1200 nodes)
> ----------------------------------------------------------------
>
> Key: AMBARI-19224
> URL: https://issues.apache.org/jira/browse/AMBARI-19224
> Project: Ambari
> Issue Type: Bug
> Components: ambari-agent
> Affects Versions: 2.5.0
> Reporter: Vitaly Brodetskyi
> Assignee: Vitaly Brodetskyi
> Priority: Critical
> Fix For: 2.5.0
>
> Attachments: AMBARI-19224.patch
>
>
> Currently, parsing of Host Checks after registration happens on UI, and on
> 1200 nodes it grows to 320mb of data, which hard to download and then parse.
> We can move parsing and aggregating data on the server, so on UI we will
> receive only required info.
> Request URL:
> {noformat}
>
> /api/v1/requests/24?fields=tasks/Tasks,Requests/inputs,Requests/request_status
> {noformat}
> Part of response:
> {code:java}
> "tasks" : [
> {
> "href" : "http://104.196.95.1:8080/api/v1/requests/24/tasks/31414";,
> "Tasks" : {
> "attempt_cnt" : 1,
> "command" : "ACTIONEXECUTE",
> "command_detail" : "check_host ACTIONEXECUTE",
> "end_time" : 1481047483941,
> "error_log" :
> "/home/testvbrodetskyi-0001/var/lib/ambari-agent/data/errors-31414.txt",
> "exit_code" : 0,
> "host_name" : "testvbrodetskyi-0001",
> "id" : 31414,
> "output_log" :
> "/home/testvbrodetskyi-0001/var/lib/ambari-agent/data/output-31414.txt",
> "request_id" : 24,
> "role" : "check_host",
> "stage_id" : 0,
> "start_time" : 1481047457478,
> "status" : "COMPLETED",
> "stderr" : "None",
> "stdout" : "2016-12-06 18:04:25,626 - Host checks
> started.\n2016-12-06 18:04:25,626 - Check execute list:
> host_resolution_check\n2016-12-06 18:04:25,626 - IP address forward
> resolution check started.\n2016-12-06 18:04:43,192 - There were 1200 host(s)
> that could not resolve to an IP address.\n2016-12-06 18:04:43,193 - IP
> address forward resolution check completed.\n2016-12-06 18:04:43,215 - Host
> checks completed.\n\nCommand completed successfully!\n",
> "structured_out" : {
> "host_resolution_check" : {
> "exit_code" : 0,
> "failed_count" : 1200,
> "failures" : [
> {
> "type" : "FORWARD_LOOKUP",
> "host" : "testvbrodetskyi-0601",
> "cause" : [
> -2,
> "Name or service not known"
> ]
> },
> {
> "type" : "FORWARD_LOOKUP",
> "host" : "testvbrodetskyi-0602",
> "cause" : [
> -2,
> "Name or service not known"
> ]
> },
> {code}
> In case of 1200 for each host we have1200 objects in failures array so in
> result we have 1200x1200 objects as:
> {code:java}
> {
> "type" : "FORWARD_LOOKUP",
> "host" : "testvbrodetskyi-0602",
> "cause" : [
> -2,
> "Name or service not known"
> ]
> },
> {code}
> However we on UI we parse only hosts' names, we can reduce response size by
> changing failures to just array of host_names like:
> "failures" : ["testvbrodetskyi-0602", "testvbrodetskyi-0603",
> "testvbrodetskyi-0604", ...]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
