Vitaly Brodetskyi created AMBARI-19224:
------------------------------------------
Summary: Perf: Host Checks response is way too big (320MB for 1200
nodes)
Key: AMBARI-19224
URL: https://issues.apache.org/jira/browse/AMBARI-19224
Project: Ambari
Issue Type: Bug
Components: ambari-agent
Affects Versions: 2.5.0
Reporter: Vitaly Brodetskyi
Assignee: Vitaly Brodetskyi
Priority: Critical
Fix For: 2.5.0
Currently, parsing of Host Checks after registration happens on UI, and on 1200
nodes it grows to 320mb of data, which hard to download and then parse. We can
move parsing and aggregating data on the server, so on UI we will receive only
required info.
Request URL:
{noformat}
/api/v1/requests/24?fields=tasks/Tasks,Requests/inputs,Requests/request_status
{noformat}
Part of response:
{code:java}
"tasks" : [
{
"href" : "http://104.196.95.1:8080/api/v1/requests/24/tasks/31414";,
"Tasks" : {
"attempt_cnt" : 1,
"command" : "ACTIONEXECUTE",
"command_detail" : "check_host ACTIONEXECUTE",
"end_time" : 1481047483941,
"error_log" :
"/home/testvbrodetskyi-0001/var/lib/ambari-agent/data/errors-31414.txt",
"exit_code" : 0,
"host_name" : "testvbrodetskyi-0001",
"id" : 31414,
"output_log" :
"/home/testvbrodetskyi-0001/var/lib/ambari-agent/data/output-31414.txt",
"request_id" : 24,
"role" : "check_host",
"stage_id" : 0,
"start_time" : 1481047457478,
"status" : "COMPLETED",
"stderr" : "None",
"stdout" : "2016-12-06 18:04:25,626 - Host checks started.\n2016-12-06
18:04:25,626 - Check execute list: host_resolution_check\n2016-12-06
18:04:25,626 - IP address forward resolution check started.\n2016-12-06
18:04:43,192 - There were 1200 host(s) that could not resolve to an IP
address.\n2016-12-06 18:04:43,193 - IP address forward resolution check
completed.\n2016-12-06 18:04:43,215 - Host checks completed.\n\nCommand
completed successfully!\n",
"structured_out" : {
"host_resolution_check" : {
"exit_code" : 0,
"failed_count" : 1200,
"failures" : [
{
"type" : "FORWARD_LOOKUP",
"host" : "testvbrodetskyi-0601",
"cause" : [
-2,
"Name or service not known"
]
},
{
"type" : "FORWARD_LOOKUP",
"host" : "testvbrodetskyi-0602",
"cause" : [
-2,
"Name or service not known"
]
},
{code}
In case of 1200 for each host we have1200 objects in failures array so in
result we have 1200x1200 objects as:
{code:java}
{
"type" : "FORWARD_LOOKUP",
"host" : "testvbrodetskyi-0602",
"cause" : [
-2,
"Name or service not known"
]
},
{code}
However we on UI we parse only hosts' names, we can reduce response size by
changing failures to just array of host_names like:
"failures" : ["testvbrodetskyi-0602", "testvbrodetskyi-0603",
"testvbrodetskyi-0604", ...]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
