>
> what kind of load do you have? Puppet? Facter? Is that ENC? Something
> else?
>
> Can you tell which requests are slow from logs or monitoring?
>
>
Yes, I should have mentioned that - there's very little puppet and ENC work
done by this cluster at this point (more is coming soon though). Host
discovery is by far the largest workload - 7600 discovered systems at this
point. The last spike that I saw the impact to overall flows was when
300-400 systems were trying to register at the same time. Because of the
deadlocks, about 200-300 systems could not register repeatedly and had to
keep retrying for a rather long time.
Rather often these registration attempts would end up creating either
duplicate entries with the same "mac<mac>" but different IDs in a DB or an
"empty" discovery host entry. Both of these would prevent a system
successfully register unless they are removed (I had to write a little
script that runs from the cron to do so). Here are the examples of an
"empty" record and duplicate ones (as they get deleted):
{
"id": 437923,
"name": "mac90e2bae6cc70",
"last_compile": null,
"last_report": null,
"updated_at": "2017-08-22T07:08:54.000Z",
"created_at": "2017-08-22T07:08:54.000Z",
"root_pass": "<removed>",
"architecture_id": null,
"operatingsystem_id": null,
"environment_id": null,
"ptable_id": null,
"medium_id": null,
"build": false,
"comment": null,
"disk": null,
"installed_at": null,
"model_id": null,
"hostgroup_id": null,
"owner_id": null,
"owner_type": null,
"enabled": true,
"puppet_ca_proxy_id": null,
"managed": false,
"use_image": null,
"image_file": null,
"uuid": null,
"compute_resource_id": null,
"puppet_proxy_id": null,
"certname": null,
"image_id": null,
"organization_id": null,
"location_id": null,
"otp": null,
"realm_id": null,
"compute_profile_id": null,
"provision_method": null,
"grub_pass": "",
"global_status": 0,
"lookup_value_matcher": null,
"discovery_rule_id": null,
"salt_proxy_id": null,
"salt_environment_id": null,
"pxe_loader": null
}
Duplicates (usually the later duplicate would be an empty one as well, but
not all the time):
{
"id": 430090,
"name": "mac3417ebe3f8f1",
"last_compile": null,
"last_report": "2017-09-14T19:47:55.000Z",
"updated_at": "2017-09-14T19:47:57.000Z",
"created_at": "2017-03-08T20:24:05.000Z",
"root_pass": "<removed>",
"architecture_id": null,
"operatingsystem_id": null,
"environment_id": null,
"ptable_id": null,
"medium_id": null,
"build": false,
"comment": null,
"disk": null,
"installed_at": null,
"model_id": 3,
"hostgroup_id": null,
"owner_id": 10,
"owner_type": "User",
"enabled": true,
"puppet_ca_proxy_id": null,
"managed": false,
"use_image": null,
"image_file": null,
"uuid": null,
"compute_resource_id": null,
"puppet_proxy_id": null,
"certname": null,
"image_id": null,
"organization_id": null,
"location_id": null,
"otp": null,
"realm_id": null,
"compute_profile_id": null,
"provision_method": null,
"grub_pass": "",
"global_status": 0,
"lookup_value_matcher": null,
"discovery_rule_id": null,
"salt_proxy_id": null,
"salt_environment_id": null,
"pxe_loader": null
}
{
"id": 438146,
"name": "mac3417ebe3f8f1",
"last_compile": null,
"last_report": "2017-09-11T08:58:05.000Z",
"updated_at": "2017-09-11T08:58:07.000Z",
"created_at": "2017-08-24T19:44:23.000Z",
"root_pass": "<removed>",
"architecture_id": null,
"operatingsystem_id": null,
"environment_id": null,
"ptable_id": null,
"medium_id": null,
"build": false,
"comment": null,
"disk": null,
"installed_at": null,
"model_id": null,
"hostgroup_id": null,
"owner_id": null,
"owner_type": null,
"enabled": true,
"puppet_ca_proxy_id": null,
"managed": false,
"use_image": null,
"image_file": null,
"uuid": null,
"compute_resource_id": null,
"puppet_proxy_id": null,
"certname": null,
"image_id": null,
"organization_id": null,
"location_id": null,
"otp": null,
"realm_id": null,
"compute_profile_id": null,
"provision_method": null,
"grub_pass": "",
"global_status": 0,
"lookup_value_matcher": null,
"discovery_rule_id": null,
"salt_proxy_id": null,
"salt_environment_id": null,
"pxe_loader": null
}
I can't tell if any queries are slow - can you remind me how to do that?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.
