Okay, now we're getting somewhere. This is an environment with five oVirt nodes in two clusters. Cluster1: 2 nodes Cluster2: 3 nodes
Each have a handful of VMs, some manually installed, and some provisioned by Foreman spanning both clusters. The oVirt nodes each appear to have maybe 10-50 nics (mostly VLAN interfaces and bridges for VMs). When I run the puppet agent on any of the three nodes in Cluster2, the Nic::Managed count shoots through the roof. Even if I kill the puppet agent, the count continues to rise. Here's a list of the NICs on one of the offending hosts: http://pastebin.com/DxZup68B Honestly, the NIC information for these hosts aren't very useful. As a temporary workaround, is there a way to exclude gathering NIC information during this process? Thanks! Justin On Tuesday, November 1, 2016 at 8:56:50 AM UTC-4, Lukas Zapletal wrote: > > Ok one minute is fine, the counters will reset in 5 minutes anyway. > > Ok, the problem is in setup_clone / setup_object_clone method which > creates a deep copy of each record for comparison. But I wonder how is > possible you have 20k calls of this clone after just 1 minute. > > Tell me more about your infrastructure. How many hosts? What is the > everage count of NICs associated with a host? Don't you have some kind > of broken host with 20k NICs associated? Remember, Puppet fact upload > will cause creation of NIC record for each NIC reported, so you could > have some broken host reporting "ethXYZ_address" each puppet run > causing the NIC table to grow. > > Also, can you tell the 100% CPU utilization is by Ruby process itself, > or is that caused by swapping? If this is really the Ruby process > doing some work, please also run > > foreman-tracer rails calls > > For a minute or two to see where is it looping in. Then pastebin again, > thanks. > > LZ > > > On Tue, Nov 1, 2016 at 12:47 PM, Justin Foreman <[email protected] > <javascript:>> wrote: > > I wasn't sure how long to run them, so I ran each for 60 seconds. > > > > foreman-tracer rails objects-total > > http://pastebin.com/QdZePcWQ > > > > foreman-tracer rails objects > > http://pastebin.com/jVbDRm3c > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to [email protected] <javascript:>. > > To post to this group, send email to [email protected] > <javascript:>. > > Visit this group at https://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > Later, > Lukas @lzap Zapletal > -- You received this message because you are subscribed to the Google Groups "Foreman users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/foreman-users. For more options, visit https://groups.google.com/d/optout.
