On 07/25/2016 03:20 PM, Tom Lane wrote:
Matthew Musgrove
<[email protected]><mailto:[email protected]> writes:
One of our instances has been behaving -- oddly. Most queries are blazing fast.
It appears to just be some of the stat views that are slow.
It sounds like requests for stats updates are not getting through to the
collector. I wonder if your kernel is blocking those packets for some
reason (probably ill-considered iptables filters?). There is a test
for packet filtering conducted during postmaster startup, but we don't
repeat it after that.
Today was the first day that I have had any significant amount of time to
investigate since probably the 12th. I had checked iptables before. When I saw
this message I checked, double-checked and even re-ordered the iptables rules
but it didn't make any difference.
I started going through the log files under /var/log again. I didn't see
anything until I made it back to /var/log/messages for July 7th. I noticed some
entries for system-config-network at 14:05.
I looked at ifconfig and the files in /etc/sysconfig/network-scripts but
everything looked fine. I checked what ports the postmaster and stats collector
were using ('lsof -i -P | grep <port>'). I'm not sure how I missed this before
but they were using an IPv6 localhost port.
I asked what network changes were made on July 7th around 14:05 and was told
that they disabled IPv6! I had them turn it back on and stats started working
immediately.
I have since changed the config from listen_addresses = '*' to listen_addresses
= '0.0.0.0' but we won't be restarting until our next maintenance window (in
about a week) when they will disable IPv6 again.
Tom and Adrian, thank you both! :)
--
Matthew Musgrove
