https://bugzilla.wikimedia.org/show_bug.cgi?id=69244
Jeff Gage <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |WORKSFORME --- Comment #5 from Jeff Gage <[email protected]> --- This is the same broker we've had timeout issues with in the past. We were hopeful that the upgrade to Kafka 0.8.1.1 might resolve them. During the upgrade we found a stale Kafka init script on analytics1021; again we hoped that fix would resolve this issue. Frustrating to see that it's still happening. On the one hand we could just reinstall the OS in order to resolve this, but on the other hand we have three other brokers so the service has remained available, and it would be nice to understand the root cause of this problem. After acknowledging the alerts and confirming what Christian observed in the logs, I upgraded all packages on the host and rebooted into a new kernel (3.2.0-67-generic) by doing (essentially): apt-get update && apt-get upgrade && apt-get dist-upgrade && reboot After reboot I observed all partitions fully replicate, triggered a replica election, and confirmed traffic flow in Ganglia. Analytics1021 is now back in service. It remains to be seen whether the package upgrades will finally resolve the timeout problems. This is the list of upgraded packages: accountsservice apt apt-transport-https apt-utils apt-xapian-index base-files bind9-host bsdutils ca-certificates consolekit curl dbus dbus-x11 dmidecode dmsetup dnsutils dpkg file gnupg gpgv grub-common grub-pc grub-pc-bin grub2-common icedtea-netx icedtea-netx-common ifupdown initramfs-tools initramfs-tools-bin iproute isc-dhcp-client isc-dhcp-common language-pack-en language-pack-en-base language-selector-common libaccountsservice0 libapt-inst1.4 libapt-pkg4.12 libasn1-8-heimdal libavahi-client3 libavahi-common-data libavahi-common3 libavahi-glib1 libbind9-80 libblkid1 libc-bin libc6 libck-connector0 libcups2 libcurl3 libcurl3-gnutls libdbus-1-3 libdevmapper-event1.02.1 libdevmapper1.02.1 libdns81 libdrm-intel1 libdrm-nouveau1a libdrm-radeon1 libdrm2 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libglib2.0-0 libgnutls26 libgssapi3-heimdal libgtk-3-0 libgtk-3-bin libgtk-3-common libgtk2.0-0 libgtk2.0-bin libgtk2.0-common libgudev-1.0-0 libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libisc83 libisccc80 libisccfg82 libjpeg-turbo8 libjson0 libkrb5-26-heimdal libldap-2.4-2 liblockfile-bin liblockfile1 liblvm2app2.2 liblwres80 libmagic1 libmount1 libmysqlclient18 libnspr4 libnss3 libnss3-1d libpam-ck-connector libparted0debian1 libperl5.14 libpixman-1-0 libpolkit-agent-1-0 libpolkit-backend-1-0 libpolkit-gobject-1-0 libpq5 libpulse0 libpython2.7 libroken18-heimdal libruby1.8 libservlet2.5-java libsnmp-base libsnmp15 libssl1.0.0 libtasn1-3 libtiff4 libudev0 libuuid1 libwbclient0 libwind0-heimdal libx11-6 libx11-data libx11-dev libx11-doc libx11-xcb1 libxfixes3 libxi6 libxml2 libyaml-0-2 linux-firmware mount multiarch-support mysql-common openjdk-6-jre openjdk-6-jre-headless openjdk-6-jre-lib openjdk-7-jre-lib openssl parted perl perl-base perl-modules policykit-1 procps python-apt python-apt-common python-jinja2 python2.7 python2.7-minimal ruby1.8 samba-common samba-common-bin smbclient sudo udev udisks update-manager-core util-linux uuid-runtime wget x11proto-input-dev xkb-data Dist-upgrade took care of the kernel: linux-headers-server linux-image-server linux-server -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. _______________________________________________ Wikibugs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
