Bug#860849: duplicity: Duplicity OOMs machines while processing backup chains (memleak?)
Hi, On 2017-04-22 03:17, Alexander Zangerl wrote: > On Thu, 20 Apr 2017 23:47:19 +0200, Maarten Aertsen writes: > >Using duplicity with a S3 remote for a while, until multiple backup > >chains exist. > > could you please provide the output of a collection-status, > 'multiple chains exist' is a bit too little information to debug. I'm afraid I can't for exactly the same reason I think this is a bug: memory usage explodes, the machine locks up and I never get a result. This on a 1024MB vhost with 515MB free -/+ buffers/cache and 2GB nearly unused swap. > how often do you remove chains? Full backups happen every week, incrementals twice per day. Cleanup happens on S3 (duplicity has append-only rights to S3), with 90 day retention of full backups, 14 day retention of incrementals. > how many chains are there? I can make a count based on S3. There's 320 files, making up 8 chains, > how many incrementals per chain? Within the 14 day retention, there's a maximum of two chains with a week's worth of incrementals (2*7*2=28 increments). Because duplicity does not perform the cleanup, at some point the oldest chain breaks (when the first incremental gets deleted). The backups are very small, a full has ~42 volumes of 200MB = ~8.4G. Total size is ~76G. Workaround is to move the files to a separate dir to make them invisible for duplicity. > > * What exactly did you do (or not do) that was effective (or ineffective)? > >/usr/bin/duplicity full --s3-use-ia --asynchronous-upload --encrypt-key > > > > --encrypt-key --include-filelist /etc/backup-targets / > > does your /etc/backup-targets exclude /proc, /sys and /dev? otherwise your > invocation as given here would attempt to back up everything, including > stuff like /dev/mem. No, /etc/backup-targets has "- /" as last line, and none of the mentioned paths is explicitly included in earlier lines. thanks, Maarten
Bug#860849: duplicity: Duplicity OOMs machines while processing backup chains (memleak?)
Package: duplicity Version: 0.7.10-1~bpo8+1 Severity: important Dear Maintainer, * What led up to the situation? Using duplicity with a S3 remote for a while, until multiple backup chains exist. * What exactly did you do (or not do) that was effective (or ineffective)? /usr/bin/duplicity full --s3-use-ia --asynchronous-upload --encrypt-key --encrypt-key --include-filelist /etc/backup-targets / (same effect when substituting full for incremental or collection-status) * What was the outcome of this action? Sudden spikes in CPU and memory consumption resulting in a machine that does not react to console or ssh logins until reboot. Screen displays kernel warnings for hung tasks / timeouts (unfortunately not recoverable from logging). * What outcome did you expect instead? Normal completion of full and incremental backups when run using a soft ulimit on memory use, I get the following error trace: >8 snip >8 Reading globbing filelist /etc/backup-targets Local and Remote metadata are synchronized, no sync needed. Traceback (most recent call last): File "/usr/bin/duplicity", line 1553, in with_tempdir(main) File "/usr/bin/duplicity", line 1547, in with_tempdir fn() File "/usr/bin/duplicity", line 1398, in main do_backup(action) File "/usr/bin/duplicity", line 1423, in do_backup globals.archive_dir).set_values() File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 710, in set_values self.get_backup_chains(partials + backend_filename_list) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 836, in get_backup_chains add_to_sets(f) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 830, in add_to_sets if new_set.add_filename(filename): File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 101, in add_filename self.set_manifest(filename) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 148, in set_manifest self.set_files_changed() File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 128, in set_files_changed mf = self.get_manifest() File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 250, in get_manifest return self.get_local_manifest() File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 224, in get_local_manifest return manifest.Manifest().from_string(manifest_buffer) File "/usr/lib/python2.7/dist-packages/duplicity/manifest.py", line 215, in from_string vi = VolumeInfo().from_string(match.group(1)) MemoryError >8 snip >8 -- System Information: Debian Release: 8.7 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 3.16.0-4-amd64 (SMP w/1 CPU core) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages duplicity depends on: ii libc62.19-18+deb8u7 ii librsync10.9.7-10 ii python 2.7.9-1 ii python-lockfile 1:0.8-2 Versions of packages duplicity recommends: ii python-oauthlib 0.6.3-1 ii python-paramiko 1.15.1-1 ii python-urllib3 1.9.1-3 ii rsync3.1.1-3 Versions of packages duplicity suggests: pn lftp pn ncftp ii python-boto 2.34.0-2 pn python-cloudfiles pn python-gdata pn python-swiftclient ii tahoe-lafs 1.10.0-2 -- no debconf information
Bug#837542: mate-power-manager: segfault (infinite recursion) after changing brightness with brightness keys
Package: mate-power-manager Version: 1.14.0-2 Followup-For: Bug #837542 Dear Maintainer, I can reproduce this bug on this Dell XPS 13 9343 as described by Samuel. I'm available to provide additional information if required. best regards, Maarten -- System Information: Debian Release: stretch/sid APT prefers testing APT policy: (750, 'testing'), (50, 'unstable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.6.0-1-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages mate-power-manager depends on: ii dbus-x111.10.10-1 ii libatk1.0-0 2.21.90-2 ii libc6 2.23-5 ii libcairo-gobject2 1.14.6-1+b1 ii libcairo2 1.14.6-1+b1 ii libcanberra-gtk3-0 0.30-3 ii libcanberra00.30-3 ii libdbus-1-3 1.10.10-1 ii libdbus-glib-1-20.106-1 ii libgdk-pixbuf2.0-0 2.34.0-1 ii libglib2.0-02.49.6-1 ii libgnome-keyring0 3.12.0-1+b1 ii libgtk-3-0 3.21.5-3 ii libmate-desktop-2-171.14.1-1 ii libmate-panel-applet-4-11.14.2-1 ii libnotify4 0.7.6-2 ii libpango-1.0-0 1.40.2-1 ii libpangocairo-1.0-0 1.40.2-1 ii libstartup-notification00.12-4 ii libunique-3.0-0 3.0.2-2 ii libupower-glib3 0.99.4-3 ii libx11-62:1.6.3-1 ii libxext62:1.3.3-1 ii libxrandr2 2:1.5.0-1 ii libxrender1 1:0.9.9-2 ii mate-notification-daemon [notification-daemon] 1.14.1-1 ii mate-power-manager-common 1.14.0-2 ii notification-daemon 3.20.0-1 ii policykit-1 0.105-16 ii systemd 231-4 ii upower 0.99.4-3 mate-power-manager recommends no packages. Versions of packages mate-power-manager suggests: ii mate-polkit 1.14.0-1 -- no debconf information
Bug#719517: jitsi: FTBFS against libav9
Package: jitsi Version: 2.3.4687.9786-1 Followup-For: Bug #719517 Dear Maintainer, with libavfilter2 no longer available in testing or unstable, jitsi has become effectively uninstallable. I have been looking through the jitsi mailinglist archives (dev, user), but cannot find any further discussion of this issue. After the debconf talks and the announcement that jitsi got accepted into Debian, I convinced a lot of my immediate friends and family to try jitsi and have succesfully used it for cross-platform rtc (brilliant!). But having recently switched from Ubuntu to Debian testing, this issue renders it impossible to continue using it myself :-) Could you please provide a status update, or perhaps point to the relevant resources tracking progress if you feel this is not the right place to do so? -- System Information: Debian Release: jessie/sid APT prefers testing APT policy: (900, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 3.10-3-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#728774: network-manager: daemon crashes when using multiple WiFi connections with same SSID (but different config)
Package: network-manager Version: 0.9.8.0-5 Severity: important Dear Maintainer, When defining multiple connections with the same SSID (e.g. for the purpose of auto-connecting one of them, with an associated VPN, and manually connecting the other without VPN to allow login on a WiFi portal), the NetworkManager daemon repeatedly crashes on use of these connections. nm-applet seems to indicate that this is a valid use-case, showing submenu's for the SSID in question. The behaviour manifests itself in when auto-connecting or when manually selecting one of the connections involved. nm-applet hides as soon as NetworkManager crashes and sometimes spawns a dialog . reproducibility: frequently, not always. Having freshly defined the connections everything seems fine. A reboot (i.e. more than a network-manager / nm-applet restart) usually is enough to trigger the problems. This behaviour also manifests itself in the network-manager in Ubuntu 13.04 (0.9.8.0-0ubuntu6). I haven't filed a bug with Ubuntu. I'd be happy to supply more information, but are not sure what to look for. Could you indicate which steps to take to supply you with more useful information? Workaround: delete duplicate connections, restart NetworkManager. -- System Information: Debian Release: jessie/sid APT prefers testing APT policy: (900, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 3.10-3-amd64 (SMP w/4 CPU cores) Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages network-manager depends on: ii adduser3.113+nmu3 ii dbus 1.6.16-1 ii isc-dhcp-client4.2.4-7 ii libc6 2.17-93 ii libdbus-1-31.6.16-1 ii libdbus-glib-1-2 0.100.2-1 ii libgcrypt111.5.3-2 ii libglib2.0-0 2.36.4-1 ii libgnutls262.12.23-8 ii libgudev-1.0-0 204-5 ii libnl-3-2003.2.21-1 ii libnl-genl-3-200 3.2.21-1 ii libnl-route-3-200 3.2.21-1 ii libnm-glib40.9.8.0-5 ii libnm-util20.9.8.0-5 ii libpolkit-gobject-1-0 0.105-4 ii libuuid1 2.20.1-5.5 ii lsb-base 4.1+Debian12 ii udev 204-5 ii wpasupplicant 1.0-3+b2 Versions of packages network-manager recommends: ii crda 1.1.2-1 ii dnsmasq-base 2.66-4 ii iptables 1.4.20-2 ii modemmanager 0.5.2.0-2 ii policykit-1 0.105-4 ii ppp 2.4.5+git20130610-1 Versions of packages network-manager suggests: ii avahi-autoipd 0.6.31-2 -- Configuration Files: /etc/polkit-1/localauthority/10-vendor.d/org.freedesktop.NetworkManager.pkla [Errno 13] Permission denied: u'/etc/polkit-1/localauthority/10-vendor.d/org.freedesktop.NetworkManager.pkla' -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#723636: [apt] apt segfault in pkgDepCache::SetCandidateVersion(pkgCache::VerIterator) ()
Package: apt Version: 0.9.12.1 Followup-For: Bug #723636 Dear Maintainer, I just ran into this same segfault while running apt-get install jitsi/unstable. I hope the backtrace is of any use. -- apt-config dump -- APT ; APT::Architecture amd64; APT::Build-Essential ; APT::Build-Essential:: build-essential; APT::Install-Recommends 1; APT::Install-Suggests 0; APT::NeverAutoRemove ; APT::NeverAutoRemove:: ^firmware-linux.*; APT::NeverAutoRemove:: ^linux-firmware$; APT::NeverAutoRemove:: ^kfreebsd-image.*; APT::NeverAutoRemove:: ^gnumach$; APT::NeverAutoRemove:: ^gnumach-image.*; APT::NeverAutoRemove:: ^linux-image-3.10-3-amd64$; APT::NeverAutoRemove:: ^linux-image-extra-3.10-3-amd64$; APT::NeverAutoRemove:: ^linux-signed-image-3.10-3-amd64$; APT::NeverAutoRemove:: ^linux-backports-modules-.*-3.10-3-amd64$; APT::NeverAutoRemove:: ^linux-headers-3.10-3-amd64$; APT::Never-MarkAuto-Sections ; APT::Never-MarkAuto-Sections:: metapackages; APT::Never-MarkAuto-Sections:: restricted/metapackages; APT::Never-MarkAuto-Sections:: universe/metapackages; APT::Never-MarkAuto-Sections:: multiverse/metapackages; APT::Never-MarkAuto-Sections:: oldlibs; APT::Never-MarkAuto-Sections:: restricted/oldlibs; APT::Never-MarkAuto-Sections:: universe/oldlibs; APT::Never-MarkAuto-Sections:: multiverse/oldlibs; APT::Update ; APT::Update::Post-Invoke-Success ; APT::Update::Post-Invoke-Success:: /usr/bin/test -e /usr/share/dbus-1/system-services/org.freedesktop.PackageKit.service /usr/bin/test -S /var/run/dbus/system_bus_socket /usr/bin/gdbus call --system --dest org.freedesktop.PackageKit --object-path /org/freedesktop/PackageKit --timeout 1 --method org.freedesktop.PackageKit.StateHasChanged cache-update /dev/null; /bin/echo /dev/null; APT::Architectures ; APT::Architectures:: amd64; APT::Compressor ; APT::Compressor::. ; APT::Compressor::.::Name .; APT::Compressor::.::Extension ; APT::Compressor::.::Binary ; APT::Compressor::.::Cost 1; APT::Compressor::gzip ; APT::Compressor::gzip::Name gzip; APT::Compressor::gzip::Extension .gz; APT::Compressor::gzip::Binary gzip; APT::Compressor::gzip::Cost 2; APT::Compressor::gzip::CompressArg ; APT::Compressor::gzip::CompressArg:: -9n; APT::Compressor::gzip::UncompressArg ; APT::Compressor::gzip::UncompressArg:: -d; APT::Compressor::bzip2 ; APT::Compressor::bzip2::Name bzip2; APT::Compressor::bzip2::Extension .bz2; APT::Compressor::bzip2::Binary bzip2; APT::Compressor::bzip2::Cost 3; APT::Compressor::bzip2::CompressArg ; APT::Compressor::bzip2::CompressArg:: -9; APT::Compressor::bzip2::UncompressArg ; APT::Compressor::bzip2::UncompressArg:: -d; APT::Compressor::xz ; APT::Compressor::xz::Name xz; APT::Compressor::xz::Extension .xz; APT::Compressor::xz::Binary xz; APT::Compressor::xz::Cost 4; APT::Compressor::xz::CompressArg ; APT::Compressor::xz::CompressArg:: -6; APT::Compressor::xz::UncompressArg ; APT::Compressor::xz::UncompressArg:: -d; APT::Compressor::lzma ; APT::Compressor::lzma::Name lzma; APT::Compressor::lzma::Extension .lzma; APT::Compressor::lzma::Binary xz; APT::Compressor::lzma::Cost 5; APT::Compressor::lzma::CompressArg ; APT::Compressor::lzma::CompressArg:: --format=lzma; APT::Compressor::lzma::CompressArg:: -9; APT::Compressor::lzma::UncompressArg ; APT::Compressor::lzma::UncompressArg:: --format=lzma; APT::Compressor::lzma::UncompressArg:: -d; APT::CompressorName ; APT::CompressorExtension .; APT::CompressorBinary ; APT::CompressorCost 100; APT::CompressorCompressArg ; APT::CompressorCompressArg:: -9; APT::CompressorUncompressArg ; APT::CompressorUncompressArg:: -d; Dir /; Dir::State var/lib/apt/; Dir::State::lists lists/; Dir::State::cdroms cdroms.list; Dir::State::mirrors mirrors/; Dir::State::extended_states extended_states; Dir::State::status /var/lib/dpkg/status; Dir::Cache var/cache/apt/; Dir::Cache::archives archives/; Dir::Cache::srcpkgcache srcpkgcache.bin; Dir::Cache::pkgcache pkgcache.bin; Dir::Etc etc/apt/; Dir::Etc::sourcelist sources.list; Dir::Etc::sourceparts sources.list.d; Dir::Etc::vendorlist vendors.list; Dir::Etc::vendorparts vendors.list.d; Dir::Etc::main apt.conf; Dir::Etc::netrc auth.conf; Dir::Etc::parts apt.conf.d; Dir::Etc::preferences preferences; Dir::Etc::preferencesparts preferences.d; Dir::Etc::trusted trusted.gpg; Dir::Etc::trustedparts trusted.gpg.d; Dir::Bin ; Dir::Bin::methods /usr/lib/apt/methods; Dir::Bin::solvers ; Dir::Bin::solvers:: /usr/lib/apt/solvers; Dir::Bin::dpkg /usr/bin/dpkg; Dir::Bin::bzip2 /bin/bzip2; Dir::Bin::xz /usr/bin/xz; Dir::Bin::lzma /usr/bin/xz; Dir::Media ; Dir::Media::MountPath /media/apt; Dir::Log var/log/apt; Dir::Log::Terminal term.log; Dir::Log::History history.log; Dir::Ignore-Files-Silently ; Dir::Ignore-Files-Silently:: ~$; Dir::Ignore-Files-Silently:: \.disabled$; Dir::Ignore-Files-Silently:: \.bak$; Dir::Ignore-Files-Silently:: \.dpkg-[a-z]+$; Dir::Ignore-Files-Silently:: \.save$; Dir::Ignore-Files-Silently:: \.orig$; Acquire ; Acquire::cdrom ; Acquire::cdrom::mount
Bug#710710: asterisk: bump version to 11 (LTS release)
Source: asterisk Version: 11 Severity: wishlist Bug to track work on packaging Asterisk 11. Current showstopper is pjproject due to inclusion of multiple external libaries instead of embedding them. Intent to package pjsip is #708122 Also needed is work on the copyright file (see FIXME's): http://anonscm.debian.org/viewvc/pkg-voip/asterisk/branches/experimental/debian/copyright_wip?view=markup Work in progress can be found in the experimental branch (currently without build-dep on pjsip) http://anonscm.debian.org/viewvc/pkg-voip/asterisk/branches/experimental/debian/changelog?view=markup -- System Information: Debian Release: 7.0 APT prefers stable APT policy: (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/1 CPU core) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
