[Bug 1316970] Re: g_dbus memory leak in lrmd
This bug was fixed in the package pacemaker - 1.1.10+git20130802-1ubuntu2.5 --- pacemaker (1.1.10+git20130802-1ubuntu2.5) trusty; urgency=medium * Fixing memory leak (LP: #1316970) - adding free function where is missing -- Seyeong Kim Tue, 20 Feb 2018 19:36:53 +0900 ** Changed in: pacemaker (Ubuntu Trusty) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Thanks Łukasz, I've tested the pacemaker packages in -proposed and they fixed the issue for me. The version I've tested is: 1.1.10+git20130802-1ubuntu2.5 . The test was performed like this: I have a test-case that I ran against the version in trusty-updates (call it baseline); then I performed the packages' update to -proposed version and re-ran the test-case (call it proposed). I've observed a memory leak of 4.5MB during 1 hour in the baseline. After update it, I've observed a 64K memory increase in 1 hour, but 60K of it was "leaked" in the first 30 minutes. I don't believe this is a real memory leak, as explained in my previous comments. Valgrind didn't show anything, seems a normal application behavior. Thanks, Guilherme ** Tags removed: sts-sru-needed verification-needed verification-needed-trusty ** Tags added: verification-done verification-done-trusty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hello Greg, or anyone else affected, Accepted pacemaker into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pacemaker/1.1.10+git20130802-1ubuntu2.5 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: pacemaker (Ubuntu Trusty) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-trusty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hi Dan, made some progress on the investigation (not definitive, but still it helps us to continue with the SRU process). By using Valgrind memcheck analyzer I couldn't observe any non-constant leaks after Seyeong's patch gets applied. Then, I started to use two more analyzers in order to obtain more information of lrmd's memory behavior: DHAT and massif, both contained in Valgrind's pack of analyzers. DHAT (dynamic heap analyzer tool) allowed me to confirm some hypothesis. By running the experiment3 times, the first for 10 minutes, then for 20 minutes and finally for 30 minutes, I observed that: 1) lrmd allocated a total of 20,671,285 bytes during 10 minutes, 38,486,917 in the 20 minute run and 56,338,077 when running for 30 minutes; 2) In all the 3 cases, the total leaked memory (from Glib library) was 104,146 bytes, a constant value; 3) Also, it measured that in all cases 20,673 bytes have lived for more than half of the run, meaning the application is allocating and de- allocating memory in reduced intervals of time, not keeping allocated memory until its end (when it would free all chunks); Number #1 above indicates the total memory allocated - it doesn't mean this entire amount was living at same time. It basically sums all the calls to malloc-like functions during the program execution. Number #2 indicates we have a constant amount of leaked memory, that is not increasing and so is not responsible for the slow memory increase we're observing. Finally, number #3 shows us that this is not a case of a program allocating memory constantly and only de-allocating all chunks in the end of application at once. This was one of my hypothesis, now proved false. That said, it's clear that heap-wise the application is not leaking an increasing amount of memory. From the stack point-of-view, by running application through massif analyzer it's possible to observe the stack behavior - the maximum size from stack was 5008 bytes, the minimum was 744 bytes. It floated between those 2 limits in a non-constant ratio, meaning it had increased and decreased over time, multiple times. This proves the stack has not much influence in the issue. So, after that I started observing the /proc/smaps of the application, and it showed an important data point: the "area" that is growing is an anonymous non-heap map, so it was allocated through the mmap() syscall. Valgrind cannot capture mmap() syscalls, so it's likely to miss a possible leak if the memory in question was allocated through mmap(). By "stracing" the application, I saw many mmap() calls, more then munmap(). And by inspecting GLib code, I could see mmap() calls there (whereas lrmd code has none itself). So, it could be a GLib wrapper causing this slow increase of memory. My last hypothesis is memory fragmentation, but I'd like to first exclude or confirm the mmap() idea before going with memory fragmentation hypothesis. That all said, I believe we should continue the SRU process since Seyeong's patch was proved a valid fix for the heap leaks we had. I intend to continue the investigation to understand exactly what kind of memory behavior lrmd has to justify this slow but steady memory growth now. Thanks, Guilherme -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Dan, thanks for your v4! I just tested it and the leak seems much less intense - the patch mitigated it very fine. But still...we're leaking 16K/hour. I'll investigate this further. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Changed in: pacemaker (Ubuntu Trusty) Assignee: Seyeong Kim (xtrusia) => Guilherme G. Piccoli (gpiccoli) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
glib2.0 memleak patch moved over to bug 1750741 ** No longer affects: glib2.0 (Ubuntu) ** No longer affects: glib2.0 (Ubuntu Trusty) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Description changed: [Impact] lrmd daemon with upstart resource has memory leak in Trusty affected to pacemaker 1.1.10. - affected to glib2.0 2.40.2-0ubuntu1 + affected to glib2.0 2.40.2-0ubuntu1 >> for glib2.0 created new lp [3] Please note that patch for pacemaker is created myself. [Test Case] - 1. deploy 3 trusty instance. - 2. install corosync, pacemaker, mysql. - 3. setting with below info - 3.1 corosync.conf, proper setting for 3 node - 3.2 crm configure < setup.crm ( which has upstart:mysql setting ) - 3.3 monitor lrmd daemon's memory usage + https://pastebin.ubuntu.com/p/fqK6Cx3SKK/ + you can check memory leak with this script [Regression] - Restarting daemon after upgrading this pkg will be needed. this patch added NULL check for several parts. prior commit[1] changed file structure. and This change makes user changes usage of upstart:mysql to lsb:mysql. So I added free function myself. This might affect to system. - For glib2.0, commit [1] is critical, but [2],[3],[4] is needed for building it. + Restarting daemon after upgrading this pkg will be needed. this patch adds free for non-freed dynamic allocated memory. so it solves memory leak. [Others] - Related commits. + this patch is from my self with testing. - [1] commit a7b61e276120184c7586a3217ed3571a982f5017 - Author: Andrew Beekhof - Date: Fri Aug 23 16:25:35 2013 +1000 - - Refactor: attrd: Move to its own directory and create a stub for - attrd-ng - - -- - $ git describe --contains a1a6922e43dfe80b23887a88401cbb93fe3645c0 - Pacemaker-1.1.11-rc3 - - $ git describe --contains a7b61e276120184c7586a3217ed3571a982f5017 - Pacemaker-1.1.11-rc1~168 - - $ rmadison pacemaker - pacemaker | 1.1.10+git20130802-1ubuntu2 | trusty - pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-security - pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-updates - - pacemaker | 1.1.14-2ubuntu1 | xenial - pacemaker | 1.1.14-2ubuntu1.3 | xenial-security - pacemaker | 1.1.14-2ubuntu1.3 | xenial-update - pacemaker | 1.1.16-1ubuntu1 | zesty - pacemaker | 1.1.16-1ubuntu1 | artful - pacemaker | 1.1.18~rc3-1ubuntu1 | bionic - -- - - For glib - [1] https://github.com/GNOME/glib/commit/db641e32920ee8b553ab6f2d318aafa156e4390c - [2] https://github.com/GNOME/glib/commit/8792609e15394967cab526838b83f90acb401663 - [3] https://github.com/GNOME/glib/commit/ec02a1875f29ecb8e46c0d8c1403cd00a0b3a9e4 - [4] https://github.com/GNOME/glib/commit/f10b6550ff2ce55d06b92d6dc3e443fc007b2f7a + Please review carefully if it is ok. [Original Description] I'm running Pacemaker 1.1.10+git20130802-1ubuntu1 on Ubuntu Saucy (13.10) and have encountered a memory leak in lrmd. The details of the bug are covered here in this thread (http://oss.clusterlabs.org/pipermail/pacemaker/2014-May/021689.html) but to summarise, the Pacemaker developers believe the leak is caused by the g_dbus API, the use of which was removed in Pacemaker 1.11. I've also attached the Valgrind output from the run that exposed the issue. Given that this issue affects production stability (a periodic restart of Pacemaker is required), will a version of 1.11 be released for Trusty? (I'm happy to upgrade the OS to Trusty to get it). If not, can you advise which version of the OS will be the first to take 1.11 please? ** Description changed: [Impact] lrmd daemon with upstart resource has memory leak in Trusty affected to pacemaker 1.1.10. - affected to glib2.0 2.40.2-0ubuntu1 >> for glib2.0 created new lp [3] + affected to glib2.0 2.40.2-0ubuntu1 >> for glib2.0 created new lp [1] Please note that patch for pacemaker is created myself. [Test Case] https://pastebin.ubuntu.com/p/fqK6Cx3SKK/ you can check memory leak with this script [Regression] - Restarting daemon after upgrading this pkg will be needed. this patch adds free for non-freed dynamic allocated memory. so it solves memory leak. + Restarting daemon after upgrading this pkg will be needed. this patch adds free for non-freed dynamic allocated memory. so it solves memory leak. [Others] this patch is from my self with testing. Please review carefully if it is ok. + + [1] https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1750741 [Original Description] I'm running Pacemaker 1.1.10+git20130802-1ubuntu1 on Ubuntu Saucy (13.10) and have encountered a memory leak in lrmd. The details of the bug are covered here in this thread (http://oss.clusterlabs.org/pipermail/pacemaker/2014-May/021689.html) but to summarise, the Pacemaker developers believe the leak is caused by the g_dbus API, the use of which was removed in Pacemaker 1.11. I've also attached the Valgrind output from the run that
[Bug 1316970] Re: g_dbus memory leak in lrmd
Seyeong, can you check this updated debdiff to verify it still fixes the mem leak? I reverted the free->g_free changes that I previously suggested, as you were correct - the existing code uses chars and normal free() so we should stick with that, i was wrong to suggest using g_free(). Also I removed one of the free() that did not seem correct per the docs. ** Patch added: "lp1316970_trusty_pacemaker_v4.debdiff" https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+attachment/5059064/+files/lp1316970_trusty_pacemaker_v4.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Ok, in that case, can you open a separate lp bug for the glib2.0 memory leak? I think it will be easier to track just fixing the pacemaker mem leaks in this bug, and track fixing the glib2.0 mem leak in a separate bug. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hello ddstreet I confirmed that glib2.0 patch is needed. I installed patched_pacemaker only on my system and running just 1 hour. it started with 4700(RES) but now it is 8000(RES). after installing glib2.0 it started with 4612, now is is 4688 Thanks [1] is script to reproduce this issue. need to adjust some code but just uploading [1] https://pastebin.ubuntu.com/p/CGrGhjdr4g/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
the glib change seems buggy, the upstream bug corresponding to the first patch https://bugzilla.gnome.org/show_bug.cgi?id=758641 states that the issue was in code commited in 2.45 or trusty has 2.40 so it seems wrong to try to use it on that codebase (unsure about the other ones) ** Bug watch added: GNOME Bug Tracker #758641 https://bugzilla.gnome.org/show_bug.cgi?id=758641 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Seyeong, can you also clarify - are the glib2.0 patches also needed to fix memory leaks? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
sure, no problem Thanks eric -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Seyeong, The SRU for LP: #1740892 is not started yet, but base on nacc's comment, It should start soon : https://bugs.launchpad.net/charm-hacluster/+bug/1740892/comments/47 I would suggest to wait for LP: #1740892 SRU to be completed first since it affects package upgrade (corosync/pacemaker relation). Additionally, IMHO, it may be "safer" to not introduce too much change in one SRU, especially considering LP: #1740892 mark as Critical. 1) Please keep monitoring LP: #1740892 and let's wait for the SRU to turn "Fix Released" for Trusty. 2) Then let's start LP: #1316970 SRU right after using either "STS-Sponsor" or "Ubuntu Sponsors Team" - Eric -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hello Eric, ddstreet #1740892 seems starting to fix Could you guys can sponsor this issue as well? Thanks a lot -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Seyeong/ddstreet, As long as this critical bug (LP: #1740892) affecting corosync/pacemaker relation is not fix, I would be reluctant to sponsor this bug for the following reasons : - To make sure it won't block potential pacemaker sru to fix this critical bug by start this one here. - To make sure to not force another package upgrade issue and impact other users until (LP: #1740892) is figure out and fix. - Eric -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
@ddstreet Could you please review V3 again? Thanks in advance! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
For V3. It seems that g_variant_unref(value); part is needed. Added it again and no memory leaks. in g_variant_lookup_value, it calls g_variant_get_child_value so as you said. it need to be unref value as well i think. Thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Patch added: "lp1316970_trusty_pacemaker_v3.debdiff" https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+attachment/5048947/+files/lp1316970_trusty_pacemaker_v3.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
hmm, this has leak more than before.. testing.. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Patch added: "lp1316970_trusty_pacemaker_v2.debdiff" https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+attachment/5047782/+files/lp1316970_trusty_pacemaker_v2.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hello ddstreet Thanks for the review. I changed some code for pacemaker. ( char -> gchar for matching type, with g_free) and glib2.0 patch is needed in my test. without it, memory leak is there. 4kB leak every 10 second at least. Thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hi Seyeong, first, I commend you for going to the trouble of fixing tests in lp1316970_trusty_glib2.0.debdiff, but I think those are unrelated to this actual bug and so the patch is not required for this. I'll focus only on the first patch. In that patch, I did a quick review of the changes, and I'm not entirely sure they are correct? LP bug comments are not a great format for reviewing patches, so please excuse formatting/line-wrapping: > --- a/lib/services/upstart.c > +++ b/lib/services/upstart.c > @@ -70,6 +70,7 @@ > if (error) { > crm_err("Can't connect obtain proxy to %s interface: %s", interface, > error->message); > g_error_free(error); > +g_object_unref(proxy); this doesn't seem right - if error is non-NULL, then proxy should be NULL, according to the docs: https://developer.gnome.org/gio/stable/GDBusProxy.html#g-dbus-proxy-new-for-bus-sync > proxy = NULL; > } > return proxy; > @@ -107,7 +108,9 @@ > /* >com.ubuntu.Upstart0_6.GetJobByName (in String name, out ObjectPath job) > */ > -GVariant *_ret = g_dbus_proxy_call_sync(proxy, "GetJobByName", > g_variant_new("(s)", arg_name), > +GVariant *_ret = NULL; > + > +_ret = g_dbus_proxy_call_sync(proxy, "GetJobByName", > g_variant_new("(s)", arg_name), > G_DBUS_CALL_FLAGS_NONE, -1, > cancellable, error); unless i'm missing something, there is no need for this change? > > if (_ret) { > @@ -200,6 +203,7 @@ > > g_variant_iter_free(iter); > g_variant_unref(_ret); > +free(path); this does not seem correct - per the docs, 'path' will be freed by each call in the while loop to g_variant_iter_loop(), and only needs to be manually freed after the while loop if the code breaks out of the loop manually. That doesn't appear to be the case, so path should not need freeing here. https://developer.gnome.org/glib/stable/glib-GVariant.html#g-variant-iter-loop > return units; > } > > @@ -224,7 +228,7 @@ > } else if (pass) { > crm_trace("Got %s", path); > } > -/* free(path) */ > +free(path); technically this should be freed only if !error - the freeing should go into the else if (pass) block above, although free(NULL) will just do nothing however, i believe this should use g_free() instead of free() > return pass; > } > > @@ -272,6 +276,8 @@ > > g_object_unref(proxy); > g_variant_unref(_ret); > +g_variant_unref(value); > +g_variant_unref(asv); asv is returned from g_variant_get_child_value, which does state in its docs that the returned value should be unref'ed, so this looks correct for 'asv'. https://developer.gnome.org/glib/stable/glib-GVariant.html#g-variant-get-child-value However value is returned from g_variant_lookup_value, which returns a value from a key-value pair in asv (as best i can tell from the docs for the function), and it does not indicate that the value is newly allocated or that it should be freed. So it does not appear that 'value' should be unref'ed here. https://developer.gnome.org/glib/stable/glib-GVariant.html#g-variant-lookup-value > return output; > } > > @@ -299,11 +305,14 @@ > GVariant *tmp2 = g_variant_get_child_value(tmp1, 0); > > instance = g_variant_dup_string(tmp2, NULL); > +g_variant_unref(tmp2); > } > +g_variant_unref(tmp1); these look correct > } > > crm_info("Result: %s", instance); > g_variant_unref(_ret); > +g_object_unref(proxy); this looks correct > return instance; > } > > @@ -338,6 +347,7 @@ > } > > crm_info("%s is%s running", name, pass ? "" : " not"); > +free(job); technically this is only needed in the else {} block, but as it should still be NULL the free will do nothing again this should be g_free() instead of free() i think > return pass; > } > > @@ -400,6 +410,7 @@ > crm_info("Call to %s passed: type '%s' %s", op->action, > g_variant_get_type_string(_ret), > path); > op->rc = PCMK_EXECRA_OK; > +free(path); looks good, but probably use g_free() > > } else { > crm_err("Call to %s passed but return type was '%s' not '(o)'", > op->action, g_variant_get_type_string(_ret)); > @@ -501,6 +512,7 @@ > crm_info("Call to %s passed: type '%s' %s", op->action, > g_variant_get_type_string(_ret), > path); > op->rc = PCMK_EXECRA_OK; > +free(path); looks good, but probably use g_free() > > } else { > crm_err("Call to %s passed but return type was '%s' not '(o)'", > op->action, g_variant_get_type_string(_ret)); -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.ne
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Description changed: [Impact] lrmd daemon with upstart resource has memory leak in Trusty affected to pacemaker 1.1.10. + affected to glib2.0 2.40.2-0ubuntu1 + + Please note that patch for pacemaker is created myself. [Test Case] 1. deploy 3 trusty instance. 2. install corosync, pacemaker, mysql. 3. setting with below info 3.1 corosync.conf, proper setting for 3 node 3.2 crm configure < setup.crm ( which has upstart:mysql setting ) 3.3 monitor lrmd daemon's memory usage [Regression] - Restarting daemon after upgrading this pkg will be needed. this patch added NULL check for several parts. prior commit[2] changed file structure ( e.g attrd created from tools/attrd.c ), This might affect to system. however, both of patches are landed to 1.11, so risk is not that high. + Restarting daemon after upgrading this pkg will be needed. this patch added NULL check for several parts. prior commit[1] changed file structure. and This change makes user changes usage of upstart:mysql to lsb:mysql. So I added free function myself. This might affect to system. + For glib2.0, commit [1] is critical, but [2],[3],[4] is needed for building it. [Others] Related commits. - [1] commit a1a6922e43dfe80b23887a88401cbb93fe3645c0 - Author: David Vossel - Date: Tue Jan 7 14:02:18 2014 -0600 - - Fix: ipc: fix memory leak for failed ipc client connections. - - When pacemaker ipc servers accept client connections, client - state data is allocated when the libqb 'accept' callback function is - invoked. It is possible however that even after the client - accepts the connection in the libqb 'accept' callback, the connection - could still fail to initialize fully if the client side hangs up. - - Currently this results in the client state data never being - destroyed because the libqb "close" callback for client connection - is never invoked on the server side. Instead, libqb jumps directly to - the "destroy" callback because the connection never actually got created. - - To account for this memory leak, pacemaker needs to verify client - state data is destroyed in the destroy callback. - [2] commit a7b61e276120184c7586a3217ed3571a982f5017 + [1] commit a7b61e276120184c7586a3217ed3571a982f5017 Author: Andrew Beekhof Date: Fri Aug 23 16:25:35 2013 +1000 Refactor: attrd: Move to its own directory and create a stub for attrd-ng -- $ git describe --contains a1a6922e43dfe80b23887a88401cbb93fe3645c0 Pacemaker-1.1.11-rc3 $ git describe --contains a7b61e276120184c7586a3217ed3571a982f5017 Pacemaker-1.1.11-rc1~168 $ rmadison pacemaker - pacemaker | 1.1.10+git20130802-1ubuntu2 | trusty - pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-security - pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-updates + pacemaker | 1.1.10+git20130802-1ubuntu2 | trusty + pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-security + pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-updates - pacemaker | 1.1.14-2ubuntu1 | xenial - pacemaker | 1.1.14-2ubuntu1.3 | xenial-security - pacemaker | 1.1.14-2ubuntu1.3 | xenial-update - pacemaker | 1.1.16-1ubuntu1 | zesty - pacemaker | 1.1.16-1ubuntu1 | artful - pacemaker | 1.1.18~rc3-1ubuntu1 | bionic + pacemaker | 1.1.14-2ubuntu1 | xenial + pacemaker | 1.1.14-2ubuntu1.3 | xenial-security + pacemaker | 1.1.14-2ubuntu1.3 | xenial-update + pacemaker | 1.1.16-1ubuntu1 | zesty + pacemaker | 1.1.16-1ubuntu1 | artful + pacemaker | 1.1.18~rc3-1ubuntu1 | bionic -- + + For glib + [1] https://github.com/GNOME/glib/commit/db641e32920ee8b553ab6f2d318aafa156e4390c + [2] https://github.com/GNOME/glib/commit/8792609e15394967cab526838b83f90acb401663 + [3] https://github.com/GNOME/glib/commit/ec02a1875f29ecb8e46c0d8c1403cd00a0b3a9e4 + [4] https://github.com/GNOME/glib/commit/f10b6550ff2ce55d06b92d6dc3e443fc007b2f7a [Original Description] I'm running Pacemaker 1.1.10+git20130802-1ubuntu1 on Ubuntu Saucy (13.10) and have encountered a memory leak in lrmd. The details of the bug are covered here in this thread (http://oss.clusterlabs.org/pipermail/pacemaker/2014-May/021689.html) but to summarise, the Pacemaker developers believe the leak is caused by the g_dbus API, the use of which was removed in Pacemaker 1.11. I've also attached the Valgrind output from the run that exposed the issue. Given that this issue affects production stability (a periodic restart of Pacemaker is required), will a version of 1.11 be released for Trusty? (I'm happy to upgrade the OS to Trusty to get it). If not, can you advise which version of the OS will be the fir
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Attachment added: "valgrind-beforepatch" https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+attachment/5047144/+files/valgrind-beforepatch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Patch added: "lp1316970_trusty_glib2.0.debdiff" https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+attachment/5047147/+files/lp1316970_trusty_glib2.0.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Patch added: "lp1316970_trusty_pacemaker.debdiff" https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+attachment/5047146/+files/lp1316970_trusty_pacemaker.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Attachment added: "valgrind-afterpatch" https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+attachment/5047145/+files/valgrind-afterpatch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Also affects: glib2.0 (Ubuntu) Importance: Undecided Status: New ** Patch removed: "lp1316970_trusty.debdiff" https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+attachment/5031302/+files/lp1316970_trusty.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/glib2.0/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Tested pacemaker 1.1.10+git20130802-1ubuntu2.4+sf160627v20180123.4 from xtrusia PPA. RSS was observed to increase slowly over time (less than 20KB/hr), and the other memory metrics[0] I gathered did not increase at all; whereas, prior to the patch, all memory metrics were increasing and more quickly. If not fixed, it certainly appears to have been significantly mitigated. [0] specified RSS, SZ, SIZE, and VSZ in ps -o option -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Sure, I'll unsubscribe sts-sponsors while you update the patch, just re- subscribe us when it's ready for review. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hello Dan & Eric I confirmed that the issue dan mentioned is happening with refactoring commit. and also the other person reported me this pkg doesn't fix this issue. it seemed fix this but actually it is not. so I think I need more time to analyze this issue. Could you please give me more time for this? Thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Seyeong, I prepared a package with the debdiff for uploading, but it is failing to build during the autopkgtests; can you check that and update the debdiff please? configure: creating ./config.status config.status: creating Makefile config.status: creating Doxyfile config.status: creating coverage.sh config.status: WARNING: 'coverage.sh.in' seems to ignore the --datarootdir setting config.status: error: cannot find input file: `cts/Makefile.in' make: *** [build-stamp] Error 1 dpkg-buildpackage: error: debian/rules build gave error exit status 2 blame: pacemaker_1.1.10+git20130802-1ubuntu2.5.dsc badpkg: rules build failed with exit code 2 You can check this by creating the patched source files and then run autopkgtest on it, e.g.: $ autopkgtest -U pacemaker_1.1.10+git20130802-1ubuntu2.5.dsc -- lxd ubuntu-daily:trusty -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hi Seyeong, the LP build farm is still unavailable, but once it's back up I'll review and sponsor this. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Tags added: sts-sponsor-ddstreet -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Changed in: pacemaker (Ubuntu) Status: Confirmed => Fix Released ** Changed in: pacemaker (Ubuntu Trusty) Assignee: (unassigned) => Seyeong Kim (xtrusia) ** Changed in: pacemaker (Ubuntu Trusty) Importance: Undecided => Medium ** Changed in: pacemaker (Ubuntu Trusty) Status: New => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Hi Seyeong, As we speak the LP build farm and autopkgtest request.cgi are disabled for maintenance; no ETA yet. No new uploads are allowed during this temporary freeze cause by the maintenance. We will gladly review your patch when everything will back to normal. - Eric ** Description changed: [Impact] lrmd daemon with upstart resource has memory leak in Trusty affected to pacemaker 1.1.10. [Test Case] 1. deploy 3 trusty instance. 2. install corosync, pacemaker, mysql. 3. setting with below info 3.1 corosync.conf, proper setting for 3 node 3.2 crm configure < setup.crm ( which has upstart:mysql setting ) 3.3 monitor lrmd daemon's memory usage [Regression] Restarting daemon after upgrading this pkg will be needed. this patch added NULL check for several parts. prior commit[2] changed file structure ( e.g attrd created from tools/attrd.c ), This might affect to system. however, both of patches are landed to 1.11, so risk is not that high. [Others] Related commits. [1] commit a1a6922e43dfe80b23887a88401cbb93fe3645c0 Author: David Vossel Date: Tue Jan 7 14:02:18 2014 -0600 Fix: ipc: fix memory leak for failed ipc client connections. When pacemaker ipc servers accept client connections, client state data is allocated when the libqb 'accept' callback function is invoked. It is possible however that even after the client accepts the connection in the libqb 'accept' callback, the connection could still fail to initialize fully if the client side hangs up. Currently this results in the client state data never being destroyed because the libqb "close" callback for client connection is never invoked on the server side. Instead, libqb jumps directly to the "destroy" callback because the connection never actually got created. To account for this memory leak, pacemaker needs to verify client state data is destroyed in the destroy callback. [2] commit a7b61e276120184c7586a3217ed3571a982f5017 Author: Andrew Beekhof Date: Fri Aug 23 16:25:35 2013 +1000 Refactor: attrd: Move to its own directory and create a stub for attrd-ng + -- + $ git describe --contains a1a6922e43dfe80b23887a88401cbb93fe3645c0 + Pacemaker-1.1.11-rc3 + + $ git describe --contains a7b61e276120184c7586a3217ed3571a982f5017 + Pacemaker-1.1.11-rc1~168 + + $ rmadison pacemaker + pacemaker | 1.1.10+git20130802-1ubuntu2 | trusty + pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-security + pacemaker | 1.1.10+git20130802-1ubuntu2.4 | trusty-updates + + pacemaker | 1.1.14-2ubuntu1 | xenial + pacemaker | 1.1.14-2ubuntu1.3 | xenial-security + pacemaker | 1.1.14-2ubuntu1.3 | xenial-update + pacemaker | 1.1.16-1ubuntu1 | zesty + pacemaker | 1.1.16-1ubuntu1 | artful + pacemaker | 1.1.18~rc3-1ubuntu1 | bionic + -- + [Original Description] I'm running Pacemaker 1.1.10+git20130802-1ubuntu1 on Ubuntu Saucy (13.10) and have encountered a memory leak in lrmd. The details of the bug are covered here in this thread (http://oss.clusterlabs.org/pipermail/pacemaker/2014-May/021689.html) but to summarise, the Pacemaker developers believe the leak is caused by the g_dbus API, the use of which was removed in Pacemaker 1.11. I've also attached the Valgrind output from the run that exposed the issue. Given that this issue affects production stability (a periodic restart of Pacemaker is required), will a version of 1.11 be released for Trusty? (I'm happy to upgrade the OS to Trusty to get it). If not, can you advise which version of the OS will be the first to take 1.11 please? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Also affects: pacemaker (Ubuntu Trusty) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Description changed: [Impact] lrmd daemon with upstart resource has memory leak in Trusty affected to pacemaker 1.1.10. - [Test Case] 1. deploy 3 trusty instance. 2. install corosync, pacemaker, mysql. 3. setting with below info - 3.1 corosync.conf ( ) - 3.2 crm configure < setup.crm ( ) + 3.1 corosync.conf, proper setting for 3 node + 3.2 crm configure < setup.crm ( which has upstart:mysql setting ) 3.3 monitor lrmd daemon's memory usage [Regression] Restarting daemon after upgrading this pkg will be needed. this patch added NULL check for several parts. prior commit[2] changed file structure ( e.g attrd created from tools/attrd.c ), This might affect to system. however, both of patches are landed to 1.11, so risk is not that high. [Others] Related commits. [1] commit a1a6922e43dfe80b23887a88401cbb93fe3645c0 Author: David Vossel Date: Tue Jan 7 14:02:18 2014 -0600 - Fix: ipc: fix memory leak for failed ipc client connections. - - When pacemaker ipc servers accept client connections, client - state data is allocated when the libqb 'accept' callback function is - invoked. It is possible however that even after the client - accepts the connection in the libqb 'accept' callback, the connection - could still fail to initialize fully if the client side hangs up. - - Currently this results in the client state data never being - destroyed because the libqb "close" callback for client connection - is never invoked on the server side. Instead, libqb jumps directly to - the "destroy" callback because the connection never actually got created. - - To account for this memory leak, pacemaker needs to verify client - state data is destroyed in the destroy callback. + Fix: ipc: fix memory leak for failed ipc client connections. + + When pacemaker ipc servers accept client connections, client + state data is allocated when the libqb 'accept' callback function is + invoked. It is possible however that even after the client + accepts the connection in the libqb 'accept' callback, the connection + could still fail to initialize fully if the client side hangs up. + + Currently this results in the client state data never being + destroyed because the libqb "close" callback for client connection + is never invoked on the server side. Instead, libqb jumps directly to + the "destroy" callback because the connection never actually got created. + + To account for this memory leak, pacemaker needs to verify client + state data is destroyed in the destroy callback. [2] commit a7b61e276120184c7586a3217ed3571a982f5017 Author: Andrew Beekhof Date: Fri Aug 23 16:25:35 2013 +1000 - Refactor: attrd: Move to its own directory and create a stub for + Refactor: attrd: Move to its own directory and create a stub for attrd-ng - [Original Description] I'm running Pacemaker 1.1.10+git20130802-1ubuntu1 on Ubuntu Saucy (13.10) and have encountered a memory leak in lrmd. The details of the bug are covered here in this thread (http://oss.clusterlabs.org/pipermail/pacemaker/2014-May/021689.html) but to summarise, the Pacemaker developers believe the leak is caused by the g_dbus API, the use of which was removed in Pacemaker 1.11. I've also attached the Valgrind output from the run that exposed the issue. Given that this issue affects production stability (a periodic restart of Pacemaker is required), will a version of 1.11 be released for Trusty? (I'm happy to upgrade the OS to Trusty to get it). If not, can you advise which version of the OS will be the first to take 1.11 please? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Patch added: "lp1316970_trusty.debdiff" https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+attachment/5031302/+files/lp1316970_trusty.debdiff ** Tags added: sts-sru-needed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
** Description changed: + [Impact] + lrmd daemon with upstart resource has memory leak in Trusty + + affected to pacemaker 1.1.10. + + + [Test Case] + + 1. deploy 3 trusty instance. + 2. install corosync, pacemaker, mysql. + 3. setting with below info + 3.1 corosync.conf ( ) + 3.2 crm configure < setup.crm ( ) + 3.3 monitor lrmd daemon's memory usage + + [Regression] + Restarting daemon after upgrading this pkg will be needed. this patch added NULL check for several parts. prior commit[2] changed file structure ( e.g attrd created from tools/attrd.c ), This might affect to system. however, both of patches are landed to 1.11, so risk is not that high. + + [Others] + + Related commits. + + [1] commit a1a6922e43dfe80b23887a88401cbb93fe3645c0 + Author: David Vossel + Date: Tue Jan 7 14:02:18 2014 -0600 + + Fix: ipc: fix memory leak for failed ipc client connections. + + When pacemaker ipc servers accept client connections, client + state data is allocated when the libqb 'accept' callback function is + invoked. It is possible however that even after the client + accepts the connection in the libqb 'accept' callback, the connection + could still fail to initialize fully if the client side hangs up. + + Currently this results in the client state data never being + destroyed because the libqb "close" callback for client connection + is never invoked on the server side. Instead, libqb jumps directly to + the "destroy" callback because the connection never actually got created. + + To account for this memory leak, pacemaker needs to verify client + state data is destroyed in the destroy callback. + [2] commit a7b61e276120184c7586a3217ed3571a982f5017 + Author: Andrew Beekhof + Date: Fri Aug 23 16:25:35 2013 +1000 + + Refactor: attrd: Move to its own directory and create a stub for + attrd-ng + + + [Original Description] + I'm running Pacemaker 1.1.10+git20130802-1ubuntu1 on Ubuntu Saucy (13.10) and have encountered a memory leak in lrmd. The details of the bug are covered here in this thread (http://oss.clusterlabs.org/pipermail/pacemaker/2014-May/021689.html) but to summarise, the Pacemaker developers believe the leak is caused by the g_dbus API, the use of which was removed in Pacemaker 1.11. I've also attached the Valgrind output from the run that exposed the issue. Given that this issue affects production stability (a periodic restart of Pacemaker is required), will a version of 1.11 be released for Trusty? (I'm happy to upgrade the OS to Trusty to get it). If not, can you advise which version of the OS will be the first to take 1.11 please? ** Tags added: sts -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1316970] Re: g_dbus memory leak in lrmd
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: pacemaker (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1316970 Title: g_dbus memory leak in lrmd To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/pacemaker/+bug/1316970/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs