Re: [Dovecot] Memory leak by getting the virtual size of a IMAP folder
On 6.9.2012, at 19.19, Dafan Zhai wrote: I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Yes, committed to hg.
Re: [Dovecot] Memory leak by getting the virtual size of a IMAP folder
Hi I think I have fixed this bug. The 'virtual_size_add_new' function in src/lib-storage/index/index-status.c creates a 'search_args' object and forgets to free it at the end. The function does call the 'mailbox_search_deinit' function but 'mailbox_search_deinit' only frees the 'search_ctx' object and reduce the refcount of 'search_args' by one but that doesn't cause 'search_args' to be freed because its refcount was 2. So an extra 'mail_search_args_unref' must be called. Timo, it would be nice if you could write a small comment if the attached patch is the right approach to address this issue. Dafan On 09/05/2012 10:53 AM, Dafan Zhai wrote: Hi everyone, I am writing a dovecot statistic plugin, which calls the 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as the 2nd parameter. enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; struct mailbox_metadata metadata; mailbox_get_metadata(mailbox, metadata_items, metadata); but Valgrind finds a memory leak when this function is called: ---snip-- ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 of 76 ==10304== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10304== by 0x416EE9F: block_alloc (mempool-alloconly.c:237) ==10304== by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) ==10304== by 0x409C07D: mail_search_build_init (mail-search-build.c:187) ==10304== by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) ==10304== by 0x4067102: maildir_mailbox_get_metadata (maildir-storage.c:486) ==10304== by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) ==10304== by 0x490492D: mailbox_status_update (statistic-plugin.c:310) ==10304== by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) ==10304== by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) ==10304== by 0x41A104F: notify_save_finish (notify-storage.c:143) ==10304== by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) ---snip-- I have looked into the source code, and found that from the 'mail_search_build_init' function call in lib-storage/index/index-status.c:200 a pool is created, but the pool is not freed in the mailbox_search_deinit function call in lib-storage/index/index-status.c:218. This may be the reason of the memory leak. But I do not know how to free the pool. It seems to me that doveadm is having the same memory leak because it is using 'mail_search_build_init' too. ---snip-- # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize INBOX ==10457== Memcheck, a memory error detector ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX ==10457== INBOX vsize=41643319 ==10457== ==10457== HEAP SUMMARY: ==10457== in use at exit: 4,356 bytes in 3 blocks ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated ==10457== ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==10457== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x41AF16C: pool_alloconly_create (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x40DC07D: mail_search_build_init (in /usr/lib/dovecot/libdovecot-storage.so.0.0.0) ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in /usr/bin/doveadm) ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) ==10457== by 0x805B863: main (in /usr/bin/doveadm) ==10457== ==10457== LEAK SUMMARY: ==10457== definitely lost: 4,096 bytes in 1 blocks ==10457== indirectly lost: 0 bytes in 0 blocks ==10457== possibly lost: 0 bytes in 0 blocks ==10457== still reachable: 260 bytes in 2 blocks ==10457== suppressed: 0 bytes in 0 blocks ==10457== Reachable blocks (those to which a pointer was found) are not shown. ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes ==10457== ==10457== For counts of detected and suppressed errors, rerun with: -v ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) ---snip-- Dovecot version: 2.1.9 OS: Linux 3.0.30-dist i686 maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX I have applied the following patches from the dovecot 2.1
[Dovecot] Memory leak by getting the virtual size of a IMAP folder
Hi everyone, I am writing a dovecot statistic plugin, which calls the 'mailbox_get_metadata' function with MAILBOX_METADATA_VIRTUAL_SIZE as the 2nd parameter. enum mailbox_status_items metadata_items = MAILBOX_METADATA_VIRTUAL_SIZE; struct mailbox_metadata metadata; mailbox_get_metadata(mailbox, metadata_items, metadata); but Valgrind finds a memory leak when this function is called: ---snip-- ==10304== 12,288 bytes in 3 blocks are definitely lost in loss record 74 of 76 ==10304==at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10304==by 0x416EE9F: block_alloc (mempool-alloconly.c:237) ==10304==by 0x416F16C: pool_alloconly_create (mempool-alloconly.c:140) ==10304==by 0x409C07D: mail_search_build_init (mail-search-build.c:187) ==10304==by 0x40C77F1: index_mailbox_get_metadata (index-status.c:200) ==10304==by 0x4067102: maildir_mailbox_get_metadata (maildir-storage.c:486) ==10304==by 0x409F78B: mailbox_get_metadata (mail-storage.c:1298) ==10304==by 0x490492D: mailbox_status_update (statistic-plugin.c:310) ==10304==by 0x4904E2E: statistic_mail_save (statistic-plugin.c:426) ==10304==by 0x41A054D: notify_contexts_mail_save (notify-plugin.c:61) ==10304==by 0x41A104F: notify_save_finish (notify-storage.c:143) ==10304==by 0x409EE94: mailbox_save_finish (mail-storage.c:1673) ---snip-- I have looked into the source code, and found that from the 'mail_search_build_init' function call in lib-storage/index/index-status.c:200 a pool is created, but the pool is not freed in the mailbox_search_deinit function call in lib-storage/index/index-status.c:218. This may be the reason of the memory leak. But I do not know how to free the pool. It seems to me that doveadm is having the same memory leak because it is using 'mail_search_build_init' too. ---snip-- # valgrind --leak-check=full doveadm mailbox status -u testuser1 vsize INBOX ==10457== Memcheck, a memory error detector ==10457== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al. ==10457== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info ==10457== Command: doveadm mailbox status -u testuser1 vsize INBOX ==10457== INBOX vsize=41643319 ==10457== ==10457== HEAP SUMMARY: ==10457== in use at exit: 4,356 bytes in 3 blocks ==10457== total heap usage: 440 allocs, 437 frees, 539,124 bytes allocated ==10457== ==10457== 4,096 bytes in 1 blocks are definitely lost in loss record 3 of 3 ==10457== at 0x40222A4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so) ==10457== by 0x41AEE9F: ??? (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x41AF16C: pool_alloconly_create (in /usr/lib/dovecot/libdovecot.so.0.0.0) ==10457== by 0x40DC07D: mail_search_build_init (in /usr/lib/dovecot/libdovecot-storage.so.0.0.0) ==10457== by 0x8057EB8: doveadm_mail_mailbox_search_args_build (in /usr/bin/doveadm) ==10457== by 0x805855A: ??? (in /usr/bin/doveadm) ==10457== by 0x8053A7B: doveadm_mail_single_user (in /usr/bin/doveadm) ==10457== by 0x8053D2B: ??? (in /usr/bin/doveadm) ==10457== by 0x80541CF: doveadm_mail_try_run (in /usr/bin/doveadm) ==10457== by 0x805B863: main (in /usr/bin/doveadm) ==10457== ==10457== LEAK SUMMARY: ==10457== definitely lost: 4,096 bytes in 1 blocks ==10457== indirectly lost: 0 bytes in 0 blocks ==10457== possibly lost: 0 bytes in 0 blocks ==10457== still reachable: 260 bytes in 2 blocks ==10457== suppressed: 0 bytes in 0 blocks ==10457== Reachable blocks (those to which a pointer was found) are not shown. ==10457== To see them, rerun with: --leak-check=full --show-reachable=yes ==10457== ==10457== For counts of detected and suppressed errors, rerun with: -v ==10457== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 122 from 35) ---snip-- Dovecot version: 2.1.9 OS: Linux 3.0.30-dist i686 maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX I have applied the following patches from the dovecot 2.1 branch fixing memory leaks: http://hg.dovecot.org/dovecot-2.1/rev/7bdbca7b0913 http://hg.dovecot.org/dovecot-2.1/rev/963482677c0b It would be nice if you could give me some hints how I can address this issue? Thanks in advance! Greetings, Dafan
