Hi Alex,
Any ideas on how to remove the memory issues for smsbox?
Rgds
----------------------------------------------------------------------
Date: Sat, 3 May 2014 10:06:54 +0700
From: Hanh Le Bich <hanhmi...@gmail.com>
To: hbil...@ecommunicate.biz
Cc: devel@kannel.org
Subject: Re: Does opensmppbox and smsbox have serious memory issues?
Message-ID:
<CA+p4DGxn78_ErwKii=yrpgrouxe9bflmc8om757yju41up8...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hello again,
Here is valgrind report for the smsbox memory leaking. This is also the
latest kannel revision 5089.
For me, it still happen, i submit (broadcast) million SMS(s) per day within
few hours at 1000 TPS of throughput. Thus each day, smsbox swallow ~ 400MB
RAM and not release until server memory exhaust after few days. At this
time, the box is auto restarted and i have my memory back, so on, again and
again.
I not sure, but in very low throughput system, i am not aware this issue.
==5730== LEAK SUMMARY:
==5730== definitely lost: 231,680 bytes in 14,480 blocks
==5730== indirectly lost: 144,790 bytes in 14,479 blocks
==5730== possibly lost: 0 bytes in 0 blocks
==5730== still reachable: 1,250 bytes in 40 blocks
==5730== suppressed: 0 bytes in 0 blocks
==5730== Reachable blocks (those to which a pointer was found) are not
shown.
==5730== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==5730== ==5730== For counts of detected and suppressed errors, rerun with:
-v ==5730== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 45 from 10)
Please find the report in the attached also this link:
https://www.dropbox.com/s/yjrc6wmdm0rwonz/smsbox_memleakcheck_20140503.txt
Regards, Hanh.
On Fri, May 2, 2014 at 9:33 PM, <hbil...@ecommunicate.biz> wrote:
> Hi Hand,
>
>
>
> Did you see the latest change by Rene committed to SVN (
> http://www.kannel.org/pipermail/devel/2014-April/005612.html)
>
> Does it fix the memory issue you reported?
>
>
>
> rgds
>
>
>
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ----
>
> Kind Regards
>
>
>
> Hillel Bilman
>
> Manager eCommunicate
>
> mailto: hbil...@ecommunicate.co.za
>
> Cell: 083-2300002
>
> Landline: 011-443-6164
>
> Fax: 088-011-443-6164
>
>
>
> Social Media marketing campaigns (Facebook, Google+, Twitter, LinkedIn
> etc) ? Interactive Websites - .mobi Sites ? Mobile Apps(Android,
> iPhone, Blackberry, Nokia) - Premium Rated SMSs and short codes - SMS
> competitions and campaigns ? Lead Generation - opt-in subscription
> Billing ? MMS campaigns - USSD campaigns - WAP - Outlook SMS ? Bulk SMS
and Bulk Email ?
> Email 2 SMS 2 Email - Developer Kit for Mobile Services integration -
> Voice Over IP services
>
>
>
> *From:* Hanh Le Bich [mailto:hanhmi...@gmail.com]
> *Sent:* 29 April 2014 12:25 PM
> *To:* Alexander Malysh
> *Cc:* hbil...@ecommunicate.biz; devel@kannel.org
> *Subject:* Re: Does opensmppbox and smsbox have serious memory issues?
>
>
>
> It's happy to get it now. Thank a lot.
>
> Regards,
> Tuan.
>
>
>
> On Tue, Apr 29, 2014 at 4:02 PM, Alexander Malysh <amal...@kannel.org>
> wrote:
>
> Hi,
>
>
>
> just checked daily snapshot, yes you can use it, this is uptodate.
>
>
>
> Alex
>
>
>
> Am 29.04.2014 um 04:44 schrieb Hanh Le Bich <hanhmi...@gmail.com>:
>
>
>
> Dear hbilman & development team,
>
> I'm willing to help to provide more evident but i have no background
> to work in IT fields.
>
> Cause my server has no internet connection thus i cannot get the
> lastest SVN trunk. Normally i download source files via the daily
> snapshot, is it ok?
>
> Regards, Hanh.
>
>
>
> On Mon, Apr 28, 2014 at 12:25 AM, <hbil...@ecommunicate.biz> wrote:
>
> Hi Kannel developers,
>
> Hanh posted his Valgrind research to the user group for smsbox and
> opensmppbox. His results seem interesting and so I'm copying them to
> this thread so the Kannel developers can view them.
> These results can be viewed by following the thread on Wed, Apr 23,
> 2014 at
> 3:41 AM, by Hanh Le Bich <hanhmi...@gmail.com> with the Subject: Re: 2
> Questions re Redis/Debian. (The email subject is not related to this
> issue.)
>
> His research shows that opensmppbox and smsbox may have serious memory
> issues.
> I use the word "may" as until others have confirmed his results, there
> could be a mistake somewhere.
> Is there anyone who has a test environment that can follow his
> approach and confirm for the Kannel community if opensmppbox and
> smsbox have serious memory issues or if his testing approach has flaws?
>
> His approach is:
> >> Let me describe a little bit for my application back end. It's
> >> pretty
> >> simple: i make a loop that for each second, it push an sms via
> >> kannel CGI for 1K mobile numbers, that mean throughput is 1000 msg/sec.
> >> My kannel configuration is simple too, it's only smsbox ->
> >> bearerbox
> >> -> SMSC (via smpp), no file storage, no SQL, no dlr (actually
> dlr-mask=8).
> > I even don't expect the sms can deliver > to all end users and the
> > app
> run some hours per day only. That why i
> > can play with the lasted SVN which don't care so much for the
> reliability.
>
> For smsbox:
> >>In the pass when using ver 1.4.3, it was fine for years. After
> >>upgrade to 1.5.0, after each few days, i realized smsbox is reset,
> >>then i found it exhaust my memory. It's funny that smsbox consume
> >>the mem and doesn't release. Example, if it occupies 50% your mem
> >>and you stop sms pushing, it will 50% forever except the box
restarting.
> >> That's all, same server with no other tasks, same back end, just
> >>different kannel version.
> >>
> >> Just paste the valgrind sum in here:
> >>
> >> ==27581== LEAK SUMMARY:
> >> ==27581== definitely lost: 1,077,904 bytes in 67,369 blocks
> >> ==27581== indirectly lost: 673,660 bytes in 67,366 blocks
> >> ==27581== possibly lost: 160 bytes in 13 blocks
> >> ==27581== still reachable: 1,240 bytes in 39 blocks
> >> ==27581== suppressed: 0 bytes in 0 blocks
> >> ==27581== Reachable blocks (those to which a pointer was found) are
> >> not shown.
> >> ==27581== To see them, rerun with: --leak-check=full
> >> --show-leak-kinds=all ==27581== ==27581== For counts of detected
> >> and suppressed errors, rerun with: -v ==27581== ERROR SUMMARY: 3
> >> errors from 3 contexts (suppressed: 45 from 10)
>
> For opensmppbox
> >opensmppbox drains your memory 10 times faster than smsbox
> ==31087== Memcheck, a memory error detector ==31087== Copyright (C)
> 2002-2013, and GNU GPL'd, by Julian Seward et al.
> ==31087== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright
> info ==31087== Command: /usr/local/sbin/opensmppbox -v -d --
> /etc/kannel/opensmppbox.conf ==31087== Parent PID: 31085 ==31087==
> ==31087== ==31087== HEAP SUMMARY:
> ==31087== in use at exit: 8,163,073 bytes in 36,550 blocks
> ==31087== total heap usage: 893,827 allocs, 857,277 frees, 295,662,079
> bytes allocated
> ==31087==
> ==31087== 49 (32 direct, 17 indirect) bytes in 2 blocks are definitely
> lost in loss record 485 of 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x80970B3: gw_native_malloc (gwmem-native.c:87)
> ==31087== by 0x80A37A1: octstr_create_from_data_real (octstr.c:263)
> ==31087== by 0x80A3D81: octstr_duplicate_real (octstr.c:377)
> ==31087== by 0x808B106: cfg_get_real (cfg.c:670)
> ==31087== by 0x8052769: main (opensmppbox.c:2291)
> ==31087==
> ==31087== 79 (64 direct, 15 indirect) bytes in 4 blocks are definitely
> lost in loss record 529 of 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x80970B3: gw_native_malloc (gwmem-native.c:87)
> ==31087== by 0x80A37A1: octstr_create_from_data_real (octstr.c:263)
> ==31087== by 0x80A3D81: octstr_duplicate_real (octstr.c:377)
> ==31087== by 0x805D52F: msg_duplicate (msg-decl.h:80)
> ==31087== by 0x805651D: catenate_msg (opensmppbox.c:525)
> ==31087== by 0x805686B: check_multipart (opensmppbox.c:1481)
> ==31087== by 0x8057AD8: smpp_to_bearerbox (opensmppbox.c:1639)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087== by 0x482F78D: clone (clone.S:130)
> ==31087==
> ==31087== 100 (80 direct, 20 indirect) bytes in 5 blocks are
> definitely lost in loss record 577 of 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x80970B3: gw_native_malloc (gwmem-native.c:87)
> ==31087== by 0x80A37A1: octstr_create_from_data_real (octstr.c:263)
> ==31087== by 0x80A3D81: octstr_duplicate_real (octstr.c:377)
> ==31087== by 0x805D6FD: msg_duplicate (msg-decl.h:80)
> ==31087== by 0x805651D: catenate_msg (opensmppbox.c:525)
> ==31087== by 0x805686B: check_multipart (opensmppbox.c:1481)
> ==31087== by 0x8057AD8: smpp_to_bearerbox (opensmppbox.c:1639)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087== by 0x482F78D: clone (clone.S:130)
> ==31087==
> ==31087== 144 bytes in 1 blocks are possibly lost in loss record 610 of
813
> ==31087== at 0x402574D: calloc (vg_replace_malloc.c:618)
> ==31087== by 0x40111FB: _dl_allocate_tls (dl-tls.c:300)
> ==31087== by 0x46FA5A0: pthread_create@@GLIBC_2.1 (allocatestack.c:580)
> ==31087== by 0x4412F2A: my_thread_global_init (in
> /usr/lib/i386-linux-gnu/libmysqlclient.so.18.0.0)
> ==31087== by 0x44112F7: my_init (in
> /usr/lib/i386-linux-gnu/libmysqlclient.so.18.0.0)
> ==31087== by 0x43EC93A: mysql_server_init (in
> /usr/lib/i386-linux-gnu/libmysqlclient.so.18.0.0)
> ==31087== by 0x43EE078: mysql_init (in
> /usr/lib/i386-linux-gnu/libmysqlclient.so.18.0.0)
> ==31087== by 0x8094F42: mysql_open_conn (dbpool_mysql.c:84)
> ==31087== by 0x80952C5: dbpool_increase (dbpool.c:194)
> ==31087== by 0x80953F6: dbpool_create (dbpool.c:160)
> ==31087== by 0x805B2B3: dlr_init_mysql (dlr_mysql.c:452)
> ==31087== by 0x8058AF3: dlr_init (dlr.c:254)
> ==31087==
> ==31087== 400 (256 direct, 144 indirect) bytes in 16 blocks are
> definitely lost in loss record 680 of 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x80970B3: gw_native_malloc (gwmem-native.c:87)
> ==31087== by 0x80A37A1: octstr_create_from_data_real (octstr.c:263)
> ==31087== by 0x80A3D81: octstr_duplicate_real (octstr.c:377)
> ==31087== by 0x805474D: run_smppbox (opensmppbox.c:2105)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087== by 0x482F78D: clone (clone.S:130)
> ==31087==
> ==31087== 2,160 bytes in 15 blocks are possibly lost in loss record
> 755 of
> 813
> ==31087== at 0x402574D: calloc (vg_replace_malloc.c:618)
> ==31087== by 0x40111FB: _dl_allocate_tls (dl-tls.c:300)
> ==31087== by 0x46FA5A0: pthread_create@@GLIBC_2.1 (allocatestack.c:580)
> ==31087== by 0x8097F4F: gwthread_create_real (gwthread-pthread.c:475)
> ==31087== by 0x80547B3: run_smppbox (opensmppbox.c:2124)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087== by 0x482F78D: clone (clone.S:130)
> ==31087==
> ==31087== 2,160 bytes in 15 blocks are possibly lost in loss record
> 756 of
> 813
> ==31087== at 0x402574D: calloc (vg_replace_malloc.c:618)
> ==31087== by 0x40111FB: _dl_allocate_tls (dl-tls.c:300)
> ==31087== by 0x46FA5A0: pthread_create@@GLIBC_2.1 (allocatestack.c:580)
> ==31087== by 0x8097F4F: gwthread_create_real (gwthread-pthread.c:475)
> ==31087== by 0x8052F97: main (opensmppbox.c:2156)
> ==31087==
> ==31087== 3,020 (1,040 direct, 1,980 indirect) bytes in 13 blocks are
> definitely lost in loss record 762 of 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x80970B3: gw_native_malloc (gwmem-native.c:87)
> ==31087== by 0x80A0DEF: gwlist_create_real (list.c:131)
> ==31087== by 0x805FC8C: sms_split (sms.c:339)
> ==31087== by 0x805517E: run_smppbox (opensmppbox.c:1008)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087== by 0x482F78D: clone (clone.S:130)
> ==31087==
> ==31087== 44,032 bytes in 43 blocks are possibly lost in loss record
> 798 of
> 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x40275BA: realloc (vg_replace_malloc.c:687)
> ==31087== by 0x809722B: gw_native_realloc (gwmem-native.c:115)
> ==31087== by 0x80A35EB: octstr_grow (octstr.c:192)
> ==31087== by 0x80A64B3: octstr_insert_data (octstr.c:1469)
> ==31087== by 0x80A672A: octstr_append_data (octstr.c:1499)
> ==31087== by 0x80A90F9: octstr_format_valist_real (octstr.c:2486)
> ==31087== by 0x80A9366: octstr_format (octstr.c:2469)
> ==31087== by 0x80534F5: boxc_route_msg_to_smsc (opensmppbox.c:1791)
> ==31087== by 0x8057AAE: smpp_to_bearerbox (opensmppbox.c:1638)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087==
> ==31087== 4,986,528 (77,472 direct, 4,909,056 indirect) bytes in 4,842
> blocks are definitely lost in loss record 813 of 813
> ==31087== at 0x4027434: malloc (vg_replace_malloc.c:291)
> ==31087== by 0x80970B3: gw_native_malloc (gwmem-native.c:87)
> ==31087== by 0x80A37A1: octstr_create_from_data_real (octstr.c:263)
> ==31087== by 0x80A3916: octstr_create_real (octstr.c:245)
> ==31087== by 0x80A908E: octstr_format_valist_real (octstr.c:2480)
> ==31087== by 0x80A9366: octstr_format (octstr.c:2469)
> ==31087== by 0x80534F5: boxc_route_msg_to_smsc (opensmppbox.c:1791)
> ==31087== by 0x8057AAE: smpp_to_bearerbox (opensmppbox.c:1638)
> ==31087== by 0x80983AE: new_thread (gwthread-pthread.c:385)
> ==31087== by 0x46F9C38: start_thread (pthread_create.c:304)
> ==31087== by 0x482F78D: clone (clone.S:130)
> ==31087==
> ==31087== LEAK SUMMARY:
> ==31087== definitely lost: 78,944 bytes in 4,882 blocks
> ==31087== indirectly lost: 4,911,232 bytes in 4,859 blocks
> ==31087== possibly lost: 48,496 bytes in 74 blocks
> ==31087== still reachable: 3,124,401 bytes in 26,735 blocks
> ==31087== suppressed: 0 bytes in 0 blocks
> ==31087== Reachable blocks (those to which a pointer was found) are
> not shown.
> ==31087== To see them, rerun with: --leak-check=full
> --show-leak-kinds=all ==31087== ==31087== For counts of detected and
> suppressed errors, rerun
> with: -v ==31087== ERROR SUMMARY: 10 errors from 10 contexts (suppressed:
> 45
> from 10)
>
> Regards
>
>
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.kannel.org/pipermail/devel/attachments/20140503/94ee18a2/attachm
ent.html>
-------------- next part --------------
==5730== Memcheck, a memory error detector ==5730== Copyright (C) 2002-2013,
and GNU GPL'd, by Julian Seward et al.
==5730== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==5730== Command: /usr/local/sbin/smsbox -v -d -- /etc/kannel/kannel.conf
==5730== Parent PID: 5728 ==5730== ==5730== ==5730== HEAP SUMMARY:
==5730== in use at exit: 377,720 bytes in 28,999 blocks
==5730== total heap usage: 2,999,707 allocs, 2,970,708 frees, 399,074,706
bytes allocated
==5730==
==5730== 376,470 (231,680 direct, 144,790 indirect) bytes in 14,480 blocks
are definitely lost in loss record 42 of 42
==5730== at 0x4027434: malloc (vg_replace_malloc.c:291)
==5730== by 0x8081BD3: gw_native_malloc (gwmem-native.c:87)
==5730== by 0x808F091: octstr_create_from_data_real (octstr.c:263)
==5730== by 0x808F671: octstr_duplicate_real (octstr.c:377)
==5730== by 0x8056966: send_message (smsbox.c:344)
==5730== by 0x805C4D4: smsbox_req_handle (smsbox.c:2303)
==5730== by 0x805D95D: sendsms_thread (smsbox.c:2558)
==5730== by 0x8082C9E: new_thread (gwthread-pthread.c:385)
==5730== by 0x4415C38: start_thread (pthread_create.c:304)
==5730== by 0x482FD4D: clone (clone.S:130)
==5730==
==5730== LEAK SUMMARY:
==5730== definitely lost: 231,680 bytes in 14,480 blocks
==5730== indirectly lost: 144,790 bytes in 14,479 blocks
==5730== possibly lost: 0 bytes in 0 blocks
==5730== still reachable: 1,250 bytes in 40 blocks
==5730== suppressed: 0 bytes in 0 blocks
==5730== Reachable blocks (those to which a pointer was found) are not
shown.
==5730== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==5730== ==5730== For counts of detected and suppressed errors, rerun with:
-v ==5730== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 45 from 10)
