Thanks for your email. What do you mean by "just [using] hosts names"? Are these in your /etc/hosts file, or are you using some other mechanism?
Please let me know. Matt From: B Ray [mailto:[email protected]] Sent: 23 May 2013 14:58 To: Matt Williams Subject: Re: [Clearwater] Registration logging Gmail is not letting me attach the file, so I've dumped a section of it below. It just repeats this over and over. One note is I'm not using DNS, just hosts names. ======================================================== 23-05-2013 13:55:00.679 Status stack.cpp: Listening on port 5060 23-05-2013 13:55:00.679 Status stack.cpp: Local host aliases: 23-05-2013 13:55:00.683 Status stack.cpp: bono.us-west-2.compute.amazonaws.com<http://bono.us-west-2.compute.amazonaws.com> 23-05-2013 13:55:00.683 Status stack.cpp: 10.254.52.157 23-05-2013 13:55:00.683 Status stack.cpp: 127.0.0.1 23-05-2013 13:55:00.683 Status stack.cpp: localhost 23-05-2013 13:55:00.683 Status stack.cpp: --trusted-port 23-05-2013 13:55:00.684 Status main.cpp: Using local store 23-05-2013 13:55:00.685 Status connection_pool.cpp: Creating connection pool to sprout.us-west-2.compute.amazonaws.com:0<http://sprout.us-west-2.compute.amazonaws.com:0> 23-05-2013 13:55:00.685 Status connection_pool.cpp: connections = 50, recycle time = 600 seconds Signal 11 caught Basic stack dump: /usr/share/clearwater/bin/sprout[0x41bb6d] /usr/share/clearwater/bin/sprout[0x47ab1f] /usr/share/clearwater/bin/sprout[0x482b69] /lib/x86_64-linux-gnu/libc.so.6(+0x364a0)[0x7f38b56f44a0] /usr/share/clearwater/bin/sprout[0x4297b9] /usr/share/clearwater/bin/sprout[0x429985] /usr/share/clearwater/bin/sprout[0x425b53] /usr/share/clearwater/bin/sprout[0x41ad17] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f38b56df76d] /usr/share/clearwater/bin/sprout[0x41ba29] Advanced stack dump (requires gdb): [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7f38ada24700 (LWP 17681)] [New Thread 0x7f38ae225700 (LWP 17680)] [New Thread 0x7f38aea26700 (LWP 17679)] [New Thread 0x7f38af227700 (LWP 17678)] [New Thread 0x7f38afa28700 (LWP 17677)] 0x00007f38b577d47d in waitpid () from /lib/x86_64-linux-gnu/libc.so.6 Thread 6 (Thread 0x7f38afa28700 (LWP 17677)): #0 0x00007f38b57b2353 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f38b6927e77 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #2 0x00007f38b6944196 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #3 0x00007f38b64f4e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #4 0x00007f38b57b1cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x0000000000000000 in ?? () Thread 5 (Thread 0x7f38af227700 (LWP 17678)): #0 0x00007f38b57b2353 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f38b6927e77 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #2 0x00007f38b6944196 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #3 0x00007f38b64f4e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #4 0x00007f38b57b1cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x0000000000000000 in ?? () Thread 4 (Thread 0x7f38aea26700 (LWP 17679)): #0 0x00007f38b57a6303 in poll () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f38b694b7d6 in zmq_poll () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #2 0x000000000047e15a in LastValueCache::run (this=0x459ca10) at zmq_lvc.cpp:151 #3 0x000000000047e829 in LastValueCache::last_value_cache_entry_func (lvc=<optimized out>) at zmq_lvc.cpp:286 #4 0x00007f38b64f4e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x00007f38b57b1cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #6 0x0000000000000000 in ?? () Thread 3 (Thread 0x7f38ae225700 (LWP 17680)): #0 0x00007f38b64f8d84 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x000000000047bb39 in pop (item=..., this=0x45a3560) at /var/lib/jenkins/workspace/sprout-ubuntu-master/include/eventq.h:1 57 #2 Statistic::reporter (this=0x45a3548) at statistic.cpp:93 #3 0x000000000047bd69 in Statistic::reporter_thread (p=<optimized out>) at statistic.cpp:144 #4 0x00007f38b64f4e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x00007f38b57b1cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #6 0x0000000000000000 in ?? () Thread 2 (Thread 0x7f38ada24700 (LWP 17681)): #0 0x00007f38b64f8d84 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x000000000047bb39 in pop (item=..., this=0x45a3b40) at /var/lib/jenkins/workspace/sprout-ubuntu-master/include/eventq.h:1 57 #2 Statistic::reporter (this=0x45a3b28) at statistic.cpp:93 #3 0x000000000047bd69 in Statistic::reporter_thread (p=<optimized out>) at statistic.cpp:144 #4 0x00007f38b64f4e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x00007f38b57b1cbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #6 0x0000000000000000 in ?? () Thread 1 (Thread 0x7f38b7d1e740 (LWP 17676)): #0 0x00007f38b577d47d in waitpid () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f38b570329e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007f38b57036b0 in system () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x000000000041bc23 in Logger::backtrace (this=0x240a5f0, data=<optimized out>) at logger.cpp:190 #4 0x000000000047ab1f in Log::backtrace (fmt=<optimized out>) at log.cpp:129 #5 0x0000000000482b69 in exception_handler (sig=11) at main.cpp:404 #6 <signal handler called> #7 0x00000000004297b9 in ConnectionPool::create_connection (this=0x45a3a70, hash_slot=0) at connection_pool.cpp:208 #8 0x0000000000429985 in ConnectionPool::init (this=0x45a3a70) at connection_pool.cpp:100 #9 0x0000000000425b53 in init_stateful_proxy (registrar_store=<optimized out>, call_services=<optimized out>, ifc_handler_in= <optimized out>, enable_edge_proxy=<optimized out>, edge_upstream_proxy=..., edge_upstream_proxy_connections=50, edge_upstream _proxy_recycle=600, enable_ibcf=0, ibcf_trusted_hosts=..., analytics=0x240a4f0, enumService=0x0, bgcfService=0x0) at stateful_ proxy.cpp:2820 #10 0x000000000041ad17 in main (argc=<optimized out>, argv=<optimized out>) at main.cpp:628 23-05-2013 13:55:11.216 Status stack.cpp: Listening on port 5060 23-05-2013 13:55:11.217 Status stack.cpp: Local host aliases: 23-05-2013 13:55:11.221 Status stack.cpp: bono.us-west-2.compute.amazonaws.com<http://bono.us-west-2.compute.amazonaws.com> 23-05-2013 13:55:11.221 Status stack.cpp: 10.254.52.157 23-05-2013 13:55:11.221 Status stack.cpp: 127.0.0.1 23-05-2013 13:55:11.221 Status stack.cpp: localhost 23-05-2013 13:55:11.221 Status stack.cpp: --trusted-port 23-05-2013 13:55:11.223 Status main.cpp: Using local store 23-05-2013 13:55:11.223 Status connection_pool.cpp: Creating connection pool to sprout.us-west-2.compute.amazonaws.com:0<http://sprout.us-west-2.compute.amazonaws.com:0> 23-05-2013 13:55:11.223 Status connection_pool.cpp: connections = 50, recycle time = 600 seconds Signal 11 caught Basic stack dump: /usr/share/clearwater/bin/sprout[0x41bb6d] /usr/share/clearwater/bin/sprout[0x47ab1f] /usr/share/clearwater/bin/sprout[0x482b69] /lib/x86_64-linux-gnu/libc.so.6(+0x364a0)[0x7f57243514a0] /usr/share/clearwater/bin/sprout[0x4297b9] /usr/share/clearwater/bin/sprout[0x429985] /usr/share/clearwater/bin/sprout[0x425b53] /usr/share/clearwater/bin/sprout[0x41ad17] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f572433c76d] /usr/share/clearwater/bin/sprout[0x41ba29] Advanced stack dump (requires gdb): [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7f571c681700 (LWP 17741)] [New Thread 0x7f571ce82700 (LWP 17740)] [New Thread 0x7f571d683700 (LWP 17739)] [New Thread 0x7f571de84700 (LWP 17738)] [New Thread 0x7f571e685700 (LWP 17737)] 0x00007f57243da47d in waitpid () from /lib/x86_64-linux-gnu/libc.so.6 Thread 6 (Thread 0x7f571e685700 (LWP 17737)): #0 0x00007f572440f353 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f5725584e77 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #2 0x00007f57255a1196 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #3 0x00007f5725151e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #4 0x00007f572440ecbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x0000000000000000 in ?? () Thread 5 (Thread 0x7f571de84700 (LWP 17738)): #0 0x00007f572440f353 in epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f5725584e77 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #2 0x00007f57255a1196 in ?? () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #3 0x00007f5725151e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #4 0x00007f572440ecbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x0000000000000000 in ?? () Thread 4 (Thread 0x7f571d683700 (LWP 17739)): #0 0x00007f5724403303 in poll () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f57255a87d6 in zmq_poll () from /usr/lib/x86_64-linux-gnu/libzmq.so.3 #2 0x000000000047e15a in LastValueCache::run (this=0x3a80a10) at zmq_lvc.cpp:151 #3 0x000000000047e829 in LastValueCache::last_value_cache_entry_func (lvc=<optimized out>) at zmq_lvc.cpp:286 #4 0x00007f5725151e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x00007f572440ecbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #6 0x0000000000000000 in ?? () Thread 3 (Thread 0x7f571ce82700 (LWP 17740)): #0 0x00007f5725155d84 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x000000000047bb39 in pop (item=..., this=0x3a87560) at /var/lib/jenkins/workspace/sprout-ubuntu-master/include/eventq.h:1 57 #2 Statistic::reporter (this=0x3a87548) at statistic.cpp:93 #3 0x000000000047bd69 in Statistic::reporter_thread (p=<optimized out>) at statistic.cpp:144 #4 0x00007f5725151e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x00007f572440ecbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #6 0x0000000000000000 in ?? () Thread 2 (Thread 0x7f571c681700 (LWP 17741)): #0 0x00007f5725155d84 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x000000000047bb39 in pop (item=..., this=0x3a87b40) at /var/lib/jenkins/workspace/sprout-ubuntu-master/include/eventq.h:1 57 #2 Statistic::reporter (this=0x3a87b28) at statistic.cpp:93 #3 0x000000000047bd69 in Statistic::reporter_thread (p=<optimized out>) at statistic.cpp:144 #4 0x00007f5725151e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x00007f572440ecbd in clone () from /lib/x86_64-linux-gnu/libc.so.6 #6 0x0000000000000000 in ?? () Thread 1 (Thread 0x7f572697b740 (LWP 17736)): #0 0x00007f57243da47d in waitpid () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f572436029e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007f57243606b0 in system () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x000000000041bc23 in Logger::backtrace (this=0x18ee5f0, data=<optimized out>) at logger.cpp:190 #4 0x000000000047ab1f in Log::backtrace (fmt=<optimized out>) at log.cpp:129 #5 0x0000000000482b69 in exception_handler (sig=11) at main.cpp:404 #6 <signal handler called> #7 0x00000000004297b9 in ConnectionPool::create_connection (this=0x3a87a70, hash_slot=0) at connection_pool.cpp:208 #8 0x0000000000429985 in ConnectionPool::init (this=0x3a87a70) at connection_pool.cpp:100 #9 0x0000000000425b53 in init_stateful_proxy (registrar_store=<optimized out>, call_services=<optimized out>, ifc_handler_in= <optimized out>, enable_edge_proxy=<optimized out>, edge_upstream_proxy=..., edge_upstream_proxy_connections=50, edge_upstream _proxy_recycle=600, enable_ibcf=0, ibcf_trusted_hosts=..., analytics=0x18ee4f0, enumService=0x0, bgcfService=0x0) at stateful_ proxy.cpp:2820 #10 0x000000000041ad17 in main (argc=<optimized out>, argv=<optimized out>) at main.cpp:628 On Wed, May 22, 2013 at 5:14 PM, Matt Williams <[email protected]<mailto:[email protected]>> wrote: That's odd - the /var/log/bono/sprout*.log shows that it's cyclically crashing - you can see the stack dump (without symbols). Please can you "apt-get install bono-dbg" so that we get debug symbols in the stack, and then post the new stack dump? Thanks, Matt ________________________________________ From: [email protected]<mailto:[email protected]> [[email protected]<mailto:[email protected]>] on behalf of B Ray [[email protected]<mailto:[email protected]>] Sent: 22 May 2013 22:11 To: [email protected]<mailto:[email protected]> Subject: Re: [Clearwater] Registration logging I can see the attempts to register hitting port 5060. However, the bono process is not running. And I don't see any process listening on port 5060/sip =============From the log=================== 22-05-2013 21:01:59.184 Status stack.cpp: Listening on port 5060 22-05-2013 21:01:59.184 Status stack.cpp: Local host aliases: 22-05-2013 21:01:59.184 Status stack.cpp: bono.us-west-2.compute.amazonaws.com<http://bono.us-west-2.compute.amazonaws.com><http://bono.us-west-2.compute.amazonaws.com> 22-05-2013 21:01:59.184 Status stack.cpp: 10.254.52.157 22-05-2013 21:01:59.184 Status stack.cpp: 127.0.0.1 22-05-2013 21:01:59.184 Status stack.cpp: localhost 22-05-2013 21:01:59.184 Status stack.cpp: --trusted-port 22-05-2013 21:01:59.185 Status main.cpp: Using local store 22-05-2013 21:01:59.185 Status connection_pool.cpp: Creating connection pool to sprout.us-west-2.compute .amazonaws.com:0<http://amazonaws.com:0><http://amazonaws.com:0> 22-05-2013 21:01:59.185 Status connection_pool.cpp: connections = 50, recycle time = 600 seconds Signal 11 caught Basic stack dump: /usr/share/clearwater/bin/sprout[0x41bb6d] /usr/share/clearwater/bin/sprout[0x47ab1f] /usr/share/clearwater/bin/sprout[0x482b69] /lib/x86_64-linux-gnu/libc.so.6(+0x364a0)[0x7fdf2de064a0] /usr/share/clearwater/bin/sprout[0x4297b9] /usr/share/clearwater/bin/sprout[0x429985] /usr/share/clearwater/bin/sprout[0x425b53] /usr/share/clearwater/bin/sprout[0x41ad17] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7fdf2ddf176d] /usr/share/clearwater/bin/sprout[0x41ba29] Advanced stack dump (requires gdb): sh: 1: /usr/bin/gdb: not found gdb failed with return code 32512 ============================================ [bono]ubuntu@ec2-54-214-142-56:/var/log/bono$ sudo monit status The Monit daemon 5.3.2 uptime: 4m Process 'restund' status Running monitoring status Monitored pid 24799 parent pid 1 uptime 4m children 0 memory kilobytes 1312 memory kilobytes total 1312 memory percent 0.0% memory percent total 0.0% cpu percent 0.0% cpu percent total 0.0% port response time 0.000s to 10.254.52.157:3478<http://10.254.52.157:3478><http://10.254.52.157:3478> [generic via UDP] data collected Wed, 22 May 2013 21:02:24 Process 'bono' status Connection failed monitoring status Monitored data collected Wed, 22 May 2013 21:02:24 System 'system_ec2-54-214-142-56' status Running monitoring status Monitored load average [1.02] [0.89] [0.81] cpu 29.5%us 10.0%sy 0.3%wa memory usage 87664 kB [5.1%] swap usage 0 kB [0.0%] data collected Wed, 22 May 2013 21:02:24 [bono]ubuntu@ec2-54-214-142-56:/etc/monit/conf.d$<mailto:ubuntu@ec2-54-214-142-56:/etc/monit/conf.d$> netstat -l Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 bono:2812 *:* LISTEN tcp 0 0 localhost:http-alt *:* LISTEN tcp 0 0 *:domain *:* LISTEN tcp 0 0 bono:3478 *:* LISTEN tcp 0 0 *:ssh *:* LISTEN tcp6 0 0 [::]:domain [::]:* LISTEN tcp6 0 0 [::]:ssh [::]:* LISTEN udp 0 0 localhost:33000 *:* udp 0 0 bono:3478 *:* udp 0 0 *:domain *:* udp 0 0 *:bootpc *:* udp 0 0 bono:ntp *:* udp 0 0 localhost:ntp *:* udp 0 0 *:ntp *:* udp6 0 0 [::]:domain [::]:* udp6 0 0 ip6-localhost:ntp [::]:* udp6 0 0 fe80::2000:aff:fefe:ntp [::]:* udp6 0 0 [::]:ntp [::]:* On Wed, May 22, 2013 at 1:20 PM, B Ray <[email protected]<mailto:[email protected]><mailto:[email protected]<mailto:[email protected]>>> wrote: I have successfully created a few users and assigned numbers to them. In attempting to register SIP subscribers, where can I see if these attempts are making it to the server?
_______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
