Re: [vpp-dev] VPP 2106 Main thread is 99% CPU

2022-05-22 Thread chetan bhasin
HI,

As per the GDB back-trace coming with *214 frames*


Program received signal SIGINT, Interrupt.
0x7fa6d0f1fca4 in clib_time_now_internal (n=35626371070243907,
c=0x7fa30edf8680) at /vpp/src/vppinfra/time.h:215
215 /vpp/src/vppinfra/time.h: No such file or directory.
(gdb) bt
#0  0x7fa6d0f1fca4 in clib_time_now_internal (n=35626371070243907,
c=0x7fa30edf8680) at /vpp/src/vppinfra/time.h:215
#1  vlib_main_or_worker_loop (is_main=1, vm=) at
/vpp/src/vlib/main.c:3076
#2  vlib_main_loop (vm=) at /vpp/src/vlib/main.c:3108
#3  vlib_main (vm=, vm@entry=0x7fa30edf8680,
input=input@entry=0x7fa308277fa0) at /vpp/src/vlib/main.c:3399
#4  0x7fa6d0f71bc8 in thread0 (arg=140338305926784) at
/vpp/src/vlib/unix/main.c:671
#5  0x7fa6cf6298ec in clib_calljmp () at /vpp/src/vppinfra/longjmp.S:123
#6  0x7ffc98d63e90 in ?? ()



(gdb) bt
#0  clib_cpu_time_now () at /vpp/src/vppinfra/time.h:85
#1  vlib_main_or_worker_loop (is_main=1, vm=) at
/vpp/src/vlib/main.c:2939
#2  vlib_main_loop (vm=) at /vpp/src/vlib/main.c:3108
#3  vlib_main (vm=, vm@entry=0x7fa30edf8680,
input=input@entry=0x7fa308277fa0) at /vpp/src/vlib/main.c:3399
#4  0x7fa6d0f71bc8 in thread0 (arg=140338305926784) at
/vpp/src/vlib/unix/main.c:671
#5  0x7fa6cf6298ec in clib_calljmp () at /vpp/src/vppinfra/longjmp.S:123
#6  0x7ffc98d63e90 in ?? ()
#7  0x7fa6d0f73147 in vlib_unix_main (argc=,
argv=) at /vpp/src/vlib/unix/main.c:751
#8  0x in ?? ()
#9  0x in ?? ()
#10 0x in ?? ()
#11 0x02e0 in ?? ()

Program received signal SIGINT, Interrupt.
0x7fa6ceff7d47 in epoll_pwait () from /lib64/libc.so.6
(gdb) thr 1
[Switching to thread 1 (Thread 0x7fa6d0ed47c0 (LWP 1533))]
#0  0x7fa6ceff7d47 in epoll_pwait () from /lib64/libc.so.6
(gdb) bt
#0  0x7fa6ceff7d47 in epoll_pwait () from /lib64/libc.so.6
#1  0x7fa6d0f70b5d in linux_epoll_input_inline (thread_index=0,
frame=, node=, vm=0x7fa30edf8680)
at /vpp/src/vlib/unix/input.c:219
#2  linux_epoll_input (vm=0x7fa30edf8680, node=,
frame=) at /vpp/src/vlib/unix/input.c:365
#3  0x7fa6d0f1f7e1 in dispatch_node (last_time_stamp=,
frame=0x0, dispatch_state=VLIB_NODE_STATE_POLLING,
type=VLIB_NODE_TYPE_PRE_INPUT, node=0x7fa31189a300, vm=)
at /vpp/src/vlib/main.c:1024
#4  vlib_main_or_worker_loop (is_main=1, vm=) at
/vpp/src/vlib/main.c:2941
#5  vlib_main_loop (vm=) at /vpp/src/vlib/main.c:3108
#6  vlib_main (vm=, vm@entry=0x7fa30edf8680,
input=input@entry=0x7fa308277fa0) at /vpp/src/vlib/main.c:3399
#7  0x7fa6d0f71bc8 in thread0 (arg=140338305926784) at
/vpp/src/vlib/unix/main.c:671
#8  0x7fa6cf6298ec in clib_calljmp () at /vpp/src/vppinfra/longjmp.S:123
#9  0x7ffc98d63e90 in ?? ()
#10 0x7fa6d0f73147 in vlib_unix_main (argc=,
argv=) at /vpp/src/vlib/unix/main.c:751
#7  0x in ?? ()
#8  0x in ?? ()
#9  0x in ?? ()
#10 0x02e0 in ?? ()
#11 0x in ?? ()
#12 0x in ?? ()
#13 0x in ?? ()
#14 0x in ?? ()
#15 0x in ?? ()
...

#208 0x7eff27bec1dc in ?? ()
#209 0x001f003e in ?? ()
#210 0x6289f8e7 in ?? ()
#211 0x0e11 in ?? ()
*#212 0x00010002 in ?? ()*
*#213 0x00020005 in ?? ()*
*#214 0x in ?? ()*


On Sat, May 21, 2022 at 5:08 PM  wrote:

> Attach to the process in gdb, switch to the main thread (if necessary),
> and type “backtrace” or “bt”. You may need to type “continue” a few times
> to build a picture of what’s going on.
>
>
>
>
>
> D.
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *chetan
> bhasin
> *Sent:* Saturday, May 21, 2022 2:50 AM
> *To:* vpp-dev 
> *Subject:* [vpp-dev] VPP 2106 Main thread is 99% CPU
>
>
>
> Hello Everyone,
>
>
>
> We are running VPP2106 with our plugin, we are observing a problem on
> load setup where CPU of main thread is reaching at 99% suddenly and stays
> there even if we stop load, although we dont have problems like any of your
> own nodes polling on main thread etc and even CLI is not blocking,
>
> Could you please provide any direction to debug where the main thread is
> getting stuck ?
>
>
>
> Thanks,
>
> Chetan
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21429): https://lists.fd.io/g/vpp-dev/message/21429
Mute This Topic: https://lists.fd.io/mt/91247657/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] VPP 2106 Main thread is 99% CPU

2022-05-21 Thread Dave Barach
Attach to the process in gdb, switch to the main thread (if necessary), and 
type “backtrace” or “bt”. You may need to type “continue” a few times to build 
a picture of what’s going on.

 

 

D.

 

From: vpp-dev@lists.fd.io  On Behalf Of chetan bhasin
Sent: Saturday, May 21, 2022 2:50 AM
To: vpp-dev 
Subject: [vpp-dev] VPP 2106 Main thread is 99% CPU

 

Hello Everyone,

 

We are running VPP2106 with our plugin, we are observing a problem on load 
setup where CPU of main thread is reaching at 99% suddenly and stays there even 
if we stop load, although we dont have problems like any of your own nodes 
polling on main thread etc and even CLI is not blocking,

Could you please provide any direction to debug where the main thread is 
getting stuck ?

 

Thanks,

Chetan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21424): https://lists.fd.io/g/vpp-dev/message/21424
Mute This Topic: https://lists.fd.io/mt/91247657/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[vpp-dev] VPP 2106 Main thread is 99% CPU

2022-05-21 Thread chetan bhasin
Hello Everyone,

We are running VPP2106 with our plugin, we are observing a problem on
load setup where CPU of main thread is reaching at 99% suddenly and stays
there even if we stop load, although we dont have problems like any of your
own nodes polling on main thread etc and even CLI is not blocking,

Could you please provide any direction to debug where the main thread is
getting stuck ?


Thanks,

Chetan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21423): https://lists.fd.io/g/vpp-dev/message/21423
Mute This Topic: https://lists.fd.io/mt/91247657/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-