Re: [vpp-dev] #vnet A bug which may cause assertion error in vnet/session

2023-03-22 Thread Chen Weihao
Thank you for you reply.
I did use a VMware virtual machine before, I switched to a physical NIC for 
testing and now there is no problem setting rx-num to 5.
Also, I tested redis using redis-benchmark, the specific command is as follows:
sudo taskset -c 9 bash -c "VCL_CONFIG=/home/chenweihao/vpp_config/vcl.conf 
LD_PRELOAD=/home/chenweihao/vpp_dev/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so
 /home/chenweihao/redis_6.0/src/redis-server --protected-mode no"
sudo taskset -c 11 bash -c "VCL_CONFIG=/home/chenweihao/vpp_config/vcl.conf 
LD_PRELOAD=/home/chenweihao/vpp_dev/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so
 /home/chenweihao/redis_6.0/src/redis-benchmark -h 127.0.0.1 -n 1 -q -t 
set,get -d 4096"

In the case of running both commands, the problems I mentioned before still 
occur (even though I applied yesterday's patch).
However, the problem is not triggered every time the test is run, and it may 
take several runs before the problem occurs once.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22745): https://lists.fd.io/g/vpp-dev/message/22745
Mute This Topic: https://lists.fd.io/mt/97707720/21656
Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] #vnet A bug which may cause assertion error in vnet/session

2023-03-21 Thread Chen Weihao
Thank you for your reply.
I tried to change num-tx-queues from 2 to 5, but it got a SIGSEGV, the 
backtrace is:
#0  0x7fffb453ff89 in rte_write32_relaxed (addr=0x80007ef0, value=0)
at ../src-dpdk/lib/eal/include/generic/rte_io.h:310
#1  rte_write32 (addr=0x80007ef0, value=0)
at ../src-dpdk/lib/eal/include/generic/rte_io.h:373
#2  vmxnet3_enable_intr (hw=0xac03b0600, intr_idx=4294967262)
at ../src-dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c:210
#3  0x7fffb4544d35 in vmxnet3_dev_rx_queue_intr_enable (
dev=0x7fffb5186980 , queue_id=0)
at ../src-dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c:1815
#4  0x7fffaff4bbf2 in rte_eth_dev_rx_intr_enable (port_id=0, queue_id=0)
at ../src-dpdk/lib/ethdev/rte_ethdev.c:4740
#5  0x7fffb49f4564 in dpdk_setup_interrupts (xd=0x7fffbdbb2940)
at /home/chenweihao/vpp_dev/src/plugins/dpdk/device/common.c:336
#6  0x7fffb49f4430 in dpdk_device_start (xd=0x7fffbdbb2940)
at /home/chenweihao/vpp_dev/src/plugins/dpdk/device/common.c:411
#7  0x7fffb49ff713 in dpdk_interface_admin_up_down (
vnm=0x77e2b828 , hw_if_index=1, flags=1)
at /home/chenweihao/vpp_dev/src/plugins/dpdk/device/device.c:476
#8  0x770d60e8 in vnet_sw_interface_set_flags_helper (
vnm=0x77e2b828 , sw_if_index=1,
flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP, helper_flags=0)
at /home/chenweihao/vpp_dev/src/vnet/interface.c:470
#9  0x770d645a in vnet_sw_interface_set_flags (
vnm=0x77e2b828 , sw_if_index=1,
flags=VNET_SW_INTERFACE_FLAG_ADMIN_UP)
at /home/chenweihao/vpp_dev/src/vnet/interface.c:524
#10 0x7710515f in set_state (vm=0x7fffb6a00740, input=0x7fffa9f84bb8,
cmd=0x7fffb7180850)
at /home/chenweihao/vpp_dev/src/vnet/interface_cli.c:946
#11 0x77e72257 in vlib_cli_dispatch_sub_commands (vm=0x7fffb6a00740,
cm=0x77f6a770 , input=0x7fffa9f84bb8,
parent_command_index=20) at /home/chenweihao/vpp_dev/src/vlib/cli.c:650
#12 0x77e71fea in vlib_cli_dispatch_sub_commands (vm=0x7fffb6a00740,
cm=0x77f6a770 , input=0x7fffa9f84bb8,
parent_command_index=7) at /home/chenweihao/vpp_dev/src/vlib/cli.c:607
#13 0x77e71fea in vlib_cli_dispatch_sub_commands (vm=0x7fffb6a00740,
cm=0x77f6a770 , input=0x7fffa9f84bb8,
parent_command_index=0) at /home/chenweihao/vpp_dev/src/vlib/cli.c:607
#14 0x77e7122a in vlib_cli_input (vm=0x7fffb6a00740,
input=0x7fffa9f84bb8, function=0x0, function_arg=0)
at /home/chenweihao/vpp_dev/src/vlib/cli.c:753
#15 0x77ef7e23 in unix_cli_exec (vm=0x7fffb6a00740,
input=0x7fffa9f84f30, cmd=0x7fffb71815b8)
at /home/chenweihao/vpp_dev/src/vlib/unix/cli.c:3431
#16 0x77e72257 in vlib_cli_dispatch_sub_commands (vm=0x7fffb6a00740,
cm=0x77f6a770 , input=0x7fffa9f84f30,
--Type  for more, q to quit, c to continue without paging--
parent_command_index=0) at /home/chenweihao/vpp_dev/src/vlib/cli.c:650
#17 0x77e7122a in vlib_cli_input (vm=0x7fffb6a00740,
input=0x7fffa9f84f30, function=0x0, function_arg=0)
at /home/chenweihao/vpp_dev/src/vlib/cli.c:753
#18 0x77efdfc5 in startup_config_process (vm=0x7fffb6a00740,
rt=0x7fffb9194080, f=0x0)
at /home/chenweihao/vpp_dev/src/vlib/unix/main.c:291
#19 0x77ea2c5d in vlib_process_bootstrap (_a=140736084405176)
at /home/chenweihao/vpp_dev/src/vlib/main.c:1221
#20 0x76f1ffd8 in clib_calljmp ()
at /home/chenweihao/vpp_dev/src/vppinfra/longjmp.S:123
#21 0x7fffac516bb0 in ?? ()
#22 0x77ea26f9 in vlib_process_startup (vm=0x8,
p=0x77ea53bb , f=0x7fffac516cc0)
at /home/chenweihao/vpp_dev/src/vlib/main.c:1246
#23 0x76f7aa1c in vec_mem_size (v=0x7fffb6a00740)
at /home/chenweihao/vpp_dev/src/vppinfra/vec.c:15
#24 0x0581655dfd1c in ?? ()
#25 0x00330004 in ?? ()
#26 0x0030 in ?? ()
#27 0x7fffbdbc7240 in ?? ()
#28 0x7fffbdbc7240 in ?? ()
#29 0x7fffb80e5498 in ?? ()
#30 0x0001 in ?? ()
#31 0x in ?? ()

I tried to change num-rx-queues and num-tx-queues to 4, then SIGSEGV not 
happened.
I applied the patch https://gerrit.fd.io/r/c/vpp/+/38529 , and the problem of 
redis 6.0 seemed still exist, the stack backtrace is same with 
https://lists.fd.io/g/vpp-dev/message/22731

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22740): https://lists.fd.io/g/vpp-dev/message/22740
Mute This Topic: https://lists.fd.io/mt/97707720/21656
Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] #vnet A bug which may cause assertion error in vnet/session

2023-03-22 Thread Chen Weihao
Thank you for your reply.
I did use a VMware virtual machine before, I switched to a physical NIC for 
testing and now there is no problem setting rx-num to 5.
Also, I tested redis using redis-benchmark, the specific command is as follows:
sudo taskset -c 9 bash -c "VCL_CONFIG=/home/chenweihao/vpp_config/vcl.conf 
LD_PRELOAD=/home/chenweihao/vpp_dev/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so
 /home/chenweihao/redis_6.0/src/redis-server --protected-mode no"
sudo taskset -c 11 bash -c "VCL_CONFIG=/home/chenweihao/vpp_config/vcl.conf 
LD_PRELOAD=/home/chenweihao/vpp_dev/build-root/install-vpp-native/vpp/lib/libvcl_ldpreload.so
 /home/chenweihao/redis_6.0/src/redis-benchmark -h 127.0.0.1 -n 1 -q -t 
set,get -d 4096"

In the case of running both commands, the problems I mentioned before still 
occur (even though I applied the patch https://gerrit.fd.io/r/c/vpp/+/38529 ).
However, the problem is not triggered every time the test is run, and it may 
take several runs before the problem occurs once.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22752): https://lists.fd.io/g/vpp-dev/message/22752
Mute This Topic: https://lists.fd.io/mt/97707720/21656
Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] #vnet A bug which may cause assertion error in vnet/session

2023-03-20 Thread Chen Weihao
Thank you for your reply.
This is the stacktrace captured by gdb:

2: /home/chenweihao/vpp_dev/src/vnet/session/session.c:233 (session_is_valid) 
assertion `! pool_is_free (session_main.wrk[thread_index].sessions, _e)' fails

Thread 4 "vpp_wk_1" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffa93f5640 (LWP 4575)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140736032888384) at 
./nptl/pthread_kill.c:44
44./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140736032888384)
at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140736032888384)
at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140736032888384, signo=signo@entry=6)
at ./nptl/pthread_kill.c:89
#3  0x76a42476 in __GI_raise (sig=sig@entry=6)
at ../sysdeps/posix/raise.c:26
#4  0x76a287f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0xb073 in os_panic ()
at /home/chenweihao/vpp_dev/src/vpp/vnet/main.c:417
#6  0x76f0a699 in debugger ()
at /home/chenweihao/vpp_dev/src/vppinfra/error.c:84
#7  0x76f0a450 in _clib_error (how_to_die=2, function_name=0x0,
line_number=0, fmt=0x77b88208 "%s:%d (%s) assertion `%s' fails")
at /home/chenweihao/vpp_dev/src/vppinfra/error.c:143
#8  0x775f31c7 in session_is_valid (si=4294967295,
thread_index=1 '\001')
at /home/chenweihao/vpp_dev/src/vnet/session/session.c:233
#9  0x775f177c in session_get (si=4294967295, thread_index=1)
at /home/chenweihao/vpp_dev/src/vnet/session/session.h:373
#10 0x775f3770 in ho_session_get (ho_index=4294967295)
at /home/chenweihao/vpp_dev/src/vnet/session/session.h:689
#11 0x775f37d8 in session_half_open_migrate_notify (tc=0x7fffbdba00c0)
at /home/chenweihao/vpp_dev/src/vnet/session/session.c:357
#12 0x77648600 in ct_accept_one (thread_index=2, ho_index=2)
at /home/chenweihao/vpp_dev/src/vnet/session/application_local.c:669
#13 0x77648243 in ct_accept_rpc_wrk_handler (rpc_args=0x2)
at /home/chenweihao/vpp_dev/src/vnet/session/application_local.c:760
#14 0x77620b42 in session_event_dispatch_ctrl (wrk=0x7fffbdb6fe00,
elt=0x7fffbdccaf2c)
at /home/chenweihao/vpp_dev/src/vnet/session/session_node.c:1656
#15 0x776175c2 in session_queue_node_fn (vm=0x7fffb82da700,
node=0x7fffbdbc21c0, frame=0x0)
at /home/chenweihao/vpp_dev/src/vnet/session/session_node.c:1962
#16 0x77ea3a62 in dispatch_node (vm=0x7fffb82da700,
node=0x7fffbdbc21c0, type=VLIB_NODE_TYPE_INPUT,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x0,
last_time_stamp=2911620275652)
at /home/chenweihao/vpp_dev/src/vlib/main.c:960
#17 0x77e9f7d1 in vlib_main_or_worker_loop (vm=0x7fffb82da700,
is_main=0) at /home/chenweihao/vpp_dev/src/vlib/main.c:1557
#18 0x77e9f1c7 in vlib_worker_loop (vm=0x7fffb82da700)
at /home/chenweihao/vpp_dev/src/vlib/main.c:1722
#19 0x77edb020 in vlib_worker_thread_fn (arg=0x7fffb9182980)
at /home/chenweihao/vpp_dev/src/vlib/threads.c:1598
#20 0x77ed62b6 in vlib_worker_thread_bootstrap_fn (arg=0x7fffb9182980)
at /home/chenweihao/vpp_dev/src/vlib/threads.c:418
#21 0x76a94b43 in start_thread (arg=)
at ./nptl/pthread_create.c:442
#22 0x76b26a00 in clone3 ()
at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
The resaon why assertion failed is that  c_s_index value is -1 (or ~0).

The redis veision I used:
git clone https://github.com/redis/redis.git
cd redis
git checkout 6.0
make

My startup.conf:
unix {
nodaemon
interactive
full-coredump
cli-listen /run/vpp/cli.sock
gid vpp
startup-config /home/chenweihao/startup.txt
}

api-trace {
on
}

cpu {
main-core 0
workers 4
}

tcp {
cc-algo cubic
}

dpdk {
uio-driver vfio-pci
dev :03:00.0 {
name eth1
num-rx-queues 2
num-tx-queues 2
}
}

buffers {
buffers-per-numa 131072
}

session { evt_qs_memfd_seg  }

My startup.txt:
set interface ip address eth1 192.168.0.2/24
set interface state eth1 up
set interface rx-mode eth1 polling
create loopback interface
set interface ip address loop0 127.0.0.1/8
set interface state loop0 up

My vcl.conf:
vcl {
rx-fifo-size 4000
tx-fifo-size 4000
app-scope-local
app-scope-global
api-socket-name /run/vpp/api.sock
use-mq-eventfd
}
I tried iperf3 as you said,indeed I did not encounter this problem.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22731): https://lists.fd.io/g/vpp-dev/message/22731
Mute This Topic: https://lists.fd.io/mt/97707720/21656
Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



Re: [vpp-dev] #vnet A bug which may cause assertion error in vnet/session

2023-03-20 Thread Chen Weihao
Thanks for your reply.
I give a more detailed backtrace and config in 
https://lists.fd.io/g/vpp-dev/message/22731. ( 
https://lists.fd.io/g/vpp-dev/message/22731, ) My installation method is to 
clone vpp from github and make build on Ubuntu 22.04(Kernel version is 
5.19),and I use make run for test and make debug for debugging. Yes, I yried to 
make the server and client are attached to the same vpp instance.I tried the 
latest version of vpp on github on yesterday, the problem is still exist.
I am looking forward to your reply.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22732): https://lists.fd.io/g/vpp-dev/message/22732
Mute This Topic: https://lists.fd.io/mt/97707720/21656
Mute #vnet:https://lists.fd.io/g/vpp-dev/mutehashtag/vnet
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-