Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
PTAL https://gerrit.fd.io/r/c/vpp/+/36334
On Fri, Jun 3, 2022 at 10:25 PM Pim van Pelt via lists.fd.io wrote:
> Hi Damjan,
>
> Just a quick note - 22.06 still has this regression
>
> 1: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci <
> n->n_errors' fails
>
>
> Is a reasonable fix for this seeing to it that the ASSERT here returns
> NULL instead and the two call sites in L95, L224 become tolerant of that?
>
> On Thu, Apr 7, 2022 at 3:11 PM Damjan Marion (damarion) <
> damar...@cisco.com> wrote:
>
>>
>> Yeah, looks like ip4_neighbor_probe is sending packet to deleted
>> interface:
>>
>> (gdb)p n->name
>> $4 = (u8 *) 0x7fff82b47578 "interface-3-output-deleted”
>>
>> So it is right that this assert kicks in.
>>
>> Likely what happens is that batch of commands are first triggering
>> generation of neighbor probe packet, then
>> immediately after that interface is deleted, but packet is still in
>> flight and drop node tries to bump counters for deleted interface.
>>
>> —
>> Damjan
>>
>>
>>
>> > On 06.04.2022., at 16:21, Pim van Pelt wrote:
>> >
>> > Hoi,
>> >
>> > Following reproduces the drop.c:77 assertion:
>> >
>> > create loopback interface instance 0
>> > set interface ip address loop0 10.0.0.1/32
>> > set interface state GigabitEthernet3/0/1 up
>> > set interface state loop0 up
>> > set interface state loop0 down
>> > set interface ip address del loop0 10.0.0.1/32
>> > delete loopback interface intfc loop0
>> > set interface state GigabitEthernet3/0/1 down
>> > set interface state GigabitEthernet3/0/1 up
>> > comment { the following crashes VPP }
>> > set interface state GigabitEthernet3/0/1 down
>> >
>> > I found that adding IPv6 addresses does not provoke the crash, while
>> adding IPv4 addresses to loop0 does provoke it.
>> >
>> > groet,
>> > Pim
>> >
>> > On Wed, Apr 6, 2022 at 3:56 PM Pim van Pelt via lists.fd.io > ipng...@lists.fd.io> wrote:
>> > Hoi,
>> >
>> > The crash I observed is now gone, thanks!
>> >
>> > VPP occasionally hits an ASSERT related to error counters at drop.c:77
>> -- I'll try to see if I can get a reproduction, but it may take a while,
>> and it may be transient.
>> >
>> > 11: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci
>> < n->n_errors' fails
>> >
>> > Thread 14 "vpp_wk_11" received signal SIGABRT, Aborted.
>> > [Switching to Thread 0x7fff4bbfd700 (LWP 182685)]
>> > __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
>> > 50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
>> > (gdb) bt
>> > #0 __GI_raise (sig=sig@entry=6) at
>> ../sysdeps/unix/sysv/linux/raise.c:50
>> > #1 0x76a5f859 in __GI_abort () at abort.c:79
>> > #2 0x004072e3 in os_panic () at
>> /home/pim/src/vpp/src/vpp/vnet/main.c:413
>> > #3 0x76daea29 in debugger () at
>> /home/pim/src/vpp/src/vppinfra/error.c:84
>> > #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
>> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
>> > at /home/pim/src/vpp/src/vppinfra/error.c:143
>> > #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
>> /home/pim/src/vpp/src/vlib/drop.c:77
>> > #6 0x76f77c57 in process_drop_punt (vm=0x7fffa09fb2c0,
>> node=0x7fffa0c79b00, frame=0x7fff97168140,
>> disposition=ERROR_DISPOSITION_DROP)
>> > at /home/pim/src/vpp/src/vlib/drop.c:224
>> > #7 0x76f77957 in error_drop_node_fn_hsw (vm=0x7fffa09fb2c0,
>> node=0x7fffa0c79b00, frame=0x7fff97168140)
>> > at /home/pim/src/vpp/src/vlib/drop.c:248
>> > #8 0x76f0b10d in dispatch_node (vm=0x7fffa09fb2c0,
>> node=0x7fffa0c79b00, type=VLIB_NODE_TYPE_INTERNAL,
>> > dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fff97168140,
>> last_time_stamp=5318787653101516) at /home/pim/src/vpp/src/vlib/main.c:961
>> > #9 0x76f0bb60 in dispatch_pending_node (vm=0x7fffa09fb2c0,
>> pending_frame_index=5, last_time_stamp=5318787653101516)
>> > at /home/pim/src/vpp/src/vlib/main.c:1120
>> > #10 0x76f06e0f in vlib_main_or_worker_loop (vm=0x7fffa09fb2c0,
>> is_main=0) at /home/pim/src/vpp/src/vlib/main.c:1587
>> > #11 0x76f06537 in vlib_worker_loop (vm=0x7fffa09fb2c0) at
>> /home/pim/src/vpp/src/vlib/main.c:1721
>> > #12 0x76f44ef4 in vlib_worker_thread_fn (arg=0x7fff98eabec0) at
>> /home/pim/src/vpp/src/vlib/threads.c:1587
>> > #13 0x76f3ffe5 in vlib_worker_thread_bootstrap_fn
>> (arg=0x7fff98eabec0) at /home/pim/src/vpp/src/vlib/threads.c:426
>> > #14 0x76e61609 in start_thread (arg=) at
>> pthread_create.c:477
>> > #15 0x76b5c163 in clone () at
>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
>> > (gdb) up 4
>> > #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
>> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
>> > at /home/pim/src/vpp/src/vppinfra/error.c:143
>> > 143 debugger ();
>> > (gdb) up
>> > #5 0x76f782d9 in counter_index
Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
Hi Damjan,
Just a quick note - 22.06 still has this regression
1: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci <
n->n_errors' fails
Is a reasonable fix for this seeing to it that the ASSERT here returns NULL
instead and the two call sites in L95, L224 become tolerant of that?
On Thu, Apr 7, 2022 at 3:11 PM Damjan Marion (damarion)
wrote:
>
> Yeah, looks like ip4_neighbor_probe is sending packet to deleted interface:
>
> (gdb)p n->name
> $4 = (u8 *) 0x7fff82b47578 "interface-3-output-deleted”
>
> So it is right that this assert kicks in.
>
> Likely what happens is that batch of commands are first triggering
> generation of neighbor probe packet, then
> immediately after that interface is deleted, but packet is still in flight
> and drop node tries to bump counters for deleted interface.
>
> —
> Damjan
>
>
>
> > On 06.04.2022., at 16:21, Pim van Pelt wrote:
> >
> > Hoi,
> >
> > Following reproduces the drop.c:77 assertion:
> >
> > create loopback interface instance 0
> > set interface ip address loop0 10.0.0.1/32
> > set interface state GigabitEthernet3/0/1 up
> > set interface state loop0 up
> > set interface state loop0 down
> > set interface ip address del loop0 10.0.0.1/32
> > delete loopback interface intfc loop0
> > set interface state GigabitEthernet3/0/1 down
> > set interface state GigabitEthernet3/0/1 up
> > comment { the following crashes VPP }
> > set interface state GigabitEthernet3/0/1 down
> >
> > I found that adding IPv6 addresses does not provoke the crash, while
> adding IPv4 addresses to loop0 does provoke it.
> >
> > groet,
> > Pim
> >
> > On Wed, Apr 6, 2022 at 3:56 PM Pim van Pelt via lists.fd.io ipng...@lists.fd.io> wrote:
> > Hoi,
> >
> > The crash I observed is now gone, thanks!
> >
> > VPP occasionally hits an ASSERT related to error counters at drop.c:77
> -- I'll try to see if I can get a reproduction, but it may take a while,
> and it may be transient.
> >
> > 11: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci <
> n->n_errors' fails
> >
> > Thread 14 "vpp_wk_11" received signal SIGABRT, Aborted.
> > [Switching to Thread 0x7fff4bbfd700 (LWP 182685)]
> > __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> > 50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> > (gdb) bt
> > #0 __GI_raise (sig=sig@entry=6) at
> ../sysdeps/unix/sysv/linux/raise.c:50
> > #1 0x76a5f859 in __GI_abort () at abort.c:79
> > #2 0x004072e3 in os_panic () at
> /home/pim/src/vpp/src/vpp/vnet/main.c:413
> > #3 0x76daea29 in debugger () at
> /home/pim/src/vpp/src/vppinfra/error.c:84
> > #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
> > at /home/pim/src/vpp/src/vppinfra/error.c:143
> > #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
> /home/pim/src/vpp/src/vlib/drop.c:77
> > #6 0x76f77c57 in process_drop_punt (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, frame=0x7fff97168140,
> disposition=ERROR_DISPOSITION_DROP)
> > at /home/pim/src/vpp/src/vlib/drop.c:224
> > #7 0x76f77957 in error_drop_node_fn_hsw (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, frame=0x7fff97168140)
> > at /home/pim/src/vpp/src/vlib/drop.c:248
> > #8 0x76f0b10d in dispatch_node (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, type=VLIB_NODE_TYPE_INTERNAL,
> > dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fff97168140,
> last_time_stamp=5318787653101516) at /home/pim/src/vpp/src/vlib/main.c:961
> > #9 0x76f0bb60 in dispatch_pending_node (vm=0x7fffa09fb2c0,
> pending_frame_index=5, last_time_stamp=5318787653101516)
> > at /home/pim/src/vpp/src/vlib/main.c:1120
> > #10 0x76f06e0f in vlib_main_or_worker_loop (vm=0x7fffa09fb2c0,
> is_main=0) at /home/pim/src/vpp/src/vlib/main.c:1587
> > #11 0x76f06537 in vlib_worker_loop (vm=0x7fffa09fb2c0) at
> /home/pim/src/vpp/src/vlib/main.c:1721
> > #12 0x76f44ef4 in vlib_worker_thread_fn (arg=0x7fff98eabec0) at
> /home/pim/src/vpp/src/vlib/threads.c:1587
> > #13 0x76f3ffe5 in vlib_worker_thread_bootstrap_fn
> (arg=0x7fff98eabec0) at /home/pim/src/vpp/src/vlib/threads.c:426
> > #14 0x76e61609 in start_thread (arg=) at
> pthread_create.c:477
> > #15 0x76b5c163 in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> > (gdb) up 4
> > #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
> > at /home/pim/src/vpp/src/vppinfra/error.c:143
> > 143 debugger ();
> > (gdb) up
> > #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
> /home/pim/src/vpp/src/vlib/drop.c:77
> > 77ASSERT (ci < n->n_errors);
> > (gdb) list
> > 72
> > 73ni = vlib_error_get_node (>node_main, e);
> > 74n = vlib_get_node (vm, ni);
> > 75
> > 76ci =
Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
Yeah, looks like ip4_neighbor_probe is sending packet to deleted interface:
(gdb)p n->name
$4 = (u8 *) 0x7fff82b47578 "interface-3-output-deleted”
So it is right that this assert kicks in.
Likely what happens is that batch of commands are first triggering generation
of neighbor probe packet, then
immediately after that interface is deleted, but packet is still in flight and
drop node tries to bump counters for deleted interface.
—
Damjan
> On 06.04.2022., at 16:21, Pim van Pelt wrote:
>
> Hoi,
>
> Following reproduces the drop.c:77 assertion:
>
> create loopback interface instance 0
> set interface ip address loop0 10.0.0.1/32
> set interface state GigabitEthernet3/0/1 up
> set interface state loop0 up
> set interface state loop0 down
> set interface ip address del loop0 10.0.0.1/32
> delete loopback interface intfc loop0
> set interface state GigabitEthernet3/0/1 down
> set interface state GigabitEthernet3/0/1 up
> comment { the following crashes VPP }
> set interface state GigabitEthernet3/0/1 down
>
> I found that adding IPv6 addresses does not provoke the crash, while adding
> IPv4 addresses to loop0 does provoke it.
>
> groet,
> Pim
>
> On Wed, Apr 6, 2022 at 3:56 PM Pim van Pelt via lists.fd.io
> wrote:
> Hoi,
>
> The crash I observed is now gone, thanks!
>
> VPP occasionally hits an ASSERT related to error counters at drop.c:77 --
> I'll try to see if I can get a reproduction, but it may take a while, and it
> may be transient.
>
> 11: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci <
> n->n_errors' fails
>
> Thread 14 "vpp_wk_11" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff4bbfd700 (LWP 182685)]
> __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> 50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> (gdb) bt
> #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> #1 0x76a5f859 in __GI_abort () at abort.c:79
> #2 0x004072e3 in os_panic () at
> /home/pim/src/vpp/src/vpp/vnet/main.c:413
> #3 0x76daea29 in debugger () at
> /home/pim/src/vpp/src/vppinfra/error.c:84
> #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
> at /home/pim/src/vpp/src/vppinfra/error.c:143
> #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
> /home/pim/src/vpp/src/vlib/drop.c:77
> #6 0x76f77c57 in process_drop_punt (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, frame=0x7fff97168140, disposition=ERROR_DISPOSITION_DROP)
> at /home/pim/src/vpp/src/vlib/drop.c:224
> #7 0x76f77957 in error_drop_node_fn_hsw (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, frame=0x7fff97168140)
> at /home/pim/src/vpp/src/vlib/drop.c:248
> #8 0x76f0b10d in dispatch_node (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, type=VLIB_NODE_TYPE_INTERNAL,
> dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fff97168140,
> last_time_stamp=5318787653101516) at /home/pim/src/vpp/src/vlib/main.c:961
> #9 0x76f0bb60 in dispatch_pending_node (vm=0x7fffa09fb2c0,
> pending_frame_index=5, last_time_stamp=5318787653101516)
> at /home/pim/src/vpp/src/vlib/main.c:1120
> #10 0x76f06e0f in vlib_main_or_worker_loop (vm=0x7fffa09fb2c0,
> is_main=0) at /home/pim/src/vpp/src/vlib/main.c:1587
> #11 0x76f06537 in vlib_worker_loop (vm=0x7fffa09fb2c0) at
> /home/pim/src/vpp/src/vlib/main.c:1721
> #12 0x76f44ef4 in vlib_worker_thread_fn (arg=0x7fff98eabec0) at
> /home/pim/src/vpp/src/vlib/threads.c:1587
> #13 0x76f3ffe5 in vlib_worker_thread_bootstrap_fn
> (arg=0x7fff98eabec0) at /home/pim/src/vpp/src/vlib/threads.c:426
> #14 0x76e61609 in start_thread (arg=) at
> pthread_create.c:477
> #15 0x76b5c163 in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> (gdb) up 4
> #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
> at /home/pim/src/vpp/src/vppinfra/error.c:143
> 143 debugger ();
> (gdb) up
> #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
> /home/pim/src/vpp/src/vlib/drop.c:77
> 77ASSERT (ci < n->n_errors);
> (gdb) list
> 72
> 73ni = vlib_error_get_node (>node_main, e);
> 74n = vlib_get_node (vm, ni);
> 75
> 76ci = vlib_error_get_code (>node_main, e);
> 77ASSERT (ci < n->n_errors);
> 78
> 79ci += n->error_heap_index;
> 80
> 81return ci;
>
> On Wed, Apr 6, 2022 at 1:53 PM Damjan Marion (damarion)
> wrote:
>
> This seems to be day one issue, and my patch just exposed it.
> Current interface deletion code is not removing node stats entries.
>
> So if you delete interface and then create one with the same name,
> stats entry is already there, and creation of new entry fails.
>
> Hope this helps:
>
>
Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
Hoi,
Following reproduces the drop.c:77 assertion:
create loopback interface instance 0
set interface ip address loop0 10.0.0.1/32
set interface state GigabitEthernet3/0/1 up
set interface state loop0 up
set interface state loop0 down
set interface ip address del loop0 10.0.0.1/32
delete loopback interface intfc loop0
set interface state GigabitEthernet3/0/1 down
set interface state GigabitEthernet3/0/1 up
comment { the following crashes VPP }
set interface state GigabitEthernet3/0/1 down
I found that adding IPv6 addresses does not provoke the crash, while adding
IPv4 addresses to loop0 does provoke it.
groet,
Pim
On Wed, Apr 6, 2022 at 3:56 PM Pim van Pelt via lists.fd.io wrote:
> Hoi,
>
> The crash I observed is now gone, thanks!
>
> VPP occasionally hits an ASSERT related to error counters at drop.c:77 --
> I'll try to see if I can get a reproduction, but it may take a while, and
> it may be transient.
>
>
> *11: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci <
> n->n_errors' fails*
>
> Thread 14 "vpp_wk_11" received signal SIGABRT, Aborted.
> [Switching to Thread 0x7fff4bbfd700 (LWP 182685)]
> __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> 50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> (gdb) bt
> #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> #1 0x76a5f859 in __GI_abort () at abort.c:79
> #2 0x004072e3 in os_panic () at
> /home/pim/src/vpp/src/vpp/vnet/main.c:413
> #3 0x76daea29 in debugger () at
> /home/pim/src/vpp/src/vppinfra/error.c:84
> #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
> at /home/pim/src/vpp/src/vppinfra/error.c:143
> #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
> /home/pim/src/vpp/src/vlib/drop.c:77
> #6 0x76f77c57 in process_drop_punt (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, frame=0x7fff97168140,
> disposition=ERROR_DISPOSITION_DROP)
> at /home/pim/src/vpp/src/vlib/drop.c:224
> #7 0x76f77957 in error_drop_node_fn_hsw (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, frame=0x7fff97168140)
> at /home/pim/src/vpp/src/vlib/drop.c:248
> #8 0x76f0b10d in dispatch_node (vm=0x7fffa09fb2c0,
> node=0x7fffa0c79b00, type=VLIB_NODE_TYPE_INTERNAL,
> dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fff97168140,
> last_time_stamp=5318787653101516) at /home/pim/src/vpp/src/vlib/main.c:961
> #9 0x76f0bb60 in dispatch_pending_node (vm=0x7fffa09fb2c0,
> pending_frame_index=5, last_time_stamp=5318787653101516)
> at /home/pim/src/vpp/src/vlib/main.c:1120
> #10 0x76f06e0f in vlib_main_or_worker_loop (vm=0x7fffa09fb2c0,
> is_main=0) at /home/pim/src/vpp/src/vlib/main.c:1587
> #11 0x76f06537 in vlib_worker_loop (vm=0x7fffa09fb2c0) at
> /home/pim/src/vpp/src/vlib/main.c:1721
> #12 0x76f44ef4 in vlib_worker_thread_fn (arg=0x7fff98eabec0) at
> /home/pim/src/vpp/src/vlib/threads.c:1587
> #13 0x76f3ffe5 in vlib_worker_thread_bootstrap_fn
> (arg=0x7fff98eabec0) at /home/pim/src/vpp/src/vlib/threads.c:426
> #14 0x76e61609 in start_thread (arg=) at
> pthread_create.c:477
> #15 0x76b5c163 in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
> (gdb) up 4
> #4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
> at /home/pim/src/vpp/src/vppinfra/error.c:143
> 143 debugger ();
> (gdb) up
> #5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
> /home/pim/src/vpp/src/vlib/drop.c:77
> 77ASSERT (ci < n->n_errors);
> (gdb) list
> 72
> 73ni = vlib_error_get_node (>node_main, e);
> 74n = vlib_get_node (vm, ni);
> 75
> 76ci = vlib_error_get_code (>node_main, e);
> 77ASSERT (ci < n->n_errors);
> 78
> 79ci += n->error_heap_index;
> 80
> 81return ci;
>
> On Wed, Apr 6, 2022 at 1:53 PM Damjan Marion (damarion) <
> damar...@cisco.com> wrote:
>
>>
>> This seems to be day one issue, and my patch just exposed it.
>> Current interface deletion code is not removing node stats entries.
>>
>> So if you delete interface and then create one with the same name,
>> stats entry is already there, and creation of new entry fails.
>>
>> Hope this helps:
>>
>> https://gerrit.fd.io/r/c/vpp/+/35900
>>
>> —
>> Damjan
>>
>>
>>
>> > On 05.04.2022., at 22:13, Pim van Pelt wrote:
>> >
>> > Hoi,
>> >
>> > Here's a minimal repro that reliably crashes VPP at head for me, does
>> not crash before gerrit 35640:
>> >
>> > create loopback interface instance 0
>> > create bond id 0 mode lacp load-balance l34
>> > create bond id 1 mode lacp load-balance l34
>> > delete loopback interface intfc loop0
>> > delete bond BondEthernet0
>> > delete bond BondEthernet1
>> > create bond id 0 mode lacp load-balance l34
>> > delete bond
Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
Hoi,
The crash I observed is now gone, thanks!
VPP occasionally hits an ASSERT related to error counters at drop.c:77 --
I'll try to see if I can get a reproduction, but it may take a while, and
it may be transient.
*11: /home/pim/src/vpp/src/vlib/drop.c:77 (counter_index) assertion `ci <
n->n_errors' fails*
Thread 14 "vpp_wk_11" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fff4bbfd700 (LWP 182685)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x76a5f859 in __GI_abort () at abort.c:79
#2 0x004072e3 in os_panic () at
/home/pim/src/vpp/src/vpp/vnet/main.c:413
#3 0x76daea29 in debugger () at
/home/pim/src/vpp/src/vppinfra/error.c:84
#4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
at /home/pim/src/vpp/src/vppinfra/error.c:143
#5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
/home/pim/src/vpp/src/vlib/drop.c:77
#6 0x76f77c57 in process_drop_punt (vm=0x7fffa09fb2c0,
node=0x7fffa0c79b00, frame=0x7fff97168140,
disposition=ERROR_DISPOSITION_DROP)
at /home/pim/src/vpp/src/vlib/drop.c:224
#7 0x76f77957 in error_drop_node_fn_hsw (vm=0x7fffa09fb2c0,
node=0x7fffa0c79b00, frame=0x7fff97168140)
at /home/pim/src/vpp/src/vlib/drop.c:248
#8 0x76f0b10d in dispatch_node (vm=0x7fffa09fb2c0,
node=0x7fffa0c79b00, type=VLIB_NODE_TYPE_INTERNAL,
dispatch_state=VLIB_NODE_STATE_POLLING, frame=0x7fff97168140,
last_time_stamp=5318787653101516) at /home/pim/src/vpp/src/vlib/main.c:961
#9 0x76f0bb60 in dispatch_pending_node (vm=0x7fffa09fb2c0,
pending_frame_index=5, last_time_stamp=5318787653101516)
at /home/pim/src/vpp/src/vlib/main.c:1120
#10 0x76f06e0f in vlib_main_or_worker_loop (vm=0x7fffa09fb2c0,
is_main=0) at /home/pim/src/vpp/src/vlib/main.c:1587
#11 0x76f06537 in vlib_worker_loop (vm=0x7fffa09fb2c0) at
/home/pim/src/vpp/src/vlib/main.c:1721
#12 0x76f44ef4 in vlib_worker_thread_fn (arg=0x7fff98eabec0) at
/home/pim/src/vpp/src/vlib/threads.c:1587
#13 0x76f3ffe5 in vlib_worker_thread_bootstrap_fn
(arg=0x7fff98eabec0) at /home/pim/src/vpp/src/vlib/threads.c:426
#14 0x76e61609 in start_thread (arg=) at
pthread_create.c:477
#15 0x76b5c163 in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) up 4
#4 0x76dae7fa in _clib_error (how_to_die=2, function_name=0x0,
line_number=0, fmt=0x76f9d19c "%s:%d (%s) assertion `%s' fails")
at /home/pim/src/vpp/src/vppinfra/error.c:143
143 debugger ();
(gdb) up
#5 0x76f782d9 in counter_index (vm=0x7fffa09fb2c0, e=3416) at
/home/pim/src/vpp/src/vlib/drop.c:77
77ASSERT (ci < n->n_errors);
(gdb) list
72
73ni = vlib_error_get_node (>node_main, e);
74n = vlib_get_node (vm, ni);
75
76ci = vlib_error_get_code (>node_main, e);
77ASSERT (ci < n->n_errors);
78
79ci += n->error_heap_index;
80
81return ci;
On Wed, Apr 6, 2022 at 1:53 PM Damjan Marion (damarion)
wrote:
>
> This seems to be day one issue, and my patch just exposed it.
> Current interface deletion code is not removing node stats entries.
>
> So if you delete interface and then create one with the same name,
> stats entry is already there, and creation of new entry fails.
>
> Hope this helps:
>
> https://gerrit.fd.io/r/c/vpp/+/35900
>
> —
> Damjan
>
>
>
> > On 05.04.2022., at 22:13, Pim van Pelt wrote:
> >
> > Hoi,
> >
> > Here's a minimal repro that reliably crashes VPP at head for me, does
> not crash before gerrit 35640:
> >
> > create loopback interface instance 0
> > create bond id 0 mode lacp load-balance l34
> > create bond id 1 mode lacp load-balance l34
> > delete loopback interface intfc loop0
> > delete bond BondEthernet0
> > delete bond BondEthernet1
> > create bond id 0 mode lacp load-balance l34
> > delete bond BondEthernet0
> > comment { the next command crashes VPP }
> > create loopback interface instance 0
> >
> >
> >
> > On Tue, Apr 5, 2022 at 9:48 PM Pim van Pelt wrote:
> > Hoi,
> >
> > There is a crashing regression in VPP after
> https://gerrit.fd.io/r/c/vpp/+/35640
> >
> > With that change merged, VPP crashes upon creation and deletion of
> interfaces. Winding back the repo until before 35640 does not crash. The
> crash happens in
> > 0: /home/pim/src/vpp/src/vlib/stats/stats.h:115 (vlib_stats_get_entry)
> assertion `entry_index < vec_len (sm->directory_vector)' fails
> >
> > (gdb) bt
> > #0 __GI_raise (sig=sig@entry=6) at
> ../sysdeps/unix/sysv/linux/raise.c:50
> > #1 0x76a5e859 in __GI_abort () at abort.c:79
> > #2 0x004072e3 in os_panic () at
> /home/pim/src/vpp/src/vpp/vnet/main.c:413
> > #3 0x76dada29 in debugger () at
>
Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
This seems to be day one issue, and my patch just exposed it.
Current interface deletion code is not removing node stats entries.
So if you delete interface and then create one with the same name,
stats entry is already there, and creation of new entry fails.
Hope this helps:
https://gerrit.fd.io/r/c/vpp/+/35900
—
Damjan
> On 05.04.2022., at 22:13, Pim van Pelt wrote:
>
> Hoi,
>
> Here's a minimal repro that reliably crashes VPP at head for me, does not
> crash before gerrit 35640:
>
> create loopback interface instance 0
> create bond id 0 mode lacp load-balance l34
> create bond id 1 mode lacp load-balance l34
> delete loopback interface intfc loop0
> delete bond BondEthernet0
> delete bond BondEthernet1
> create bond id 0 mode lacp load-balance l34
> delete bond BondEthernet0
> comment { the next command crashes VPP }
> create loopback interface instance 0
>
>
>
> On Tue, Apr 5, 2022 at 9:48 PM Pim van Pelt wrote:
> Hoi,
>
> There is a crashing regression in VPP after
> https://gerrit.fd.io/r/c/vpp/+/35640
>
> With that change merged, VPP crashes upon creation and deletion of
> interfaces. Winding back the repo until before 35640 does not crash. The
> crash happens in
> 0: /home/pim/src/vpp/src/vlib/stats/stats.h:115 (vlib_stats_get_entry)
> assertion `entry_index < vec_len (sm->directory_vector)' fails
>
> (gdb) bt
> #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> #1 0x76a5e859 in __GI_abort () at abort.c:79
> #2 0x004072e3 in os_panic () at
> /home/pim/src/vpp/src/vpp/vnet/main.c:413
> #3 0x76dada29 in debugger () at
> /home/pim/src/vpp/src/vppinfra/error.c:84
> #4 0x76dad7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9c19c "%s:%d (%s) assertion `%s' fails")
>at /home/pim/src/vpp/src/vppinfra/error.c:143
> #5 0x76f39605 in vlib_stats_get_entry (sm=0x76fce5e8
> , entry_index=4294967295)
>at /home/pim/src/vpp/src/vlib/stats/stats.h:115
> #6 0x76f39273 in vlib_stats_remove_entry (entry_index=4294967295) at
> /home/pim/src/vpp/src/vlib/stats/stats.c:135
> #7 0x76ee36d9 in vlib_register_errors (vm=0x7fff96800740,
> node_index=718, n_errors=0, error_strings=0x0, counters=0x0)
>at /home/pim/src/vpp/src/vlib/error.c:149
> #8 0x770b8e0c in setup_tx_node (vm=0x7fff96800740, node_index=718,
> dev_class=0x7fff973f9fb0) at /home/pim/src/vpp/src/vnet/interface.c:816
> #9 0x770b7f26 in vnet_register_interface (vnm=0x77f579a0
> , dev_class_index=31, dev_instance=0, hw_class_index=29,
>hw_instance=7) at /home/pim/src/vpp/src/vnet/interface.c:1085
> #10 0x77129efd in vnet_eth_register_interface (vnm=0x77f579a0
> , r=0x7fff4b288f18)
>at /home/pim/src/vpp/src/vnet/ethernet/interface.c:376
> #11 0x7712bd05 in vnet_create_loopback_interface
> (sw_if_indexp=0x7fff4b288fb8, mac_address=0x7fff4b288fb2 "", is_specified=1
> '\001',
>user_instance=0) at /home/pim/src/vpp/src/vnet/ethernet/interface.c:883
> #12 0x7712fecf in create_simulated_ethernet_interfaces
> (vm=0x7fff96800740, input=0x7fff4b2899d0, cmd=0x7fff973c7e38)
>at /home/pim/src/vpp/src/vnet/ethernet/interface.c:930
> #13 0x76ed65e8 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740,
> cm=0x42c2f0 , input=0x7fff4b2899d0,
>parent_command_index=1161) at /home/pim/src/vpp/src/vlib/cli.c:592
> #14 0x76ed6358 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740,
> cm=0x42c2f0 , input=0x7fff4b2899d0,
>parent_command_index=33) at /home/pim/src/vpp/src/vlib/cli.c:549
> #15 0x76ed6358 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740,
> cm=0x42c2f0 , input=0x7fff4b2899d0,
>parent_command_index=0) at /home/pim/src/vpp/src/vlib/cli.c:549
> #16 0x76ed5528 in vlib_cli_input (vm=0x7fff96800740,
> input=0x7fff4b2899d0, function=0x0, function_arg=0)
>at /home/pim/src/vpp/src/vlib/cli.c:695
> #17 0x76f61f21 in unix_cli_exec (vm=0x7fff96800740,
> input=0x7fff4b289e78, cmd=0x7fff973c99d8) at
> /home/pim/src/vpp/src/vlib/unix/cli.c:3454
> #18 0x76ed65e8 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740,
> cm=0x42c2f0 , input=0x7fff4b289e78,
>parent_command_index=0) at /home/pim/src/vpp/src/vlib/cli.c:592
> #19 0x76ed5528 in vlib_cli_input (vm=0x7fff96800740,
> input=0x7fff4b289e78, function=0x76f55960 ,
> function_arg=1)
>at /home/pim/src/vpp/src/vlib/cli.c:695
>
> This is caught by a local regression test
> (https://github.com/pimvanpelt/vppcfg/tree/main/intest) that executes a bunch
> of CLI statements, and I have a set of transitions there which I can probably
> narrow down to an exact repro case.
>
> On Fri, Apr 1, 2022 at 3:08 PM Pim van Pelt via lists.fd.io
> wrote:
> Hoi,
>
> As a followup - I tried to remember why I copied class VPPStats() and friends
> into my own repository, but that may be because it's not exported in
Re: 35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
Hoi,
Here's a minimal repro that reliably crashes VPP at head for me, does not
crash before gerrit 35640:
create loopback interface instance 0
create bond id 0 mode lacp load-balance l34
create bond id 1 mode lacp load-balance l34
delete loopback interface intfc loop0
delete bond BondEthernet0
delete bond BondEthernet1
create bond id 0 mode lacp load-balance l34
delete bond BondEthernet0
comment { the next command crashes VPP }
create loopback interface instance 0
On Tue, Apr 5, 2022 at 9:48 PM Pim van Pelt wrote:
> Hoi,
>
> There is a crashing regression in VPP after
> https://gerrit.fd.io/r/c/vpp/+/35640
>
> With that change merged, VPP crashes upon creation and deletion of
> interfaces. Winding back the repo until before 35640 does not crash. The
> crash happens in
> 0: /home/pim/src/vpp/src/vlib/stats/stats.h:115 (vlib_stats_get_entry)
> assertion `entry_index < vec_len (sm->directory_vector)' fails
>
> (gdb) bt
> #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> #1 0x76a5e859 in __GI_abort () at abort.c:79
> #2 0x004072e3 in os_panic () at
> /home/pim/src/vpp/src/vpp/vnet/main.c:413
> #3 0x76dada29 in debugger () at
> /home/pim/src/vpp/src/vppinfra/error.c:84
> #4 0x76dad7fa in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x76f9c19c "%s:%d (%s) assertion `%s' fails")
> at /home/pim/src/vpp/src/vppinfra/error.c:143
> #5 0x76f39605 in vlib_stats_get_entry (sm=0x76fce5e8
> , entry_index=4294967295)
> at /home/pim/src/vpp/src/vlib/stats/stats.h:115
> #6 0x76f39273 in vlib_stats_remove_entry (entry_index=4294967295)
> at /home/pim/src/vpp/src/vlib/stats/stats.c:135
> #7 0x76ee36d9 in vlib_register_errors (vm=0x7fff96800740,
> node_index=718, n_errors=0, error_strings=0x0, counters=0x0)
> at /home/pim/src/vpp/src/vlib/error.c:149
> #8 0x770b8e0c in setup_tx_node (vm=0x7fff96800740,
> node_index=718, dev_class=0x7fff973f9fb0) at
> /home/pim/src/vpp/src/vnet/interface.c:816
> #9 0x770b7f26 in vnet_register_interface (vnm=0x77f579a0
> , dev_class_index=31, dev_instance=0, hw_class_index=29,
> hw_instance=7) at /home/pim/src/vpp/src/vnet/interface.c:1085
> #10 0x77129efd in vnet_eth_register_interface (vnm=0x77f579a0
> , r=0x7fff4b288f18)
> at /home/pim/src/vpp/src/vnet/ethernet/interface.c:376
> #11 0x7712bd05 in vnet_create_loopback_interface
> (sw_if_indexp=0x7fff4b288fb8, mac_address=0x7fff4b288fb2 "", is_specified=1
> '\001',
> user_instance=0) at /home/pim/src/vpp/src/vnet/ethernet/interface.c:883
> #12 0x7712fecf in create_simulated_ethernet_interfaces
> (vm=0x7fff96800740, input=0x7fff4b2899d0, cmd=0x7fff973c7e38)
> at /home/pim/src/vpp/src/vnet/ethernet/interface.c:930
> #13 0x76ed65e8 in vlib_cli_dispatch_sub_commands
> (vm=0x7fff96800740, cm=0x42c2f0 ,
> input=0x7fff4b2899d0,
> parent_command_index=1161) at /home/pim/src/vpp/src/vlib/cli.c:592
> #14 0x76ed6358 in vlib_cli_dispatch_sub_commands
> (vm=0x7fff96800740, cm=0x42c2f0 ,
> input=0x7fff4b2899d0,
> parent_command_index=33) at /home/pim/src/vpp/src/vlib/cli.c:549
> #15 0x76ed6358 in vlib_cli_dispatch_sub_commands
> (vm=0x7fff96800740, cm=0x42c2f0 ,
> input=0x7fff4b2899d0,
> parent_command_index=0) at /home/pim/src/vpp/src/vlib/cli.c:549
> #16 0x76ed5528 in vlib_cli_input (vm=0x7fff96800740,
> input=0x7fff4b2899d0, function=0x0, function_arg=0)
> at /home/pim/src/vpp/src/vlib/cli.c:695
> #17 0x76f61f21 in unix_cli_exec (vm=0x7fff96800740,
> input=0x7fff4b289e78, cmd=0x7fff973c99d8) at
> /home/pim/src/vpp/src/vlib/unix/cli.c:3454
> #18 0x76ed65e8 in vlib_cli_dispatch_sub_commands
> (vm=0x7fff96800740, cm=0x42c2f0 ,
> input=0x7fff4b289e78,
> parent_command_index=0) at /home/pim/src/vpp/src/vlib/cli.c:592
> #19 0x76ed5528 in vlib_cli_input (vm=0x7fff96800740,
> input=0x7fff4b289e78, function=0x76f55960 ,
> function_arg=1)
> at /home/pim/src/vpp/src/vlib/cli.c:695
>
> This is caught by a local regression test (
> https://github.com/pimvanpelt/vppcfg/tree/main/intest) that executes a
> bunch of CLI statements, and I have a set of transitions there which I can
> probably narrow down to an exact repro case.
>
> On Fri, Apr 1, 2022 at 3:08 PM Pim van Pelt via lists.fd.io ipng...@lists.fd.io> wrote:
>
>> Hoi,
>>
>> As a followup - I tried to remember why I copied class VPPStats() and
>> friends into my own repository, but that may be because it's not exported
>> in __init__.py. Should it be? I pulled in the latest changed Damjan made to
>> vpp_stats.py into my own repo, and my app runs again. Is it possibly worth
>> our while to add the VPPStats() class to the exported classes in vpp_papi ?
>>
>> groet,
>> Pim
>>
>> On Fri, Apr 1, 2022 at 2:50 PM Pim van Pelt via lists.fd.io > ipng...@lists.fd.io> wrote:
>>
>>> Hoi,
>>>
>>> I noticed that my VPP SNMP Agent no longer
35640 has a crashing regression, was Re: [vpp-dev] vpp-papi stats is broken
Hoi, There is a crashing regression in VPP after https://gerrit.fd.io/r/c/vpp/+/35640 With that change merged, VPP crashes upon creation and deletion of interfaces. Winding back the repo until before 35640 does not crash. The crash happens in 0: /home/pim/src/vpp/src/vlib/stats/stats.h:115 (vlib_stats_get_entry) assertion `entry_index < vec_len (sm->directory_vector)' fails (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x76a5e859 in __GI_abort () at abort.c:79 #2 0x004072e3 in os_panic () at /home/pim/src/vpp/src/vpp/vnet/main.c:413 #3 0x76dada29 in debugger () at /home/pim/src/vpp/src/vppinfra/error.c:84 #4 0x76dad7fa in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x76f9c19c "%s:%d (%s) assertion `%s' fails") at /home/pim/src/vpp/src/vppinfra/error.c:143 #5 0x76f39605 in vlib_stats_get_entry (sm=0x76fce5e8 , entry_index=4294967295) at /home/pim/src/vpp/src/vlib/stats/stats.h:115 #6 0x76f39273 in vlib_stats_remove_entry (entry_index=4294967295) at /home/pim/src/vpp/src/vlib/stats/stats.c:135 #7 0x76ee36d9 in vlib_register_errors (vm=0x7fff96800740, node_index=718, n_errors=0, error_strings=0x0, counters=0x0) at /home/pim/src/vpp/src/vlib/error.c:149 #8 0x770b8e0c in setup_tx_node (vm=0x7fff96800740, node_index=718, dev_class=0x7fff973f9fb0) at /home/pim/src/vpp/src/vnet/interface.c:816 #9 0x770b7f26 in vnet_register_interface (vnm=0x77f579a0 , dev_class_index=31, dev_instance=0, hw_class_index=29, hw_instance=7) at /home/pim/src/vpp/src/vnet/interface.c:1085 #10 0x77129efd in vnet_eth_register_interface (vnm=0x77f579a0 , r=0x7fff4b288f18) at /home/pim/src/vpp/src/vnet/ethernet/interface.c:376 #11 0x7712bd05 in vnet_create_loopback_interface (sw_if_indexp=0x7fff4b288fb8, mac_address=0x7fff4b288fb2 "", is_specified=1 '\001', user_instance=0) at /home/pim/src/vpp/src/vnet/ethernet/interface.c:883 #12 0x7712fecf in create_simulated_ethernet_interfaces (vm=0x7fff96800740, input=0x7fff4b2899d0, cmd=0x7fff973c7e38) at /home/pim/src/vpp/src/vnet/ethernet/interface.c:930 #13 0x76ed65e8 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740, cm=0x42c2f0 , input=0x7fff4b2899d0, parent_command_index=1161) at /home/pim/src/vpp/src/vlib/cli.c:592 #14 0x76ed6358 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740, cm=0x42c2f0 , input=0x7fff4b2899d0, parent_command_index=33) at /home/pim/src/vpp/src/vlib/cli.c:549 #15 0x76ed6358 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740, cm=0x42c2f0 , input=0x7fff4b2899d0, parent_command_index=0) at /home/pim/src/vpp/src/vlib/cli.c:549 #16 0x76ed5528 in vlib_cli_input (vm=0x7fff96800740, input=0x7fff4b2899d0, function=0x0, function_arg=0) at /home/pim/src/vpp/src/vlib/cli.c:695 #17 0x76f61f21 in unix_cli_exec (vm=0x7fff96800740, input=0x7fff4b289e78, cmd=0x7fff973c99d8) at /home/pim/src/vpp/src/vlib/unix/cli.c:3454 #18 0x76ed65e8 in vlib_cli_dispatch_sub_commands (vm=0x7fff96800740, cm=0x42c2f0 , input=0x7fff4b289e78, parent_command_index=0) at /home/pim/src/vpp/src/vlib/cli.c:592 #19 0x76ed5528 in vlib_cli_input (vm=0x7fff96800740, input=0x7fff4b289e78, function=0x76f55960 , function_arg=1) at /home/pim/src/vpp/src/vlib/cli.c:695 This is caught by a local regression test ( https://github.com/pimvanpelt/vppcfg/tree/main/intest) that executes a bunch of CLI statements, and I have a set of transitions there which I can probably narrow down to an exact repro case. On Fri, Apr 1, 2022 at 3:08 PM Pim van Pelt via lists.fd.io wrote: > Hoi, > > As a followup - I tried to remember why I copied class VPPStats() and > friends into my own repository, but that may be because it's not exported > in __init__.py. Should it be? I pulled in the latest changed Damjan made to > vpp_stats.py into my own repo, and my app runs again. Is it possibly worth > our while to add the VPPStats() class to the exported classes in vpp_papi ? > > groet, > Pim > > On Fri, Apr 1, 2022 at 2:50 PM Pim van Pelt via lists.fd.io ipng...@lists.fd.io> wrote: > >> Hoi, >> >> I noticed that my VPP SNMP Agent no longer works with the python API at >> HEAD, and my attention was drawn to this change: >> https://gerrit.fd.io/r/c/vpp/+/35640 >> stats: convert error counters to normal counters >> At HEAD, src/vpp-api/python/vpp_papi/vpp_stats.py now fails 4 out of 6 >> tests with the same error as my application: >> struct.error: offset -140393469444104 out of range for 1073741824-byte >> buffer >> .. >> Ran 6 tests in 0.612s >> FAILED (errors=4) >> >> Damjan, Ole, any clues? >> >> groet, >> Pim >> -- >> Pim van Pelt >> PBVP1-RIPE - http://www.ipng.nl/ >> >> >> >> > > -- > Pim van Pelt > PBVP1-RIPE - http://www.ipng.nl/ > > > > -- Pim van Pelt PBVP1-RIPE - http://www.ipng.nl/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all
