Public bug reported:
Tests inside
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase
seems to fail intermittently. The root cause seems like to be the same
for all the tests...
Here are two different stack traces from two different patches...
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-19 09:49:54.288 | 2016-02-19 09:49:54.264 |
==============================
2016-02-19 09:49:54.312 | 2016-02-19 09:49:54.279 | Failed 1 tests - output
below:
2016-02-19 09:49:54.321 | 2016-02-19 09:49:54.292 |
==============================
2016-02-19 09:49:54.332 | 2016-02-19 09:49:54.308 |
2016-02-19 09:49:54.335 | 2016-02-19 09:49:54.310 |
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase.test_arp_spoof_disable_network_port(native)
2016-02-19 09:49:54.339 | 2016-02-19 09:49:54.312 |
---------------------------------------------------------------------------------------------------------------
2016-02-19 09:49:54.339 | 2016-02-19 09:49:54.314 |
2016-02-19 09:49:54.343 | 2016-02-19 09:49:54.316 | Captured traceback:
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.317 | ~~~~~~~~~~~~~~~~~~~
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.322 | Traceback (most recent
call last):
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.328 | File
"neutron/tests/functional/agent/test_ovs_flows.py", line 242, in
test_arp_spoof_disable_network_port
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.351 |
net_helpers.assert_ping(self.src_namespace, self.dst_addr, count=2)
2016-02-19 09:49:54.378 | 2016-02-19 09:49:54.354 | File
"neutron/tests/common/net_helpers.py", line 93, in assert_ping
2016-02-19 09:49:54.380 | 2016-02-19 09:49:54.357 | dst_ip])
2016-02-19 09:49:54.385 | 2016-02-19 09:49:54.361 | File
"neutron/agent/linux/ip_lib.py", line 885, in execute
2016-02-19 09:49:54.386 | 2016-02-19 09:49:54.363 |
log_fail_as_error=log_fail_as_error, **kwargs)
2016-02-19 09:49:54.389 | 2016-02-19 09:49:54.365 | File
"neutron/agent/linux/utils.py", line 140, in execute
2016-02-19 09:49:54.392 | 2016-02-19 09:49:54.368 | raise
RuntimeError(msg)
2016-02-19 09:49:54.395 | 2016-02-19 09:49:54.371 | RuntimeError: Exit
code: 1; Stdin: ; Stdout: PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
2016-02-19 09:49:54.397 | 2016-02-19 09:49:54.374 |
2016-02-19 09:49:54.420 | 2016-02-19 09:49:54.376 | --- 192.168.0.2 ping
statistics ---
2016-02-19 09:49:54.421 | 2016-02-19 09:49:54.378 | 2 packets transmitted,
0 received, 100% packet loss, time 1005ms
2016-02-19 09:49:54.421 | 2016-02-19 09:49:54.379 |
2016-02-19 09:49:54.421 | 2016-02-19 09:49:54.382 | ; Stderr:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:55.965 | 2016-02-19 09:12:55.947 |
==============================
2016-02-19 09:12:55.972 | 2016-02-19 09:12:55.951 | Failed 1 tests - output
below:
2016-02-19 09:12:55.977 | 2016-02-19 09:12:55.959 |
==============================
2016-02-19 09:12:55.986 | 2016-02-19 09:12:55.967 |
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.976 |
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase.test_arp_spoof_doesnt_block_normal_traffic(native)
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.978 |
----------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:56.009 | 2016-02-19 09:12:55.990 |
2016-02-19 09:12:56.025 | 2016-02-19 09:12:56.006 | Captured pythonlogging:
2016-02-19 09:12:56.034 | 2016-02-19 09:12:56.016 | ~~~~~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.036 | 2016-02-19 09:12:56.018 | DEBUG
[oslo_policy._cache_handler] Reloading cached file
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.038 | 2016-02-19 09:12:56.020 | DEBUG
[oslo_policy.policy] Reloaded policy file:
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.046 | 2016-02-19 09:12:56.028 |
2016-02-19 09:12:56.051 | 2016-02-19 09:12:56.030 |
2016-02-19 09:12:56.060 | 2016-02-19 09:12:56.036 | Captured traceback:
2016-02-19 09:12:56.062 | 2016-02-19 09:12:56.044 | ~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.066 | 2016-02-19 09:12:56.048 | Traceback (most recent
call last):
2016-02-19 09:12:56.071 | 2016-02-19 09:12:56.053 | File
"neutron/tests/functional/agent/test_ovs_flows.py", line 145, in
test_arp_spoof_doesnt_block_normal_traffic
2016-02-19 09:12:56.073 | 2016-02-19 09:12:56.055 |
net_helpers.assert_ping(self.src_namespace, self.dst_addr, count=2)
2016-02-19 09:12:56.078 | 2016-02-19 09:12:56.058 | File
"neutron/tests/common/net_helpers.py", line 93, in assert_ping
2016-02-19 09:12:56.080 | 2016-02-19 09:12:56.062 | dst_ip])
2016-02-19 09:12:56.082 | 2016-02-19 09:12:56.064 | File
"neutron/agent/linux/ip_lib.py", line 885, in execute
2016-02-19 09:12:56.084 | 2016-02-19 09:12:56.066 |
log_fail_as_error=log_fail_as_error, **kwargs)
2016-02-19 09:12:56.086 | 2016-02-19 09:12:56.068 | File
"neutron/agent/linux/utils.py", line 140, in execute
2016-02-19 09:12:56.094 | 2016-02-19 09:12:56.076 | raise
RuntimeError(msg)
2016-02-19 09:12:56.102 | 2016-02-19 09:12:56.080 | RuntimeError: Exit
code: 1; Stdin: ; Stdout: PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
2016-02-19 09:12:56.109 | 2016-02-19 09:12:56.091 |
2016-02-19 09:12:56.121 | 2016-02-19 09:12:56.101 | --- 192.168.0.2 ping
statistics ---
2016-02-19 09:12:56.132 | 2016-02-19 09:12:56.113 | 2 packets transmitted,
0 received, 100% packet loss, time 1003ms
2016-02-19 09:12:56.137 | 2016-02-19 09:12:56.118 |
2016-02-19 09:12:56.138 | 2016-02-19 09:12:56.121 | ; Stderr:
2016-02-19 09:12:56.141 | 2016-02-19 09:12:56.123 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:55.965 | 2016-02-19 09:12:55.947 |
==============================
2016-02-19 09:12:55.972 | 2016-02-19 09:12:55.951 | Failed 1 tests - output
below:
2016-02-19 09:12:55.977 | 2016-02-19 09:12:55.959 |
==============================
2016-02-19 09:12:55.986 | 2016-02-19 09:12:55.967 |
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.976 |
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase.test_arp_spoof_doesnt_block_normal_traffic(native)
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.978 |
----------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:56.009 | 2016-02-19 09:12:55.990 |
2016-02-19 09:12:56.025 | 2016-02-19 09:12:56.006 | Captured pythonlogging:
2016-02-19 09:12:56.034 | 2016-02-19 09:12:56.016 | ~~~~~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.036 | 2016-02-19 09:12:56.018 | DEBUG
[oslo_policy._cache_handler] Reloading cached file
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.038 | 2016-02-19 09:12:56.020 | DEBUG
[oslo_policy.policy] Reloaded policy file:
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.046 | 2016-02-19 09:12:56.028 |
2016-02-19 09:12:56.051 | 2016-02-19 09:12:56.030 |
2016-02-19 09:12:56.060 | 2016-02-19 09:12:56.036 | Captured traceback:
2016-02-19 09:12:56.062 | 2016-02-19 09:12:56.044 | ~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.066 | 2016-02-19 09:12:56.048 | Traceback (most recent
call last):
2016-02-19 09:12:56.071 | 2016-02-19 09:12:56.053 | File
"neutron/tests/functional/agent/test_ovs_flows.py", line 145, in
test_arp_spoof_doesnt_block_normal_traffic
2016-02-19 09:12:56.073 | 2016-02-19 09:12:56.055 |
net_helpers.assert_ping(self.src_namespace, self.dst_addr, count=2)
2016-02-19 09:12:56.078 | 2016-02-19 09:12:56.058 | File
"neutron/tests/common/net_helpers.py", line 93, in assert_ping
2016-02-19 09:12:56.080 | 2016-02-19 09:12:56.062 | dst_ip])
2016-02-19 09:12:56.082 | 2016-02-19 09:12:56.064 | File
"neutron/agent/linux/ip_lib.py", line 885, in execute
2016-02-19 09:12:56.084 | 2016-02-19 09:12:56.066 |
log_fail_as_error=log_fail_as_error, **kwargs)
2016-02-19 09:12:56.086 | 2016-02-19 09:12:56.068 | File
"neutron/agent/linux/utils.py", line 140, in execute
2016-02-19 09:12:56.094 | 2016-02-19 09:12:56.076 | raise
RuntimeError(msg)
2016-02-19 09:12:56.102 | 2016-02-19 09:12:56.080 | RuntimeError: Exit
code: 1; Stdin: ; Stdout: PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
2016-02-19 09:12:56.109 | 2016-02-19 09:12:56.091 |
2016-02-19 09:12:56.121 | 2016-02-19 09:12:56.101 | --- 192.168.0.2 ping
statistics ---
2016-02-19 09:12:56.132 | 2016-02-19 09:12:56.113 | 2 packets transmitted,
0 received, 100% packet loss, time 1003ms
2016-02-19 09:12:56.137 | 2016-02-19 09:12:56.118 |
2016-02-19 09:12:56.138 | 2016-02-19 09:12:56.121 | ; Stderr:
2016-02-19 09:12:56.141 | 2016-02-19 09:12:56.123 |
** Affects: neutron
Importance: Undecided
Status: New
** Tags: gate-failure
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1547486
Title:
ARPSpoofOFCtlTestCase functional gate tests failing intermittently
Status in neutron:
New
Bug description:
Tests inside
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase
seems to fail intermittently. The root cause seems like to be the same
for all the tests...
Here are two different stack traces from two different patches...
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-19 09:49:54.288 | 2016-02-19 09:49:54.264 |
==============================
2016-02-19 09:49:54.312 | 2016-02-19 09:49:54.279 | Failed 1 tests - output
below:
2016-02-19 09:49:54.321 | 2016-02-19 09:49:54.292 |
==============================
2016-02-19 09:49:54.332 | 2016-02-19 09:49:54.308 |
2016-02-19 09:49:54.335 | 2016-02-19 09:49:54.310 |
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase.test_arp_spoof_disable_network_port(native)
2016-02-19 09:49:54.339 | 2016-02-19 09:49:54.312 |
---------------------------------------------------------------------------------------------------------------
2016-02-19 09:49:54.339 | 2016-02-19 09:49:54.314 |
2016-02-19 09:49:54.343 | 2016-02-19 09:49:54.316 | Captured traceback:
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.317 | ~~~~~~~~~~~~~~~~~~~
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.322 | Traceback (most
recent call last):
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.328 | File
"neutron/tests/functional/agent/test_ovs_flows.py", line 242, in
test_arp_spoof_disable_network_port
2016-02-19 09:49:54.375 | 2016-02-19 09:49:54.351 |
net_helpers.assert_ping(self.src_namespace, self.dst_addr, count=2)
2016-02-19 09:49:54.378 | 2016-02-19 09:49:54.354 | File
"neutron/tests/common/net_helpers.py", line 93, in assert_ping
2016-02-19 09:49:54.380 | 2016-02-19 09:49:54.357 | dst_ip])
2016-02-19 09:49:54.385 | 2016-02-19 09:49:54.361 | File
"neutron/agent/linux/ip_lib.py", line 885, in execute
2016-02-19 09:49:54.386 | 2016-02-19 09:49:54.363 |
log_fail_as_error=log_fail_as_error, **kwargs)
2016-02-19 09:49:54.389 | 2016-02-19 09:49:54.365 | File
"neutron/agent/linux/utils.py", line 140, in execute
2016-02-19 09:49:54.392 | 2016-02-19 09:49:54.368 | raise
RuntimeError(msg)
2016-02-19 09:49:54.395 | 2016-02-19 09:49:54.371 | RuntimeError: Exit
code: 1; Stdin: ; Stdout: PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
2016-02-19 09:49:54.397 | 2016-02-19 09:49:54.374 |
2016-02-19 09:49:54.420 | 2016-02-19 09:49:54.376 | --- 192.168.0.2 ping
statistics ---
2016-02-19 09:49:54.421 | 2016-02-19 09:49:54.378 | 2 packets
transmitted, 0 received, 100% packet loss, time 1005ms
2016-02-19 09:49:54.421 | 2016-02-19 09:49:54.379 |
2016-02-19 09:49:54.421 | 2016-02-19 09:49:54.382 | ; Stderr:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:55.965 | 2016-02-19 09:12:55.947 |
==============================
2016-02-19 09:12:55.972 | 2016-02-19 09:12:55.951 | Failed 1 tests - output
below:
2016-02-19 09:12:55.977 | 2016-02-19 09:12:55.959 |
==============================
2016-02-19 09:12:55.986 | 2016-02-19 09:12:55.967 |
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.976 |
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase.test_arp_spoof_doesnt_block_normal_traffic(native)
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.978 |
----------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:56.009 | 2016-02-19 09:12:55.990 |
2016-02-19 09:12:56.025 | 2016-02-19 09:12:56.006 | Captured pythonlogging:
2016-02-19 09:12:56.034 | 2016-02-19 09:12:56.016 | ~~~~~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.036 | 2016-02-19 09:12:56.018 | DEBUG
[oslo_policy._cache_handler] Reloading cached file
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.038 | 2016-02-19 09:12:56.020 | DEBUG
[oslo_policy.policy] Reloaded policy file:
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.046 | 2016-02-19 09:12:56.028 |
2016-02-19 09:12:56.051 | 2016-02-19 09:12:56.030 |
2016-02-19 09:12:56.060 | 2016-02-19 09:12:56.036 | Captured traceback:
2016-02-19 09:12:56.062 | 2016-02-19 09:12:56.044 | ~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.066 | 2016-02-19 09:12:56.048 | Traceback (most
recent call last):
2016-02-19 09:12:56.071 | 2016-02-19 09:12:56.053 | File
"neutron/tests/functional/agent/test_ovs_flows.py", line 145, in
test_arp_spoof_doesnt_block_normal_traffic
2016-02-19 09:12:56.073 | 2016-02-19 09:12:56.055 |
net_helpers.assert_ping(self.src_namespace, self.dst_addr, count=2)
2016-02-19 09:12:56.078 | 2016-02-19 09:12:56.058 | File
"neutron/tests/common/net_helpers.py", line 93, in assert_ping
2016-02-19 09:12:56.080 | 2016-02-19 09:12:56.062 | dst_ip])
2016-02-19 09:12:56.082 | 2016-02-19 09:12:56.064 | File
"neutron/agent/linux/ip_lib.py", line 885, in execute
2016-02-19 09:12:56.084 | 2016-02-19 09:12:56.066 |
log_fail_as_error=log_fail_as_error, **kwargs)
2016-02-19 09:12:56.086 | 2016-02-19 09:12:56.068 | File
"neutron/agent/linux/utils.py", line 140, in execute
2016-02-19 09:12:56.094 | 2016-02-19 09:12:56.076 | raise
RuntimeError(msg)
2016-02-19 09:12:56.102 | 2016-02-19 09:12:56.080 | RuntimeError: Exit
code: 1; Stdin: ; Stdout: PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
2016-02-19 09:12:56.109 | 2016-02-19 09:12:56.091 |
2016-02-19 09:12:56.121 | 2016-02-19 09:12:56.101 | --- 192.168.0.2 ping
statistics ---
2016-02-19 09:12:56.132 | 2016-02-19 09:12:56.113 | 2 packets
transmitted, 0 received, 100% packet loss, time 1003ms
2016-02-19 09:12:56.137 | 2016-02-19 09:12:56.118 |
2016-02-19 09:12:56.138 | 2016-02-19 09:12:56.121 | ; Stderr:
2016-02-19 09:12:56.141 | 2016-02-19 09:12:56.123 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:55.965 | 2016-02-19 09:12:55.947 |
==============================
2016-02-19 09:12:55.972 | 2016-02-19 09:12:55.951 | Failed 1 tests - output
below:
2016-02-19 09:12:55.977 | 2016-02-19 09:12:55.959 |
==============================
2016-02-19 09:12:55.986 | 2016-02-19 09:12:55.967 |
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.976 |
neutron.tests.functional.agent.test_ovs_flows.ARPSpoofOFCtlTestCase.test_arp_spoof_doesnt_block_normal_traffic(native)
2016-02-19 09:12:56.007 | 2016-02-19 09:12:55.978 |
----------------------------------------------------------------------------------------------------------------------
2016-02-19 09:12:56.009 | 2016-02-19 09:12:55.990 |
2016-02-19 09:12:56.025 | 2016-02-19 09:12:56.006 | Captured pythonlogging:
2016-02-19 09:12:56.034 | 2016-02-19 09:12:56.016 | ~~~~~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.036 | 2016-02-19 09:12:56.018 | DEBUG
[oslo_policy._cache_handler] Reloading cached file
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.038 | 2016-02-19 09:12:56.020 | DEBUG
[oslo_policy.policy] Reloaded policy file:
/opt/stack/new/neutron/neutron/tests/etc/policy.json
2016-02-19 09:12:56.046 | 2016-02-19 09:12:56.028 |
2016-02-19 09:12:56.051 | 2016-02-19 09:12:56.030 |
2016-02-19 09:12:56.060 | 2016-02-19 09:12:56.036 | Captured traceback:
2016-02-19 09:12:56.062 | 2016-02-19 09:12:56.044 | ~~~~~~~~~~~~~~~~~~~
2016-02-19 09:12:56.066 | 2016-02-19 09:12:56.048 | Traceback (most
recent call last):
2016-02-19 09:12:56.071 | 2016-02-19 09:12:56.053 | File
"neutron/tests/functional/agent/test_ovs_flows.py", line 145, in
test_arp_spoof_doesnt_block_normal_traffic
2016-02-19 09:12:56.073 | 2016-02-19 09:12:56.055 |
net_helpers.assert_ping(self.src_namespace, self.dst_addr, count=2)
2016-02-19 09:12:56.078 | 2016-02-19 09:12:56.058 | File
"neutron/tests/common/net_helpers.py", line 93, in assert_ping
2016-02-19 09:12:56.080 | 2016-02-19 09:12:56.062 | dst_ip])
2016-02-19 09:12:56.082 | 2016-02-19 09:12:56.064 | File
"neutron/agent/linux/ip_lib.py", line 885, in execute
2016-02-19 09:12:56.084 | 2016-02-19 09:12:56.066 |
log_fail_as_error=log_fail_as_error, **kwargs)
2016-02-19 09:12:56.086 | 2016-02-19 09:12:56.068 | File
"neutron/agent/linux/utils.py", line 140, in execute
2016-02-19 09:12:56.094 | 2016-02-19 09:12:56.076 | raise
RuntimeError(msg)
2016-02-19 09:12:56.102 | 2016-02-19 09:12:56.080 | RuntimeError: Exit
code: 1; Stdin: ; Stdout: PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
2016-02-19 09:12:56.109 | 2016-02-19 09:12:56.091 |
2016-02-19 09:12:56.121 | 2016-02-19 09:12:56.101 | --- 192.168.0.2 ping
statistics ---
2016-02-19 09:12:56.132 | 2016-02-19 09:12:56.113 | 2 packets
transmitted, 0 received, 100% packet loss, time 1003ms
2016-02-19 09:12:56.137 | 2016-02-19 09:12:56.118 |
2016-02-19 09:12:56.138 | 2016-02-19 09:12:56.121 | ; Stderr:
2016-02-19 09:12:56.141 | 2016-02-19 09:12:56.123 |
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1547486/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
