Dear developpers,
I found that brctl addif command was delayed only when I use X540-AT2 network
adapter.
I am making a software for critical system with using virtual bridge.
And ioctl(SIOCBRADDBR) is used to connect the network interface to the bridge.
The ioctl is expected to return within a few milliseconds.
But now, I have a trouble. The ioctl has 20msec delay until return,
only when X540-AT2 network adapter is used.
No other network adapter I have causes such a delay.
That is become a failure of my software.
Is this a bug?
This trouble also appears brctl addif command. It uses same ioctl.
So I tried some performance analysis to investigate what causes such delay.
i) CPU times
With using X540-AT2, CPU spending time is shown as follows:
# time brctl addif br3 p1p1
real 0m0.021s
user 0m0.000s
sys 0m0.001s
For reference, CPU spending time with using 82598EB AT2 Server Adapter is shown
as follows:
# time brctl addif br3 p4p1
real 0m0.001s
user 0m0.000s
sys 0m0.001s
ii)Stack trace
The kernel stack trace obtained in perf command is shown as follows:
# captured on: Mon Jun 8 16:51:11 2015
# hostname : localhost.localdomain
# os release : 3.10.0-123.el7.x86_64
# perf version : 3.10.0-123.el7.x86_64.debug # arch : x86_64 # nrcpus online :
8 # nrcpus avail : 8 # cpudesc : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz #
cpuid : GenuineIntel,6,60,3 # total memory : 3822860 kB # cmdline :
/usr/bin/perf record -g -T -c 1 brctl addif br3 p1p1 # event : name = cycles,
type = 0, config = 0x0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern =
0, excl_host = 0, excl_guest = 1, precise_ip = 0,
attr_mmap2 = 0, attr_mmap = 1, attr_mmap_data = 0 # HEADER_CPU_TOPOLOGY info
available, use -I to display # HEADER_NUMA_TOPOLOGY info available, use -I to
display # pmu mappings: cpu = 4, software = 1, tracepoint = 2, breakpoint = 5 #
======== # # Samples: 3 of event 'cycles'
# Event count (approx.): 3
#
# Overhead Command Shared Object Symbol
# ........ ....... ................. .........................
#
100.00% brctl [kernel.kallsyms] [k] native_write_msr_safe
|
|--66.67%-- 0x313d524e54565f
| 0x4019f1
| __GI___ioctl
| system_call
| sys_ioctl
| do_vfs_ioctl
| sock_ioctl
| sock_do_ioctl
| dev_ioctl
| dev_ifsioc
| |
| |--50.00%-- br_ioctl_deviceless_stub
| | add_del_if
| | br_add_if
| | port_cost
| | __ethtool_get_settings
| | ixgbe_init_thermal_sensor_thresh_generic
| | ixgbe_get_copper_link_capabilities_generic
| | ixgbe_read_phy_reg_generic
| | ixgbe_acquire_swfw_sync_X540
| | usleep_range
| | schedule_hrtimeout_range
| | schedule_hrtimeout_range_clock
| | schedule
| | __schedule
| | finish_task_switch
| | __perf_event_task_sched_in
| | perf_event_context_sched_in
| | perf_pmu_enable
| | x86_pmu_enable
| | intel_pmu_enable_all
| | native_write_msr_safe
| |
| --50.00%-- br_dev_ioctl
| add_del_if
| br_add_if
| port_cost
| __ethtool_get_settings
| ixgbe_get_settings
| ixgbe_get_copper_link_capabilities_generic
| ixgbe_read_phy_reg_generic
| ixgbe_release_swfw_sync_X540
| usleep_range
| schedule_hrtimeout_range
| schedule_hrtimeout_range_clock
| schedule
| __schedule
| finish_task_switch
| __perf_event_task_sched_in
| perf_event_context_sched_in
| perf_pmu_enable
| x86_pmu_enable
| intel_pmu_enable_all
| native_write_msr_safe
|
--33.33%-- 0x7f8066428c47
stub_execve
sys_execve
do_execve_common.isra.22
search_binary_handler
load_elf_binary
setup_new_exec
set_task_comm
perf_event_comm
perf_event_context_sched_in
perf_pmu_enable
x86_pmu_enable
intel_pmu_enable_all
native_write_msr_safe
#
# (For a higher level overview, try: perf report --sort comm,dso) #
These results show that almost delay is spent at ioctl(SIOCBRADDBR),
and usleep_range is called twice to ixgbe driver.
Both of ixgbe_acquire_swfw_sync_X540 and ixgbe_release_swfw_sync_X540 call
usleep_range(5000, 10000) just before return. It causes non-operation time
about 20msec.
I want to ask you why usleep_range has to be called there.
And can you remove or mitigate the delay?
Environment)
HW: DELL Precision T1600
OS: Redhat Enterprise Linux 7.0
kernel: 3.10.0-123.el7.x86_64
Best regards,
Shota Uehara
-----
CONFIDENTIAL: This e-mail may contain information that is confidential or
otherwise protected from disclosure and intended only for the party to whom it
is addressed. If you are not the intended recipient, please notify the sender
by return and delete this e-mail. You are hereby formally advised that any
unauthorized use, disclosure or copying of this email is strictly prohibited
and may be unlawful.
------------------------------------------------------------------------------
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
