Hi Lukas, thanks for the quick reply!
On 08.01.2013, at 19:44, Lukas Tribus <[email protected]> wrote: > >> Jan 8 18:30:59 srv11 kernel: [ 3878.272003] ------------[ cut here >> ]------------ >> Jan 8 18:30:59 srv11 kernel: [ 3878.295572] WARNING: at net/ipv4/tcp.c:1330 >> tcp_cleanup_rbuf+0x4d/0xfc() >> Jan 8 18:30:59 srv11 kernel: [ 3878.319107] Hardware name: System x3690 X5 >> -[7148Z68]- >> Jan 8 18:30:59 srv11 kernel: [ 3878.340686] cleanup rbuf bug: copied >> 7B02E4F6 seq 7B01F558 rcvnxt 7B02E4F6 >> Jan 8 18:30:59 srv11 kernel: [ 3878.363160] Modules linked in: 8021q garp >> stp llc nls_utf8 nls_cp437 vfat fat acpi_cpufreq snd_pcm cdc_ether usbnet >> mii coretemp kvm_intel kvm snd_timer snd crc32c_intel evdev joydev >> hid_generic soundcore microcode snd_page_alloc serio_raw pcspkr mperf >> tpm_tis processor ioatdma lpc_ich i2c_i801 tpm shpchp mfd_core tpm_bios >> pci_hotplug i2c_core dca thermal_sys button ext4 mbcache jbd2 crc16 dm_mod >> sg sr_mod cdrom sd_mod crc_t10dif ata_generic usbhid hid uhci_hcd ata_piix >> libata megaraid_sas ehci_hcd bnx2 usbcore scsi_mod usb_common be2net >> Jan 8 18:30:59 srv11 kernel: [ 3878.513462] Pid: 30307, comm: haproxy >> Tainted: G W 3.7.1 #1 >> Jan 8 18:30:59 srv11 kernel: [ 3878.540064] Call Trace: >> Jan 8 18:30:59 srv11 kernel: [ 3878.564947] [<ffffffff8103ef70>] ? >> warn_slowpath_common+0x78/0x8c >> Jan 8 18:30:59 srv11 kernel: [ 3878.591417] [<ffffffff8103f023>] ? >> warn_slowpath_fmt+0x45/0x4a >> Jan 8 18:30:59 srv11 kernel: [ 3878.617965] [<ffffffff812d3e02>] ? >> tcp_cleanup_rbuf+0x4d/0xfc >> Jan 8 18:30:59 srv11 kernel: [ 3878.645352] [<ffffffff812d4034>] ? >> tcp_read_sock+0x183/0x194 >> Jan 8 18:30:59 srv11 kernel: [ 3878.670621] [<ffffffff812d487d>] ? >> tcp_sendpage+0x45b/0x45b >> Jan 8 18:30:59 srv11 kernel: [ 3878.696935] [<ffffffff812d4118>] ? >> tcp_splice_read+0xd3/0x223 >> Jan 8 18:30:59 srv11 kernel: [ 3878.721845] [<ffffffff8112d9ae>] ? >> sys_splice+0x345/0x3bf >> Jan 8 18:30:59 srv11 kernel: [ 3878.746239] [<ffffffff813651a9>] ? >> system_call_fastpath+0x16/0x1b >> Jan 8 18:30:59 srv11 kernel: [ 3878.770749] ---[ end trace 91a60bafa2f9d85e >> ]--- > > > I recall we had a similar report on the ML back in September with a 3.5.0 > kernel (thread @[1]). Like the old report your one shows a tainted kernel as > well. However both reports have the G and the W flag, that probably means the > kernel is only "Tainted" because its not the first kernel warning since boot > (first warning sets WARN_ON - the W flag; G means everything in your kernel > is GPL or GPL compliant). > > Can you confirm you are running a vanilla kernel from kernel.org? If my > suspicion is correct, you should find the same kernel warning in the > beginning of your dmesg, but without the "Tainted" sign. I suggest you post > that kernel warning on the linux netdev mailing list, as Willy suggested in > the older thread. I checked the syslog and the initial not tainted message is the same as i´ve posted above. After your suggestion i´ve checked the netdev mailing list and found an interesting thread created by Willy. Major network performance regression in 3.7 - http://marc.info/?t=135742278000001&r=1&w=2 Since i´m not really aware of kernel programming, i´ll forward my initial request to this thread tomorrow - unless somebody confirms this as related in the meantime. > > >> After removing the option splice-response and option splice-request the cpu >> load instantly dropped to a almost normal level. >> So i guess there are either in haproxy or in the kernel some splice issues. > > In fact, this seems to be a kernel issue with TCP splicing. > > >> i´ll try to downgrade the kernel tomorrow to 3.7 or maybe lower. > > I believe the regression is older than that. If you have the possibility, you > could give 3.4.24 and 3.5.7 a try. If 3.4.24 works and 3.5.7 doesn't, then > the regression has been introduced with the 3.5 kernel. > > > [1] http://permalink.gmane.org/gmane.comp.web.haproxy/9560 > > > > Regards, > > Lukas > > > Regards, Christian
