New ASLR Patch

2014-09-05 Thread Shawn Webb
Hey All,

I've submitted a new revision of our ASLR patch to Phabric. It can be
applied to 11-CURRENT. The main changes include removal of the MAP_32BIT
hack for amd64, a couple bug fixes, and stylistic changes requested by a
few people. I'm looking for commentary and volunteers for testing. The link
to Phabric is below and you can download the raw patch from there.

https://reviews.freebsd.org/D473

Thanks,

Shawn
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ddb_enable=YES by default?

2014-09-05 Thread Brooks Davis
On Thu, Sep 04, 2014 at 05:02:55PM -0700, Craig Rodrigues wrote:
 Brooks,
 
 In r178450, you set the default of ddb_enable to NO:
 
 
 r178450 | brooks | 2008-04-23 15:40:59 -0700 (Wed, 23 Apr 2008) | 4 lines
 Changed paths:
M /head/etc/defaults/rc.conf
 
 Revert rev 1.332 and keep ddb scripts off by default for now.  Minidumps
 are more flexable and much text-dump like output can be produced from
 them so there's a good argument they are a better default.
 
 Index: head/etc/defaults/rc.conf
 ===
 --- head/etc/defaults/rc.conf(revision 178449)
 +++ head/etc/defaults/rc.conf(revision 178450)
 @@ -33,7 +33,7 @@
  apm_enable=NO# Set to YES to enable APM BIOS functions (or NO).
  apmd_enable=NO# Run apmd to handle APM event from userland.
  apmd_flags=# Flags to apmd (if enabled).
 -ddb_enable=YES# Load ddb scripts at boot.
 +ddb_enable=NO# Set to YES to load ddb scripts at boot.
  ddb_config=/etc/ddb.conf# ddb(8) config file.
  devd_enable=YES # Run devd, to trigger programs on device tree changes.
  devd_flags=# Additional flags for devd(8).
 
 
 
 Do you think this is OK to enable by default now?
 Developers who know what they are doing can turn it off in /etc/rc.conf.

 For the average end-user, this is super useful, because
 it loads the ddb rules in /etc/ddb.conf, which do
 useful things like enable textdumps, show all the locks, show all the
 locked vnodes,
 and reboots the box.
 
 This will allow end-users who have a problem in the field with FreeBSD,
 and are not kernel debugging experts, to get a lot of useful diagnostic
 info that can be reported back to developers on the mailing lists.
 
 Right now, a lot of times, people take camera pictures of their screen at the
 ddb prompt.  That's pretty painful. :)

IIRC John was the one who convinced me it was better to post process
text dumps than to run potentially risky ddb scripts.  I've cc'd him for
his opinion.  One way or another I belive we should default to producing
a useful crash report even at the risk of filling /var.

-- Brooks


pgpA7K6pa2EH4.pgp
Description: PGP signature


panic m_demote: m_nextpkt not NULL

2014-09-05 Thread Mark Atkinson
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

r271093 GENERIC amd64.   Received this panic in the tcp reassembly code:

Unread portion of the kernel message buffer:
panic: m_demote: m_nextpkt not NULL
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
0xfe011331b410
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe011331b4c0
vpanic() at vpanic+0x189/frame 0xfe011331b540
kassert_panic() at kassert_panic+0x139/frame 0xfe011331b5b0
m_demote() at m_demote+0x79/frame 0xfe011331b5e0
sbappendstream_locked() at sbappendstream_locked+0x4b/frame
0xfe011331b600
tcp_reass() at tcp_reass+0x3bd/frame 0xfe011331b660
tcp_do_segment() at tcp_do_segment+0x1b01/frame 0xfe011331b750
tcp_input() at tcp_input+0xf67/frame 0xfe011331b890
ip_input() at ip_input+0xce/frame 0xfe011331b8e0
netisr_dispatch_src() at netisr_dispatch_src+0x86/frame 0xfe011331b950
ether_demux() at ether_demux+0x141/frame 0xfe011331b980
ether_nh_input() at ether_nh_input+0x32a/frame 0xfe011331b9b0
netisr_dispatch_src() at netisr_dispatch_src+0x86/frame 0xfe011331ba20
ether_input() at ether_input+0x4f/frame 0xfe011331ba50
if_input() at if_input+0xa/frame 0xfe011331ba60
em_rxeof() at em_rxeof+0x2bd/frame 0xfe011331bae0
em_handle_que() at em_handle_que+0x40/frame 0xfe011331bb20
taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame
0xfe011331bb80
taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame
0xfe011331bbb0
fork_exit() at fork_exit+0x84/frame 0xfe011331bbf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfe011331bbf0
- --- trap 0, rip = 0, rsp = 0xfe011331bcb0, rbp = 0 ---
Uptime: 41m32s
Dumping 357 out of 3937
MB:..5%..14%..23%..32%..41%..54%..63%..72%..81%..95%


(kgdb) bt
#0  doadump (textdump=1) at pcpu.h:219
#1  0x8090d6b7 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:447
#2  0x8090dc58 in vpanic (fmt=value optimized out,
ap=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:746
#3  0x8090da89 in kassert_panic (fmt=value optimized out)
at /usr/src/sys/kern/kern_shutdown.c:634
#4  0x80983b59 in m_demote (m0=0xf80067044c00,
all=value optimized out) at /usr/src/sys/kern/uipc_mbuf.c:402
#5  0x8098b0ab in sbappendstream_locked (sb=0xf8009914c090,
m=0xf80067044c00) at /usr/src/sys/kern/uipc_sockbuf.c:532
#6  0x80ac15ed in tcp_reass (tp=0xf80099115000,
th=value optimized out, tlenp=value optimized out,
m=value optimized out) at /usr/src/sys/netinet/tcp_reass.c:264
#7  0x80abbb41 in tcp_do_segment (m=0xf80067044c00,
th=0xf80067091022, so=0xf8009914c000, tp=value optimized
out,
drop_hdrlen=value optimized out, tlen=1448, ti_locked=1)
at /usr/src/sys/netinet/tcp_input.c:2917
#8  0x80ab9667 in tcp_input (mp=value optimized out,
offp=value optimized out, proto=0)
at /usr/src/sys/netinet/tcp_input.c:1383
#9  0x80a4f6fe in ip_input (m=0x0)
at /usr/src/sys/netinet/ip_input.c:729
#10 0x809e9046 in netisr_dispatch_src (proto=value optimized
out,
source=value optimized out, m=0xf80067044c00)
at /usr/src/sys/net/netisr.c:968
#11 0x809dfe91 in ether_demux (ifp=value optimized out,
m=0xf80067044c00) at /usr/src/sys/net/if_ethersubr.c:775
#12 0x809e0bea in ether_nh_input (m=value optimized out)
at /usr/src/sys/net/if_ethersubr.c:582
#13 0x809e9046 in netisr_dispatch_src (proto=value optimized
out,
source=value optimized out, m=0xf80067044c00)
at /usr/src/sys/net/netisr.c:968
#14 0x809e019f in ether_input (ifp=0xf80002c08000, m=0x0)
at /usr/src/sys/net/if_ethersubr.c:683
#15 0x809dd1da in if_input (ifp=0x0, sendmp=0x0)
at /usr/src/sys/net/if.c:3909
#16 0x804dd51d in em_rxeof (count=99)
at /usr/src/sys/dev/e1000/if_em.c:4485
#17 0x804dce00 in em_handle_que (context=0xfed23000,
pending=value optimized out) at /usr/src/sys/dev/e1000/if_em.c:1522
#18 0x80956bb0 in taskqueue_run_locked (queue=0xf80002957000)
at /usr/src/sys/kern/subr_taskqueue.c:356
#19 0x809576ab in taskqueue_thread_loop (arg=value optimized
out)
at /usr/src/sys/kern/subr_taskqueue.c:623
#20 0x808db5f4 in fork_exit (
callout=0x80957610 taskqueue_thread_loop,
arg=0xfed25738, frame=0xfe011331bc00)
at /usr/src/sys/kern/kern_fork.c:977
#21 0x80d0607e in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:605
#22 0x in ?? ()
(kgdb) frame 4
#4  0x80983b59 in m_demote (m0=0xf80067044c00,
all=value optimized out) at /usr/src/sys/kern/uipc_mbuf.c:402
402 KASSERT(m-m_nextpkt == NULL,
(kgdb) list
397 m_tag_delete_chain(m, NULL);
398 m-m_flags = ~M_PKTHDR;
399 

Re: ddb_enable=YES by default?

2014-09-05 Thread John Baldwin
On Thursday, September 04, 2014 8:02:55 pm Craig Rodrigues wrote:
 Brooks,
 
 In r178450, you set the default of ddb_enable to NO:
 
 
 r178450 | brooks | 2008-04-23 15:40:59 -0700 (Wed, 23 Apr 2008) | 4 lines
 Changed paths:
M /head/etc/defaults/rc.conf
 
 Revert rev 1.332 and keep ddb scripts off by default for now.  Minidumps
 are more flexable and much text-dump like output can be produced from
 them so there's a good argument they are a better default.
 
 Index: head/etc/defaults/rc.conf
 ===
 --- head/etc/defaults/rc.conf(revision 178449)
 +++ head/etc/defaults/rc.conf(revision 178450)
 @@ -33,7 +33,7 @@
  apm_enable=NO# Set to YES to enable APM BIOS functions (or NO).
  apmd_enable=NO# Run apmd to handle APM event from userland.
  apmd_flags=# Flags to apmd (if enabled).
 -ddb_enable=YES# Load ddb scripts at boot.
 +ddb_enable=NO# Set to YES to load ddb scripts at boot.
  ddb_config=/etc/ddb.conf# ddb(8) config file.
  devd_enable=YES # Run devd, to trigger programs on device tree changes.
  devd_flags=# Additional flags for devd(8).
 
 
 
 Do you think this is OK to enable by default now?
 Developers who know what they are doing can turn it off in /etc/rc.conf.
 
 For the average end-user, this is super useful, because
 it loads the ddb rules in /etc/ddb.conf, which do
 useful things like enable textdumps, show all the locks, show all the
 locked vnodes,
 and reboots the box.
 
 This will allow end-users who have a problem in the field with FreeBSD,
 and are not kernel debugging experts, to get a lot of useful diagnostic
 info that can be reported back to developers on the mailing lists.
 
 Right now, a lot of times, people take camera pictures of their screen at the
 ddb prompt.  That's pretty painful. :)

Probably at least 50% of the time when I work with a user on a bug report,
I ask them to go into kgdb and run specific commands to extract more detailed
info (print some struct, etc.).  You can'd do that with text dumps.  All that
info is thrown away when the machine reboots.  OTOH, /var/crash/core.txt.N
provides much of the same information automatically from a minidump (and I've
seen users posting URLs to the file).  You can also implement many of the
more advanced ddb commands like 'show lockedvnodes' using gdb scripts against
a minidump (see 'lockedvnodes' command in www.freebsd.org/~jhb/gdb/gdb6).

In short, minidumps still provide far more information and can almost completely
replicate the functionality of textdumps.

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


request for documentation of m_uiotombuf and m_mbuftouio...

2014-09-05 Thread John-Mark Gurney
If someone would like to write a man page for these functions, I'll
fix them up and commit them...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic m_demote: m_nextpkt not NULL

2014-09-05 Thread Jean-Sébastien Pédron
On 05.09.2014 17:38, Mark Atkinson wrote:
 r271093 GENERIC amd64.   Received this panic in the tcp reassembly code:

Gleb Smirnof fixed this in r271123. I had this same panic yesterday
after around 3h of uptime each time, but today, everything's fine.

-- 
Jean-Sébastien Pédron



signature.asc
Description: OpenPGP digital signature


UEFI display frozen on Retina MacBook Pro

2014-09-05 Thread John Nielsen
I have a MacBook Pro Retina, Mid 2012 (MacBookPro10,1) on which I'd like to 
be able to boot FreeBSD from an external USB drive. For testing I've been using 
the mini-memstick images from the -CURRENT snapshots, most recently the one 
from 20140903.

I am able to select EFI Boot on the USB device from the Mac's boot menu, and 
it does _something_, but the screen never changes--the image of the boot menu 
is displayed indefinitely. I think it is actually booting since there is drive 
activity and the caps lock key indicator starts working a few seconds in, but 
the screen just stays the same. Thinking the resolution of the Retina display 
may have been an issue, I tried booting with it disabled (lid closed) and an 
external monitor and keyboard. The result was the same--Mac boot menu frozen on 
the external display.

Is there anything I should try to troubleshoot or debug this issue? Anything 
else I should include in a PR? I can test patches if needed (probably after 
building an image including the patch from a VM).

Thanks,

JN

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: UEFI display frozen on Retina MacBook Pro

2014-09-05 Thread Glen Barber
On Fri, Sep 05, 2014 at 11:20:21AM -0600, John Nielsen wrote:
 I have a MacBook Pro Retina, Mid 2012 (MacBookPro10,1) on which I'd like to 
 be able to boot FreeBSD from an external USB drive. For testing I've been 
 using the mini-memstick images from the -CURRENT snapshots, most recently the 
 one from 20140903.
 
 I am able to select EFI Boot on the USB device from the Mac's boot menu, 
 and it does _something_, but the screen never changes--the image of the boot 
 menu is displayed indefinitely. I think it is actually booting since there is 
 drive activity and the caps lock key indicator starts working a few seconds 
 in, but the screen just stays the same. Thinking the resolution of the Retina 
 display may have been an issue, I tried booting with it disabled (lid closed) 
 and an external monitor and keyboard. The result was the same--Mac boot menu 
 frozen on the external display.
 
 Is there anything I should try to troubleshoot or debug this issue? Anything 
 else I should include in a PR? I can test patches if needed (probably after 
 building an image including the patch from a VM).
 

To be clear, which boot menu do you see?  If you see the FreeBSD loader
menu, escape to the loader prompt and try:

set kern.vty=vt
set hw.vga.textmode=1
boot

I am a bit unclear under which conditions 'hw.vga.textmode=1' is
required, though.

Glen



pgpt99PpkqpL7.pgp
Description: PGP signature


Re: UEFI display frozen on Retina MacBook Pro

2014-09-05 Thread John Nielsen
On Sep 5, 2014, at 11:30 AM, Glen Barber g...@freebsd.org wrote:

 On Fri, Sep 05, 2014 at 11:20:21AM -0600, John Nielsen wrote:
 I have a MacBook Pro Retina, Mid 2012 (MacBookPro10,1) on which I'd like 
 to be able to boot FreeBSD from an external USB drive. For testing I've been 
 using the mini-memstick images from the -CURRENT snapshots, most recently 
 the one from 20140903.
 
 I am able to select EFI Boot on the USB device from the Mac's boot menu, 
 and it does _something_, but the screen never changes--the image of the boot 
 menu is displayed indefinitely. I think it is actually booting since there 
 is drive activity and the caps lock key indicator starts working a few 
 seconds in, but the screen just stays the same. Thinking the resolution of 
 the Retina display may have been an issue, I tried booting with it disabled 
 (lid closed) and an external monitor and keyboard. The result was the 
 same--Mac boot menu frozen on the external display.
 
 Is there anything I should try to troubleshoot or debug this issue? Anything 
 else I should include in a PR? I can test patches if needed (probably after 
 building an image including the patch from a VM).
 
 
 To be clear, which boot menu do you see?  If you see the FreeBSD loader
 menu, escape to the loader prompt and try:
 
set kern.vty=vt
set hw.vga.textmode=1
boot
 
 I am a bit unclear under which conditions 'hw.vga.textmode=1' is
 required, though.

No, I don't ever see the FreeBSD loader. I see the menu you get on a Mac when 
you hold down the option (alt) key while booting--big disk icons representing 
the bootable disks/partitions in the system. In my case it was the Macintosh 
HD volume (Mac OS Mavericks), my Windows partition, and the USB stick with the 
FreeBSD memstick image on it, which the Mac just called EFI Boot (and the 
icon was that of a USB disk). There is also a little section at the bottom that 
allows wifi network booting (if you've done all the black magic (not PXE) to 
get that to happen). It shows a circular activity animation while it scans for 
wireless networks. That animation stops when I select the USB EFI icon and 
press enter (and that is the only visual indication I get that I made a 
selection).

JN

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


mk output during builds: duplicate script for target .... ignored

2014-09-05 Thread Bjoern A. Zeeb
Started the last 48 hours at some time:

bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target clean ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.obj.mk line 130: warning: using 
previous script for clean defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target cleandepend ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.dep.mk line 207: warning: using 
previous script for cleandepend defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target distribute ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.subdir.mk line 42: warning: 
using previous script for distribute defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target lint ignored
bmake: /sys/conf/kmod.mk line 464: warning: using previous script for lint 
defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target obj ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.obj.mk line 91: warning: using 
previous script for obj defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target objlink ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.obj.mk line 101: warning: using 
previous script for objlink defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target tags ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.dep.mk line 62: warning: using 
previous script for tags defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target files ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.subdir.mk line 121: warning: 
using previous script for files defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target includes ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.subdir.mk line 121: warning: 
using previous script for includes defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target clean ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.obj.mk line 130: warning: using 
previous script for clean defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target cleandepend ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.dep.mk line 207: warning: using 
previous script for cleandepend defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target distribute ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.subdir.mk line 42: warning: 
using previous script for distribute defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target lint ignored
bmake: /sys/conf/kmod.mk line 464: warning: using previous script for lint 
defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target obj ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.obj.mk line 91: warning: using 
previous script for obj defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target objlink ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.obj.mk line 101: warning: using 
previous script for objlink defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target tags ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.dep.mk line 62: warning: using 
previous script for tags defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target files ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.subdir.mk line 121: warning: 
using previous script for files defined here
bmake: /scratch/tmp/bz/head.svn/Makefile line 282: warning: duplicate script 
for target includes ignored
bmake: /scratch/tmp/bz/head.svn/share/mk/bsd.subdir.mk line 121: warning: 
using previous script for includes defined here


— 
Bjoern A. Zeeb Come on. Learn, goddamn it., WarGames, 1983

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


[RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Hans Petter Selasky

Hi,

I've tested the attached patch with success and would like to have some 
feedback from other FreeBSD network developers. The problem is that the 
current TSO limitation only limits the number of bytes that can be 
transferred in a TSO packet and not the number of mbuf's.


The current solution is to have a quick and dirty custom m_dup() in the 
TX path to re-allocate the mbuf chains into 4K ones to make it simple. 
All of this hack can be avoided if the definition of the TSO limit can 
be changed a bit, like shown here:



 /*
+ * Structure defining hardware TSO limits.
+ */
+struct if_tso_limit {
+   u_int raw_value[0]; /* access all fields as one */
+   u_char frag_count;  /* maximum number of fragments: 1..255 */
+   u_char frag_size_log2;  /* maximum fragment size: 2 ** (12..16) */
+   u_char hdr_size_log2;   /* maximum header size: 2 ** (2..8) */
+   u_char reserved;/* zero */
+};


First we need to know the maximum fragment count. Typical value is 32.
Second we need to know the maximum fragment size. Typical value is 4K.
Last we need to know of any headers that should be subtracted from the 
maximum. Hence this code is running in the fast path, I would like to 
use u_char for all fields and allow copy-only access as a u_int as 
an optimization. This avoids cludges and messing with additional header 
files.


I would like to push this patch after some more testing to -current and 
then to 10-stable hopefully before the coming 10-release, because the 
current solution is affecting performance of the Mellanox based network 
adapters in an unfair way. For example by setting the current TSO limit 
to 32KBytes which will be OK for all-2K fragments, we see a severe 
degradation in performance. Even though the hardware is fully capable of 
transmitting 16 4K mbufs.


Comments and reviews are welcome!

--HPS
=== sys/dev/oce/oce_if.c
==
--- sys/dev/oce/oce_if.c	(revision 270996)
+++ sys/dev/oce/oce_if.c	(local)
@@ -1731,7 +1731,9 @@
 	sc-ifp-if_baudrate = IF_Gbps(10);
 
 #if __FreeBSD_version = 100
-	sc-ifp-if_hw_tsomax = OCE_MAX_TSO_SIZE;
+	sc-ifp-if_hw_tsomax.frag_count = 29;		/* 29 elements */
+	sc-ifp-if_hw_tsomax.frag_size_log2 = 12;	/* 4K */
+	sc-ifp-if_hw_tsomax.hdr_size_log2 = 5;	/* ETH+VLAN  2**5 */
 #endif
 
 	ether_ifattach(sc-ifp, sc-macaddr.mac_addr);
=== sys/dev/oce/oce_if.h
==
--- sys/dev/oce/oce_if.h	(revision 270996)
+++ sys/dev/oce/oce_if.h	(local)
@@ -152,7 +152,6 @@
 #define OCE_MAX_TX_ELEMENTS		29
 #define OCE_MAX_TX_DESC			1024
 #define OCE_MAX_TX_SIZE			65535
-#define OCE_MAX_TSO_SIZE		(65535 - ETHER_HDR_LEN)
 #define OCE_MAX_RX_SIZE			4096
 #define OCE_MAX_RQ_POSTS		255
 #define OCE_DEFAULT_PROMISCUOUS		0
=== sys/dev/vmware/vmxnet3/if_vmx.c
==
--- sys/dev/vmware/vmxnet3/if_vmx.c	(revision 270996)
+++ sys/dev/vmware/vmxnet3/if_vmx.c	(local)
@@ -1722,7 +1722,9 @@
 	ifp-if_flags = IFF_BROADCAST | IFF_SIMPLEX | IFF_MULTICAST;
 	ifp-if_init = vmxnet3_init;
 	ifp-if_ioctl = vmxnet3_ioctl;
-	ifp-if_hw_tsomax = VMXNET3_TSO_MAXSIZE;
+	ifp-if_hw_tsomax.frag_count = VMXNET3_TX_MAXSEGS;
+	ifp-if_hw_tsomax.frag_size_log2 = VMXNET3_TX_MAXSEGSHIFT;
+	ifp-if_hw_tsomax.hdr_size_log2 = 5;	/* ETH+VLAN  2**5 */
 
 #ifdef VMXNET3_LEGACY_TX
 	ifp-if_start = vmxnet3_start;
=== sys/dev/vmware/vmxnet3/if_vmxvar.h
==
--- sys/dev/vmware/vmxnet3/if_vmxvar.h	(revision 270996)
+++ sys/dev/vmware/vmxnet3/if_vmxvar.h	(local)
@@ -277,14 +277,13 @@
  */
 #define VMXNET3_TX_MAXSEGS		32
 #define VMXNET3_TX_MAXSIZE		(VMXNET3_TX_MAXSEGS * MCLBYTES)
-#define VMXNET3_TSO_MAXSIZE \
-(VMXNET3_TX_MAXSIZE - sizeof(struct ether_vlan_header))
 
 /*
  * Maximum support Tx segments size. The length field in the
  * Tx descriptor is 14 bits.
  */
-#define VMXNET3_TX_MAXSEGSIZE		(1  14)
+#define VMXNET3_TX_MAXSEGSHIFT		14
+#define VMXNET3_TX_MAXSEGSIZE		(1  VMXNET3_TX_MAXSEGSHIFT)
 
 /*
  * The maximum number of Rx segments we accept. When LRO is enabled,
=== sys/dev/xen/netfront/netfront.c
==
--- sys/dev/xen/netfront/netfront.c	(revision 270996)
+++ sys/dev/xen/netfront/netfront.c	(local)
@@ -134,7 +134,6 @@
  * to mirror the Linux MAX_SKB_FRAGS constant.
  */
 #define	MAX_TX_REQ_FRAGS (65536 / PAGE_SIZE + 2)
-#define	NF_TSO_MAXBURST ((IP_MAXPACKET / PAGE_SIZE) * MCLBYTES)
 
 #define RX_COPY_THRESHOLD 256
 
@@ -2102,7 +2101,9 @@
 	
 	ifp-if_hwassist = XN_CSUM_FEATURES;
 	ifp-if_capabilities = IFCAP_HWCSUM;
-	ifp-if_hw_tsomax = NF_TSO_MAXBURST;
+	ifp-if_hw_tsomax.frag_count = MAX_TX_REQ_FRAGS;
+	ifp-if_hw_tsomax.frag_size_log2 = PAGE_SHIFT;
+	ifp-if_hw_tsomax.hdr_size_log2 = 5;	/* ETH+VLAN  2**5 */
 	
 	ether_ifattach(ifp, np-mac);
 	

Re: request for documentation of m_uiotombuf and m_mbuftouio...

2014-09-05 Thread Gleb Smirnoff
On Fri, Sep 05, 2014 at 09:44:32AM -0700, John-Mark Gurney wrote:
J If someone would like to write a man page for these functions, I'll
J fix them up and commit them...

Well, the entire mbuf(9) needs to be examined and fixed. I promised
myself to do that before stable/11.

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: panic m_demote: m_nextpkt not NULL

2014-09-05 Thread Gleb Smirnoff
On Fri, Sep 05, 2014 at 08:38:10AM -0700, Mark Atkinson wrote:
M r271093 GENERIC amd64.   Received this panic in the tcp reassembly code:
M Unread portion of the kernel message buffer:
M panic: m_demote: m_nextpkt not NULL
M cpuid = 0

Sorry for that. Was fixed in r271123.

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ddb_enable=YES by default?

2014-09-05 Thread Craig Rodrigues
On Fri, Sep 5, 2014 at 7:54 AM, John Baldwin j...@freebsd.org wrote:

 Probably at least 50% of the time when I work with a user on a bug report,
 I ask them to go into kgdb and run specific commands to extract more detailed
 info (print some struct, etc.).

Sure, I understand, but you are not working with every user who
encounters a kernel panic in FreeBSD.  For the average or casual
FreeBSD user, such as desktop
users of FreeBSD or PC-BSD, wouldn't it be better
to have ddb_enable=YES be the default in FreeBSD?  The ddb script
there does a fairly reasonable
job of gathering some useful info which can be analyzed later, and
then rebooting the box.

For more expert users, or people developing products, they can set
ddb_enable=NO
and do more advanced debugging.  Or hook into /etc/rc.d/ddb and define
a different
ddb script which doesn't do textdumps on kernel panic.

--
Craig
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ddb_enable=YES by default?

2014-09-05 Thread Peter Wemm
On Friday 05 September 2014 13:51:24 Craig Rodrigues wrote:
 On Fri, Sep 5, 2014 at 7:54 AM, John Baldwin j...@freebsd.org wrote:
  Probably at least 50% of the time when I work with a user on a bug report,
  I ask them to go into kgdb and run specific commands to extract more
  detailed info (print some struct, etc.).
 
 Sure, I understand, but you are not working with every user who
 encounters a kernel panic in FreeBSD.  For the average or casual
 FreeBSD user, such as desktop
 users of FreeBSD or PC-BSD, wouldn't it be better
 to have ddb_enable=YES be the default in FreeBSD?  The ddb script
 there does a fairly reasonable
 job of gathering some useful info which can be analyzed later, and
 then rebooting the box.
 
 For more expert users, or people developing products, they can set
 ddb_enable=NO
 and do more advanced debugging.  Or hook into /etc/rc.d/ddb and define
 a different
 ddb script which doesn't do textdumps on kernel panic.

I think what John was saying was at that point it's too late.  The loss of the 
crash dump means the one shot at getting more information is gone.

For reproducable crashes, yes, an end user could just flip the bit.  But for a 
one-off, it's too late.

-- 
Peter Wemm - pe...@wemm.org; pe...@freebsd.org; pe...@yahoo-inc.com; KI6FJV
UTF-8: for when a ' or ... just won\342\200\231t do\342\200\246

signature.asc
Description: This is a digitally signed message part.


Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Eric Joyner
There are some concerns if we use this with devices that ixl supports:

- The maximum fragment size is 16KB-1, which isn't a power of 2.

- You can't get the maximum TSO size for ixl devices by multiplying the
maximum number of fragments by the maximum size.
Instead the number of fragments is AFAIK unlimited, but a segment can only
span 8 mbufs (including the [up to 3] mbufs containing the header), and the
maximum TSO size is 256KB.

And one question:

- Is hdr_size_log2 supposed to be the length of the L2 header? We can fit
254 L2 bytes in our hardware during a TSO, so if that's the value, I guess
that's fine, barring the it-not-being-a-power-of-2 issue.

With all that said, the 8 mbuf limit per segment issue is a TSO limitation
that we'd like to notify the stack about, so I wonder if that could be
incorporated along with this. Right now, our driver checks to see if a
segment in a TSO spans more than six mbufs and then m_defrag()'s the entire
chain if one exists; it's less than optimal but necessary to prevent errors.

- Eric

---
- Eric Joyner


On Fri, Sep 5, 2014 at 11:37 AM, Hans Petter Selasky h...@selasky.org
wrote:

 Hi,

 I've tested the attached patch with success and would like to have some
 feedback from other FreeBSD network developers. The problem is that the
 current TSO limitation only limits the number of bytes that can be
 transferred in a TSO packet and not the number of mbuf's.

 The current solution is to have a quick and dirty custom m_dup() in the TX
 path to re-allocate the mbuf chains into 4K ones to make it simple. All of
 this hack can be avoided if the definition of the TSO limit can be changed
 a bit, like shown here:


  /*
 + * Structure defining hardware TSO limits.
 + */
 +struct if_tso_limit {
 +   u_int raw_value[0]; /* access all fields as one */
 +   u_char frag_count;  /* maximum number of fragments: 1..255 */
 +   u_char frag_size_log2;  /* maximum fragment size: 2 ** (12..16) */
 +   u_char hdr_size_log2;   /* maximum header size: 2 ** (2..8) */
 +   u_char reserved;/* zero */
 +};


 First we need to know the maximum fragment count. Typical value is 32.
 Second we need to know the maximum fragment size. Typical value is 4K.
 Last we need to know of any headers that should be subtracted from the
 maximum. Hence this code is running in the fast path, I would like to use
 u_char for all fields and allow copy-only access as a u_int as an
 optimization. This avoids cludges and messing with additional header files.

 I would like to push this patch after some more testing to -current and
 then to 10-stable hopefully before the coming 10-release, because the
 current solution is affecting performance of the Mellanox based network
 adapters in an unfair way. For example by setting the current TSO limit to
 32KBytes which will be OK for all-2K fragments, we see a severe degradation
 in performance. Even though the hardware is fully capable of transmitting
 16 4K mbufs.

 Comments and reviews are welcome!

 --HPS

 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ddb_enable=YES by default?

2014-09-05 Thread John Baldwin
On Friday, September 05, 2014 5:08:07 pm Peter Wemm wrote:
 On Friday 05 September 2014 13:51:24 Craig Rodrigues wrote:
  On Fri, Sep 5, 2014 at 7:54 AM, John Baldwin j...@freebsd.org wrote:
   Probably at least 50% of the time when I work with a user on a bug report,
   I ask them to go into kgdb and run specific commands to extract more
   detailed info (print some struct, etc.).
  
  Sure, I understand, but you are not working with every user who
  encounters a kernel panic in FreeBSD.  For the average or casual
  FreeBSD user, such as desktop
  users of FreeBSD or PC-BSD, wouldn't it be better
  to have ddb_enable=YES be the default in FreeBSD?  The ddb script
  there does a fairly reasonable
  job of gathering some useful info which can be analyzed later, and
  then rebooting the box.
  
  For more expert users, or people developing products, they can set
  ddb_enable=NO
  and do more advanced debugging.  Or hook into /etc/rc.d/ddb and define
  a different
  ddb script which doesn't do textdumps on kernel panic.
 
 I think what John was saying was at that point it's too late.  The loss of 
 the 
 crash dump means the one shot at getting more information is gone.
 
 For reproducable crashes, yes, an end user could just flip the bit.  But for 
 a 
 one-off, it's too late.

Also, crashinfo is already enabled by default.  If a user enables crash dumps
in the installer, they will have a nice /var/crash/core.txt.N that they can
post to the mailing lists just as easily as the text dump you envision.  And in
fact, I've seen our users already doing this.  (Have you looked at a
/var/crash/core.txt.N file yet?)

-- 
John Baldwin
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ddb_enable=YES by default?

2014-09-05 Thread Larry Rosenman

On 2014-09-05 16:35, John Baldwin wrote:

On Friday, September 05, 2014 5:08:07 pm Peter Wemm wrote:

On Friday 05 September 2014 13:51:24 Craig Rodrigues wrote:
 On Fri, Sep 5, 2014 at 7:54 AM, John Baldwin j...@freebsd.org wrote:
  Probably at least 50% of the time when I work with a user on a bug report,
  I ask them to go into kgdb and run specific commands to extract more
  detailed info (print some struct, etc.).

 Sure, I understand, but you are not working with every user who
 encounters a kernel panic in FreeBSD.  For the average or casual
 FreeBSD user, such as desktop
 users of FreeBSD or PC-BSD, wouldn't it be better
 to have ddb_enable=YES be the default in FreeBSD?  The ddb script
 there does a fairly reasonable
 job of gathering some useful info which can be analyzed later, and
 then rebooting the box.

 For more expert users, or people developing products, they can set
 ddb_enable=NO
 and do more advanced debugging.  Or hook into /etc/rc.d/ddb and define
 a different
 ddb script which doesn't do textdumps on kernel panic.

I think what John was saying was at that point it's too late.  The 
loss of the

crash dump means the one shot at getting more information is gone.

For reproducable crashes, yes, an end user could just flip the bit.  
But for a

one-off, it's too late.


Also, crashinfo is already enabled by default.  If a user enables crash 
dumps
in the installer, they will have a nice /var/crash/core.txt.N that they 
can
post to the mailing lists just as easily as the text dump you envision. 
 And in

fact, I've seen our users already doing this.  (Have you looked at a
/var/crash/core.txt.N file yet?)
I've in fact done just that (posted the top part of a core.txt.N file, 
and gotten VERY good results

from the list(s).

--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: l...@lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Hans Petter Selasky

On 09/05/14 23:19, Eric Joyner wrote:

There are some concerns if we use this with devices that ixl supports:

- The maximum fragment size is 16KB-1, which isn't a power of 2.



Hi Eric,

Multiplying by powers of two are more fast, than non-powers of two. So 
in this case you would have to use 8KB as a maximum.



- You can't get the maximum TSO size for ixl devices by multiplying the
maximum number of fragments by the maximum size.
Instead the number of fragments is AFAIK unlimited, but a segment can only
span 8 mbufs (including the [up to 3] mbufs containing the header), and the
maximum TSO size is 256KB.

And one question:

- Is hdr_size_log2 supposed to be the length of the L2 header? We can fit
254 L2 bytes in our hardware during a TSO, so if that's the value, I guess
that's fine, barring the it-not-being-a-power-of-2 issue.


This is the ethernet / vlan headers. It is added with the TCP/IP-header 
in the end.




With all that said, the 8 mbuf limit per segment issue is a TSO limitation
that we'd like to notify the stack about, so I wonder if that could be
incorporated along with this. Right now, our driver checks to see if a
segment in a TSO spans more than six mbufs and then m_defrag()'s the entire
chain if one exists; it's less than optimal but necessary to prevent errors.


It is not impossible to move from log2 syntax to non-log2 syntax, hence 
the logic will be exactly the same, only that the required division and 
multiplication will have a bit overhead I guess.


Could you make a patch on top of my patch with the changes you think are 
required to fully support the ixl hardware? Or propose a new patch which 
also serves the MLX needs?


Thank you!

--HPS

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Rick Macklem
Hans Petter Selesky wrote:
 On 09/05/14 23:19, Eric Joyner wrote:
  There are some concerns if we use this with devices that ixl
  supports:
 
  - The maximum fragment size is 16KB-1, which isn't a power of 2.
 
 
 Hi Eric,
 
 Multiplying by powers of two are more fast, than non-powers of two.
 So
 in this case you would have to use 8KB as a maximum.
 
Well, I'm no architecture expert, but I really doubt the CPU delay of a
non-power of 2 multiply/divide is significant related to doing smaller
TSO segments. Long ago (as in 1970s) I did work on machines where shifts
for power of 2 multiply/divide was preferable, but these days I doubt it
is going to matter??

  - You can't get the maximum TSO size for ixl devices by multiplying
  the
  maximum number of fragments by the maximum size.
  Instead the number of fragments is AFAIK unlimited, but a segment
  can only
  span 8 mbufs (including the [up to 3] mbufs containing the header),
  and the
  maximum TSO size is 256KB.
 
  And one question:
 
  - Is hdr_size_log2 supposed to be the length of the L2 header? We
  can fit
  254 L2 bytes in our hardware during a TSO, so if that's the value,
  I guess
  that's fine, barring the it-not-being-a-power-of-2 issue.
 
 This is the ethernet / vlan headers. It is added with the
 TCP/IP-header
 in the end.
 
 
  With all that said, the 8 mbuf limit per segment issue is a TSO
  limitation
  that we'd like to notify the stack about, so I wonder if that could
  be
  incorporated along with this. Right now, our driver checks to see
  if a
  segment in a TSO spans more than six mbufs and then m_defrag()'s
  the entire
  chain if one exists; it's less than optimal but necessary to
  prevent errors.
 
At this time, if there is a limit of 8 TSO segments (mbufs) in a
transmit list, you will need to set:
 ifp-if_hw_tsomax = 8 * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN);

- just before the call to
 ether_ifattach(ifp);

I do have an untested patch (attached in case anyone is interested) which
adds if_hw_tsomaxseg that drivers can set to their maximum number of transmit
segments (mbufs) fot TSO. This value is then used by tcp_output() to generate
appropriately sized TSO segments.
However, I'm just working on getting a way to test this patch, so I can't say
if/when it will be in head.

rick

 

 It is not impossible to move from log2 syntax to non-log2 syntax,
 hence
 the logic will be exactly the same, only that the required division
 and
 multiplication will have a bit overhead I guess.
 
 Could you make a patch on top of my patch with the changes you think
 are
 required to fully support the ixl hardware? Or propose a new patch
 which
 also serves the MLX needs?
 
 Thank you!
 
 --HPS
 
 ___
 freebsd-...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-net
 To unsubscribe, send any mail to
 freebsd-net-unsubscr...@freebsd.org
 
--- kern/uipc_sockbuf.c.sav	2014-01-30 20:27:17.0 -0500
+++ kern/uipc_sockbuf.c	2014-01-30 22:12:08.0 -0500
@@ -965,6 +965,39 @@ sbsndptr(struct sockbuf *sb, u_int off, 
 }
 
 /*
+ * Return the first mbuf for the provided offset.
+ */
+struct mbuf *
+sbsndmbuf(struct sockbuf *sb, u_int off, long *first_len)
+{
+	struct mbuf *m;
+
+	KASSERT(sb-sb_mb != NULL, (%s: sb_mb is NULL, __func__));
+
+	*first_len = 0;
+	/*
+	 * Is off below stored offset? Happens on retransmits.
+	 * If so, just use sb_mb.
+	 */
+	if (sb-sb_sndptr == NULL || sb-sb_sndptroff  off)
+		m = sb-sb_mb;
+	else {
+		m = sb-sb_sndptr;
+		off -= sb-sb_sndptroff;
+	}
+	while (off  0  m != NULL) {
+		if (off  m-m_len)
+			break;
+		off -= m-m_len;
+		m = m-m_next;
+	}
+	if (m != NULL)
+		*first_len = m-m_len - off;
+
+	return (m);
+}
+
+/*
  * Drop a record off the front of a sockbuf and move the next record to the
  * front.
  */
--- sys/sockbuf.h.sav	2014-01-30 20:42:28.0 -0500
+++ sys/sockbuf.h	2014-01-30 22:08:43.0 -0500
@@ -153,6 +153,8 @@ int	sbreserve_locked(struct sockbuf *sb,
 	struct thread *td);
 struct mbuf *
 	sbsndptr(struct sockbuf *sb, u_int off, u_int len, u_int *moff);
+struct mbuf *
+	sbsndmbuf(struct sockbuf *sb, u_int off, long *first_len);
 void	sbtoxsockbuf(struct sockbuf *sb, struct xsockbuf *xsb);
 int	sbwait(struct sockbuf *sb);
 int	sblock(struct sockbuf *sb, int flags);
--- netinet/tcp_input.c.sav	2014-01-30 19:37:52.0 -0500
+++ netinet/tcp_input.c	2014-01-30 19:39:07.0 -0500
@@ -3627,6 +3627,7 @@ tcp_mss(struct tcpcb *tp, int offer)
 	if (cap.ifcap  CSUM_TSO) {
 		tp-t_flags |= TF_TSO;
 		tp-t_tsomax = cap.tsomax;
+		tp-t_tsomaxsegs = cap.tsomaxsegs;
 	}
 }
 
--- netinet/tcp_output.c.sav	2014-01-30 18:55:15.0 -0500
+++ netinet/tcp_output.c	2014-01-30 22:18:56.0 -0500
@@ -166,8 +166,8 @@ int
 tcp_output(struct tcpcb *tp)
 {
 	struct socket *so = tp-t_inpcb-inp_socket;
-	long len, recwin, sendwin;
-	int off, flags, error = 0;	/* Keep compiler happy */
+	long len, recwin, 

Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Rick Macklem
Hans Petter Selasky wrote:
 Hi,
 
 I've tested the attached patch with success and would like to have
 some
 feedback from other FreeBSD network developers. The problem is that
 the
 current TSO limitation only limits the number of bytes that can be
 transferred in a TSO packet and not the number of mbuf's.
 
 The current solution is to have a quick and dirty custom m_dup() in
 the
 TX path to re-allocate the mbuf chains into 4K ones to make it
 simple.
 All of this hack can be avoided if the definition of the TSO limit
 can
 be changed a bit, like shown here:
 
 
   /*
 + * Structure defining hardware TSO limits.
 + */
 +struct if_tso_limit {
 +   u_int raw_value[0]; /* access all fields as one */
 +   u_char frag_count;  /* maximum number of fragments:
 1..255 */
 +   u_char frag_size_log2;  /* maximum fragment size: 2 **
 (12..16) */
 +   u_char hdr_size_log2;   /* maximum header size: 2 ** (2..8)
 */
 +   u_char reserved;/* zero */
 +};
 
 
 First we need to know the maximum fragment count. Typical value is
 32.
 Second we need to know the maximum fragment size. Typical value is
 4K.
 Last we need to know of any headers that should be subtracted from
 the
 maximum. Hence this code is running in the fast path, I would like to
 use u_char for all fields and allow copy-only access as a u_int
 as
 an optimization. This avoids cludges and messing with additional
 header
 files.
 
 I would like to push this patch after some more testing to -current
 and
 then to 10-stable hopefully before the coming 10-release, because the
 current solution is affecting performance of the Mellanox based
 network
 adapters in an unfair way. For example by setting the current TSO
 limit
 to 32KBytes which will be OK for all-2K fragments, we see a severe
 degradation in performance. Even though the hardware is fully capable
 of
 transmitting 16 4K mbufs.
 
Ok, I didn't see this until now, but I will take a look at the patch.

My main comment is that I tried using a mix of 2K and 4K mbuf clusters in
NFS and was able (with some effort) get the UMA allocator all messed up, where
it would basically be stuck because it couldn't allocate boundary tags.

As such, until this issue w.r.t. UMA is rssolved, mixing MCLBYTES and MPAGESIZE
clusters is very dangerous imho. (alc@ did send me a simple patch related to 
this
UMA problem, but I haven't been able to test it yet.)

rick
ps: For the M_WAITOK case, the allocator problem shows up as threads sleeping
on btallo which happens in vmem_bt_alloc() in kern/subr_vmem.c.
It may never happen on 64bit arches, but it can definitely happen on i386.

 Comments and reviews are welcome!
 
 --HPS
 
 ___
 freebsd-...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-net
 To unsubscribe, send any mail to
 freebsd-net-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Hans Petter Selasky

On 09/06/14 00:09, Rick Macklem wrote:

Hans Petter Selesky wrote:

On 09/05/14 23:19, Eric Joyner wrote:

There are some concerns if we use this with devices that ixl
supports:

- The maximum fragment size is 16KB-1, which isn't a power of 2.



Hi Eric,

Multiplying by powers of two are more fast, than non-powers of two.
So
in this case you would have to use 8KB as a maximum.


Well, I'm no architecture expert, but I really doubt the CPU delay of a
non-power of 2 multiply/divide is significant related to doing smaller
TSO segments. Long ago (as in 1970s) I did work on machines where shifts
for power of 2 multiply/divide was preferable, but these days I doubt it
is going to matter??



Hi,

You also need to patch LAGG and VLAN drivers?

--HPS

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread Rick Macklem
Hans Petter Selasky wrote:
 On 09/06/14 00:09, Rick Macklem wrote:
  Hans Petter Selesky wrote:
  On 09/05/14 23:19, Eric Joyner wrote:
  There are some concerns if we use this with devices that ixl
  supports:
 
  - The maximum fragment size is 16KB-1, which isn't a power of 2.
 
 
  Hi Eric,
 
  Multiplying by powers of two are more fast, than non-powers of
  two.
  So
  in this case you would have to use 8KB as a maximum.
 
  Well, I'm no architecture expert, but I really doubt the CPU delay
  of a
  non-power of 2 multiply/divide is significant related to doing
  smaller
  TSO segments. Long ago (as in 1970s) I did work on machines where
  shifts
  for power of 2 multiply/divide was preferable, but these days I
  doubt it
  is going to matter??
 
 
 Hi,
 
 You also need to patch LAGG and VLAN drivers?
 
Yep. I already ran into the fact that these drivers didn't pass
if_hw_tsomax up and patched them for that recently. The same will
be necessary for if_hw_tsomaxseg if/when it goes into head.

As I said, this patch is currently completely untested and, even
once I get it tested/working, there will need to be a discussion
on freebsd-net@ w.r.t. whether it is appropriate for head.

I will take a look at your patch around Monday.

Btw, when setting if_hw_tsomax as I suggested in the first post,
you will still end up doing a lot of m_defrag() calls for NFS
RPC messages, but at least they will get through.

rick

 --HPS
 
 ___
 freebsd-...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-net
 To unsubscribe, send any mail to
 freebsd-net-unsubscr...@freebsd.org
 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: UEFI display frozen on Retina MacBook Pro

2014-09-05 Thread Huang Wen Hui
Hi,


2014-09-06 1:20 GMT+08:00 John Nielsen li...@jnielsen.net:

 I have a MacBook Pro Retina, Mid 2012 (MacBookPro10,1) on which I'd like
 to be able to boot FreeBSD from an external USB drive. For testing I've
 been using the mini-memstick images from the -CURRENT snapshots, most
 recently the one from 20140903.


This IMAC model can UEFI boot when I tried before. I install rEFInd(
http://www.rodsbooks.com/refind/) as boot manager, You can see FreeBSD boot
information  under rEFInd.

Cheers,

Huang Wen Hui

 I am able to select EFI Boot on the USB device from the Mac's boot menu,
 and it does _something_, but the screen never changes--the image of the
 boot menu is displayed indefinitely. I think it is actually booting since
 there is drive activity and the caps lock key indicator starts working a
 few seconds in, but the screen just stays the same. Thinking the resolution
 of the Retina display may have been an issue, I tried booting with it
 disabled (lid closed) and an external monitor and keyboard. The result was
 the same--Mac boot menu frozen on the external display.

 Is there anything I should try to troubleshoot or debug this issue?
 Anything else I should include in a PR? I can test patches if needed
 (probably after building an image including the patch from a VM).

 Thanks,

 JN

 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: [RFC] Patch to improve TSO limitation formula in general

2014-09-05 Thread John-Mark Gurney
Hans Petter Selasky wrote this message on Fri, Sep 05, 2014 at 20:37 +0200:
 I've tested the attached patch with success and would like to have some 
 feedback from other FreeBSD network developers. The problem is that the 
 current TSO limitation only limits the number of bytes that can be 
 transferred in a TSO packet and not the number of mbuf's.
 
 The current solution is to have a quick and dirty custom m_dup() in the 
 TX path to re-allocate the mbuf chains into 4K ones to make it simple. 
 All of this hack can be avoided if the definition of the TSO limit can 
 be changed a bit, like shown here:
 
 
  /*
 + * Structure defining hardware TSO limits.
 + */
 +struct if_tso_limit {
 +   u_int raw_value[0]; /* access all fields as one */
 +   u_char frag_count;  /* maximum number of fragments: 1..255 */
 +   u_char frag_size_log2;  /* maximum fragment size: 2 ** (12..16) */
 +   u_char hdr_size_log2;   /* maximum header size: 2 ** (2..8) */
 +   u_char reserved;/* zero */
 +};

Please make this a union if you really need to access the raw_value,
or just drop it...  Is this done to fit in the u_int t_tsomax that
is in tcpcb?  Also, I couldn't find code, but if the tcp connection
needs to be sent out a different interface that has more restrictive
tso requirements, do we properly handle this case?  My quick reading
of the code seems to imply that we only get the TSO requirements on
connection and never update it...

As these are per if, saving memory by packing them isn't really that
effective these days...  Per the later comments, yes, a shift MAY be
faster than a full mul/div by a cycle or two, but this won't make that
huge of a difference when dealing with it..  If the programmer has to
use crazy macros or do the math EVERY time they use the fields, this
will end up w/ less readable/maintainable code at the cost of improving
performance by maybe .001%, so my vote is for u_int's instead, and
convert to their sizes properly...

Comments on the patch:
You can drop the .reserve initalization...  It is common C knowlege
that unassigned members are assigned zero...

The IF_TSO_LIMIT_CMP macros seems excesive... Do you ever see a need
to use other operators? and if so, would they be useful?  I'd just
convert it to:
#define IF_TSO_LIMIT_EQ(a, b)   ((a)-raw_value[0] == (b)-raw_value[0])

I am a bit puzzled by this code:
+   /* Check if fragment limit will be exceeded */
+   if (cur_frags = rem_frags) {
+   max_len += min(cur_length, rem_frags  
if_hw_tsomax.frag_size_log2);
+   break;
+   }

specificly the max_len += line...  The code seems to say if we would
overrun the remaining frags (maybe you want a  instead of =) we
increase max_len...  seems like if we include this frag that would
put us over the limit that we should just skip it?  (break w/o increasing
max_len)..

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 All that I will do, has been done, All that I have, has not.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Looking For a New Casino?

2014-09-05 Thread Joe Nosay
http://divineglobal.com/ckib/icrxfpwm.briszwj
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: Looking For a New Casino?

2014-09-05 Thread Joe Nosay
This is not from me.

On Fri, Sep 5, 2014 at 10:03 PM, Joe Nosay superbisq...@gmail.com wrote:

 http://divineglobal.com/ckib/icrxfpwm.briszwj
 ___
 freebsd-...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-ppc
 To unsubscribe, send any mail to freebsd-ppc-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: make installworld commands used to generate manifest for makefs?

2014-09-05 Thread Kevin Oberman
On Thu, Sep 4, 2014 at 8:36 AM, Boris Samorodov b...@passap.ru wrote:

 03.09.2014 21:01, Nenhum_de_Nos пишет:
 
 
  On September 3, 2014 12:02:24 PM GMT-03:00, Boris Samorodov 
 b...@passap.ru wrote:
  28.08.2014 23:02, Craig Rodrigues пишет:
 
  I did this:
 
  make -DDB_FROM_SRC -DNO_ROOT installkernel DESTDIR=/tmp/test4
  make -DDB_FROM_SRC -DNO_ROOT installworld DESTDIR=/tmp/test4
  make -DDB_FROM_SRC -DNO_ROOT distribution DESTDIR=/tmp/test4
 
  /tmp/test4/METALOG was created, but it did not seem to have
  /boot/kernel/kernel or
  any kernel modules.  Is that expected?
 
  For a new installation installworld should be done first (it creates
  the needed directory infrastructure). And then one may do
  installkernel.
 
  As I read from so much time ago to install first kernel, starting from
 which FreeBSD version I should change?

 It seems to be true for ages. Take a look at /usr/src/Makefile,
 section To cross-install current onto a separate partition.

 --
 WBR, Boris Samorodov (bsam)
 FreeBSD Committer, http://www.FreeBSD.org The Power To Serve


Please understand that this is true when doing a cross-install where a
system with the required directory structure to install a kernel is not yet
present. It is not true when updating a system on which is existing FreeBSD
is running. When updating a system from sources, you should always
installkernel first and then reboot to make sure you have a working kernel
before installworld blows away the tools needed to fix a kernel problem.
--
R. Kevin Oberman, Network Engineer, Retired
E-mail: rkober...@gmail.com
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: make installworld commands used to generate manifest for makefs?

2014-09-05 Thread Garrett Cooper

 On Sep 4, 2014, at 8:36, Boris Samorodov b...@passap.ru wrote:
 
 03.09.2014 21:01, Nenhum_de_Nos пишет:
 
 
 On September 3, 2014 12:02:24 PM GMT-03:00, Boris Samorodov 
 b...@passap.ru wrote:
 28.08.2014 23:02, Craig Rodrigues пишет:
 
 I did this:
 
 make -DDB_FROM_SRC -DNO_ROOT installkernel DESTDIR=/tmp/test4
 make -DDB_FROM_SRC -DNO_ROOT installworld DESTDIR=/tmp/test4
 make -DDB_FROM_SRC -DNO_ROOT distribution DESTDIR=/tmp/test4
 
 /tmp/test4/METALOG was created, but it did not seem to have
 /boot/kernel/kernel or
 any kernel modules.  Is that expected?
 
 For a new installation installworld should be done first (it creates
 the needed directory infrastructure). And then one may do
 installkernel.
 
 As I read from so much time ago to install first kernel, starting from which 
 FreeBSD version I should change?
 
 It seems to be true for ages. Take a look at /usr/src/Makefile,
 section To cross-install current onto a separate partition.

If you mkdir DESTDIR first, then run the canonical build steps, that would work 
as well :).
Cheers!
-Garrett
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org