Re: svn commit: r314831 - head/usr.bin/fortune

[Alfred Perlstein]

On 3/6/17 10:07 PM, Jordan Hubbard wrote:

[ Charset ISO-Latin1 unsupported, converting… ]

Is it true you still use mutt to read your e-mail? :-)

- Jordan


I had to use mutt(1) to send a patch to the git project to support 
FreeBSD's propset commands a couple years back.


was... fun?  I sorta miss it!

-Alfred
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r314289 - head/usr.bin/fortune/datfiles

[Alfred Perlstein]

Author: alfred
Date: Sun Feb 26 06:05:29 2017
New Revision: 314289
URL: https://svnweb.freebsd.org/changeset/base/314289

Log:
  spelling fix
  
  Submitted by: adamw

Modified:
  head/usr.bin/fortune/datfiles/freebsd-tips

Modified: head/usr.bin/fortune/datfiles/freebsd-tips
==
--- head/usr.bin/fortune/datfiles/freebsd-tips  Sun Feb 26 04:41:37 2017
(r314288)
+++ head/usr.bin/fortune/datfiles/freebsd-tips  Sun Feb 26 06:05:29 2017
(r314289)
@@ -494,7 +494,7 @@ attributes use
-- Lars Engels 
 %
 Do you wonder what a terminal program is doing at the moment? dd(1) does not
-show any troughput? Hit "^T" (Control + t) to send SIGINFO to the process
+show any throughput? Hit "^T" (Control + t) to send SIGINFO to the process
 and see what it is doing.
 
-- Lars Engels 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r314288 - head/usr.bin/fortune/datfiles

[Alfred Perlstein]

Author: alfred
Date: Sun Feb 26 04:41:37 2017
New Revision: 314288
URL: https://svnweb.freebsd.org/changeset/base/314288

Log:
  More FreeBSD tips for fortune(6)
  
  Submitted by: lme
  PR: 192373

Modified:
  head/usr.bin/fortune/datfiles/freebsd-tips

Modified: head/usr.bin/fortune/datfiles/freebsd-tips
==
--- head/usr.bin/fortune/datfiles/freebsd-tips  Sun Feb 26 01:05:27 2017
(r314287)
+++ head/usr.bin/fortune/datfiles/freebsd-tips  Sun Feb 26 04:41:37 2017
(r314288)
@@ -7,6 +7,7 @@ a root login. You can add a user to the 
 %
 By pressing "Scroll Lock" you can use the arrow keys to scroll backward
 through the console output.  Press "Scroll Lock" again to turn it off.
+Don't have a "Scroll Lock" key? The "Pause / Break" key acts alike.
 %
 Can't remember if you've installed a certain port or not? Try "pkg info
 -x port_name".
@@ -40,8 +41,8 @@ Having trouble using fetch through a fir
 variable FTP_PASSIVE_MODE to yes, and see fetch(3) for more details.
 %
 If other operating systems have damaged your Master Boot Record, you can
-reinstall it with boot0cfg(8). See
-"man boot0cfg" for details.
+reinstall it with gpart(8). See
+"man gpart" for details.
 %
 If you accidentally end up inside vi, you can quit it by pressing Escape, colon
 (:), q (q), bang (!) and pressing return.
@@ -116,7 +117,7 @@ In order to support national characters 
 less without creating other nationalisation aspects, set the environment
 variable LC_ALL to 'en_US.ISO8859-1'.
 %
-"man firewall" will give advice for building a FreeBSD firewall
+"man firewall" will give advice for building a FreeBSD firewall using ipfw(8).
-- David Scheidt 
 %
 "man hier" will explain the way FreeBSD filesystems are normally laid out.
@@ -141,7 +142,8 @@ FreeBSD system.
-- David Scheidt 
 %
 Need to do a search in a manpage or in a file you've sent to a pager? Use
-"/search_word". To repeat the same search, type "n" for next.
+"/search_word". To repeat the same search, type "n" for next or "p" for
+previous.
-- Dru 
 %
 Need to find the location of a program? Use "locate program_name".
@@ -183,7 +185,7 @@ flag is your gateway.
 Nice bash prompt: PS1='(\[$(tput md)\]\t <\w>\[$(tput me)\]) $(echo $?) \$ '
-- Mathieu 
 %
-Over quota?  "du -s * | sort -n " will give you a sorted list of your
+Over quota?  "du -sh * | sort -h " will give you a sorted list of your
 directory sizes.
-- David Scheidt 
 %
@@ -191,7 +193,8 @@ nc(1) (or netcat) is useful not only for
 TCP or UDP connections, but also for proxying them with inetd(8).
 %
 sh (the default Bourne shell in FreeBSD) supports command-line editing.  Just
-``set -o emacs'' or ``set -o vi'' to enable it.
+``set -o emacs'' or ``set -o vi'' to enable it. Use "" key to complete
+paths.
 %
 Simple tcsh prompt: set prompt = '%# '
 %
@@ -215,6 +218,8 @@ the scroll lock key and use your page up
 press the scroll lock key again to get your prompt back.
-- Dru 
 %
+You can press Ctrl-L while in the shell to clear the screen.
+%
 To determine whether a file is a text file, executable, or some other type
 of file, use
 
@@ -231,10 +236,10 @@ is running FreeBSD at the time) to quick
 To erase a line you've written at the command prompt, use "Ctrl-U".
-- Dru 
 %
-To find the hostname associated with an IP address, use
+To find out the hostname associated with an IP address, use
 
drill -x IP_address
-   -- Allan Jude 
+   -- Dru 
 %
 To obtain a neat PostScript rendering of a manual page, use ``-t'' switch
 of the man(1) utility: ``man -t ''.  For example:
@@ -247,7 +252,8 @@ To quickly create an empty file, use "to
-- Dru 
 %
 To read a compressed file without having to first uncompress it, use
-"zcat" or "zless" to view it.
+"zcat" or "zless" to view it. There is also "bzcat", "bzless", "xzcat"
+and "xzless".
-- Dru 
 %
 To repeat the last command in the C shell, type "!!".
@@ -283,7 +289,7 @@ To see how much disk space is left on yo
 %
 To see the 10 largest files on a directory or partition, use
 
-   du /partition_or_directory_name | sort -rn | head
+   du -h /partition_or_directory_name | sort -rh | head
-- Dru 
 %
 To see the IP addresses currently set on your active interfaces, type
@@ -291,7 +297,8 @@ To see the IP addresses currently set on
-- Dru 
 %
 To see the last 10 lines of a long file, use "tail filename". To see the
-first 10 lines, use "head filename".
+first 10 lines, use "head filename". To see new lines as they're appended

Re: svn commit: r308314 - head/usr.bin/sed

[Alfred Perlstein]

On 11/5/16 2:07 AM, Konstantin Belousov wrote:

On Fri, Nov 04, 2016 at 08:49:59PM +, Pedro F. Giffuni wrote:

Author: pfg
Date: Fri Nov  4 20:49:59 2016
New Revision: 308314
URL: https://svnweb.freebsd.org/changeset/base/308314

Log:
   sed(1): add LEGACY_BSDSED_COMPAT compile-time flag.
   
   In r297602, which included a __FreeBSD_version bump to 1100105, we changed

   sed 'i' and 'a' from discarding whitespaces to conform with what GNU and
   sysvish sed do.
   
   There are arguments in favor of keeping the old behavior but the new

   behavior is also useful for migration purposes. It seems important to at
   least consider the case of developers depending on the previous behavior,
   so add a CFLAG to enable the old behaviour.

If such legacy behavior appears to be useful or even important for
real-world scenarios, I think that an environment variable controlling
it is more practical and traditional than the recompilation.


+1
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r295136 - in head: sys/kern sys/netinet sys/sys usr.bin/netstat

[Alfred Perlstein]

On 2/2/16 12:23 PM, Xin LI wrote:



On Tuesday, February 2, 2016, Alfred Perlstein <alf...@freebsd.org 
<mailto:alf...@freebsd.org>> wrote:




On 2/2/16 11:39 AM, John Baldwin wrote:

On Tuesday, February 02, 2016 05:57:59 AM Alfred Perlstein wrote:

Author: alfred
Date: Tue Feb  2 05:57:59 2016
New Revision: 295136
URL: https://svnweb.freebsd.org/changeset/base/295136

Log:
   Increase max allowed backlog for listen sockets
   from short to int.
  PR: 203922
   Submitted by: White Knight <white_kni...@2ch.net>
   MFC After: 4 weeks

You do realize that this breaks the ABI of the sysctls used to
fetch
connection lists (and so will break existing binaries like
ucd-snmpd, etc.)
and thus can't be MFC'd right?

OK, I will not MFC it.

Is it worthwhile to extend the xsocket to have padding so that in
11.x and beyond we can allow for some changes in the structure?

Another idea I had was to include a version number with the sysctl
request so that we can send back versioned structures, let me know
what you think about that.

The first idea will take not more than a few days for me to
accomplish, the second (versioning the sysctl) probably a few more
days than that.


We have similar construction (versioned ioctl) in FreeBSD ZFS but the 
main goal is to keep system bootable, not to support all 
functionalities.  Do we change the structure often and is it important 
enough to warrant the complexity?  I think kmem interface have a 
simple size check to guard against world/kernel inconsistency.


I would second John's comment on the necessity of the change though, 
if one already have 32K of *backlogged* connections, it's probably not 
very useful to allow more coming in.  It sounds like the application 
itself is seriously broken, and unless expanding the field have some 
performance benefit, I don't think it should stay.


Imagine a hugely busy image board like 2ch.net, if there is a single 
hiccup, it's very possible to start dropping connections.


I stand by the scalability improvement offered here even though it is an 
edge case.  Linux appears to offer 32 bits of backlog and so should we.


-Alfred
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r295136 - in head: sys/kern sys/netinet sys/sys usr.bin/netstat

[Alfred Perlstein]

On 2/2/16 1:09 PM, Slawa Olhovchenkov wrote:

On Tue, Feb 02, 2016 at 12:35:47PM -0800, Alfred Perlstein wrote:


I would second John's comment on the necessity of the change though,
if one already have 32K of *backlogged* connections, it's probably not
very useful to allow more coming in.  It sounds like the application
itself is seriously broken, and unless expanding the field have some
performance benefit, I don't think it should stay.

Imagine a hugely busy image board like 2ch.net, if there is a single
hiccup, it's very possible to start dropping connections.

In reality start dropping connections in any case: nobody will be
infinity wait of accept (user close browser and go away, etc).

Also, if you have more then 4K backloged connections -- you have
problem, you can't process all connections request and in next second
you will be have 8K, after next second -- 12K and etc.


Thank you Slawa,

I am pretty familiar with what you are describing which are "cascade 
failures", however in order to understand why such a change makes sense 
I can give you a little early history lesson on a project I developed 
under FreeBSD, and then explain why such a project would probably not 
work with FreeBSD as a platform today (we would have to use Linux or 
custom patches).


Here is that use case:

Back in 1999 I wrote a custom webserver using FreeBSD that was 
processing over 1500 connections per second.


What we were doing was tracking web hits using "hidden gifs".  Now this 
was 1999 with only 100mbit hardware and a pentium 400mhz.  Mind you I 
was doing this with cpu to spare, so having an influx of additional hits 
was OK.


Meaning I could easily deal with backlog.

Now what was important about this case was that EVERY time we served the 
data we were able to monitize it and pay for my salary at the time which 
was working on SMP for FreeBSD and a bunch of other patches.  Any lost 
hits / broken connections would easily cost us money, which in turn 
meant less time on FreeBSD and less time fixing things to scale.


In our case the user would not really know if our "page" didn't load 
because we were just an invisible gif.


So back to the example, let's scale that out to today's numbers.

100mbps -> 10gigE, so that would be 1500 conn/sec -> 150,000 conn/sec.  
so basically at 0.20 of a second of any sort of latency I will be 
overflowing the listen queue and dropping connections.


Now when you still have CPU to spare because connections *are* precious, 
then the model makes sense to slightly over-provision the servers to 
allow for somebacklog to be processed.


So, in today's day and age, it really does make sense to allow for 
buffering more than 32k connections, particularly if the developer knows 
what he is doing.


Does this help explain the reasoning?

thanks!

-Alfred

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r295136 - in head: sys/kern sys/netinet sys/sys usr.bin/netstat

[Alfred Perlstein]

On 2/2/16 11:39 AM, John Baldwin wrote:

On Tuesday, February 02, 2016 05:57:59 AM Alfred Perlstein wrote:

Author: alfred
Date: Tue Feb  2 05:57:59 2016
New Revision: 295136
URL: https://svnweb.freebsd.org/changeset/base/295136

Log:
   Increase max allowed backlog for listen sockets
   from short to int.
   
   PR: 203922

   Submitted by: White Knight <white_kni...@2ch.net>
   MFC After: 4 weeks

You do realize that this breaks the ABI of the sysctls used to fetch
connection lists (and so will break existing binaries like ucd-snmpd, etc.)
and thus can't be MFC'd right?

OK, I will not MFC it.

Is it worthwhile to extend the xsocket to have padding so that in 11.x 
and beyond we can allow for some changes in the structure?


Another idea I had was to include a version number with the sysctl 
request so that we can send back versioned structures, let me know what 
you think about that.


The first idea will take not more than a few days for me to accomplish, 
the second (versioning the sysctl) probably a few more days than that.


thanks,
-Alfred

___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

svn commit: r295136 - in head: sys/kern sys/netinet sys/sys usr.bin/netstat

[Alfred Perlstein]

Author: alfred
Date: Tue Feb  2 05:57:59 2016
New Revision: 295136
URL: https://svnweb.freebsd.org/changeset/base/295136

Log:
  Increase max allowed backlog for listen sockets
  from short to int.
  
  PR: 203922
  Submitted by: White Knight 
  MFC After: 4 weeks

Modified:
  head/sys/kern/uipc_debug.c
  head/sys/kern/uipc_socket.c
  head/sys/netinet/sctp_sysctl.c
  head/sys/netinet/sctp_uio.h
  head/sys/sys/socketvar.h
  head/usr.bin/netstat/inet.c
  head/usr.bin/netstat/sctp.c
  head/usr.bin/netstat/unix.c

Modified: head/sys/kern/uipc_debug.c
==
--- head/sys/kern/uipc_debug.c  Tue Feb  2 03:08:37 2016(r295135)
+++ head/sys/kern/uipc_debug.c  Tue Feb  2 05:57:59 2016(r295136)
@@ -461,9 +461,9 @@ db_print_socket(struct socket *so, const
 
db_print_indent(indent);
/* so_list skipped */
-   db_printf("so_qlen: %d   ", so->so_qlen);
-   db_printf("so_incqlen: %d   ", so->so_incqlen);
-   db_printf("so_qlimit: %d   ", so->so_qlimit);
+   db_printf("so_qlen: %u   ", so->so_qlen);
+   db_printf("so_incqlen: %u   ", so->so_incqlen);
+   db_printf("so_qlimit: %u   ", so->so_qlimit);
db_printf("so_timeo: %d   ", so->so_timeo);
db_printf("so_error: %d\n", so->so_error);
 

Modified: head/sys/kern/uipc_socket.c
==
--- head/sys/kern/uipc_socket.c Tue Feb  2 03:08:37 2016(r295135)
+++ head/sys/kern/uipc_socket.c Tue Feb  2 05:57:59 2016(r295136)
@@ -196,7 +196,7 @@ VNET_DEFINE(struct hhook_head *, socket_
  * NB: The orginal sysctl somaxconn is still available but hidden
  * to prevent confusion about the actual purpose of this number.
  */
-static int somaxconn = SOMAXCONN;
+static u_int somaxconn = SOMAXCONN;
 
 static int
 sysctl_somaxconn(SYSCTL_HANDLER_ARGS)
@@ -209,7 +209,13 @@ sysctl_somaxconn(SYSCTL_HANDLER_ARGS)
if (error || !req->newptr )
return (error);
 
-   if (val < 1 || val > USHRT_MAX)
+   /*
+* The purpose of the UINT_MAX / 3 limit, is so that the formula
+*   3 * so_qlimit / 2
+* below, will not overflow.
+ */
+
+   if (val < 1 || val > UINT_MAX / 3)
return (EINVAL);
 
somaxconn = val;

Modified: head/sys/netinet/sctp_sysctl.c
==
--- head/sys/netinet/sctp_sysctl.c  Tue Feb  2 03:08:37 2016
(r295135)
+++ head/sys/netinet/sctp_sysctl.c  Tue Feb  2 05:57:59 2016
(r295136)
@@ -426,7 +426,11 @@ sctp_sysctl_handle_assoclist(SYSCTL_HAND
xinpcb.maxqlen = 0;
} else {
xinpcb.qlen = so->so_qlen;
+   xinpcb.qlen_old = so->so_qlen > USHRT_MAX ? 
+   USHRT_MAX : (uint16_t) so->so_qlen;
xinpcb.maxqlen = so->so_qlimit;
+   xinpcb.maxqlen_old = so->so_qlimit > USHRT_MAX ? 
+   USHRT_MAX : (uint16_t) so->so_qlimit;
}
SCTP_INP_INCR_REF(inp);
SCTP_INP_RUNLOCK(inp);

Modified: head/sys/netinet/sctp_uio.h
==
--- head/sys/netinet/sctp_uio.h Tue Feb  2 03:08:37 2016(r295135)
+++ head/sys/netinet/sctp_uio.h Tue Feb  2 05:57:59 2016(r295136)
@@ -1170,13 +1170,15 @@ struct xsctp_inpcb {
uint32_t total_nospaces;
uint32_t fragmentation_point;
uint16_t local_port;
-   uint16_t qlen;
-   uint16_t maxqlen;
+   uint16_t qlen_old;
+   uint16_t maxqlen_old;
void *socket;
+   uint32_t qlen;
+   uint32_t maxqlen;
 #if defined(__LP64__)
-   uint32_t extra_padding[29]; /* future */
+   uint32_t extra_padding[27]; /* future */
 #else
-   uint32_t extra_padding[30]; /* future */
+   uint32_t extra_padding[28]; /* future */
 #endif
 };
 

Modified: head/sys/sys/socketvar.h
==
--- head/sys/sys/socketvar.hTue Feb  2 03:08:37 2016(r295135)
+++ head/sys/sys/socketvar.hTue Feb  2 05:57:59 2016(r295136)
@@ -95,10 +95,10 @@ struct socket {
TAILQ_HEAD(, socket) so_incomp; /* (e) queue of partial unaccepted 
connections */
TAILQ_HEAD(, socket) so_comp;   /* (e) queue of complete unaccepted 
connections */
TAILQ_ENTRY(socket) so_list;/* (e) list of unaccepted connections */
-   u_short so_qlen;/* (e) number of unaccepted connections 
*/
-   u_short so_incqlen; /* (e) number of unaccepted incomplete
+   u_int   so_qlen;/* (e) number of unaccepted connections 
*/
+   u_int   so_incqlen; /* (e) number of 

Re: svn commit: r287606 - head/sys/kern

[Alfred Perlstein]

64k hard is too low a number for large memory machines.

-Alfred

On 9/10/15 9:18 AM, Adrian Chadd wrote:

On 10 September 2015 at 09:04, Warner Losh  wrote:


On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:

On 10 September 2015 at 04:05, Adrian Chadd  wrote:

Author: adrian
Date: Thu Sep 10 04:05:58 2015
New Revision: 287606
URL: https://svnweb.freebsd.org/changeset/base/287606

Log:
   Also make kern.maxfilesperproc a boot time tunable.
...
   TODO:

Also "we" should
* Submit patches upstream or to the ports tree to use closefrom


I thought the consensus was that we'd fix things to have fewer FDs
by default, but instead allow individual processes to raise it via the
usual methods.

I'm looking at how to do this in a somewhat sensible fashion. Right
now we just have openfiles=unlimited; in /etc/login.conf which seems a
little odd. I don't know yet if that affects the default set that
services started via /etc/rc get - init gets the whole default
maxfilesperproc and stuff seems to inherit from that unless told
otherwise.

I think the more sensible default would be:

* set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
* root can always override its settings up to kern.maxfilesperproc;
* modify /etc/rc to set some default rlimits as appropriate;
* introduce configuration options ({daemon_rlimit_XXX}?) in
/etc/rc.conf that lets someone override what the default rlimits
should be for a given process,, as (and I'm not making this up) if you
run 'service XXX restart' from a root login you get the rlimits from
the shell, which may differ from the system startup.

That way we can setup various services to have higher openfile limits
via /etc/rc.conf entries for those services rather than having to hack
each startup script. It also means that no matter what is running
'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.

Thoughts?


-adrian



___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Alfred Perlstein]

The idea is sane defaults. Surely 256k makes sense for a machine with that much 
memory. 

Sent from my iPhone

> On Sep 11, 2015, at 2:02 PM, Adrian Chadd <adr...@freebsd.org> wrote:
> 
>> On 11 September 2015 at 13:46, Alfred Perlstein <bri...@mu.org> wrote:
>> 64k hard is too low a number for large memory machines.
> 
> Root can always bump it up all the way to kern.maxfilesperproc.
> 
> I'm also a big fan of having the description of config of service
> stuff be in /etc/rc.conf, rather than splattered around the place. So
> I also like the idea of _rlimit_openfiles="" so it can be
> clearly overridden for services that require it.
> 
> I'm open to other suggestions!
> 
> 
> 
> -adrian
> 
>> -Alfred
>> 
>> 
>>> On 9/10/15 9:18 AM, Adrian Chadd wrote:
>>> 
>>>> On 10 September 2015 at 09:04, Warner Losh <i...@bsdimp.com> wrote:
>>>> 
>>>> 
>>>>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste <ema...@freebsd.org> wrote:
>>>>> 
>>>>>> On 10 September 2015 at 04:05, Adrian Chadd <adr...@freebsd.org> wrote:
>>>>>> 
>>>>>> Author: adrian
>>>>>> Date: Thu Sep 10 04:05:58 2015
>>>>>> New Revision: 287606
>>>>>> URL: https://svnweb.freebsd.org/changeset/base/287606
>>>>>> 
>>>>>> Log:
>>>>>>   Also make kern.maxfilesperproc a boot time tunable.
>>>>>> ...
>>>>>>   TODO:
>>>>> 
>>>>> Also "we" should
>>>>> * Submit patches upstream or to the ports tree to use closefrom
>>>> 
>>>> 
>>>> I thought the consensus was that we'd fix things to have fewer FDs
>>>> by default, but instead allow individual processes to raise it via the
>>>> usual methods.
>>> 
>>> I'm looking at how to do this in a somewhat sensible fashion. Right
>>> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>>> little odd. I don't know yet if that affects the default set that
>>> services started via /etc/rc get - init gets the whole default
>>> maxfilesperproc and stuff seems to inherit from that unless told
>>> otherwise.
>>> 
>>> I think the more sensible default would be:
>>> 
>>> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
>>> * root can always override its settings up to kern.maxfilesperproc;
>>> * modify /etc/rc to set some default rlimits as appropriate;
>>> * introduce configuration options ({daemon_rlimit_XXX}?) in
>>> /etc/rc.conf that lets someone override what the default rlimits
>>> should be for a given process,, as (and I'm not making this up) if you
>>> run 'service XXX restart' from a root login you get the rlimits from
>>> the shell, which may differ from the system startup.
>>> 
>>> That way we can setup various services to have higher openfile limits
>>> via /etc/rc.conf entries for those services rather than having to hack
>>> each startup script. It also means that no matter what is running
>>> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.
>>> 
>>> Thoughts?
>>> 
>>> 
>>> -adrian
> 
___
svn-src-head@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"

Re: svn commit: r280120 - head/sys/dev/wpi

[Alfred Perlstein]

Just use github, you can then just follow my directions on the 
https://wiki.freebsd.org/GitWorkflow page.

you can commit them as individual commits if you would like or make a giant 
squash commit.

Better would really be to replay all the commits into an svn branch, then make 
a single svn merge commit.  Although svn is meh.

-Alfred

On Mar 15, 2015, at 2:35 PM, Adrian Chadd wrote:

 .. promise I'm done for now.
 
 (God, it'd be nice to use git, or some web ui that lets me batch
 review and commit things like this.)
 
 
 -a
 
 
 On 15 March 2015 at 14:32, Adrian Chadd adr...@freebsd.org wrote:
 Author: adrian
 Date: Sun Mar 15 21:32:11 2015
 New Revision: 280120
 URL: https://svnweb.freebsd.org/changeset/base/280120
 
 Log:
  Add a new taskqueue (device specific, not net80211 ic-tq); use it for
  device restart.
 
  (Committers note - once scan overhaul and a few other things have been
  fixed in net80211 to not block things in the taskqueue, this can disappear
  and the device specific taskqueues in other drivers can also go away.)
 
  PR:   kern/197143
  Submitted by: Andriy Voskoboinyk s3er...@gmail.com
 
 Modified:
  head/sys/dev/wpi/if_wpi.c
  head/sys/dev/wpi/if_wpivar.h
 
 Modified: head/sys/dev/wpi/if_wpi.c
 ==
 --- head/sys/dev/wpi/if_wpi.c   Sun Mar 15 21:30:20 2015(r280119)
 +++ head/sys/dev/wpi/if_wpi.c   Sun Mar 15 21:32:11 2015(r280120)
 @@ -532,6 +532,14 @@ wpi_attach(device_t dev)
TASK_INIT(sc-sc_radioon_task, 0, wpi_radio_on, sc);
TASK_INIT(sc-sc_start_task, 0, wpi_start_task, sc);
 
 +   sc-sc_tq = taskqueue_create(wpi_taskq, M_WAITOK,
 +   taskqueue_thread_enqueue, sc-sc_tq);
 +   error = taskqueue_start_threads(sc-sc_tq, 1, 0, wpi_taskq);
 +   if (error != 0) {
 +   device_printf(dev, can't start threads, error %d\n, error);
 +   goto fail;
 +   }
 +
wpi_sysctlattach(sc);
 
/*
 @@ -688,6 +696,9 @@ wpi_detach(device_t dev)
 
wpi_stop(sc);
 
 +   taskqueue_drain_all(sc-sc_tq);
 +   taskqueue_free(sc-sc_tq);
 +
callout_drain(sc-watchdog_rfkill);
callout_drain(sc-tx_timeout);
callout_drain(sc-scan_timeout);
 @@ -2387,8 +2398,6 @@ wpi_intr(void *arg)
WPI_WRITE(sc, WPI_FH_INT, r2);
 
if (r1  (WPI_INT_SW_ERR | WPI_INT_HW_ERR)) {
 -   struct ieee80211com *ic = ifp-if_l2com;
 -
device_printf(sc-sc_dev, fatal firmware error\n);
 #ifdef WPI_DEBUG
wpi_debug_registers(sc);
 @@ -2397,7 +2406,7 @@ wpi_intr(void *arg)
DPRINTF(sc, WPI_DEBUG_HW,
(%s)\n, (r1  WPI_INT_SW_ERR) ? (Software Error) :
(Hardware Error));
 -   ieee80211_runtask(ic, sc-sc_reinittask);
 +   taskqueue_enqueue(sc-sc_tq, sc-sc_reinittask);
goto end;
}
 
 @@ -2950,10 +2959,9 @@ wpi_scan_timeout(void *arg)
 {
struct wpi_softc *sc = arg;
struct ifnet *ifp = sc-sc_ifp;
 -   struct ieee80211com *ic = ifp-if_l2com;
 
if_printf(ifp, scan timeout\n);
 -   ieee80211_runtask(ic, sc-sc_reinittask);
 +   taskqueue_enqueue(sc-sc_tq, sc-sc_reinittask);
 }
 
 static void
 @@ -2961,11 +2969,10 @@ wpi_tx_timeout(void *arg)
 {
struct wpi_softc *sc = arg;
struct ifnet *ifp = sc-sc_ifp;
 -   struct ieee80211com *ic = ifp-if_l2com;
 
if_printf(ifp, device timeout\n);
if_inc_counter(ifp, IFCOUNTER_OERRORS, 1);
 -   ieee80211_runtask(ic, sc-sc_reinittask);
 +   taskqueue_enqueue(sc-sc_tq, sc-sc_reinittask);
 }
 
 static int
 
 Modified: head/sys/dev/wpi/if_wpivar.h
 ==
 --- head/sys/dev/wpi/if_wpivar.hSun Mar 15 21:30:20 2015
 (r280119)
 +++ head/sys/dev/wpi/if_wpivar.hSun Mar 15 21:32:11 2015
 (r280120)
 @@ -228,6 +228,9 @@ struct wpi_softc {
struct task sc_radioon_task;
struct task sc_start_task;
 
 +   /* Taskqueue */
 +   struct taskqueue*sc_tq;
 +
/* Eeprom info. */
uint8_t cap;
uint16_trev;
 
 

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r279603 - in head: bin/rcp usr.bin/rlogin usr.bin/rsh

[Alfred Perlstein]

On 3/5/15 6:09 AM, David Chisnall wrote:

On 5 Mar 2015, at 14:04, Dmitry Sivachenko de...@freebsd.org wrote:


It is so nice to have most useful stuff out of the box.

The question is whether a tool for logging into remote machines without 
encryption is 'the most useful stuff'.  The tool is also [ab]used for network 
testing, but we already provide a better tool for that in the form of nc(1).

David



Agree.

Moving it out of base +1000.

It's time we made FreeBSD into an actual distro which is what is 
happening anyhow.


This is a great time!
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r279603 - in head: bin/rcp usr.bin/rlogin usr.bin/rsh

[Alfred Perlstein]

On 3/5/15 4:30 AM, Gleb Smirnoff wrote:

On Thu, Mar 05, 2015 at 03:30:16PM +0300, Slawa Olhovchenkov wrote:
S  On Thu, Mar 05, 2015 at 03:21:03PM +0300, Slawa Olhovchenkov wrote:
S  S  On Wed, Mar 04, 2015 at 10:01:45PM +, Baptiste Daroussin wrote:
S  S  B Author: bapt
S  S  B Date: Wed Mar  4 22:01:44 2015
S  S  B New Revision: 279603
S  S  B URL: https://svnweb.freebsd.org/changeset/base/279603
S  S  B
S  S  B Log:
S  S  B   r* commands are not precious anymore
S  S  B
S  S  B Modified:
S  S  B   head/bin/rcp/Makefile
S  S  B   head/usr.bin/rlogin/Makefile
S  S 
S  S  I guess when they are going to be not precious enough to be removed? 
:)
S  S 
S  S  In modern world of ssh and https, does any OS require them in base?
S  S
S  S yes.
S  S Some telecom equipment require rlogin.
S 
S  Other telecom equipment require a JRE and special .jar to
S  configure them. Does that mean operating systems must ship
S  them?
S
S Yes, if ships before (don't break if working).
S Some Linux distro remove telnet from default install.
S Do you like to remove telnet also?

Yes.


+1 :)  (well at least not have it in source tree as-is) :)
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r279603 - in head: bin/rcp usr.bin/rlogin usr.bin/rsh

[Alfred Perlstein]

On 3/5/15 4:49 AM, Edward Tomasz Napierała wrote:

On 0305T1555, Gleb Smirnoff wrote:

On Thu, Mar 05, 2015 at 03:40:37PM +0300, Slawa Olhovchenkov wrote:
S  nc(1), which is a pure socket testing tool. For telnet(1) this
S  capability is a side effect.
S
S You don't try this in practice.
S % nc zxy.spb.ru 81
S %
S % telnet zxy.spb.ru 81
S Trying 195.70.199.98...
S telnet: connect to address 195.70.199.98: Connection refused
S telnet: Unable to connect to remote host
S
S nc is not usable.

Yes, this how unix way works. Command has return value, or
you can use '-v', of you want it to be chatty.


Actually, not giving error message on error by default seems quite
broken to me.  Standard UNIX utilities don't fail silently:



You need to read unix hater's handbook. :)

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r279603 - in head: bin/rcp usr.bin/rlogin usr.bin/rsh

[Alfred Perlstein]

On 3/5/15 5:06 AM, David Chisnall wrote:



On 5 Mar 2015, at 12:42, Slawa Olhovchenkov s...@zxy.spb.ru wrote:

nc don't work with unix socket.


Okay, now you're changing your requirements - you first spoke of remote 
equipment and network testing.  However, UNIX domain sockets appear as files in 
the filesystem, and we have a host of utilities that are capable of interacting 
with files (unless they're message-oriented, but then telnet doesn't help 
either).


nc -U

?

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r279603 - in head: bin/rcp usr.bin/rlogin usr.bin/rsh

[Alfred Perlstein]

On 3/5/15 4:28 AM, Slawa Olhovchenkov wrote:

On Thu, Mar 05, 2015 at 12:24:05PM +, David Chisnall wrote:


On 5 Mar 2015, at 12:21, Slawa Olhovchenkov s...@zxy.spb.ru wrote:



I guess when they are going to be not precious enough to be removed? :)

In modern world of ssh and https, does any OS require them in base?


yes.
Some telecom equipment require rlogin.


'Some relatively obscure use case needs them' is not usually the
requirement for keeping something in the base system.  Presumably
people who interact with telecoms equipment are capable of
installing packages...


And install from package ssh, inetd, systemd, binutils...



How does one patch systemd under FreeBSD?

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r277694 - head/sys/amd64/conf

[Alfred Perlstein]

Never mind, apologies, thought this was i386+amd64.  I'm going to guess 
anything emulating amd64 has pci NICs in it. :)

On Jan 26, 2015, at 9:21 AM, Alfred Perlstein wrote:

 Don't have a strong opinion on this because I'm not in the know lately, but 
 was wondering:
 
 1) if we wanted to keep NE1000/2000 support for FreeBSD in emulators?  
 Probably not I assume?
 2) Does Linux nuke these devices?  why/why-not?
 
 
 On Jan 25, 2015, at 9:58 AM, Warner Losh wrote:
 
 
 On Jan 25, 2015, at 7:54 AM, Garrett Cooper yaneurab...@gmail.com wrote:
 
 
 On Jan 25, 2015, at 04:43, Sergey Kandaurov pluk...@freebsd.org wrote:
 
 On 25 January 2015 at 15:02, Dag-Erling Smørgrav d...@freebsd.org wrote:
 Author: des
 Date: Sun Jan 25 12:02:38 2015
 New Revision: 277694
 URL: https://svnweb.freebsd.org/changeset/base/277694
 
 Log:
 Remove ISA NICs.  Anyone still using these on amd64 can build their
 own kernel.
 
 Modified:
 head/sys/amd64/conf/GENERIC
 
 If so, what about i386? (I'd rather not pc98)
 What about device isa in DEFAULTS?
 
 isa is still needed in some scenarios. I just don't remember the full 
 details offhand (something about internal buses iirc... And IPMI for 
 starters...)
 
 isa” in this context should be read as “mainbus” not as “ISA slots”. You 
 can’t
 remove it without some significant work, and even then the interface changes
 to long established interfaces isn’t worth the pain.
 
 Warner
 
 
 

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r277694 - head/sys/amd64/conf

[Alfred Perlstein]

Don't have a strong opinion on this because I'm not in the know lately, but was 
wondering:

1) if we wanted to keep NE1000/2000 support for FreeBSD in emulators?  Probably 
not I assume?
2) Does Linux nuke these devices?  why/why-not?


On Jan 25, 2015, at 9:58 AM, Warner Losh wrote:

 
 On Jan 25, 2015, at 7:54 AM, Garrett Cooper yaneurab...@gmail.com wrote:
 
 
 On Jan 25, 2015, at 04:43, Sergey Kandaurov pluk...@freebsd.org wrote:
 
 On 25 January 2015 at 15:02, Dag-Erling Smørgrav d...@freebsd.org wrote:
 Author: des
 Date: Sun Jan 25 12:02:38 2015
 New Revision: 277694
 URL: https://svnweb.freebsd.org/changeset/base/277694
 
 Log:
 Remove ISA NICs.  Anyone still using these on amd64 can build their
 own kernel.
 
 Modified:
 head/sys/amd64/conf/GENERIC
 
 If so, what about i386? (I'd rather not pc98)
 What about device isa in DEFAULTS?
 
 isa is still needed in some scenarios. I just don't remember the full 
 details offhand (something about internal buses iirc... And IPMI for 
 starters...)
 
 isa” in this context should be read as “mainbus” not as “ISA slots”. You 
 can’t
 remove it without some significant work, and even then the interface changes
 to long established interfaces isn’t worth the pain.
 
 Warner
 
 

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r277653 - head/sys/dev/netmap

[Alfred Perlstein]

Wasn't this pointed out by James K?

On 1/24/15 11:49 AM, Adrian Chadd wrote:

Author: adrian
Date: Sat Jan 24 19:49:27 2015
New Revision: 277653
URL: https://svnweb.freebsd.org/changeset/base/277653

Log:
   Change the permissions from 0660 to 0600.
   
   Otherwise people in wheel can do things with netmap, including

   but not limited to promisc transmit/receive.
   
   Approved by:	luigi

   MFC after:   1 week

Modified:
   head/sys/dev/netmap/netmap.c

Modified: head/sys/dev/netmap/netmap.c
==
--- head/sys/dev/netmap/netmap.cSat Jan 24 19:13:03 2015
(r277652)
+++ head/sys/dev/netmap/netmap.cSat Jan 24 19:49:27 2015
(r277653)
@@ -3075,10 +3075,10 @@ netmap_init(void)
  #ifdef __FreeBSD__
/* support for the 'eternal' flag */
netmap_dev = make_dev_credf(MAKEDEV_ETERNAL_KLD,
-   netmap_cdevsw, 0, NULL, UID_ROOT, GID_WHEEL, 0660,
+   netmap_cdevsw, 0, NULL, UID_ROOT, GID_WHEEL, 0600,
  netmap);
  #else
-   netmap_dev = make_dev(netmap_cdevsw, 0, UID_ROOT, GID_WHEEL, 0660,
+   netmap_dev = make_dev(netmap_cdevsw, 0, UID_ROOT, GID_WHEEL, 0600,
  netmap);
  #endif
if (!netmap_dev)



___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r276565 - in head/gnu: lib/libstdc++/doc usr.bin/diff usr.bin/diff/doc usr.bin/gdb usr.bin/gperf usr.bin/gperf/doc

[Alfred Perlstein]

On 1/2/15 1:33 PM, Steve Kargl wrote:

On Fri, Jan 02, 2015 at 03:59:11PM -0500, Benjamin Kaduk wrote:

On Fri, Jan 2, 2015 at 3:44 PM, Steve Kargl 
s...@troutmask.apl.washington.edu wrote:


On Fri, Jan 02, 2015 at 08:34:56PM +, Garrett Cooper wrote:

Author: ngie
Date: Fri Jan  2 20:34:55 2015
New Revision: 276565
URL: https://svnweb.freebsd.org/changeset/base/276565

Log:
   Remove gnu/ info pages to unbreak the build with MK_GCC != no, etc


Does this mean that FreeBSD is now shipping utilities without
documentation?


I'm pretty sure we've been doing that for a long, long time.

Note that MK_GCC=no is the default on head for the tier-1 platforms.


Yeah, I know what the default is for tier-1, which is somewhat
irrelevant to my question.  bapt@ has removed info pages (and
Garrett seems to have cleanup after bapt).  Noting that FreeBSD
has more than just tier-1, the question remains.  Is FreeBSD
now shipping utilities without proper documentation?


The docs are there, you just need to install a package to read them.

It was explained in the commit log.

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r276195 - in head: . lib/libxo

[Alfred Perlstein]

You are correct.

Reason for move was due to programs inside of /bin and /sbin depending 
on libxo.


-Alfred

On 12/29/14 8:49 AM, Bjoern A. Zeeb wrote:

On 25 Dec 2014, at 03:15 , Alfred Perlstein alf...@freebsd.org wrote:

Author: alfred
Date: Thu Dec 25 03:15:56 2014
New Revision: 276195
URL: https://svnweb.freebsd.org/changeset/base/276195

Log:
  Move libxo to /lib

  Update ObsoleteFiles to reflect libxo move.

What this commit message doesn’t say is “WHY?”  Everything else you described 
is kind of obvious from the commit diff ;-)



  Reviewed by: ngie
  Differential Revision: https://reviews.freebsd.org/D1370

Modified:
  head/ObsoleteFiles.inc
  head/lib/libxo/Makefile

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Thu Dec 25 02:17:17 2014(r276194)
+++ head/ObsoleteFiles.inc  Thu Dec 25 03:15:56 2014(r276195)
@@ -38,6 +38,9 @@
#   xargs -n1 | sort | uniq -d;
# done

+# 20141224: libxo moved to /lib
+OLD_FILES+=usr/lib/libxo.a
+OLD_FILES+=usr/lib/libxo_p.a
# 20141223: remove in6_gif.h, in_gif.h and if_stf.h
OLD_FILES+=usr/include/net/if_stf.h
OLD_FILES+=usr/include/netinet/in_gif.h

Modified: head/lib/libxo/Makefile
==
--- head/lib/libxo/Makefile Thu Dec 25 02:17:17 2014(r276194)
+++ head/lib/libxo/Makefile Thu Dec 25 03:15:56 2014(r276195)
@@ -7,6 +7,8 @@ LIBXO=  ${.CURDIR:H:H}/contrib/libxo
LIB=xo
SHLIB_MAJOR=0

+SHLIBDIR?=  /lib
+
SRCS=   libxo.c

CFLAGS+=-I${LIBXO}/libxo


—
Bjoern A. Zeeb  Charles Haddon Spurgeon:
Friendship is one of the sweetest joys of life.  Many might have failed
  beneath the bitterness of their trial  had they not found a friend.




___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r276273 - head/contrib/libxo/libxo

[Alfred Perlstein]

Author: alfred
Date: Sat Dec 27 01:06:19 2014
New Revision: 276273
URL: https://svnweb.freebsd.org/changeset/base/276273

Log:
  Output strerror from xo_warn
  
  Reported by: bapt
  Reviewed by: bapt, ngie
  
  Differential Revision: https://reviews.freebsd.org/D1378

Modified:
  head/contrib/libxo/libxo/libxo.c

Modified: head/contrib/libxo/libxo/libxo.c
==
--- head/contrib/libxo/libxo/libxo.cSat Dec 27 00:55:14 2014
(r276272)
+++ head/contrib/libxo/libxo/libxo.cSat Dec 27 01:06:19 2014
(r276273)
@@ -956,9 +956,6 @@ xo_warn_hcv (xo_handle_t *xop, int code,
 }
 memcpy(newfmt + plen, fmt, len);
 
-/* Add a newline to the fmt string */
-if (!(xop-xo_flags  XOF_WARN_XML))
-   newfmt[len++ + plen] = '\n';
 newfmt[len + plen] = '\0';
 
 if (xop-xo_flags  XOF_WARN_XML) {
@@ -1010,6 +1007,7 @@ xo_warn_hcv (xo_handle_t *xop, int code,
 
 } else {
vfprintf(stderr, newfmt, vap);
+   fprintf(stderr, : %s\n, strerror(code));
 }
 }
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r276209 - head

[Alfred Perlstein]

Author: alfred
Date: Thu Dec 25 17:53:43 2014
New Revision: 276209
URL: https://svnweb.freebsd.org/changeset/base/276209

Log:
  Fix OLD_LIBS for libxo moved to /lib
  
  Pointed out by: kib

Modified:
  head/ObsoleteFiles.inc

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Thu Dec 25 17:50:04 2014(r276208)
+++ head/ObsoleteFiles.inc  Thu Dec 25 17:53:43 2014(r276209)
@@ -39,8 +39,7 @@
 # done
 
 # 20141224: libxo moved to /lib
-OLD_FILES+=usr/lib/libxo.a
-OLD_FILES+=usr/lib/libxo_p.a
+OLD_LIBS+=usr/lib/libxo.so.0
 # 20141223: remove in6_gif.h, in_gif.h and if_stf.h
 OLD_FILES+=usr/include/net/if_stf.h
 OLD_FILES+=usr/include/netinet/in_gif.h
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r276195 - in head: . lib/libxo

[Alfred Perlstein]

On 12/25/14 12:07 PM, Garrett Cooper wrote:

On Dec 25, 2014, at 02:21, Konstantin Belousov kostik...@gmail.com wrote:


On Wed, Dec 24, 2014 at 11:04:01PM -0800, Garrett Cooper wrote:


On Dec 24, 2014, at 19:56, Alfred Perlstein alf...@freebsd.org wrote:


On 12/24/14 7:36 PM, Garrett Cooper wrote:

...


There isn't a leftover versioned .so in /usr/lib ?

usr/lib/libxo.so.0 (without leading slash) must go into OLD_LIBS.
It is all explained in the first paragraph of the file comment.

+1 to what kib said :)..

I *think* i fixed it, but tbh the docs in that file make very little 
sense to the layperson.


The suggested sanity scripts should be either make(1) targets, or 
perhaps scripts in /usr/share to run, not inline comments.


Having a make sanity-obsolete would be nice.

-Alfred


___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r276195 - in head: . lib/libxo

[Alfred Perlstein]

Author: alfred
Date: Thu Dec 25 03:15:56 2014
New Revision: 276195
URL: https://svnweb.freebsd.org/changeset/base/276195

Log:
  Move libxo to /lib
  
  Update ObsoleteFiles to reflect libxo move.
  
  Reviewed by: ngie
  Differential Revision: https://reviews.freebsd.org/D1370

Modified:
  head/ObsoleteFiles.inc
  head/lib/libxo/Makefile

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Thu Dec 25 02:17:17 2014(r276194)
+++ head/ObsoleteFiles.inc  Thu Dec 25 03:15:56 2014(r276195)
@@ -38,6 +38,9 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20141224: libxo moved to /lib
+OLD_FILES+=usr/lib/libxo.a
+OLD_FILES+=usr/lib/libxo_p.a
 # 20141223: remove in6_gif.h, in_gif.h and if_stf.h
 OLD_FILES+=usr/include/net/if_stf.h
 OLD_FILES+=usr/include/netinet/in_gif.h

Modified: head/lib/libxo/Makefile
==
--- head/lib/libxo/Makefile Thu Dec 25 02:17:17 2014(r276194)
+++ head/lib/libxo/Makefile Thu Dec 25 03:15:56 2014(r276195)
@@ -7,6 +7,8 @@ LIBXO=  ${.CURDIR:H:H}/contrib/libxo
 LIB=   xo
 SHLIB_MAJOR=0
 
+SHLIBDIR?=  /lib
+
 SRCS=  libxo.c
 
 CFLAGS+=-I${LIBXO}/libxo
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r276195 - in head: . lib/libxo

[Alfred Perlstein]

On 12/24/14 7:36 PM, Garrett Cooper wrote:

On Dec 24, 2014, at 19:15, Alfred Perlstein alf...@freebsd.org wrote:

Author: alfred
Date: Thu Dec 25 03:15:56 2014
New Revision: 276195
URL: https://svnweb.freebsd.org/changeset/base/276195

Log:
  Move libxo to /lib

  Update ObsoleteFiles to reflect libxo move.

  Reviewed by: ngie
  Differential Revision: https://reviews.freebsd.org/D1370

Modified:
  head/ObsoleteFiles.inc
  head/lib/libxo/Makefile

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.incThu Dec 25 02:17:17 2014(r276194)
+++ head/ObsoleteFiles.incThu Dec 25 03:15:56 2014(r276195)
@@ -38,6 +38,9 @@
#   xargs -n1 | sort | uniq -d;
# done

+# 20141224: libxo moved to /lib
+OLD_FILES+=usr/lib/libxo.a
+OLD_FILES+=usr/lib/libxo_p.a

This should actually be the .so file only. Static libraries are always in 
/usr/lib .

Sorry for not being explicit about this (I'm currently writing from the boonies 
so bandwidth is limited) :/.




I think the .so gets clobbered with a symlink, so it should be fine?

Meaning that after I moved the lib to /lib this is what I see in /usr/lib:
/usr/src/contrib/libxo % ls -l /usr/lib/libxo*
-r--r--r--  1 root  wheel  83764 Dec 24 19:31 /usr/lib/libxo.a
lrwxr-xr-x  1 root  wheel 15 Dec 24 19:31 /usr/lib/libxo.so - 
/lib/libxo.so.0

-r--r--r--  1 root  wheel  95164 Dec 24 19:31 /usr/lib/libxo_p.a


Or should I reference it explicitly?

What is the end result here?



-Alfred

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275804 - head/gnu/usr.bin/cc/cc1plus

[Alfred Perlstein]

On 12/15/14 1:44 PM, Craig Rodrigues wrote:



On Mon, Dec 15, 2014 at 1:38 PM, Ed Maste ema...@freebsd.org 
mailto:ema...@freebsd.org wrote:



  cfns.h: cfns.gperf
 gperf -o -C -E -k '1-6,$$' -j1 -D -N 'libc_name_p' -L
ANSI-C \
${.ALLSRC}  ${.TARGET}_temp
mv ${.TARGET}_temp ${.TARGET}

Yeah.  There are already examples of both approaches in the tree; I
don't see a reason to strongly prefer one over the other.


For very large build systems, sometimes it is easier to debug
weird build problems when the make operations do not have
conditional logic in them.

+1

  Also, not removing temporary files upon failure
makes things easier to debug.


+100


--
Craig


___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275431 - in head/sys/dev: e1000 ixgbe

[Alfred Perlstein]

 here:

https://reviews.freebsd.org/D1149

The gist of this change is just to allow later tuning of the num_queues 
parameter for Intel NICS.


Can you please sign up for a phabricator account? 
https://wiki.freebsd.org/CodeReview


-Alfred


 Original Message 
Subject: 	[Differential] [Commented On] D1149: ixgbe and igb should 
check tunables at load time. Also support per-device queue count.

Date:   Sat, 15 Nov 2014 20:13:51 +
From:   alfredperlstein (Alfred Perlstein) phabric-nore...@freebsd.org
To: alf...@freebsd.org



alfredperlstein added a comment.

I don't know how to do that.  Please make 'jfv' an account.

REVISION DETAIL
  https://reviews.freebsd.org/D1149

To: alfredperlstein, glebius, adrian, melifaro, hrs, wollman, bryanv, rpaulo, 
bz, gnn, hiren, rwatson, smh
Cc: ngie, bryanv



---End Message---
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

John,

Will work on a new revision based on feedback. 

Two things to note however:

Already explored the idea of using kernel_sysctlbyname but rejected due to 
following:

It makes little sense to have a rw sysctl that only takes effect some times. 
This violates POLA at the expense of making code appear cleaner. Expectation is 
that writable sysctls take effect or are read only. They are not to be write 
sometimes unless we are to introduce a new flag. Instead of going to a 
confusing model we consider some form of rw sysctl that can set itself ro 
somehow. Otherwise people will be confused as to why nic queues says N while 
actually M.  What the rw-ro api would look like I have no idea. Suggestions?

Btw the review was cc'd to jvf and smh and gleb in phabricator.  Smh responded. 
Jvf + gleb timed out after 2+ weeks for a relatively trivial change.  

Will look at the device_get_int stuff soon. Wasn't aware that function existed. 

Sent from my iPhone

 On Dec 1, 2014, at 6:32 AM, John Baldwin j...@freebsd.org wrote:
 
 On Wednesday, November 26, 2014 08:19:36 PM Alfred Perlstein wrote:
 Author: alfred
 Date: Wed Nov 26 20:19:36 2014
 New Revision: 275136
 URL: https://svnweb.freebsd.org/changeset/base/275136
 
 Log:
  Make igb and ixgbe check tunables at probe time.
 
  This allows one to make a kernel module to tune the
  number of queues before the driver loads.
 
  This is needed so that a module at SI_SUB_CPU can set
  tunables for these drivers to take.  Otherwise getenv
  is called too early by the TUNABLE macros.
 
  Reviewed by: smh
  Phabric: https://reviews.freebsd.org/D1149
 
 The explicit TUNABLE_INT_FETCH strikes me as the wrong approach and hackish.  
 That is, each time you want to frob a tunable like this you will have to go 
 add a bunch of code to explicitly re-read the tunable, etc.  Instead, you 
 could have simply changed your helper module to use kernel_sysctlbyname() to 
 set hw.igb.num_queues.  That would have required only a single character 
 change to make the SYSCTL read/write instead of read/only for each tunable in 
 question.  To be completely future proof (i.e. to handle loading the module 
 in 
 question post-boot), your module could both do setenv() and 
 kernel_sysctlbyname().  That seems to be a more extensible approach in terms 
 of allowing more of these to be changed in the future without having to do a 
 manual bypass of the existing tunable infrastructure for each tunable.  I 
 would much prefer that you revert this part and change the relevant tunables 
 to CTLFLAG_RWTUN and update your out-of-tree module to use 
 kernel_sysctlbyname().
 
 Also, if you didn't run this by the Intel folks (e.g. jfv@) that might have 
 been nice as a courtesy.
 
 Also, please use the existing resource_int_value() that uses 'hint.igb.0.foo'
 instead of inventing a different wheel (device_get_int()).  This is what all 
 the other drivers in the tree that provide per-instance tunables use.  You
 could perhaps have device_get_int() as a wrapper around resource_int_value(),
 but we should use the existing convention, not invent a conflicting one.
 
 Modified:
  head/sys/dev/e1000/if_igb.c
  head/sys/dev/ixgbe/ixgbe.c
  head/sys/kern/subr_bus.c
  head/sys/sys/bus.h
 
 Modified: head/sys/dev/e1000/if_igb.c
 
 == --- head/sys/dev/e1000/if_igb.cWed Nov 26 18:03:25 2014(r275135)
 +++
 head/sys/dev/e1000/if_igb.cWed Nov 26 20:19:36 2014(r275136) @@
 -188,6
 +188,7 @@ static char *igb_strings[] = {
 /*
  *  Function prototypes
  */
 +static intigb_per_unit_num_queues(SYSCTL_HANDLER_ARGS);
 static intigb_probe(device_t);
 static intigb_attach(device_t);
 static intigb_detach(device_t);
 @@ -493,6 +494,11 @@ igb_attach(device_t dev)
OID_AUTO, nvm, CTLTYPE_INT|CTLFLAG_RW, adapter, 0,
igb_sysctl_nvm_info, I, NVM Information);
 
 +SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev),
 +SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
 +OID_AUTO, num_queues, CTLTYPE_INT | CTLFLAG_RD,
 +adapter, 0, igb_per_unit_num_queues, I, Number of Queues);
 +
 
 You don't need igb_per_unit_num_queues().
 SYSCTL_ADD_INT(..., adapter-num_queues) should have been used instead.
 
 @@ -2831,6 +2837,7 @@ igb_setup_msix(struct adapter *adapter)
 {
device_tdev = adapter-dev;
intbar, want, queues, msgs, maxqueues;
 +intn_queues;
 
/* tuneable override */
if (igb_enable_msix == 0)
 @@ -2858,8 +2865,18 @@ igb_setup_msix(struct adapter *adapter)
goto msi;
}
 
 -/* Figure out a reasonable auto config value */
 -queues = (mp_ncpus  (msgs-1)) ? (msgs-1) : mp_ncpus;
 +n_queues = 0;
 +/* try more specific tunable, then global, then finally default to boot
 time

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 7:26 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:19, Alfred Perlstein wrote:
 It makes little sense to have a rw sysctl that only takes effect some 
 times. This violates POLA at the expense of making code appear cleaner. 
 Expectation is that writable sysctls take
 
 Hi,
 
 I think you are missing a new feature in 11-current, that if you add 
 CTLFLAG_TUN to even dynamic sysctls, they get initialized from the 
 enviroment, if any. That way you can just skip the TUNABLE_INT_FETCH() stuff!
 

Ok I can probably switch to that. 

Any objection if I mfc this feature to -stable if it does what I need?




 --HPS
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 7:34 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:29, Alfred Perlstein wrote:
 
 
 On Dec 1, 2014, at 7:26 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:19, Alfred Perlstein wrote:
 It makes little sense to have a rw sysctl that only takes effect some 
 times. This violates POLA at the expense of making code appear cleaner. 
 Expectation is that writable sysctls take
 
 Hi,
 
 I think you are missing a new feature in 11-current, that if you add 
 CTLFLAG_TUN to even dynamic sysctls, they get initialized from the 
 enviroment, if any. That way you can just skip the TUNABLE_INT_FETCH() 
 stuff!
 
 Ok I can probably switch to that.
 
 Any objection if I mfc this feature to -stable if it does what I need?
 
 Hi,
 
 No objections from me at least, but it might require some work from your 
 side, because there was a lot of cleanup about removing duplicate 
 definitions, like static SYSCTLS which have already CTLFLAG_TUN and a TUNABLE 
 fetch statement, which makes the variable init twice. Just look at the 
 revision history for kern/kern_sysctl.c in 11-current.
 
 --HPS
 
 

One question though... For the global sysctl for all nodes

When is the var fetched?  If it's before SI_SUB_CPU it is not useful.  
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 7:43 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:39, Alfred Perlstein wrote:
 
 
 On Dec 1, 2014, at 7:34 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:29, Alfred Perlstein wrote:
 
 
 On Dec 1, 2014, at 7:26 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:19, Alfred Perlstein wrote:
 It makes little sense to have a rw sysctl that only takes effect some 
 times. This violates POLA at the expense of making code appear cleaner. 
 Expectation is that writable sysctls take
 
 Hi,
 
 I think you are missing a new feature in 11-current, that if you add 
 CTLFLAG_TUN to even dynamic sysctls, they get initialized from the 
 enviroment, if any. That way you can just skip the TUNABLE_INT_FETCH() 
 stuff!
 
 Ok I can probably switch to that.
 
 Any objection if I mfc this feature to -stable if it does what I need?
 
 Hi,
 
 No objections from me at least, but it might require some work from your 
 side, because there was a lot of cleanup about removing duplicate 
 definitions, like static SYSCTLS which have already CTLFLAG_TUN and a 
 TUNABLE fetch statement, which makes the variable init twice. Just look at 
 the revision history for kern/kern_sysctl.c in 11-current.
 
 --HPS
 
 One question though... For the global sysctl for all nodes
 
 When is the var fetched?  If it's before SI_SUB_CPU it is not useful.
 
 Hi,
 
 It is quite early, actually:
 
 SYSINIT(sysctl, SI_SUB_KMEM, SI_ORDER_FIRST, sysctl_register_all, 0);
 
 In some parts of the machine independent, MI, code you neee to keep the 
 TUNABLE_FETCH'es, because its run before SI_SUB_KMEM !

Then it will not work unless I move the global n_queues sysctl creation into 
the driver's mod load function. 

Is that ok?



 
 --HPS
 
 
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 7:49 AM, John Baldwin j...@freebsd.org wrote:
 
 On Monday, December 01, 2014 07:19:13 AM Alfred Perlstein wrote:
 John,
 
 Will work on a new revision based on feedback.
 
 Two things to note however:
 
 Already explored the idea of using kernel_sysctlbyname but rejected due to
 following:
 
 It makes little sense to have a rw sysctl that only takes effect some
 times. This violates POLA at the expense of making code appear cleaner.
 Expectation is that writable sysctls take effect or are read only. They are
 not to be write sometimes unless we are to introduce a new flag. Instead
 of going to a confusing model we consider some form of rw sysctl that can
 set itself ro somehow. Otherwise people will be confused as to why nic
 queues says N while actually M.  What the rw-ro api would look like I have
 no idea. Suggestions?
 
 This is only somewhat true.  In the near distant future we will have a devctl 
 tool which would let you do 'devctl detach igb0  devctl attach igb0' which 
 would honor your post-boot setting of hw.igb.num_queues.  Instead what is 
 important to understand about this particular sysctl node is that it only 
 takes affect when a device is attached.  However, there are other control 
 knobs that also only affect future operations and not existing instances of 
 objects, so I don't think this is that big of a leap.
 

Strongly disagree here.  If I were not able to grok the c code I would be very 
confused by such a thing. In fact even with the fact that I do grok c code I 
would be very discouraged to find such behavior and strongly object to writable 
sysctls that do nothing. 

The ux is that the user has a bunch of dials on their dashboard that function 
as a busybox as opposed to doing what they are advertised to do. Sort of like 
those crossing light buttons in New York City that aren't actually hooked to 
anything. 

So: No. Frankly would rather back out the change entirely and keep this change 
local to us than expose users to such a UX. 

I will however like to discuss the possibility of a tunable/sysctl system that 
makes sense. 

-Alfred. 



 -- 
 John Baldwin
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 7:49 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:45, Alfred Perlstein wrote:
 
  Hi,
 
 It is quite early, actually:
 
 SYSINIT(sysctl, SI_SUB_KMEM, SI_ORDER_FIRST, sysctl_register_all, 0);
 
 In some parts of the machine independent, MI, code you neee to keep the 
 TUNABLE_FETCH'es, because its run before SI_SUB_KMEM !
 
 Then it will not work unless I move the global n_queues sysctl creation into 
 the driver's mod load function.
 
 Is that ok?
 
 Are you asking me?

In soviet russia no one is ever sure whom to ask for permission to proceed. 

(Also you have significant commits to the driver so it makes sense. )


 
 --HPS
 
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

Jack you were asked.   Please see the review system. 

Sent from my iPhone

 On Dec 1, 2014, at 8:05 AM, Jack Vogel jfvo...@gmail.com wrote:
 
 There is no mystery about who's drivers these are, and its not like it would
 take a lot of effort to figure out ownership and ask us for review. 
 
 Remove this commit until I have had time to look it over!
 
 Jack
 
 
 On Mon, Dec 1, 2014 at 7:56 AM, Alfred Perlstein bri...@mu.org wrote:
 
 
  On Dec 1, 2014, at 7:49 AM, Hans Petter Selasky h...@selasky.org wrote:
 
  On 12/01/14 16:45, Alfred Perlstein wrote:
 
   Hi,
 
  It is quite early, actually:
 
  SYSINIT(sysctl, SI_SUB_KMEM, SI_ORDER_FIRST, sysctl_register_all, 0);
 
  In some parts of the machine independent, MI, code you neee to keep the 
  TUNABLE_FETCH'es, because its run before SI_SUB_KMEM !
 
  Then it will not work unless I move the global n_queues sysctl creation 
  into the driver's mod load function.
 
  Is that ok?
 
  Are you asking me?
 
 In soviet russia no one is ever sure whom to ask for permission to proceed.
 
 (Also you have significant commits to the driver so it makes sense. )
 
 
 
  --HPS
 
 
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 8:28 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:56, Alfred Perlstein wrote:
 
 
 On Dec 1, 2014, at 7:49 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 On 12/01/14 16:45, Alfred Perlstein wrote:
 
 Hi,
 
 It is quite early, actually:
 
 SYSINIT(sysctl, SI_SUB_KMEM, SI_ORDER_FIRST, sysctl_register_all, 0);
 
 In some parts of the machine independent, MI, code you neee to keep the 
 TUNABLE_FETCH'es, because its run before SI_SUB_KMEM !
 
 Then it will not work unless I move the global n_queues sysctl creation 
 into the driver's mod load function.
 
 Is that ok?
 
 Are you asking me?
 
 In soviet russia no one is ever sure whom to ask for permission to proceed.
 
 (Also you have significant commits to the driver so it makes sense. )
 
 Ok,
 
 You need to check sys/sys/kernel.h for the full init order and figure out 
 where exactly your driver code is running and make sure that is running after 
 the SYSCTL/TUNABLE gets init.

Yes that is why it is being done by hand in the probe routine. I think proper 
thing might be a way to sort out how to get tunables to run at a driver load 
event?  Is that possible?

 
 --HPS
 
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

 On Dec 1, 2014, at 8:37 AM, Hans Petter Selasky h...@selasky.org wrote:
 
 Hi,
 
 I think you maybe missed a point 
 
 On 12/01/14 17:31, Alfred Perlstein wrote:
 
 Yes that is why it is being done by hand in the probe routine. I think 
 proper thing might be a way to sort out how to get tunables to run at a 
 driver load event?  Is that possible?
 
 All sysctls are tried init when they are created, both so-called static and 
 dynamic ones.
 
 If the sysctl is created inside the probe routine and has the tunable flag 
 set, it will get init before the creation is complete, if present in the boot 
 environment.
 
 If the sysctl is of a static kind, it will be created and initialized when 
 SI_SUB_KMEM is executing!

I totally understand this. It is in the phabricator review. :)

 
 --HPS
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r275326 - in head: sys/dev/cxgbe/tom sys/kern sys/netinet sys/sys usr.bin/bluetooth/btsockstat usr.bin/netstat usr.bin/systat

[Alfred Perlstein]

Agree, this was already discussed as quite possibly an incorrect way 
forward some months ago.


Where is the actual review for this huge change to the networking stack?

Splitting this into the mbuf layer adds a huge level of complexity where 
again, there are already completion paths in the socket layer to do 
this.  I am completely confused as to why this couldn't just be done 
with the socket callback system already in place.  Very open to being 
educated on this!


The concept of not filled mbufs in a socket buffer seems absolutely 
wrong at a glance, I'm sure with some better explanation this would all 
make sense, but really am still not convinced this is at all the right 
way to go on this.


Does any other OS do this for any reason?  Or is this just a short 
sighted hack for an experiment in sendfile?


I am really trying very hard to rationalize this change, so I will ask, 
is there something about keeping TCP windows open that you are hoping to 
accomplish that you can not otherwise do without sb_ccc and sb_acc?  If 
not then why is all this stuff being stuffed into mbufs as opposed to 
using callbacks?  It really seems wrong, my thoughts are this is like 
kse for mbufs something done with good intentions, but is complex and 
will have to be ripped out later.  Am I wrong here?


-Alfred

On 11/30/14, 9:55 AM, Adrian Chadd wrote:

Hi,

I really wished that these commits got individual reviews before they went in.

The whole idea of an mbuf whose data isn't quite there yet makes
debugging issues rather amusing, as now you may have VM issues to
debug at the same time as you're debugging network related stuff.

I really think this could've been done without all the back-handed VM
work. The mbufs and IO buffers both have completion function calls; it
would've been much less intrusive to do it that way.



-adrian


On 30 November 2014 at 04:52, Gleb Smirnoff gleb...@freebsd.org wrote:

Author: glebius
Date: Sun Nov 30 12:52:33 2014
New Revision: 275326
URL: https://svnweb.freebsd.org/changeset/base/275326

Log:
   Merge from projects/sendfile:

   o Introduce a notion of not ready mbufs in socket buffers.  These
   mbufs are now being populated by some I/O in background and are
   referenced outside.  This forces following implications:
   - An mbuf which is not ready can't be taken out of the buffer.
   - An mbuf that is behind a not ready in the queue neither.
   - If sockbet buffer is flushed, then not ready mbufs shouln't be
 freed.

   o In struct sockbuf the sb_cc field is split into sb_ccc and sb_acc.
 The sb_ccc stands for claimed character count, or committed
 character count.  And the sb_acc is available character count.
 Consumers of socket buffer API shouldn't already access them directly,
 but use sbused() and sbavail() respectively.
   o Not ready mbufs are marked with M_NOTREADY, and ready but blocked ones
 with M_BLOCKED.
   o New field sb_fnrdy points to the first not ready mbuf, to avoid linear
 search.
   o New function sbready() is provided to activate certain amount of mbufs
 in a socket buffer.

   A special note on SCTP:
 SCTP has its own sockbufs.  Unfortunately, FreeBSD stack doesn't yet
   allow protocol specific sockbufs.  Thus, SCTP does some hacks to make
   itself compatible with FreeBSD: it manages sockbufs on its own, but keeps
   sb_cc updated to inform the stack of amount of data in them.  The new
   notion of not ready data isn't supported by SCTP.  Instead, only a
   mechanical substitute is done: s/sb_cc/sb_ccc/.
 A proper solution would be to take away struct sockbuf from struct
   socket and allow protocols to implement their own socket buffers, like
   SCTP already does.  This was discussed with rrs@.

   Sponsored by: Netflix
   Sponsored by: Nginx, Inc.

Modified:
   head/sys/dev/cxgbe/tom/t4_ddp.c
   head/sys/kern/uipc_debug.c
   head/sys/kern/uipc_sockbuf.c
   head/sys/kern/uipc_socket.c
   head/sys/netinet/sctp_indata.c
   head/sys/netinet/sctp_input.c
   head/sys/netinet/sctp_os_bsd.h
   head/sys/netinet/sctp_output.c
   head/sys/netinet/sctp_pcb.c
   head/sys/netinet/sctp_pcb.h
   head/sys/netinet/sctp_structs.h
   head/sys/netinet/sctp_usrreq.c
   head/sys/netinet/sctp_var.h
   head/sys/netinet/sctputil.c
   head/sys/netinet/sctputil.h
   head/sys/sys/sockbuf.h
   head/usr.bin/bluetooth/btsockstat/btsockstat.c
   head/usr.bin/netstat/inet.c
   head/usr.bin/netstat/netgraph.c
   head/usr.bin/netstat/unix.c
   head/usr.bin/systat/netstat.c

Modified: head/sys/dev/cxgbe/tom/t4_ddp.c
==
--- head/sys/dev/cxgbe/tom/t4_ddp.c Sun Nov 30 12:37:20 2014
(r275325)
+++ head/sys/dev/cxgbe/tom/t4_ddp.c Sun Nov 30 12:52:33 2014
(r275326)
@@ -971,8 +971,9 @@ handle_ddp(struct socket *so, struct uio
  */
 rc = sbwait(sb);
 while (toep-ddp_flags  buf_flag) {
+   /* XXXGL: shouldn't here be sbwait() 

Re: svn commit: r275326 - in head: sys/dev/cxgbe/tom sys/kern sys/netinet sys/sys usr.bin/bluetooth/btsockstat usr.bin/netstat usr.bin/systat

[Alfred Perlstein]

On 11/30/14, 10:38 AM, Gleb Smirnoff wrote:

   Adrian  Alfred,

On Sun, Nov 30, 2014 at 09:55:05AM -0800, Adrian Chadd wrote:
A I really wished that these commits got individual reviews before they went 
in.

On Sun, Nov 30, 2014 at 10:10:57AM -0800, Alfred Perlstein wrote:
A Agree, this was already discussed as quite possibly an incorrect way
A forward some months ago.
A
A Where is the actual review for this huge change to the networking stack?
A

The development was going on in the open branch, and I've been sending the
diff on the new sendfile constantly during this year. Last time I sent the
diff, the discussion very quickly moved to Lua in kernel, script(2) and
other buzz, actually ignoring my review and test request and hijacking
my thread:

https://lists.freebsd.org/pipermail/freebsd-arch/2014-September/015861.html

Yes, Alfred did ask me why can't this be accomplished with socket
upcalls, and replied I to him, but discussion ended there:

https://lists.freebsd.org/pipermail/freebsd-arch/2014-September/015858.html

Probably the script(2) topic was more appealing.

If you don't mind, I will make a separate replies on your actual
technical questions.


Gleb,

Maybe I misread your reply to 
'https://lists.freebsd.org/pipermail/freebsd-arch/2014-September/015858.html' 
but my interpretation of your reply was that you would reconsider your 
approach because the suggestion to use socket callback was a valid one.


I am very open to being wrong on this issue, but with all due respect I 
don't see a good reason why not to use socket callbacks to even 
implement what you have right now.


I appreciate what seem to be kind words saying my idea is good, but it 
didn't really answer the question nor address the issue I raised.


I am currently sorting out why this approach was taken and here's what I 
have so far, this is me trying to find the best here:


1) Possible that the socket buffer accounting needs to be done to keep 
TCP window open for send?  (Could this be forced via setsockopt(2) 
option instead)?

2) Lock order issues in the socket callback.
3) There is only a single socket callback func and arg (maybe we need more?)
4) Properly asserting backpressure and actual blocking in sendfile(2) 
becomes very difficult otherwise, meaning that if you don't have the 
complexity in socketbuffer, then you need to sort of implement this 
inside a private control block AND do accounting there.
5) This is a cool feature and maybe other things can make use of it 
eventually?  - hoping this is more of a good thing as opposed to 
(sorry) kse.
6) Alllows one to mix async sendfile write WITH sync write and the 
correct thing happens?  Am I right in assuming that socketbuffers can 
wind up with mbufs as such:  M_NOTREADY, M_READY, M_NOTREADY, M_READY, 
M_NOTREADY, M_READY ?  If so that is pretty interesting and useful.


Again my proposal would be to use socket callback and callback arg would 
basically be something like:


socket_sendfile_cb_arg {
  struct mbuf *head;  /* list of mbufs queued */
}

When the callback is fired just pop off the front of the queue and put 
on the socketbuffer.


This however has the following drawbacks:
1) May have trouble if anyone else is using the sbcallback as well? 
aio?  (might need to allow multiple callbacks now).
2) Can't do M_NOTREADY, M_READY, M_NOTREADY, M_READY, M_NOTREADY, 
M_READY, basically write(2) can get in front of async sendfile.


I know you have put a lot of effort into this and these are just my 
semi-random thoughts on the issue, so I apologize if I'm making you 
repeat a lot of your thinking on the issue, I would just like to 
understand a bit better why this was done and if we really need to do it.


-Alfred





___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r275136 - in head/sys: dev/e1000 dev/ixgbe kern sys

[Alfred Perlstein]

Author: alfred
Date: Wed Nov 26 20:19:36 2014
New Revision: 275136
URL: https://svnweb.freebsd.org/changeset/base/275136

Log:
  Make igb and ixgbe check tunables at probe time.
  
  This allows one to make a kernel module to tune the
  number of queues before the driver loads.
  
  This is needed so that a module at SI_SUB_CPU can set
  tunables for these drivers to take.  Otherwise getenv
  is called too early by the TUNABLE macros.
  
  Reviewed by: smh
  Phabric: https://reviews.freebsd.org/D1149

Modified:
  head/sys/dev/e1000/if_igb.c
  head/sys/dev/ixgbe/ixgbe.c
  head/sys/kern/subr_bus.c
  head/sys/sys/bus.h

Modified: head/sys/dev/e1000/if_igb.c
==
--- head/sys/dev/e1000/if_igb.c Wed Nov 26 18:03:25 2014(r275135)
+++ head/sys/dev/e1000/if_igb.c Wed Nov 26 20:19:36 2014(r275136)
@@ -188,6 +188,7 @@ static char *igb_strings[] = {
 /*
  *  Function prototypes
  */
+static int igb_per_unit_num_queues(SYSCTL_HANDLER_ARGS);
 static int igb_probe(device_t);
 static int igb_attach(device_t);
 static int igb_detach(device_t);
@@ -493,6 +494,11 @@ igb_attach(device_t dev)
OID_AUTO, nvm, CTLTYPE_INT|CTLFLAG_RW, adapter, 0,
igb_sysctl_nvm_info, I, NVM Information);
 
+SYSCTL_ADD_PROC(device_get_sysctl_ctx(dev),
+SYSCTL_CHILDREN(device_get_sysctl_tree(dev)),
+   OID_AUTO, num_queues, CTLTYPE_INT | CTLFLAG_RD,
+   adapter, 0, igb_per_unit_num_queues, I, Number of 
Queues);
+
igb_set_sysctl_value(adapter, enable_aim,
Interrupt Moderation, adapter-enable_aim,
igb_enable_aim);
@@ -2831,6 +2837,7 @@ igb_setup_msix(struct adapter *adapter)
 {
device_tdev = adapter-dev;
int bar, want, queues, msgs, maxqueues;
+   int n_queues;
 
/* tuneable override */
if (igb_enable_msix == 0)
@@ -2858,8 +2865,18 @@ igb_setup_msix(struct adapter *adapter)
goto msi;
}
 
-   /* Figure out a reasonable auto config value */
-   queues = (mp_ncpus  (msgs-1)) ? (msgs-1) : mp_ncpus;
+   n_queues = 0;
+   /* try more specific tunable, then global, then finally default to boot 
time tunable if set. */
+   if (device_getenv_int(dev, num_queues, n_queues) != 0) {
+   device_printf(dev, using specific tunable num_queues=%d, 
n_queues);
+   } else if (TUNABLE_INT_FETCH(hw.igb.num_queues, n_queues) != 0) {
+   if (igb_num_queues != n_queues) {
+   device_printf(dev, using global tunable 
hw.igb.num_queues=%d, n_queues);
+   igb_num_queues = n_queues;
+   }
+   } else {
+   n_queues = igb_num_queues;
+   }
 
 #ifdef RSS
/* If we're doing RSS, clamp at the number of RSS buckets */
@@ -2867,10 +2884,12 @@ igb_setup_msix(struct adapter *adapter)
queues = rss_getnumbuckets();
 #endif
 
-
-   /* Manual override */
-   if (igb_num_queues != 0)
-   queues = igb_num_queues;
+   if (n_queues != 0) {
+   queues = n_queues;
+   } else {
+   /* Figure out a reasonable auto config value */
+   queues = (mp_ncpus  (msgs-1)) ? (msgs-1) : mp_ncpus;
+   }
 
/* Sanity check based on HW */
switch (adapter-hw.mac.type) {
@@ -2893,12 +2912,17 @@ igb_setup_msix(struct adapter *adapter)
maxqueues = 1;
break;
}
-   if (queues  maxqueues)
+   if (queues  maxqueues) {
+   device_printf(adapter-dev, requested %d queues, but max for 
this adapter is %d\n,
+   queues, maxqueues);
queues = maxqueues;
-
-   /* Manual override */
-   if (igb_num_queues != 0)
-   queues = igb_num_queues;
+   } else if (queues == 0) {
+   queues = 1;
+   } else if (queues  0) {
+   device_printf(adapter-dev, requested %d queues, but min for 
this adapter is %d\n,
+   queues, 1);
+   queues = 1;
+   }
 
/*
** One vector (RX/TX pair) per queue
@@ -6384,3 +6408,14 @@ igb_sysctl_eee(SYSCTL_HANDLER_ARGS)
IGB_CORE_UNLOCK(adapter);
return (0);
 }
+
+static int
+igb_per_unit_num_queues(SYSCTL_HANDLER_ARGS)
+{
+   struct adapter  *adapter;
+
+   adapter = (struct adapter *) arg1;
+
+   return sysctl_handle_int(oidp, adapter-num_queues, 0, req);
+}
+

Modified: head/sys/dev/ixgbe/ixgbe.c
==
--- head/sys/dev/ixgbe/ixgbe.c  Wed Nov 26 18:03:25 2014(r275135)
+++ head/sys/dev/ixgbe/ixgbe.c  Wed Nov 26 20:19:36 

Re: svn commit: r274672 - in head/contrib/libxo: . libxo xolint

[Alfred Perlstein]

Marcel, is there a way to get libxo programs to emit time series data?  

Any examples of this in the tree right now?

Example would be if netstat -1 was libxo-ified, it would also emit a 
timestamp.

-Alfred

On Nov 18, 2014, at 10:03 AM, Marcel Moolenaar wrote:

 Author: marcel
 Date: Tue Nov 18 18:03:40 2014
 New Revision: 274672
 URL: https://svnweb.freebsd.org/changeset/base/274672
 
 Log:
  Upgrade libxo to 0.1.6.
 
  Summary of changes:
  1.  Coverity defect fixes
 
  Obtained from:  https://github.com/Juniper/libxo/releases/tag/0.1.6
 
 Modified:
  head/contrib/libxo/configure.ac
  head/contrib/libxo/libxo/libxo.c
  head/contrib/libxo/libxo/xoconfig.h
  head/contrib/libxo/libxo/xoversion.h
  head/contrib/libxo/xolint/xolint.pl
 
 Modified: head/contrib/libxo/configure.ac
 ==
 --- head/contrib/libxo/configure.ac   Tue Nov 18 17:37:33 2014
 (r274671)
 +++ head/contrib/libxo/configure.ac   Tue Nov 18 18:03:40 2014
 (r274672)
 @@ -12,7 +12,7 @@
 #
 
 AC_PREREQ(2.2)
 -AC_INIT([libxo], [0.1.5], [p...@juniper.net])
 +AC_INIT([libxo], [0.1.6], [p...@juniper.net])
 AM_INIT_AUTOMAKE([-Wall -Werror foreign -Wno-portability])
 
 # Support silent build rules.  Requires at least automake-1.11.
 
 Modified: head/contrib/libxo/libxo/libxo.c
 ==
 --- head/contrib/libxo/libxo/libxo.c  Tue Nov 18 17:37:33 2014
 (r274671)
 +++ head/contrib/libxo/libxo/libxo.c  Tue Nov 18 18:03:40 2014
 (r274672)
 @@ -317,7 +317,7 @@ xo_init_handle (xo_handle_t *xop)
   cp = getenv(LC_ALL);
   if (cp == NULL)
   cp = UTF-8;   /* Optimistic? */
 - cp = setlocale(LC_CTYPE, cp);
 + (void) setlocale(LC_CTYPE, cp);
 }
 
 /*
 @@ -607,8 +607,10 @@ xo_vsnprintf (xo_handle_t *xop, xo_buffe
   rc = vsnprintf(xbp-xb_curp, left, fmt, va_local);
 
 if (rc  xbp-xb_size) {
 - if (!xo_buf_has_room(xbp, rc))
 + if (!xo_buf_has_room(xbp, rc)) {
 + va_end(va_local);
   return -1;
 + }
 
   /*
* After we call vsnprintf(), the stage of vap is not defined.
 @@ -648,8 +650,10 @@ xo_printf_v (xo_handle_t *xop, const cha
 rc = vsnprintf(xbp-xb_curp, left, fmt, va_local);
 
 if (rc  xbp-xb_size) {
 - if (!xo_buf_has_room(xbp, rc))
 + if (!xo_buf_has_room(xbp, rc)) {
 + va_end(va_local);
   return -1;
 + }
 
   va_end(va_local);   /* Reset vap to the start */
   va_copy(va_local, vap);
 @@ -974,8 +978,10 @@ xo_warn_hcv (xo_handle_t *xop, int code,
   int left = xbp-xb_size - (xbp-xb_curp - xbp-xb_bufp);
   int rc = vsnprintf(xbp-xb_curp, left, newfmt, vap);
   if (rc  xbp-xb_size) {
 - if (!xo_buf_has_room(xbp, rc))
 + if (!xo_buf_has_room(xbp, rc)) {
 + va_end(va_local);
   return;
 + }
 
   va_end(vap);/* Reset vap to the start */
   va_copy(vap, va_local);
 @@ -1118,8 +1124,10 @@ xo_message_hcv (xo_handle_t *xop, int co
   int left = xbp-xb_size - (xbp-xb_curp - xbp-xb_bufp);
   rc = vsnprintf(xbp-xb_curp, left, fmt, vap);
   if (rc  xbp-xb_size) {
 - if (!xo_buf_has_room(xbp, rc))
 + if (!xo_buf_has_room(xbp, rc)) {
 + va_end(va_local);
   return;
 + }
 
   va_end(vap);/* Reset vap to the start */
   va_copy(vap, va_local);
 @@ -1154,14 +1162,15 @@ xo_message_hcv (xo_handle_t *xop, int co
 
   va_copy(va_local, vap);
 
 - rc = vsnprintf(buf, bufsiz, fmt, va_local);
 + rc = vsnprintf(bp, bufsiz, fmt, va_local);
   if (rc  bufsiz) {
   bufsiz = rc + BUFSIZ;
   bp = alloca(bufsiz);
   va_end(va_local);
   va_copy(va_local, vap);
 - rc = vsnprintf(buf, bufsiz, fmt, va_local);
 + rc = vsnprintf(bp, bufsiz, fmt, va_local);
   }
 + va_end(va_local);
   cp = bp + rc;
 
   if (need_nl) {
 @@ -1302,9 +1311,9 @@ xo_create_to_file (FILE *fp, xo_style_t 
  * @xop XO handle to alter (or NULL for default handle)
  */
 void
 -xo_destroy (xo_handle_t *xop)
 +xo_destroy (xo_handle_t *xop_arg)
 {
 -xop = xo_default(xop);
 +xo_handle_t *xop = xo_default(xop_arg);
 
 if (xop-xo_close  (xop-xo_flags  XOF_CLOSE_FP))
   xop-xo_close(xop-xo_opaque);
 @@ -1315,7 +1324,7 @@ xo_destroy (xo_handle_t *xop)
 xo_buf_cleanup(xop-xo_predicate);
 xo_buf_cleanup(xop-xo_attrs);
 
 -if (xop == xo_default_handle) {
 +if (xop_arg == NULL) {
   bzero(xo_default_handle, sizeof(xo_default_handle));
   xo_default_inited = 0;
 } else
 @@ -1743,7 +1752,7 @@ xo_format_string_direct (xo_handle_t *xo
int need_enc, int have_enc)
 {
 int cols = 0;
 -wchar_t wc;
 +wchar_t wc = 0;
 int ilen, olen, width;
 int 

Re: svn commit: r274573 - head/contrib/netbsd-tests/lib/libpthread

[Alfred Perlstein]

This looks easy enough to fix under _thr_find_thread() in libthread.

Any interest in fixing it?

Might be worth hacking _thr_find_thread() to take an ERRNO to return 
based on NULL until we chase down all the paths into it just in case 
EINVAL is a valid ptr.


Also, just wondering what happens on other platforms, does it elicit a 
crash?  Ie. is NULL a safe value to pass in on other platforms?


-Alfred

On 11/15/14, 9:08 PM, Garrett Cooper wrote:

Author: ngie
Date: Sun Nov 16 05:08:19 2014
New Revision: 274573
URL: https://svnweb.freebsd.org/changeset/base/274573

Log:
   Expect :pthread_detach to fail with EINVAL instead of ESRCH on FreeBSD
   
   PR: 191906

   In collaboration with: pho

Modified:
   head/contrib/netbsd-tests/lib/libpthread/t_detach.c

Modified: head/contrib/netbsd-tests/lib/libpthread/t_detach.c
==
--- head/contrib/netbsd-tests/lib/libpthread/t_detach.c Sun Nov 16 05:06:35 
2014(r274572)
+++ head/contrib/netbsd-tests/lib/libpthread/t_detach.c Sun Nov 16 05:08:19 
2014(r274573)
@@ -75,6 +75,10 @@ ATF_TC_BODY(pthread_detach, tc)
rv = pthread_join(t, NULL);
ATF_REQUIRE(rv == EINVAL);
  
+#if defined(__FreeBSD__)

+   atf_tc_expect_fail(PR # 191906: fails with EINVAL, not ESRCH);
+#endif
+
/*
 * As usual, ESRCH should follow if
 * we try to detach an invalid thread.



___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r274573 - head/contrib/netbsd-tests/lib/libpthread

[Alfred Perlstein]

On 11/15/14, 9:22 PM, Garrett Cooper wrote:

On Nov 15, 2014, at 21:19, Alfred Perlstein alf...@freebsd.org wrote:


This looks easy enough to fix under _thr_find_thread() in libthread.

Any interest in fixing it?

Yes, if it’s POSIXly correct and doesn’t break everything else.


Might be worth hacking _thr_find_thread() to take an ERRNO to return based on 
NULL until we chase down all the paths into it just in case EINVAL is a valid 
ptr.

K. Thanks for the hint!


Also, just wondering what happens on other platforms, does it elicit a crash?  
Ie. is NULL a safe value to pass in on other platforms?

I wish I knew what happened on !x86 platforms… I honestly don’t have access to 
ARM/MIPS/PowerPC, so I can’t say :/.

Thanks!


Oh, I meant Linux and Solaris, or even other BSD.

-Alfred

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r274017 - head/sys/kern

[Alfred Perlstein]

 On Nov 4, 2014, at 1:25 AM, Konstantin Belousov kostik...@gmail.com wrote:
 
 On Mon, Nov 03, 2014 at 01:45:21PM -0800, Alfred Perlstein wrote:
 Isn't there a problem where the stack can be swapped out?
 
 I seem to recall a problem where a swapped out process was causing 
 problems due to a buffer passed being stack allocated and that process 
 being swapped out...
 
 If this is not the case then please disregard.
 
 Sure, stack can be swapped out, but buffer passing is usually not a problem.
 At least, I am not aware of cases.
 
 In fact, many compat layers do exactly this, allocate the native-ABI
 structure on the stack, copyin the foreighn-ABI structure in pieces
 into the native-ABI one, and pass native to kern_foo() implementations.
 
 So I think you worries are not realized.
 

Ok then as long as system will not pass this buffer as Dma target down to 
driver we are ok.  
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r274017 - head/sys/kern

[Alfred Perlstein]

 On Nov 4, 2014, at 6:22 AM, Alfred Perlstein bri...@mu.org wrote:
 
 
 
 On Nov 4, 2014, at 1:25 AM, Konstantin Belousov kostik...@gmail.com wrote:
 
 On Mon, Nov 03, 2014 at 01:45:21PM -0800, Alfred Perlstein wrote:
 Isn't there a problem where the stack can be swapped out?
 
 I seem to recall a problem where a swapped out process was causing 
 problems due to a buffer passed being stack allocated and that process 
 being swapped out...
 
 If this is not the case then please disregard.
 
 Sure, stack can be swapped out, but buffer passing is usually not a problem.
 At least, I am not aware of cases.
 
 In fact, many compat layers do exactly this, allocate the native-ABI
 structure on the stack, copyin the foreighn-ABI structure in pieces
 into the native-ABI one, and pass native to kern_foo() implementations.
 
 So I think you worries are not realized.
 
 Ok then as long as system will not pass this buffer as Dma target down to 
 driver we are ok.  

Also thank you for explanation. 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r274017 - head/sys/kern

[Alfred Perlstein]

Isn't there a problem where the stack can be swapped out?

I seem to recall a problem where a swapped out process was causing 
problems due to a buffer passed being stack allocated and that process 
being swapped out...


If this is not the case then please disregard.

-Alfred

On 11/2/14, 11:46 PM, Mateusz Guzik wrote:

Author: mjg
Date: Mon Nov  3 07:46:51 2014
New Revision: 274017
URL: https://svnweb.freebsd.org/changeset/base/274017

Log:
   Provide an on-stack temporary buffer for small ioctl requests.

Modified:
   head/sys/kern/sys_generic.c

Modified: head/sys/kern/sys_generic.c
==
--- head/sys/kern/sys_generic.c Mon Nov  3 07:18:42 2014(r274016)
+++ head/sys/kern/sys_generic.c Mon Nov  3 07:46:51 2014(r274017)
@@ -649,6 +649,7 @@ sys_ioctl(struct thread *td, struct ioct
u_long com;
int arg, error;
u_int size;
+   u_char smalldata[128];
caddr_t data;
  
  	if (uap-com  0x) {

@@ -680,17 +681,18 @@ sys_ioctl(struct thread *td, struct ioct
arg = (intptr_t)uap-data;
data = (void *)arg;
size = 0;
-   } else
-   data = malloc((u_long)size, M_IOCTLOPS, M_WAITOK);
+   } else {
+   if (size = sizeof(smalldata))
+   data = smalldata;
+   else
+   data = malloc((u_long)size, M_IOCTLOPS, 
M_WAITOK);
+   }
} else
data = (void *)uap-data;
if (com  IOC_IN) {
error = copyin(uap-data, data, (u_int)size);
-   if (error) {
-   if (size  0)
-   free(data, M_IOCTLOPS);
-   return (error);
-   }
+   if (error != 0)
+   goto out;
} else if (com  IOC_OUT) {
/*
 * Zero the buffer so the user always
@@ -704,7 +706,8 @@ sys_ioctl(struct thread *td, struct ioct
if (error == 0  (com  IOC_OUT))
error = copyout(data, uap-data, (u_int)size);
  
-	if (size  0)

+out:
+   if (size  0  data != (caddr_t)smalldata)
free(data, M_IOCTLOPS);
return (error);
  }



___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

[Alfred Perlstein]

On 6/25/14 8:30 AM, Attilio Rao wrote:

On Wed, Jun 25, 2014 at 5:16 PM, Alfred Perlstein alf...@freebsd.org wrote:

On 6/25/14 5:41 AM, Attilio Rao wrote:

On Wed, Jun 25, 2014 at 2:09 PM, Gleb Smirnoff gleb...@freebsd.org
wrote:

On Wed, Jun 25, 2014 at 01:58:29PM +0200, Attilio Rao wrote:
A  Log:
Axen/virtio: fix balloon drivers to not mark pages as WIRED
A 
APrevent the Xen and VirtIO balloon drivers from marking pages as
Awired. This prevents them from increasing the system wired page
count,
Awhich can lead to mlock failing because of hitting the limit in
Avm.max_wired.
A
A This change is conceptually wrong.
A The pages balloon is allocating are unmanaged and they should be wired
A by definition. Alan and I are considering enforcing this (mandatory
A wired pages for unmanaged pages allocation) directly in the KPI.
A This in practice just seem an artifact to deal with scarce  wired
A memory limit. I suggest that for the XEN case this limit gets bumped
A rather relying on similar type of hacks.

Proper limit would be to count pages wired by userland via mlock(2)
and enforce limit only on those pages. Pages wired by kernel should
be either unlimited or controled by a separate limit.

FWIW, I mostly agree with this. I think that the kernel and userland
limits should be split apart. But for the time being, rising the limit
is better.

Attilio



Can you explain?  I would think that if you were designing some kind of
embedded device you would want to know exactly how much locked pages there
are overall, not just in userland.

Meaning you would not want to overcommit and have too many locked pages due
to kernel+user.

Well, assuming you trace them indipendently I don't think this is
going to be problematic to aggregate them, is it?

I am not sure as I am not as strong in this area as you are.


As far as I understand it, right now we have RMEM_LIMIT to someway
limit per-process amount of wired memory and finally max_wired as a
global accounted wired memory limit.

I think that the idea now is that RMEM_LIMIT is enough to correctly
control all the front-end check, coming from untrusted sources
(userland, non-privileged syscalls like mlock(), mmap(), etc.).
Possibly that's not always the case and I think that the hypervisor
can be a fair example of this.

Having more granular accountability, means that rather than having a
global limit (or, rather, along with it) we can grow a per-process
limit to control kernel-allocated wired memory.


Perhaps that needs an API as well?

I don't have anything in my mind yet. My initial point was more trying
to get a better semantic on a paradigm that is at least dangerous.

Attilio


My concern is a group of daemons working to provide system services 
playing nicely with each other and the system as a whole.


I think the point is that let's say you have a concert of userspace 
daemons importantd(8) and imperatived(8) running on a system.


Both importantd(8) and imperatived(8) need pages wired for dealing with 
important timing/throughput issues.  The kernel obviously needs such 
pages as well.


importantd(8) and imperatived(8) do not want to blow up the system by 
requesting more than a fixed amount otherwise supposed bad things will 
happen, or perhaps they deadlock against each other.


A global count seems (to me) to make sense at this point even if the 
kernel ignores it as a way for everything to act in concert.


Is there a way for kernel, importantd(8), and imperatived(8) to play 
nice together, meaning they can take each other's wired count into 
account if we get rid of the global?  My feeling is no, that they will 
then need another rendezvous to do global accounting, if we retire this 
facility.


I'm likely wrong, but wanted to bring this up as a concern.

-Alfred


___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

[Alfred Perlstein]

On 6/25/14 5:41 AM, Attilio Rao wrote:

On Wed, Jun 25, 2014 at 2:09 PM, Gleb Smirnoff gleb...@freebsd.org wrote:

On Wed, Jun 25, 2014 at 01:58:29PM +0200, Attilio Rao wrote:
A  Log:
Axen/virtio: fix balloon drivers to not mark pages as WIRED
A 
APrevent the Xen and VirtIO balloon drivers from marking pages as
Awired. This prevents them from increasing the system wired page count,
Awhich can lead to mlock failing because of hitting the limit in
Avm.max_wired.
A
A This change is conceptually wrong.
A The pages balloon is allocating are unmanaged and they should be wired
A by definition. Alan and I are considering enforcing this (mandatory
A wired pages for unmanaged pages allocation) directly in the KPI.
A This in practice just seem an artifact to deal with scarce  wired
A memory limit. I suggest that for the XEN case this limit gets bumped
A rather relying on similar type of hacks.

Proper limit would be to count pages wired by userland via mlock(2)
and enforce limit only on those pages. Pages wired by kernel should
be either unlimited or controled by a separate limit.

FWIW, I mostly agree with this. I think that the kernel and userland
limits should be split apart. But for the time being, rising the limit
is better.

Attilio


Can you explain?  I would think that if you were designing some kind of 
embedded device you would want to know exactly how much locked pages 
there are overall, not just in userland.


Meaning you would not want to overcommit and have too many locked pages 
due to kernel+user.


Perhaps that needs an API as well?
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

[Alfred Perlstein]

On 6/25/14 6:49 AM, Andriy Gapon wrote:

On 25/06/2014 17:44, Attilio Rao wrote:

Why? If VM needs more wired memory I assume that we can tune up the
default value of max_wired?

I think that however your case makes an interesting point: if we want
to make unmanaged pages as inherently wired, we likely need a little
bit higher max_wired value. When I completed a patch for this, pho@
couldn't reproduce any similar issue even with stress-testing (and
also, the places to allocate unmanaged pages and not requesting
VM_ALLOC_WIRED were very little, almost 0, with the exception of
vm_page_alloc_contig() calls) but I think it is a valid proposition.

However I would still like to have more control on kernel-specific
wired memory for processes. I'm for example thinking to ARC vs. buffer
cache, where I expect the wired memory consumption to be much bigger
for the former case.

My humble opinion is that userland page wiring should be tuned via
resource limits and that vm.max_wired could be retired altogether.
Kernel wiring ignores the knob anyway.

I think the goal of the limit as it stands would not to let the total 
amount of wired pages reach a bad point due to a userland program 
pushing it over the limit.


Basically userland wants to know how many pages the kernel has wired in 
order to avoid asking for too many pages and making the system implode.

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r267559 - head/share/examples/bhyve

[Alfred Perlstein]

Author: alfred
Date: Tue Jun 17 00:53:00 2014
New Revision: 267559
URL: http://svnweb.freebsd.org/changeset/base/267559

Log:
  Support for multiple disks and tap devices.
  
  This allows you to give a bhyve instance multiple network devices
  and disk devices easily by specifying additional -d  and -t 
  options.
  
  Reviewed by: neel
  Sponsored by: Norse

Modified:
  head/share/examples/bhyve/vmrun.sh

Modified: head/share/examples/bhyve/vmrun.sh
==
--- head/share/examples/bhyve/vmrun.sh  Mon Jun 16 22:59:18 2014
(r267558)
+++ head/share/examples/bhyve/vmrun.sh  Tue Jun 17 00:53:00 2014
(r267559)
@@ -78,8 +78,8 @@ isofile=${DEFAULT_ISOFILE}
 memsize=${DEFAULT_MEMSIZE}
 console=${DEFAULT_CONSOLE}
 cpus=${DEFAULT_CPUS}
-virtio_diskdev=${DEFAULT_VIRTIO_DISK}
-tapdev=${DEFAULT_TAPDEV}
+tap_total=0
+disk_total=0
 apic_opt=
 gdbport=0
 loader_opt=
@@ -96,7 +96,8 @@ while getopts ac:C:d:e:g:hH:iI:m:t: c ; 
console=${OPTARG}
;;
d)
-   virtio_diskdev=${OPTARG}
+   eval disk_dev${disk_total}=\${OPTARG}\
+   disk_total=$(($disk_total + 1))
;;
e)
loader_opt=${loader_opt} -e ${OPTARG}
@@ -117,7 +118,8 @@ while getopts ac:C:d:e:g:hH:iI:m:t: c ; 
memsize=${OPTARG}
;;
t)
-   tapdev=${OPTARG}
+   eval tap_dev${tap_total}=\${OPTARG}\
+   tap_total=$(($tap_total + 1))
;;
*)
usage
@@ -125,6 +127,16 @@ while getopts ac:C:d:e:g:hH:iI:m:t: c ; 
esac
 done
 
+if [ $tap_total -eq 0 ] ; then
+tap_total=1
+tap_dev0=${DEFAULT_TAPDEV}
+fi
+if [ $disk_total -eq 0 ] ; then
+disk_total=1
+disk_dev0=${DEFAULT_VIRTIO_DISK}
+
+fi
+
 shift $((${OPTIND} - 1))
 
 if [ $# -ne 1 ]; then
@@ -136,25 +148,31 @@ if [ -n ${host_base} ]; then
loader_opt=${loader_opt} -h ${host_base}
 fi
 
-# Create the virtio diskdev file if needed
-if [ ! -f ${virtio_diskdev} ]; then
-   echo virtio disk device file \${virtio_diskdev}\ does not exist.
-   echo Creating it ...
-   truncate -s 8G ${virtio_diskdev}  /dev/null
-fi
-
-if [ ! -r ${virtio_diskdev} ]; then
-   echo virtio disk device file \${virtio_diskdev}\ is not readable
-   exit 1
-fi
-
-if [ ! -w ${virtio_diskdev} ]; then
-   echo virtio disk device file \${virtio_diskdev}\ is not writable
-   exit 1
-fi
+make_and_check_diskdev()
+{
+local virtio_diskdev=$1
+# Create the virtio diskdev file if needed
+if [ ! -f ${virtio_diskdev} ]; then
+   echo virtio disk device file \${virtio_diskdev}\ does not exist.
+   echo Creating it ...
+   truncate -s 8G ${virtio_diskdev}  /dev/null
+fi
+
+if [ ! -r ${virtio_diskdev} ]; then
+   echo virtio disk device file \${virtio_diskdev}\ is not readable
+   exit 1
+fi
+
+if [ ! -w ${virtio_diskdev} ]; then
+   echo virtio disk device file \${virtio_diskdev}\ is not writable
+   exit 1
+fi
+}
 
 echo Launching virtual machine \$vmname\ ...
 
+virtio_diskdev=$disk_dev0
+
 while [ 1 ]; do
${BHYVECTL} --vm=${vmname} --destroy  /dev/null 21
 
@@ -189,12 +207,33 @@ while [ 1 ]; do
break
fi
 
+   #
+   # Build up args for additional tap and disk devices now.
+   #
+   nextslot=2  # slot 0 is hostbridge, slot 1 is lpc
+   devargs=  # accumulate disk/tap args here
+   i=0
+   while [ $i -lt $tap_total ] ; do
+   eval tapname=\$tap_dev${i}
+   devargs=$devargs -s $nextslot:0,virtio-net,${tapname} 
+   nextslot=$(($nextslot + 1))
+   i=$(($i + 1))
+   done
+
+   i=0
+   while [ $i -lt $disk_total ] ; do
+   eval disk=\$disk_dev${i}
+   make_and_check_diskdev ${disk}
+   devargs=$devargs -s $nextslot:0,virtio-blk,${disk} 
+   nextslot=$(($nextslot + 1))
+   i=$(($i + 1))
+   done
+
${FBSDRUN} -c ${cpus} -m ${memsize} ${apic_opt} -A -H -P\
-g ${gdbport}   \
-s 0:0,hostbridge   \
-s 1:0,lpc  \
-   -s 2:0,virtio-net,${tapdev} \
-   -s 3:0,virtio-blk,${virtio_diskdev} \
+   ${devargs}  \
-l com1,${console}  \
${installer_opt}\
${vmname}
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r267233 - in head: . bin/rmail gnu/usr.bin/binutils/addr2line gnu/usr.bin/binutils/nm gnu/usr.bin/binutils/objcopy gnu/usr.bin/binutils/objdump gnu/usr.bin/binutils/readelf gnu/usr.bin

[Alfred Perlstein]

On 6/8/14 11:27 AM, Konstantin Belousov wrote:

On Sun, Jun 08, 2014 at 05:38:49PM +, Bjoern A. Zeeb wrote:

On 08 Jun 2014, at 17:29 , Bryan Drewery bdrew...@freebsd.org wrote:


Author: bdrewery
Date: Sun Jun  8 17:29:31 2014
New Revision: 267233
URL: http://svnweb.freebsd.org/changeset/base/267233

Log:
  In preparation for ASLR [1] support add WITH_PIE to support building with 
-fPIE.

  This is currently an opt-in build flag. Once ASLR support is ready and stable
  it should changed to opt-out and be enabled by default along with ASLR.

  Each application Makefile uses opt-out to ensure that ASLR will be enabled by
  default in new directories when the system is compiled with PIE/ASLR. [2]

  Mark known build failures as NO_PIE for now.

No, no, no, no more NOs!

I?ll leave it to others who understand the current build system in days when 
it?s not broken to fix this entire splattering across all these Makefiles;  we 
really need a better way for this.

I have no words to express my dissatisfaction with this commit.
If change to the build of _some_ usermode binaries require patching
of loader', csu and rtld Makefiles, obviously it is done wrong.

Why almost half of the binaries require opt-out ?

PLEASE REVERT THIS.
Wait.  Does this not serve as a useful stake in the ground for people to 
come in and update things?  Instead of asking to back out, shouldn't we 
be doing an announcement ok folks, it's now time to fix this! and move 
forward?  Otherwise we may never get any pie.


-Alfred





--
Alfred Perlstein

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r267233 - in head: . bin/rmail gnu/usr.bin/binutils/addr2line gnu/usr.bin/binutils/nm gnu/usr.bin/binutils/objcopy gnu/usr.bin/binutils/objdump gnu/usr.bin/binutils/readelf gnu/usr.bin

[Alfred Perlstein]

On 6/8/14 11:44 AM, Konstantin Belousov wrote:

On Sun, Jun 08, 2014 at 11:30:42AM -0700, Alfred Perlstein wrote:

On 6/8/14 11:27 AM, Konstantin Belousov wrote:

On Sun, Jun 08, 2014 at 05:38:49PM +, Bjoern A. Zeeb wrote:

On 08 Jun 2014, at 17:29 , Bryan Drewery bdrew...@freebsd.org wrote:


Author: bdrewery
Date: Sun Jun  8 17:29:31 2014
New Revision: 267233
URL: http://svnweb.freebsd.org/changeset/base/267233

Log:
   In preparation for ASLR [1] support add WITH_PIE to support building with 
-fPIE.

   This is currently an opt-in build flag. Once ASLR support is ready and stable
   it should changed to opt-out and be enabled by default along with ASLR.

   Each application Makefile uses opt-out to ensure that ASLR will be enabled by
   default in new directories when the system is compiled with PIE/ASLR. [2]

   Mark known build failures as NO_PIE for now.

No, no, no, no more NOs!

I?ll leave it to others who understand the current build system in days when 
it?s not broken to fix this entire splattering across all these Makefiles;  we 
really need a better way for this.

I have no words to express my dissatisfaction with this commit.
If change to the build of _some_ usermode binaries require patching
of loader', csu and rtld Makefiles, obviously it is done wrong.

Why almost half of the binaries require opt-out ?

PLEASE REVERT THIS.

Wait.  Does this not serve as a useful stake in the ground for people to
come in and update things?  Instead of asking to back out, shouldn't we
be doing an announcement ok folks, it's now time to fix this! and move
forward?  Otherwise we may never get any pie.

Let me reformulate.

Somebody commits broken change, despite it was pointed out by many
before the commit. From the changes it is obvious that people which
proposed it do not understand what they hack on. And then, somebody else
must run and 'fix' previously non-broken code.

Sure, you get the pie.

Sure, but hasn't the default stayed unchanged?

It seems like you have to enable ASLR first before you see all the 
breakage.  Right now it seems like goal was to document what even 
compiles versus doesn't compile with ASLR.  Afaik there is not setting 
of ASLR on by default.


There has to be a way to call out what works and what doesn't work and 
form a transition from a world with no ASLR to one with some ASLR and 
eventually one with almost entirely ASLR coverage.  I'm not sure it can 
be done in one fell swoop.  Hooks like this in -current allow for this 
to be done as a group effort.


It would be very unlikely that we retain the semantics all the way until 
a -stable release.


-Alfred

--
Alfred Perlstein

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r267233 - in head: . bin/rmail gnu/usr.bin/binutils/addr2line gnu/usr.bin/binutils/nm gnu/usr.bin/binutils/objcopy gnu/usr.bin/binutils/objdump gnu/usr.bin/binutils/readelf gnu/usr.bin

[Alfred Perlstein]

On 6/8/14 1:13 PM, Pedro Giffuni wrote:

Hello;

El 6/8/2014 2:14 PM, Alfred Perlstein escribió:


There has to be a way to call out what works and what doesn't work and
form a transition from a world with no ASLR to one with some ASLR and
eventually one with almost entirely ASLR coverage.  I'm not sure it can
be done in one fell swoop.  Hooks like this in -current allow for this
to be done as a group effort.

It would be very unlikely that we retain the semantics all the way until
a -stable release.



I am not (yet) criticizing the patches to the build system as I want 
to preserve my innocence ;) ... but perhaps if the semantics are not 
finalized this should be done in a branch. It is my opinion that in 
general we are not using SVN branches as much as we should.


IMO branching is great for something that causes instability, known 
performance issues or won't build.  This is not the same as changes 
build system.


Putting things like this on branches is likely a good way to imo kill 
discussion.


Right now we have discussion, it's rather healthy.  Let's take a while 
to think about this before saying this all should be done in a branch.


-Alfred

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r260898 - head/sys/kern

[Alfred Perlstein]

On 1/22/14, 12:27 PM, John Baldwin wrote:

On Wednesday, January 22, 2014 2:06:39 pm Alfred Perlstein wrote:

Hmm, what if locks had a pointer to a 2 element char * array, the first
being the name, the second the type.  That would keep the size of the
lock down and most locks could share a common tuple of name/type in each
subsystem.  This would allow us to get rid of the pending static list.

effectively:
struct lock_object {
  char *lo_name;  /* Individual lock name. */
  u_int   lo_flags;
  u_int   lo_data;/* General class specific data. */
  struct  witness *lo_witness;/* Data for witness. */
};

would change to:
struct lock_object {
  char **lo_name_type;  /* Individual lock
name[0]/type[1]. */
  u_int   lo_flags;
  u_int   lo_data;/* General class specific data. */
  struct  witness *lo_witness;/* Data for witness. */
};

This may be somewhat disruptive, I haven't played with how it would
actually change driver/etc/code.

Where would the memory for the char* array come from?

That is a good question.  I suspect it would be up to the subsystem to 
allocate it.


Wouldn't it be trivial for *most* of the subsystems to simply have this 
either as a static global or static function variable:


static char *mutex_typename = { kqueue, foo };

Under kern I see this:
grep mtx_init * | grep -v NULL
...
kern_rmlock.c:mtx_init(rm-rm_lock_mtx, name, rmlock_mtx, 
MTX_NOWITNESS);

subr_bus.c:mtx_init(devsoftc.mtx, dev mtx, devd, MTX_DEF);

Those are solved with statics.

Another example:

sys/dev/ae/if_ae.c
mtx_init(sc-mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK, 
MTX_DEF);


I think the array could be in the softc here? sc-mutex_name_type[0] = 
device_get_nameunit(dev); sc-mutex_name_type[1] = MTX_NETWORK_LOCK;


Do we want to do that?  It moves wasting space to another variable.

I'm not sure where there isn't the possibility of using either static 
(for a global mutex) or space inside the equiv of the softc (or proc or 
whatever) for this?


I'm not sure this is a good idea, just an idea.  Are there places where 
it's not as simple as doing this?


-Alfred

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r260898 - head/sys/kern

[Alfred Perlstein]

On 1/22/14, 1:22 PM, John Baldwin wrote:

On Wednesday, January 22, 2014 3:59:37 pm Alfred Perlstein wrote:

On 1/22/14, 12:27 PM, John Baldwin wrote:

On Wednesday, January 22, 2014 2:06:39 pm Alfred Perlstein wrote:

Hmm, what if locks had a pointer to a 2 element char * array, the first
being the name, the second the type.  That would keep the size of the
lock down and most locks could share a common tuple of name/type in each
subsystem.  This would allow us to get rid of the pending static list.

effectively:
struct lock_object {
   char *lo_name;  /* Individual lock name. */
   u_int   lo_flags;
   u_int   lo_data;/* General class specific data.

*/

   struct  witness *lo_witness;/* Data for witness. */
};

would change to:
struct lock_object {
   char **lo_name_type;  /* Individual lock
name[0]/type[1]. */
   u_int   lo_flags;
   u_int   lo_data;/* General class specific data.

*/

   struct  witness *lo_witness;/* Data for witness. */
};

This may be somewhat disruptive, I haven't played with how it would
actually change driver/etc/code.

Where would the memory for the char* array come from?


That is a good question.  I suspect it would be up to the subsystem to
allocate it.

Wouldn't it be trivial for *most* of the subsystems to simply have this
either as a static global or static function variable:

static char *mutex_typename = { kqueue, foo };

Under kern I see this:
grep mtx_init * | grep -v NULL
...
kern_rmlock.c:mtx_init(rm-rm_lock_mtx, name, rmlock_mtx,
MTX_NOWITNESS);
subr_bus.c:mtx_init(devsoftc.mtx, dev mtx, devd, MTX_DEF);

Those are solved with statics.

Another example:

sys/dev/ae/if_ae.c
  mtx_init(sc-mtx, device_get_nameunit(dev), MTX_NETWORK_LOCK,
MTX_DEF);

I think the array could be in the softc here? sc-mutex_name_type[0] =
device_get_nameunit(dev); sc-mutex_name_type[1] = MTX_NETWORK_LOCK;

Do we want to do that?  It moves wasting space to another variable.

I'm not sure where there isn't the possibility of using either static
(for a global mutex) or space inside the equiv of the softc (or proc or
whatever) for this?

I'm not sure this is a good idea, just an idea.  Are there places where
it's not as simple as doing this?

To be honest, the whole name vs type thing isn't widely used, and it makes
the mtx_init() function kind of fugly.  I think what I would actually prefer
is to just kill it, changing the various places that pass a separate name to
just pass the type instead.  Note that none of the other lock APIs even allow
setting a separate type.  This would let us remove the static pending list
array as well.

(And yes, I added the name vs type thing, but at this point I think it did
not turn out nearly as useful as I had thought it would be.)

The original issue of picking useful-to-witness lock names (i.e. not just
using device_get_nameunit()) still remains of course.

I really want to agree, but anything that reduces the immediate ability 
for people to diagnose problems really makes me worry.


This would mean that you would see network device lock or some type 
but not know the actual owner.


I would say that maybe given this it's just better to grow 
WITNESS_PENDING based on maxcpu like the PR I pointed out, that way we 
do not introduce churn AND we maintain the debug-ability.


http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/185831

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r260898 - head/sys/kern

[Alfred Perlstein]

On 1/22/14, 8:34 PM, Rui Paulo wrote:

On 22 Jan 2014, at 20:05, Adrian Chadd adrian.ch...@gmail.com wrote:


.. Make it be an offset into the table rather than a pointer, then we can do 
dirty rcu style hacks to just replace and grow the table as we need more memory.

Don't we have a standard way to pull memory from the top of the physmem area 
early on for allocations like this?

Perhaps a bit overkill for this problem?


Probably.. I keep thinking we should just increase the size by 2x but 
allow platforms to override. for SMALL.


-Alfred

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r255219 - in head: contrib/tcpdump lib/libc lib/libc/capability lib/libc/include lib/libc/sys lib/libprocstat sbin/dhclient sbin/hastd sys/amd64/linux32 sys/bsm sys/cddl/compat/opensol

[Alfred Perlstein]

On 1/2/14 1:33 AM, Pawel Jakub Dawidek wrote:

On Wed, Jan 01, 2014 at 11:16:22PM -0800, Stanislav Sedov wrote:

On Sep 4, 2013, at 5:09 PM, Pawel Jakub Dawidek p...@freebsd.org wrote:


  This commit also breaks compatibility with some existing Capsicum system 
calls,
  but I see no other way to do that. This should be fine as Capsicum is still
  experimental and this change is not going to 9.x.

Hi!

This change also increases the size of kinfo_file structure, which won’t allow
programs not compiled against HEAD and working with kern.info.filedesc sysctl
to run properly on HEAD (e.g. 8.x, 9.x and 10.x jails won’t run properly on 
HEAD,
and it also broke valgrind).  Is there absolutely no way to avoid extending the 
size
of this struct?

Well, I made this change to have space for future cap_rights_t
expension. I did that change for a major branch, so we don't have to do
it in the middle of 10.x or to not block the work until 11.0.

Note that the structure changed size not only because of _kf_cap_spare[3]
field, but also because cap_rights_t is not uint64_t anymore, it is now
struct that contains two uint64_t (1424 - 1392 = 4 * 8).

I'm afraid it is too late to change it for 10.0 at this point anyway.
Not sure if you are aware this was merged to 10, because you write about
10.x jails not working properly on HEAD. 10.x jails will work properly
on HEAD.

BTW. I'd love if we stop using such structures for a running kernel.
We should really move to using libnv to export data like that.


Aren't there enough bits in int _kf_ispare[4];  /* Space 
for more stuff. */
to make this work for the time being until you can provide an alternate 
way to fetch the cap stuff from the kernel.


Afaik you could just remove the spare and steal 2 or 4 entries from 
_kf_ispare until it is sorted.


Can you please make use of that and discuss merge to 10 with re@?

It really sounds like breaking top/etc under jails is something that 
should and can be avoided.


Thank you,
-Alfred




  #if defined(__amd64__) || defined(__i386__)
-#defineKINFO_FILE_SIZE 1392
+#defineKINFO_FILE_SIZE 1424
  #endif
  
  struct kinfo_file {

@@ -389,6 +390,7 @@
 uint16_tkf_pad1;/* Round to 32 bit alignment. 
*/
 int _kf_ispare0;/* Space for more stuff. */
 cap_rights_tkf_cap_rights;  /* Capability rights. */
+   uint64_t_kf_cap_spare[3];   /* Space for future 
cap_rights_t. */
 int _kf_ispare[4];  /* Space for more stuff. */
 /* Truncated before copyout in sysctl */
 charkf_path[PATH_MAX];  /* Path to file, if any. */



--
Alfred Perlstein

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r255219 - in head: contrib/tcpdump lib/libc lib/libc/capability lib/libc/include lib/libc/sys lib/libprocstat sbin/dhclient sbin/hastd sys/amd64/linux32 sys/bsm sys/cddl/compat/opensol

[Alfred Perlstein]

On 1/2/14 2:49 AM, Pawel Jakub Dawidek wrote:

On Thu, Jan 02, 2014 at 02:28:57AM -0800, Alfred Perlstein wrote:

On 1/2/14 1:33 AM, Pawel Jakub Dawidek wrote:

On Wed, Jan 01, 2014 at 11:16:22PM -0800, Stanislav Sedov wrote:

On Sep 4, 2013, at 5:09 PM, Pawel Jakub Dawidek p...@freebsd.org wrote:


   This commit also breaks compatibility with some existing Capsicum system 
calls,
   but I see no other way to do that. This should be fine as Capsicum is still
   experimental and this change is not going to 9.x.

Hi!

This change also increases the size of kinfo_file structure, which won’t allow
programs not compiled against HEAD and working with kern.info.filedesc sysctl
to run properly on HEAD (e.g. 8.x, 9.x and 10.x jails won’t run properly on 
HEAD,
and it also broke valgrind).  Is there absolutely no way to avoid extending the 
size
of this struct?

Well, I made this change to have space for future cap_rights_t
expension. I did that change for a major branch, so we don't have to do
it in the middle of 10.x or to not block the work until 11.0.

Note that the structure changed size not only because of _kf_cap_spare[3]
field, but also because cap_rights_t is not uint64_t anymore, it is now
struct that contains two uint64_t (1424 - 1392 = 4 * 8).

I'm afraid it is too late to change it for 10.0 at this point anyway.
Not sure if you are aware this was merged to 10, because you write about
10.x jails not working properly on HEAD. 10.x jails will work properly
on HEAD.

BTW. I'd love if we stop using such structures for a running kernel.
We should really move to using libnv to export data like that.

Aren't there enough bits in int _kf_ispare[4];  /* Space
for more stuff. */
to make this work for the time being until you can provide an alternate
way to fetch the cap stuff from the kernel.

I don't plan to provide alternative way to fetch the cap stuff. Well, I
implemented libnv, which can be used to reimplement how we fetch all
data like kinfo_file in a ABI friendly way, but I don't plan to modify
this specific code myself.


Afaik you could just remove the spare and steal 2 or 4 entries from
_kf_ispare until it is sorted.

Yes, this would work for current cap_rights_t structure, at least for
i386 and amd64, but would only allow to expand the structure by one
uint64_t in the future (which might or might not be enough). The
cap_rights_t structure is designed to be expanded to 5 uint64_ts without
breaking ABI. I don't want to stuck with current cap_rights_t that is
designed to expand, but cannot be, because kinfo_file wasn't modified at
the start of a major branch.


Can you please make use of that and discuss merge to 10 with re@?

I'm Bccing re@, but I'm pretty sure it is too late for such a change,
especially that it breaks ABI with all 10-RCs. I'm also not changing my
mind. I'd like to structure to stay as-is.


It really sounds like breaking top/etc under jails is something that
should and can be avoided.

I agree. Maybe it should be done every 10 major releases (I'm still fine
with that rule), but we cannot just stuck with it forever.

My suggestions would be:
1. Move to libnv.
2. Detect that the given binary was compiled against some older version
of this structure and copy old structure to userland. Not sure if we
can do that now or not, but I'd expect we can detect that.


Well I agree strongly with what you are doing except the part where 9.x 
jails and earlier are broken because of this change.


It seems like there is a way out and you agree.  Perhaps since the cap 
fits in the spare area we can make do for the time being?


The way to do a major re-org of the kinfo_file/proc/whatever is to 
either use libnv as you suggest, check the binary version (as you 
suggest) or to make an entirely new one and make the old one 
kinfo_file_old and make a new way to fetch the new data as we did with 
the various syscalls ostatfs, ostat, etc.


It still seems like we have a way out that would even give capabilities 
another version (there are enough cells in _kf_ispare for the next 
version of the capabilities code.


-Alfred








   #if defined(__amd64__) || defined(__i386__)
-#defineKINFO_FILE_SIZE 1392
+#defineKINFO_FILE_SIZE 1424
   #endif
   
   struct kinfo_file {

@@ -389,6 +390,7 @@
  uint16_tkf_pad1;/* Round to 32 bit alignment. 
*/
  int _kf_ispare0;/* Space for more stuff. */
  cap_rights_tkf_cap_rights;  /* Capability rights. */
+   uint64_t_kf_cap_spare[3];   /* Space for future 
cap_rights_t. */
  int _kf_ispare[4];  /* Space for more stuff. */
  /* Truncated before copyout in sysctl */
  charkf_path[PATH_MAX];  /* Path to file, if any. */



--
Alfred Perlstein

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src

Re: svn commit: r255219 - in head: contrib/tcpdump lib/libc lib/libc/capability lib/libc/include lib/libc/sys lib/libprocstat sbin/dhclient sbin/hastd sys/amd64/linux32 sys/bsm sys/cddl/compat/opensol

[Alfred Perlstein]

On 1/2/14, 11:14 AM, John-Mark Gurney wrote:

Konstantin Belousov wrote this message on Thu, Jan 02, 2014 at 15:13 +0200:

Afaik you could just remove the spare and steal 2 or 4 entries from
_kf_ispare until it is sorted.

Yes, this would work for current cap_rights_t structure, at least for
i386 and amd64, but would only allow to expand the structure by one
uint64_t in the future (which might or might not be enough). The
cap_rights_t structure is designed to be expanded to 5 uint64_ts without
breaking ABI. I don't want to stuck with current cap_rights_t that is
designed to expand, but cannot be, because kinfo_file wasn't modified at
the start of a major branch.

The ABI stability is not limited to the single branch.  It must be
preserved across whole project lifetime.

Umm. when did this policy change happen?  I thought ABI compatibility
was limited to major releases of FreeBSD?  How are you suppose to do
any work if you can't break ABI ever?

I did a quick search for freebsd policy abi breakage and found some
mailing list posts about this, but no authoritative statement...

Of course the problem is that when we move to
(ASN.1/libnv/ctf/YAML/JSON/XML/etc) we will break ABI compatibility too,
or introduce tons of compatibility code that will rot...

I agree, however there is a very easy way to fix it for the time being.  
Let's not be binary about it well it's going to have to break, so let's 
break it! when such an easy way to not break it exists.  It should be 
let's see if there's a non-intrusive way of not breaking it and the 
answer to that seems to be yes.


-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r255219 - in head: contrib/tcpdump lib/libc lib/libc/capability lib/libc/include lib/libc/sys lib/libprocstat sbin/dhclient sbin/hastd sys/amd64/linux32 sys/bsm sys/cddl/compat/opensol

[Alfred Perlstein]

On 1/2/14, 11:34 AM, Konstantin Belousov wrote:

On Thu, Jan 02, 2014 at 11:23:55AM -0800, Alfred Perlstein wrote:

On 1/2/14, 11:14 AM, John-Mark Gurney wrote:

Konstantin Belousov wrote this message on Thu, Jan 02, 2014 at 15:13 +0200:

Afaik you could just remove the spare and steal 2 or 4 entries from
_kf_ispare until it is sorted.

Yes, this would work for current cap_rights_t structure, at least for
i386 and amd64, but would only allow to expand the structure by one
uint64_t in the future (which might or might not be enough). The
cap_rights_t structure is designed to be expanded to 5 uint64_ts without
breaking ABI. I don't want to stuck with current cap_rights_t that is
designed to expand, but cannot be, because kinfo_file wasn't modified at
the start of a major branch.

The ABI stability is not limited to the single branch.  It must be
preserved across whole project lifetime.

Umm. when did this policy change happen?  I thought ABI compatibility
was limited to major releases of FreeBSD?  How are you suppose to do
any work if you can't break ABI ever?

I did a quick search for freebsd policy abi breakage and found some
mailing list posts about this, but no authoritative statement...

Of course the problem is that when we move to
(ASN.1/libnv/ctf/YAML/JSON/XML/etc) we will break ABI compatibility too,
or introduce tons of compatibility code that will rot...


I agree, however there is a very easy way to fix it for the time being.
Let's not be binary about it well it's going to have to break, so let's
break it! when such an easy way to not break it exists.  It should be
let's see if there's a non-intrusive way of not breaking it and the
answer to that seems to be yes.

If parts of ABI is broken, then why spend efforts trying to keep other
parts stable ?  You already have random set of binaries broken, sometimes
in subtle way.  Then, making other interfaces stable is just a waste.

ABI stability is a yes/no proposition, you cannot have it partly done.
Personally, I do not want to spend a time on hobbyist system.

BTW, to point out obvious thing, Linux has almost perfect ABI stability
and forward compatibility.  It is pity to see that our people do not
understand the importance and benefits of it.

I agree strongly.

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r259411 - head/sys/ofed/drivers/net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Sun Dec 15 07:07:13 2013
New Revision: 259411
URL: http://svnweb.freebsd.org/changeset/base/259411

Log:
  Defer start/stop port to workqueues.
  
  We need to do this because the Linux compat layer uses sx(9) for
  mutex, however the lagg code uses rmlocks and calls into the mellanox
  driver.  This causes deadlock due to sleeping while holding a rmlock.
  
  Submitted by: Shahar Klein (shahark mellanox.com)
  MFC After: 3 days.

Modified:
  head/sys/ofed/drivers/net/mlx4/en_netdev.c
  head/sys/ofed/drivers/net/mlx4/mlx4_en.h

Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
==
--- head/sys/ofed/drivers/net/mlx4/en_netdev.c  Sun Dec 15 07:04:59 2013
(r259410)
+++ head/sys/ofed/drivers/net/mlx4/en_netdev.c  Sun Dec 15 07:07:13 2013
(r259411)
@@ -153,6 +153,19 @@ restart:
return (i);
 }
 
+static void mlx4_en_stop_port(struct net_device *dev)
+{
+   struct mlx4_en_priv *priv = netdev_priv(dev);
+ 
+   queue_work(priv-mdev-workqueue, priv-stop_port_task);
+}
+
+static void mlx4_en_start_port(struct net_device *dev)
+{
+   struct mlx4_en_priv *priv = netdev_priv(dev);
+
+   queue_work(priv-mdev-workqueue, priv-start_port_task);
+}
 
 static void mlx4_en_set_multicast(struct net_device *dev)
 {
@@ -473,6 +486,7 @@ static void mlx4_en_do_get_stats(struct 
 
queue_delayed_work(mdev-workqueue, priv-stats_task, 
STATS_DELAY);
}
+   mlx4_en_QUERY_PORT(priv-mdev, priv-port);
mutex_unlock(mdev-state_lock);
 }
 
@@ -498,8 +512,31 @@ static void mlx4_en_linkstate(struct wor
mutex_unlock(mdev-state_lock);
 }
 
+static void mlx4_en_lock_and_stop_port(struct work_struct *work)
+{
+   struct mlx4_en_priv *priv = container_of(work, struct mlx4_en_priv,
+stop_port_task);
+   struct net_device *dev = priv-dev;
+   struct mlx4_en_dev *mdev = priv-mdev;
+ 
+   mutex_lock(mdev-state_lock);
+   mlx4_en_do_stop_port(dev);
+   mutex_unlock(mdev-state_lock);
+}
+
+static void mlx4_en_lock_and_start_port(struct work_struct *work)
+{
+   struct mlx4_en_priv *priv = container_of(work, struct mlx4_en_priv,
+start_port_task);
+   struct net_device *dev = priv-dev;
+   struct mlx4_en_dev *mdev = priv-mdev;
+
+   mutex_lock(mdev-state_lock);
+   mlx4_en_do_start_port(dev);
+   mutex_unlock(mdev-state_lock);
+}
 
-int mlx4_en_start_port(struct net_device *dev)
+int mlx4_en_do_start_port(struct net_device *dev)
 {
struct mlx4_en_priv *priv = netdev_priv(dev);
struct mlx4_en_dev *mdev = priv-mdev;
@@ -691,7 +728,7 @@ cq_err:
 }
 
 
-void mlx4_en_stop_port(struct net_device *dev)
+void mlx4_en_do_stop_port(struct net_device *dev)
 {
struct mlx4_en_priv *priv = netdev_priv(dev);
struct mlx4_en_dev *mdev = priv-mdev;
@@ -761,8 +798,8 @@ reset:
 
mutex_lock(mdev-state_lock);
if (priv-port_up) {
-   mlx4_en_stop_port(dev);
-   if (mlx4_en_start_port(dev))
+   mlx4_en_do_stop_port(dev);
+   if (mlx4_en_do_start_port(dev))
en_err(priv, Failed restarting port %d\n, priv-port);
}
mutex_unlock(mdev-state_lock);
@@ -793,7 +830,7 @@ mlx4_en_init_locked(struct mlx4_en_priv 
dev = priv-dev;
mdev = priv-mdev;
if (dev-if_drv_flags  IFF_DRV_RUNNING)
-   mlx4_en_stop_port(dev);
+   mlx4_en_do_stop_port(dev);
 
if (!mdev-device_up) {
en_err(priv, Cannot open - device down/disabled\n);
@@ -816,7 +853,7 @@ mlx4_en_init_locked(struct mlx4_en_priv 
}
 
mlx4_en_set_default_moderation(priv);
-   if (mlx4_en_start_port(dev))
+   if (mlx4_en_do_start_port(dev))
en_err(priv, Failed starting port:%d\n, priv-port);
 }
 
@@ -905,7 +942,7 @@ void mlx4_en_destroy_netdev(struct net_d
mlx4_free_hwq_res(mdev-dev, priv-res, MLX4_EN_PAGE_SIZE);
 
mutex_lock(mdev-state_lock);
-   mlx4_en_stop_port(dev);
+   mlx4_en_do_stop_port(dev);
mutex_unlock(mdev-state_lock);
 
cancel_delayed_work(priv-stats_task);
@@ -925,7 +962,6 @@ void mlx4_en_destroy_netdev(struct net_d
 
mtx_destroy(priv-stats_lock.m);
mtx_destroy(priv-vlan_lock.m);
-   mtx_destroy(priv-ioctl_lock.m);
kfree(priv);
if_free(dev);
 }
@@ -951,9 +987,9 @@ static int mlx4_en_change_mtu(struct net
 * the port */
en_dbg(DRV, priv, Change MTU called with card 
down!?\n);
} else {
-   mlx4_en_stop_port(dev);
+   mlx4_en_do_stop_port(dev);
mlx4_en_set_default_moderation(priv);
-   err = mlx4_en_start_port(dev);
+   

svn commit: r259114 - head/sys/modules/crypto

[Alfred Perlstein]

Author: alfred
Date: Mon Dec  9 02:06:52 2013
New Revision: 259114
URL: http://svnweb.freebsd.org/changeset/base/259114

Log:
  Chase down cryptodeflate.c change from r259109.

Modified:
  head/sys/modules/crypto/Makefile

Modified: head/sys/modules/crypto/Makefile
==
--- head/sys/modules/crypto/MakefileMon Dec  9 01:30:20 2013
(r259113)
+++ head/sys/modules/crypto/MakefileMon Dec  9 02:06:52 2013
(r259114)
@@ -11,7 +11,7 @@
 KMOD   = crypto
 SRCS   = crypto.c cryptodev_if.c
 SRCS   += criov.c cryptosoft.c xform.c
-SRCS   += cast.c deflate.c rmd160.c rijndael-alg-fst.c rijndael-api.c
+SRCS   += cast.c cryptodeflate.c rmd160.c rijndael-alg-fst.c rijndael-api.c
 SRCS   += skipjack.c bf_enc.c bf_ecb.c bf_skey.c
 SRCS   += des_ecb.c des_enc.c des_setkey.c
 SRCS   += sha1.c sha2.c
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r258276 - head/sys/ofed/drivers/net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Sun Nov 17 20:58:31 2013
New Revision: 258276
URL: http://svnweb.freebsd.org/changeset/base/258276

Log:
  Fix creating a vlan over lagg over mlxen crash.
  
  PR:   181931
  Submitted by: Shahar Klein (shahark mellanox.com)

Modified:
  head/sys/ofed/drivers/net/mlx4/en_netdev.c

Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
==
--- head/sys/ofed/drivers/net/mlx4/en_netdev.c  Sun Nov 17 20:29:33 2013
(r258275)
+++ head/sys/ofed/drivers/net/mlx4/en_netdev.c  Sun Nov 17 20:58:31 2013
(r258276)
@@ -52,6 +52,9 @@ static void mlx4_en_vlan_rx_add_vid(void
int idx;
u8 field;
 
+   if (arg != priv)
+   return;
+
if ((vid == 0) || (vid  4095))/* Invalid */
return;
en_dbg(HW, priv, adding VLAN:%d\n, vid);
@@ -73,6 +76,9 @@ static void mlx4_en_vlan_rx_kill_vid(voi
int idx;
u8 field;
 
+   if (arg != priv)
+   return;
+
if ((vid == 0) || (vid  4095))/* Invalid */
return;
en_dbg(HW, priv, Killing VID:%d\n, vid);
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r257862 - head/sys/ofed/include/linux

[Alfred Perlstein]

Author: alfred
Date: Fri Nov  8 18:20:19 2013
New Revision: 257862
URL: http://svnweb.freebsd.org/changeset/base/257862

Log:
  Use explicit long cast to avoid overflow in bitopts.
  
  This was causing problems with the buddy allocator inside of
  ofed.
  
  Submitted by: odeds

Modified:
  head/sys/ofed/include/linux/bitops.h

Modified: head/sys/ofed/include/linux/bitops.h
==
--- head/sys/ofed/include/linux/bitops.hFri Nov  8 17:27:38 2013
(r257861)
+++ head/sys/ofed/include/linux/bitops.hFri Nov  8 18:20:19 2013
(r257862)
@@ -286,14 +286,14 @@ bitmap_empty(unsigned long *addr, int si
 #defineNBLONG  (NBBY * sizeof(long))
 
 #defineset_bit(i, a)   
\
-atomic_set_long(((volatile long *)(a))[(i)/NBLONG], 1  (i) % NBLONG)
+atomic_set_long(((volatile long *)(a))[(i)/NBLONG], 1UL  (i) % NBLONG)
 
 #defineclear_bit(i, a) 
\
-atomic_clear_long(((volatile long *)(a))[(i)/NBLONG], 1  (i) % NBLONG)
+atomic_clear_long(((volatile long *)(a))[(i)/NBLONG], 1UL  (i) % NBLONG)
 
 #definetest_bit(i, a)  
\
 !!(atomic_load_acq_long(((volatile long *)(a))[(i)/NBLONG])  \
-1  ((i) % NBLONG))
+1UL  ((i) % NBLONG))
 
 static inline long
 test_and_clear_bit(long bit, long *var)
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r257863 - head/sys/ofed/drivers/net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Fri Nov  8 18:26:28 2013
New Revision: 257863
URL: http://svnweb.freebsd.org/changeset/base/257863

Log:
  Fix for bad performance when mtu is increased.
  
  Update the auto moderation behavior in the mlxen driver to match
  the new LINUX OFED code.
  
  Submitted by: odeds

Modified:
  head/sys/ofed/drivers/net/mlx4/en_ethtool.c
  head/sys/ofed/drivers/net/mlx4/en_netdev.c
  head/sys/ofed/drivers/net/mlx4/mlx4_en.h

Modified: head/sys/ofed/drivers/net/mlx4/en_ethtool.c
==
--- head/sys/ofed/drivers/net/mlx4/en_ethtool.c Fri Nov  8 18:20:19 2013
(r257862)
+++ head/sys/ofed/drivers/net/mlx4/en_ethtool.c Fri Nov  8 18:26:28 2013
(r257863)
@@ -366,13 +366,13 @@ static int mlx4_en_set_coalesce(struct n
priv-rx_usecs_high = coal-rx_coalesce_usecs_high;
priv-sample_interval = coal-rate_sample_interval;
priv-adaptive_rx_coal = coal-use_adaptive_rx_coalesce;
-   priv-last_moder_time = MLX4_EN_AUTO_CONF;
if (priv-adaptive_rx_coal)
return 0;
 
for (i = 0; i  priv-rx_ring_num; i++) {
priv-rx_cq[i].moder_cnt = priv-rx_frames;
priv-rx_cq[i].moder_time = priv-rx_usecs;
+   priv-last_moder_time[i] = MLX4_EN_AUTO_CONF;
err = mlx4_en_set_cq_moder(priv, priv-rx_cq[i]);
if (err)
return err;
@@ -418,6 +418,7 @@ static int mlx4_en_set_ringparam(struct 
u32 rx_size, tx_size;
int port_up = 0;
int err = 0;
+   int i;
 
if (param-rx_jumbo_pending || param-rx_mini_pending)
return -EINVAL;
@@ -456,6 +457,15 @@ static int mlx4_en_set_ringparam(struct 
en_err(priv, Failed starting port\n);
}
 
+   for (i = 0; i  priv-rx_ring_num; i++) {
+   priv-rx_cq[i].moder_cnt = priv-rx_frames;
+   priv-rx_cq[i].moder_time = priv-rx_usecs;
+   priv-last_moder_time[i] = MLX4_EN_AUTO_CONF;
+   err = mlx4_en_set_cq_moder(priv, priv-rx_cq[i]);
+   if (err)
+   goto out;
+   }
+
 out:
mutex_unlock(mdev-state_lock);
return err;

Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
==
--- head/sys/ofed/drivers/net/mlx4/en_netdev.c  Fri Nov  8 18:20:19 2013
(r257862)
+++ head/sys/ofed/drivers/net/mlx4/en_netdev.c  Fri Nov  8 18:26:28 2013
(r257863)
@@ -318,6 +318,9 @@ static void mlx4_en_set_default_moderati
cq = priv-rx_cq[i];
cq-moder_cnt = priv-rx_frames;
cq-moder_time = priv-rx_usecs;
+   priv-last_moder_time[i] = MLX4_EN_AUTO_CONF;
+   priv-last_moder_packets[i] = 0;
+   priv-last_moder_bytes[i] = 0;
}
 
for (i = 0; i  priv-tx_ring_num; i++) {
@@ -333,11 +336,8 @@ static void mlx4_en_set_default_moderati
priv-rx_usecs_high = MLX4_EN_RX_COAL_TIME_HIGH;
priv-sample_interval = MLX4_EN_SAMPLE_INTERVAL;
priv-adaptive_rx_coal = 1;
-   priv-last_moder_time = MLX4_EN_AUTO_CONF;
priv-last_moder_jiffies = 0;
-   priv-last_moder_packets = 0;
priv-last_moder_tx_packets = 0;
-   priv-last_moder_bytes = 0;
 }
 
 static void mlx4_en_auto_moderation(struct mlx4_en_priv *priv)
@@ -349,43 +349,29 @@ static void mlx4_en_auto_moderation(stru
unsigned long avg_pkt_size;
unsigned long rx_packets;
unsigned long rx_bytes;
-   unsigned long tx_packets;
-   unsigned long tx_pkt_diff;
unsigned long rx_pkt_diff;
int moder_time;
-   int i, err;
+   int ring, err;
 
if (!priv-adaptive_rx_coal || period  priv-sample_interval * HZ)
return;
-
-   spin_lock(priv-stats_lock);
-   rx_packets = priv-dev-if_ipackets;
-   rx_bytes = priv-dev-if_ibytes;
-   tx_packets = priv-dev-if_opackets;
-   spin_unlock(priv-stats_lock);
-
-   if (!priv-last_moder_jiffies || !period)
-   goto out;
-
-   tx_pkt_diff = ((unsigned long) (tx_packets -
-   priv-last_moder_tx_packets));
-   rx_pkt_diff = ((unsigned long) (rx_packets -
-   priv-last_moder_packets));
-   packets = max(tx_pkt_diff, rx_pkt_diff);
-   rate = packets * HZ / period;
-   avg_pkt_size = packets ? ((unsigned long) (rx_bytes -
-priv-last_moder_bytes)) / packets : 0;
-
-   /* Apply auto-moderation only when packet rate exceeds a rate that
-* it matters */
-   if (rate  MLX4_EN_RX_RATE_THRESH) {
-   /* If tx and rx packet rates are not balanced, assume that
-* traffic is mainly BW bound and apply maximum moderation.
-* Otherwise, moderate according to 

svn commit: r257864 - head/sys/ofed/drivers/net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Fri Nov  8 18:28:48 2013
New Revision: 257864
URL: http://svnweb.freebsd.org/changeset/base/257864

Log:
  Do not use a sleep lock when protecting the driver flags.
  
  This was causing a locking issue with lagg
  
  Submitted by: odeds

Modified:
  head/sys/ofed/drivers/net/mlx4/en_netdev.c
  head/sys/ofed/drivers/net/mlx4/mlx4_en.h

Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
==
--- head/sys/ofed/drivers/net/mlx4/en_netdev.c  Fri Nov  8 18:26:28 2013
(r257863)
+++ head/sys/ofed/drivers/net/mlx4/en_netdev.c  Fri Nov  8 18:28:48 2013
(r257864)
@@ -919,6 +919,7 @@ void mlx4_en_destroy_netdev(struct net_d
 
mtx_destroy(priv-stats_lock.m);
mtx_destroy(priv-vlan_lock.m);
+   mtx_destroy(priv-ioctl_lock.m);
kfree(priv);
if_free(dev);
 }
@@ -1087,9 +1088,9 @@ static int mlx4_en_ioctl(struct ifnet *d
break;
case SIOCADDMULTI:
case SIOCDELMULTI:
-   mutex_lock(mdev-state_lock);
+spin_lock(priv-ioctl_lock);
mlx4_en_set_multicast(dev);
-   mutex_unlock(mdev-state_lock);
+spin_unlock(priv-ioctl_lock);
break;
case SIOCSIFMEDIA:
case SIOCGIFMEDIA:
@@ -1510,6 +1511,7 @@ int mlx4_en_init_netdev(struct mlx4_en_d
priv-msg_enable = MLX4_EN_MSG_LEVEL;
priv-ip_reasm = priv-mdev-profile.ip_reasm;
mtx_init(priv-stats_lock.m, mlx4 stats, NULL, MTX_DEF);
+   mtx_init(priv-ioctl_lock.m, mlx4 ioctl, NULL, MTX_DEF);
mtx_init(priv-vlan_lock.m, mlx4 vlan, NULL, MTX_DEF);
INIT_WORK(priv-mcast_task, mlx4_en_do_set_multicast);
INIT_WORK(priv-watchdog_task, mlx4_en_restart);

Modified: head/sys/ofed/drivers/net/mlx4/mlx4_en.h
==
--- head/sys/ofed/drivers/net/mlx4/mlx4_en.hFri Nov  8 18:26:28 2013
(r257863)
+++ head/sys/ofed/drivers/net/mlx4/mlx4_en.hFri Nov  8 18:28:48 2013
(r257864)
@@ -493,6 +493,7 @@ struct mlx4_en_priv {
spinlock_t vlan_lock;
struct mlx4_en_port_state port_state;
spinlock_t stats_lock;
+   spinlock_t ioctl_lock;
 
unsigned long last_moder_packets[MAX_RX_RINGS];
unsigned long last_moder_tx_packets;
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r257542 - head/sys/ofed/drivers/net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Sat Nov  2 10:49:47 2013
New Revision: 257542
URL: http://svnweb.freebsd.org/changeset/base/257542

Log:
  Fix API mismatch exposed by lagg.
  
  When destroying a lagg the driver tries to restore the old mac and
  fails due to API mismatch

Modified:
  head/sys/ofed/drivers/net/mlx4/en_netdev.c

Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
==
--- head/sys/ofed/drivers/net/mlx4/en_netdev.c  Sat Nov  2 09:16:11 2013
(r257541)
+++ head/sys/ofed/drivers/net/mlx4/en_netdev.c  Sat Nov  2 10:49:47 2013
(r257542)
@@ -633,8 +633,8 @@ int mlx4_en_start_port(struct net_device
en_dbg(DRV, priv, Setting mac for port %d\n, priv-port);
err = mlx4_register_mac(mdev-dev, priv-port,
mlx4_en_mac_to_u64(IF_LLADDR(dev)));
-   if (err) {
-   en_err(priv, Failed setting port mac\n);
+   if (err  0) {
+   en_err(priv, Failed setting port mac err=%d\n, err);
goto tx_err;
}
mdev-mac_removed[priv-port] = 0;
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r257543 - head/sys/dev/usb/wlan

[Alfred Perlstein]

Author: alfred
Date: Sat Nov  2 11:37:16 2013
New Revision: 257543
URL: http://svnweb.freebsd.org/changeset/base/257543

Log:
  Add device ID for 'Sanoxy 802.11N' usb

Modified:
  head/sys/dev/usb/wlan/if_urtwn.c

Modified: head/sys/dev/usb/wlan/if_urtwn.c
==
--- head/sys/dev/usb/wlan/if_urtwn.cSat Nov  2 10:49:47 2013
(r257542)
+++ head/sys/dev/usb/wlan/if_urtwn.cSat Nov  2 11:37:16 2013
(r257543)
@@ -139,6 +139,7 @@ static const STRUCT_USB_HOST_ID urtwn_de
URTWN_DEV(REALTEK,  RTL8191CU),
URTWN_DEV(REALTEK,  RTL8192CE),
URTWN_DEV(REALTEK,  RTL8192CU),
+   URTWN_DEV(REALTEK,  RTL8188CU_0),
URTWN_DEV(SITECOMEU,RTL8188CU_1),
URTWN_DEV(SITECOMEU,RTL8188CU_2),
URTWN_DEV(SITECOMEU,RTL8192CU),
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r256682 - head/sys/ofed/drivers/net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Thu Oct 17 12:19:36 2013
New Revision: 256682
URL: http://svnweb.freebsd.org/changeset/base/256682

Log:
  Fix resource free.
  
  The order of releasing resources in mlxen was wrong, which caused
  panic on reload of the module.
  
  conf_ctx list should be released before stat_ctx list, otherwise
  the leafs in conf_ctx list won't be released because of the dependancy.
  
  The fix is to change the order of the releases.
  
  Submitted by: Shahar Klein (shahark at mellanox.com)

Modified:
  head/sys/ofed/drivers/net/mlx4/en_netdev.c

Modified: head/sys/ofed/drivers/net/mlx4/en_netdev.c
==
--- head/sys/ofed/drivers/net/mlx4/en_netdev.c  Thu Oct 17 12:15:21 2013
(r256681)
+++ head/sys/ofed/drivers/net/mlx4/en_netdev.c  Thu Oct 17 12:19:36 2013
(r256682)
@@ -927,9 +927,6 @@ void mlx4_en_destroy_netdev(struct net_d
if (priv-allocated)
mlx4_free_hwq_res(mdev-dev, priv-res, MLX4_EN_PAGE_SIZE);
 
-   if (priv-sysctl)
-   sysctl_ctx_free(priv-conf_ctx);
-
mutex_lock(mdev-state_lock);
mlx4_en_stop_port(dev);
mutex_unlock(mdev-state_lock);
@@ -946,6 +943,9 @@ void mlx4_en_destroy_netdev(struct net_d
 
mlx4_en_free_resources(priv);
 
+   if (priv-sysctl)
+   sysctl_ctx_free(priv-conf_ctx);
+
mtx_destroy(priv-stats_lock.m);
mtx_destroy(priv-vlan_lock.m);
kfree(priv);
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r256546 - head/sys/ofed/include/linux

[Alfred Perlstein]

Author: alfred
Date: Tue Oct 15 15:50:43 2013
New Revision: 256546
URL: http://svnweb.freebsd.org/changeset/base/256546

Log:
  Fix __free_pages() in the linux shim.
  
  __free_pages() is actaully supposed to take a struct page * not
  an address.

Modified:
  head/sys/ofed/include/linux/gfp.h

Modified: head/sys/ofed/include/linux/gfp.h
==
--- head/sys/ofed/include/linux/gfp.h   Tue Oct 15 15:43:29 2013
(r256545)
+++ head/sys/ofed/include/linux/gfp.h   Tue Oct 15 15:50:43 2013
(r256546)
@@ -92,14 +92,14 @@ __free_page(struct page *m)
 }
 
 static inline void
-__free_pages(void *p, unsigned int order)
+__free_pages(struct page *m, unsigned int order)
 {
size_t size;
 
-   if (p == 0)
+   if (m == NULL)
return;
size = PAGE_SIZE  order;
-   kmem_free(kmem_arena, (vm_offset_t)p, size);
+   kmem_free(kmem_arena, (vm_offset_t)page_address(m), size);
 }
 
 /*
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r256269 - head/sys/ofed/drivers/infiniband/hw/mlx4

[Alfred Perlstein]

Author: alfred
Date: Thu Oct 10 14:03:03 2013
New Revision: 256269
URL: http://svnweb.freebsd.org/changeset/base/256269

Log:
  Fix for When more than one NIC is present.
  
  The device name was incorrect due to a specific function we ported
  from the Linux driver that is not FBSD compatible.  This resulted
  with a false sysctl registration and some more problematic issues.
  
  The patch basically revokes it all together.
  
  Submitted by: Meny Yossefi (menyy mellanox.com)
  
  Approved by:  re

Modified:
  head/sys/ofed/drivers/infiniband/hw/mlx4/main.c

Modified: head/sys/ofed/drivers/infiniband/hw/mlx4/main.c
==
--- head/sys/ofed/drivers/infiniband/hw/mlx4/main.c Thu Oct 10 12:47:34 
2013(r256268)
+++ head/sys/ofed/drivers/infiniband/hw/mlx4/main.c Thu Oct 10 14:03:03 
2013(r256269)
@@ -1859,33 +1859,6 @@ err:
is incorrect. The parameter value is discarded!);
 }
 
-static int mlx4_ib_dev_idx(struct mlx4_dev *dev)
-{
-   int /*bus,*/ slot, fn;
-   int i;
-
-   if (!dev)
-   return -1;
-   else if (!dev-pdev)
-   return -1;
-   //else if (!dev-pdev-bus)
-   //  return -1;
-
-   //bus   = dev-pdev-bus-conf.pc_sel.pc_bus;
-   slot= PCI_SLOT(dev-pdev-devfn);
-   fn  = PCI_FUNC(dev-pdev-devfn);
-
-   for (i = 0; i  MAX_DR; ++i) {
-   if (/*dr[i].bus == bus */
-   dr[i].dev == slot 
-   dr[i].func == fn) {
-   return dr[i].nr;
-   }
-   }
-
-   return -1;
-}
-
 static void *mlx4_ib_add(struct mlx4_dev *dev)
 {
struct mlx4_ib_dev *ibdev;
@@ -1893,7 +1866,6 @@ static void *mlx4_ib_add(struct mlx4_dev
int i, j;
int err;
struct mlx4_ib_iboe *iboe;
-   int dev_idx;
 
printk(KERN_INFO %s, mlx4_ib_version);
 
@@ -1928,12 +1900,7 @@ static void *mlx4_ib_add(struct mlx4_dev
 
ibdev-dev = dev;
 
-   dev_idx = mlx4_ib_dev_idx(dev);
-   if (dev_idx = 0)
-   sprintf(ibdev-ib_dev.name, mlx4_%d, dev_idx);
-   else
-   strlcpy(ibdev-ib_dev.name, mlx4_%d, IB_DEVICE_NAME_MAX);
-
+   strlcpy(ibdev-ib_dev.name, mlx4_%d, IB_DEVICE_NAME_MAX);
ibdev-ib_dev.owner = THIS_MODULE;
ibdev-ib_dev.node_type = RDMA_NODE_IB_CA;
ibdev-ib_dev.local_dma_lkey= dev-caps.reserved_lkey;
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r255968 - head/sys/ofed/include/linux

[Alfred Perlstein]

Author: alfred
Date: Tue Oct  1 15:33:00 2013
New Revision: 255968
URL: http://svnweb.freebsd.org/changeset/base/255968

Log:
  Fix mis-merge of upstream fix.
  
  We would accidentally make the string one byte too short.
  
  Submitted by: Orit Moskovich (oritm mellanox.com)
  
  Approved by:  re

Modified:
  head/sys/ofed/include/linux/sysfs.h

Modified: head/sys/ofed/include/linux/sysfs.h
==
--- head/sys/ofed/include/linux/sysfs.h Tue Oct  1 12:01:20 2013
(r255967)
+++ head/sys/ofed/include/linux/sysfs.h Tue Oct  1 15:33:00 2013
(r255968)
@@ -105,10 +105,6 @@ sysctl_handle_attr(SYSCTL_HANDLER_ARGS)
/* Trim trailing newline. */
buf[len] = '\0';
}
-
-   /* Trim trailing newline. */
-   len--;
-   ((char*)buf)[len] = '\0';
}
 
/* Leave one trailing byte to append a newline. */
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r255970 - head/sys/ofed/drivers/infiniband/hw/mlx4

[Alfred Perlstein]

Author: alfred
Date: Tue Oct  1 15:38:29 2013
New Revision: 255970
URL: http://svnweb.freebsd.org/changeset/base/255970

Log:
  Fixed 'Couldn't Create QP' issue when running rc_pingpong, uc_pingpong,
  srq_pingpong IBverbs
  
  Removed refrences using 'ifdef __linux__' to qpg functions and
  related fields in struct
  ib_qp_init_attr.
  
  Submitted by: Orit Moskovich (oritm mellanox.com)
  
  Approved by:  re

Modified:
  head/sys/ofed/drivers/infiniband/hw/mlx4/qp.c

Modified: head/sys/ofed/drivers/infiniband/hw/mlx4/qp.c
==
--- head/sys/ofed/drivers/infiniband/hw/mlx4/qp.c   Tue Oct  1 15:36:51 
2013(r255969)
+++ head/sys/ofed/drivers/infiniband/hw/mlx4/qp.c   Tue Oct  1 15:38:29 
2013(r255970)
@@ -611,6 +611,7 @@ static int qp_has_rq(struct ib_qp_init_a
return !attr-srq;
 }
 
+#ifdef __linux__
 static int init_qpg_parent(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *pqp,
   struct ib_qp_init_attr *attr, int *qpn)
 {
@@ -791,6 +792,7 @@ static void free_qpg_qpn(struct mlx4_ib_
break;
}
 }
+#endif
 
 static int alloc_qpn_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp,
struct ib_qp_init_attr *attr, int *qpn)
@@ -811,11 +813,15 @@ static int alloc_qpn_common(struct mlx4_
}
break;
case IB_QPG_PARENT:
+#ifdef __linux__
err = init_qpg_parent(dev, qp, attr, qpn);
+#endif
break;
case IB_QPG_CHILD_TX:
case IB_QPG_CHILD_RX:
+#ifdef __linux__
err = alloc_qpg_qpn(attr, qp, qpn);
+#endif
break;
default:
qp-qpg_type = IB_QPG_NONE;
@@ -839,11 +845,15 @@ static void free_qpn_common(struct mlx4_
mlx4_qp_release_range(dev-dev, qpn, 1);
break;
case IB_QPG_PARENT:
+#ifdef __linux__
free_qpg_parent(dev, qp);
+#endif
break;
case IB_QPG_CHILD_TX:
case IB_QPG_CHILD_RX:
+#ifdef __linux__
free_qpg_qpn(qp, qpn);
+#endif
break;
default:
break;
@@ -872,6 +882,10 @@ static int create_qp_common(struct mlx4_
struct mlx4_ib_qp *qp;
enum mlx4_ib_qp_type qp_type = (enum mlx4_ib_qp_type) 
init_attr-qp_type;
 
+#ifndef __linux__
+init_attr-qpg_type = IB_QPG_NONE;
+#endif
+
/* When tunneling special qps, we use a plain UD qp */
if (sqpn) {
if (mlx4_is_mfunc(dev-dev) 
@@ -1287,6 +1301,7 @@ static u32 get_sqp_num(struct mlx4_ib_de
return dev-dev-caps.qp1_proxy[attr-port_num - 1];
 }
 
+#ifdef __linux__
 static int check_qpg_attr(struct mlx4_ib_dev *dev,
  struct ib_qp_init_attr *attr)
 {
@@ -1332,6 +1347,7 @@ static int check_qpg_attr(struct mlx4_ib
}
return 0;
 }
+#endif
 
 #define RESERVED_FLAGS_MASK unsigned int)IB_QP_CREATE_RESERVED_END - 1) | 
IB_QP_CREATE_RESERVED_END)   \
 
~(IB_QP_CREATE_RESERVED_START - 1))
@@ -1390,9 +1406,11 @@ struct ib_qp *mlx4_ib_create_qp(struct i
  init_attr-qp_type  IB_QPT_GSI)))
return ERR_PTR(-EINVAL);
 
+#ifdef __linux__
err = check_qpg_attr(to_mdev(device), init_attr);
if (err)
return ERR_PTR(err);
+#endif
 
switch (init_attr-qp_type) {
case IB_QPT_XRC_TGT:
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r255972 - in head/sys/ofed: drivers/infiniband/core drivers/infiniband/hw/mlx4 include/rdma

[Alfred Perlstein]

Author: alfred
Date: Tue Oct  1 15:42:38 2013
New Revision: 255972
URL: http://svnweb.freebsd.org/changeset/base/255972

Log:
  Enable ib_dev.mmap function
  
  Removed the ifdef linux from this function.
  Added stub function for contiguous pages to avoid compilation
  errors.
  
  Submitted by: Orit Moskovich (oritm mellanox.com)
  Approved by:  re

Modified:
  head/sys/ofed/drivers/infiniband/core/umem.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/main.c
  head/sys/ofed/include/rdma/ib_umem.h

Modified: head/sys/ofed/drivers/infiniband/core/umem.c
==
--- head/sys/ofed/drivers/infiniband/core/umem.cTue Oct  1 15:40:27 
2013(r255971)
+++ head/sys/ofed/drivers/infiniband/core/umem.cTue Oct  1 15:42:38 
2013(r255972)
@@ -530,3 +530,46 @@ int ib_umem_page_count(struct ib_umem *u
return n;
 }
 EXPORT_SYMBOL(ib_umem_page_count);
+
+/**/
+/* 
+ * Stub functions for contiguous pages - 
+ * We currently do not support this feature
+ */
+/**/
+
+/**
+ * ib_cmem_release_contiguous_pages - release memory allocated by
+ *  ib_cmem_alloc_contiguous_pages.
+ * @cmem: cmem struct to release
+ */
+void ib_cmem_release_contiguous_pages(struct ib_cmem *cmem)
+{
+}
+EXPORT_SYMBOL(ib_cmem_release_contiguous_pages);
+
+/**
+ *  * ib_cmem_alloc_contiguous_pages - allocate contiguous pages
+ *  *  @context: userspace context to allocate memory for
+ *   * @total_size: total required size for that allocation.
+ ** @page_size_order: order of one contiguous page.
+ * */
+struct ib_cmem *ib_cmem_alloc_contiguous_pages(struct ib_ucontext *context,
+unsigned long total_size,
+   unsigned long 
page_size_order)
+{
+   return NULL;
+}
+EXPORT_SYMBOL(ib_cmem_alloc_contiguous_pages);
+
+/**
+ *  * ib_cmem_map_contiguous_pages_to_vma - map contiguous pages into VMA
+ *   * @ib_cmem: cmem structure returned by ib_cmem_alloc_contiguous_pages
+ ** @vma: VMA to inject pages into.
+ * */
+int ib_cmem_map_contiguous_pages_to_vma(struct ib_cmem *ib_cmem,
+struct vm_area_struct *vma)
+{
+   return 0;
+}
+EXPORT_SYMBOL(ib_cmem_map_contiguous_pages_to_vma);

Modified: head/sys/ofed/drivers/infiniband/hw/mlx4/main.c
==
--- head/sys/ofed/drivers/infiniband/hw/mlx4/main.c Tue Oct  1 15:40:27 
2013(r255971)
+++ head/sys/ofed/drivers/infiniband/hw/mlx4/main.c Tue Oct  1 15:42:38 
2013(r255972)
@@ -726,6 +726,7 @@ full_search:
addr = ALIGN(vma-vm_end, 1  page_size_order);
}
 }
+#endif
 
 static int mlx4_ib_mmap(struct ib_ucontext *context, struct vm_area_struct 
*vma)
 {
@@ -780,7 +781,6 @@ static int mlx4_ib_mmap(struct ib_uconte
 
return 0;
 }
-#endif
 
 static struct ib_pd *mlx4_ib_alloc_pd(struct ib_device *ibdev,
  struct ib_ucontext *context,
@@ -1984,8 +1984,8 @@ static void *mlx4_ib_add(struct mlx4_dev
ibdev-ib_dev.modify_port   = mlx4_ib_modify_port;
ibdev-ib_dev.alloc_ucontext= mlx4_ib_alloc_ucontext;
ibdev-ib_dev.dealloc_ucontext  = mlx4_ib_dealloc_ucontext;
-#ifdef __linux__
ibdev-ib_dev.mmap  = mlx4_ib_mmap;
+#ifdef __linux__
ibdev-ib_dev.get_unmapped_area = mlx4_ib_get_unmapped_area;
 #endif
ibdev-ib_dev.alloc_pd  = mlx4_ib_alloc_pd;

Modified: head/sys/ofed/include/rdma/ib_umem.h
==
--- head/sys/ofed/include/rdma/ib_umem.hTue Oct  1 15:40:27 2013
(r255971)
+++ head/sys/ofed/include/rdma/ib_umem.hTue Oct  1 15:42:38 2013
(r255972)
@@ -57,6 +57,24 @@ struct ib_umem {
unsigned long   diff;
 };
 
+struct ib_cmem {
+
+struct ib_ucontext *context;
+size_t  length;
+/* Link list of contiguous blocks being part of that cmem  */
+struct list_head ib_cmem_block;
+
+/* Order of cmem block,  2^ block_order will equal number
+ of physical pages per block
+*/
+unsigned longblock_order;
+/* Refernce counter for that memory area
+  - When value became 0 pages will be returned to the kernel.
+*/
+struct kref refcount;
+};
+
+
 struct ib_umem_chunk {
struct list_headlist;
int nents;
@@ -70,4 +88,14 @@ struct ib_umem *ib_umem_get(struct ib_uc
 void ib_umem_release(struct ib_umem *umem);
 int ib_umem_page_count(struct ib_umem *umem);
 
+int ib_cmem_map_contiguous_pages_to_vma(struct ib_cmem *ib_cmem,
+struct 

svn commit: r255973 - head/sys/ofed/drivers/infiniband/core

[Alfred Perlstein]

Author: alfred
Date: Tue Oct  1 15:43:23 2013
New Revision: 255973
URL: http://svnweb.freebsd.org/changeset/base/255973

Log:
  Fixed kernel crash when running devinfo
  
  When calling to ib_uverbs_cleanup_ucontext, there is a call to
  mutex_lock of xrcd_table_mutex, which was not initialized.
  Added missing initialization for xrcd_table_mutex.
  
  Submitted by: Orit Moskovich (oritm mellanox.com)
  
  Approved by:  re

Modified:
  head/sys/ofed/drivers/infiniband/core/device.c

Modified: head/sys/ofed/drivers/infiniband/core/device.c
==
--- head/sys/ofed/drivers/infiniband/core/device.c  Tue Oct  1 15:42:38 
2013(r255972)
+++ head/sys/ofed/drivers/infiniband/core/device.c  Tue Oct  1 15:43:23 
2013(r255973)
@@ -296,6 +296,8 @@ int ib_register_device(struct ib_device 
INIT_LIST_HEAD(device-client_data_list);
spin_lock_init(device-event_handler_lock);
spin_lock_init(device-client_data_lock);
+   device-ib_uverbs_xrcd_table = RB_ROOT;
+   mutex_init(device-xrcd_table_mutex);
 
ret = read_port_table_lengths(device);
if (ret) {
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r255969 - in head/sys/ofed/drivers: infiniband/hw/mlx4 net/mlx4

[Alfred Perlstein]

Author: alfred
Date: Tue Oct  1 15:36:51 2013
New Revision: 255969
URL: http://svnweb.freebsd.org/changeset/base/255969

Log:
  Fixed kernel crash when removing IPOIB_CM option from configuration file
  
  Changed module init from module_init() to module_init_order() with
  SI_ORDER_MIDDLE flag
  Submitted by: Orit Moskovich (oritm mellanox.com)
  Approved by:  re

Modified:
  head/sys/ofed/drivers/infiniband/hw/mlx4/main.c
  head/sys/ofed/drivers/net/mlx4/main.c

Modified: head/sys/ofed/drivers/infiniband/hw/mlx4/main.c
==
--- head/sys/ofed/drivers/infiniband/hw/mlx4/main.c Tue Oct  1 15:33:00 
2013(r255968)
+++ head/sys/ofed/drivers/infiniband/hw/mlx4/main.c Tue Oct  1 15:36:51 
2013(r255969)
@@ -2431,7 +2431,7 @@ static void __exit mlx4_ib_cleanup(void)
 
 }
 
-module_init(mlx4_ib_init);
+module_init_order(mlx4_ib_init, SI_ORDER_MIDDLE);
 module_exit(mlx4_ib_cleanup);
 
 #undef MODULE_VERSION

Modified: head/sys/ofed/drivers/net/mlx4/main.c
==
--- head/sys/ofed/drivers/net/mlx4/main.c   Tue Oct  1 15:33:00 2013
(r255968)
+++ head/sys/ofed/drivers/net/mlx4/main.c   Tue Oct  1 15:36:51 2013
(r255969)
@@ -2859,7 +2859,7 @@ static void __exit mlx4_cleanup(void)
destroy_workqueue(mlx4_wq);
 }
 
-module_init(mlx4_init);
+module_init_order(mlx4_init, SI_ORDER_MIDDLE);
 module_exit(mlx4_cleanup);
 
 #undef MODULE_VERSION
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r255932 - in head/sys: conf contrib/rdma dev/cxgb/ulp/iw_cxgb modules modules/ibcore modules/ipoib modules/mlx4 modules/mlx4ib ofed/drivers/infiniband/core ofed/drivers/infiniband/hw/ml...

[Alfred Perlstein]

Author: alfred
Date: Sun Sep 29 00:35:03 2013
New Revision: 255932
URL: http://svnweb.freebsd.org/changeset/base/255932

Log:
  Update OFED to Linux 3.7 and update Mellanox drivers.
  
  Update the OFED Infiniband core to the version supplied in Linux
  version 3.7.
  
  The update to OFED is nearly all additional defines and functions
  with the exception of the addition of additional parameters to
  ib_register_device() and the reg_user_mr callback.
  
  In addition the ibcore (Infiniband core) and ipoib (IP over Infiniband)
  have both been made into completely loadable modules to facilitate
  testing of the OFED stack in FreeBSD.
  
  Finally the Mellanox Infiniband drivers are now updated to the
  latest version shipping with Linux 3.7.
  
  Submitted by: Mellanox FreeBSD driver team:
  Oded Shanoon (odeds mellanox.com),
  Meny Yossefi (menyy mellanox.com),
  Orit Moskovich (oritm mellanox.com)
  
  Approved by: re

Added:
  head/sys/modules/ibcore/
  head/sys/modules/ibcore/Makefile   (contents, props changed)
  head/sys/modules/ipoib/
  head/sys/modules/ipoib/Makefile   (contents, props changed)
  head/sys/ofed/drivers/infiniband/hw/mlx4/alias_GUID.c   (contents, props 
changed)
  head/sys/ofed/drivers/infiniband/hw/mlx4/cm.c   (contents, props changed)
  head/sys/ofed/drivers/infiniband/hw/mlx4/mcg.c   (contents, props changed)
  head/sys/ofed/drivers/infiniband/hw/mlx4/sysfs.c   (contents, props changed)
  head/sys/ofed/drivers/net/mlx4/resource_tracker.c   (contents, props changed)
  head/sys/ofed/drivers/net/mlx4/sys_tune.c   (contents, props changed)
  head/sys/ofed/include/linux/atomic.h   (contents, props changed)
  head/sys/ofed/include/linux/clocksource.h   (contents, props changed)
  head/sys/ofed/include/rdma/ib_pma.h   (contents, props changed)
Modified:
  head/sys/conf/files
  head/sys/contrib/rdma/ib_umem.h
  head/sys/dev/cxgb/ulp/iw_cxgb/iw_cxgb_provider.c
  head/sys/modules/Makefile
  head/sys/modules/mlx4/Makefile
  head/sys/modules/mlx4ib/Makefile
  head/sys/ofed/drivers/infiniband/core/addr.c
  head/sys/ofed/drivers/infiniband/core/cma.c
  head/sys/ofed/drivers/infiniband/core/core_priv.h
  head/sys/ofed/drivers/infiniband/core/device.c
  head/sys/ofed/drivers/infiniband/core/sa_query.c
  head/sys/ofed/drivers/infiniband/core/sysfs.c
  head/sys/ofed/drivers/infiniband/core/uverbs_cmd.c
  head/sys/ofed/drivers/infiniband/core/uverbs_main.c
  head/sys/ofed/drivers/infiniband/core/verbs.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/Kconfig
  head/sys/ofed/drivers/infiniband/hw/mlx4/Makefile
  head/sys/ofed/drivers/infiniband/hw/mlx4/ah.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/cq.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/mad.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/main.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/mlx4_ib.h
  head/sys/ofed/drivers/infiniband/hw/mlx4/mr.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/qp.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/srq.c
  head/sys/ofed/drivers/infiniband/hw/mlx4/user.h
  head/sys/ofed/drivers/infiniband/hw/mlx4/wc.c
  head/sys/ofed/drivers/infiniband/hw/mthca/mthca_cmd.c
  head/sys/ofed/drivers/infiniband/hw/mthca/mthca_main.c
  head/sys/ofed/drivers/infiniband/hw/mthca/mthca_memfree.c
  head/sys/ofed/drivers/infiniband/hw/mthca/mthca_provider.c
  head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib.h
  head/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c
  head/sys/ofed/drivers/net/mlx4/Makefile
  head/sys/ofed/drivers/net/mlx4/alloc.c
  head/sys/ofed/drivers/net/mlx4/catas.c
  head/sys/ofed/drivers/net/mlx4/cmd.c
  head/sys/ofed/drivers/net/mlx4/cq.c
  head/sys/ofed/drivers/net/mlx4/en_cq.c
  head/sys/ofed/drivers/net/mlx4/en_main.c
  head/sys/ofed/drivers/net/mlx4/en_netdev.c
  head/sys/ofed/drivers/net/mlx4/en_port.c
  head/sys/ofed/drivers/net/mlx4/en_port.h
  head/sys/ofed/drivers/net/mlx4/en_rx.c
  head/sys/ofed/drivers/net/mlx4/en_tx.c
  head/sys/ofed/drivers/net/mlx4/eq.c
  head/sys/ofed/drivers/net/mlx4/fw.c
  head/sys/ofed/drivers/net/mlx4/fw.h
  head/sys/ofed/drivers/net/mlx4/icm.c
  head/sys/ofed/drivers/net/mlx4/icm.h
  head/sys/ofed/drivers/net/mlx4/intf.c
  head/sys/ofed/drivers/net/mlx4/main.c
  head/sys/ofed/drivers/net/mlx4/mcg.c
  head/sys/ofed/drivers/net/mlx4/mlx4.h
  head/sys/ofed/drivers/net/mlx4/mlx4_en.h
  head/sys/ofed/drivers/net/mlx4/mr.c
  head/sys/ofed/drivers/net/mlx4/pd.c
  head/sys/ofed/drivers/net/mlx4/port.c
  head/sys/ofed/drivers/net/mlx4/profile.c
  head/sys/ofed/drivers/net/mlx4/qp.c
  head/sys/ofed/drivers/net/mlx4/reset.c
  head/sys/ofed/drivers/net/mlx4/sense.c
  head/sys/ofed/drivers/net/mlx4/srq.c
  head/sys/ofed/include/asm/atomic.h
  head/sys/ofed/include/asm/byteorder.h
  head/sys/ofed/include/linux/bitops.h
  head/sys/ofed/include/linux/compat.h
  head/sys/ofed/include/linux/device.h
  head/sys/ofed/include/linux/dma-mapping.h
  head/sys/ofed/include/linux/gfp.h
  head/sys/ofed/include/linux/idr.h
  

svn commit: r254963 - head/sys/net

[Alfred Perlstein]

Author: alfred
Date: Tue Aug 27 16:45:00 2013
New Revision: 254963
URL: http://svnweb.freebsd.org/changeset/base/254963

Log:
  Remove include opt_ofed.h since OFED is unifdef'd.
  
  Pointed out by: glebius

Modified:
  head/sys/net/if_llatbl.h

Modified: head/sys/net/if_llatbl.h
==
--- head/sys/net/if_llatbl.hTue Aug 27 16:30:50 2013(r254962)
+++ head/sys/net/if_llatbl.hTue Aug 27 16:45:00 2013(r254963)
@@ -30,8 +30,6 @@ __FBSDID($FreeBSD$);
 #ifndef_NET_IF_LLATBL_H_
 #define_NET_IF_LLATBL_H_
 
-#include opt_ofed.h
-
 #include sys/_rwlock.h
 #include netinet/in.h
 
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r254823 - head/sys/net

[Alfred Perlstein]

Thanks Gleb.  Will do.

-Alfred

On 8/26/13 4:52 AM, Gleb Smirnoff wrote:

On Sun, Aug 25, 2013 at 01:55:15AM +, Alfred Perlstein wrote:
A Author: alfred
A Date: Sun Aug 25 01:55:14 2013
A New Revision: 254823
A URL: http://svnweb.freebsd.org/changeset/base/254823
A
A Log:
A   Remove the #ifdef OFED from the 20 byte mac in struct llentry.
A
A   With this change it is now possible to build the entire infiniband
A   stack as modules and load it dynamically including IP over IB.
A
A Modified:
A   head/sys/net/if_llatbl.h
A
A Modified: head/sys/net/if_llatbl.h
A 
==
A --- head/sys/net/if_llatbl.h  Sun Aug 25 00:34:44 2013(r254822)
A +++ head/sys/net/if_llatbl.h  Sun Aug 25 01:55:14 2013(r254823)
A @@ -75,9 +75,7 @@ struct llentry {
A   union {
A   uint64_tmac_aligned;
A   uint16_tmac16[3];
A -#ifdef OFED
A   uint8_t mac8[20];   /* IB needs 20 bytes. */
A -#endif
A   } ll_addr;
A
A   /* XXX af-private? */

#include opt_ofed.h should be removed from the file as well.



___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r254823 - head/sys/net

[Alfred Perlstein]

Author: alfred
Date: Sun Aug 25 01:55:14 2013
New Revision: 254823
URL: http://svnweb.freebsd.org/changeset/base/254823

Log:
  Remove the #ifdef OFED from the 20 byte mac in struct llentry.
  
  With this change it is now possible to build the entire infiniband
  stack as modules and load it dynamically including IP over IB.

Modified:
  head/sys/net/if_llatbl.h

Modified: head/sys/net/if_llatbl.h
==
--- head/sys/net/if_llatbl.hSun Aug 25 00:34:44 2013(r254822)
+++ head/sys/net/if_llatbl.hSun Aug 25 01:55:14 2013(r254823)
@@ -75,9 +75,7 @@ struct llentry {
union {
uint64_tmac_aligned;
uint16_tmac16[3];
-#ifdef OFED
uint8_t mac8[20];   /* IB needs 20 bytes. */
-#endif
} ll_addr;
 
/* XXX af-private? */
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r254756 - head/sys/sys

[Alfred Perlstein]

Author: alfred
Date: Sat Aug 24 00:30:32 2013
New Revision: 254756
URL: http://svnweb.freebsd.org/changeset/base/254756

Log:
  Grow some spares in struct vfsops.
  
  This should hopefully prevent ABI breakage
  on adding new vfsops in 10.x.

Modified:
  head/sys/sys/mount.h

Modified: head/sys/sys/mount.h
==
--- head/sys/sys/mount.hSat Aug 24 00:29:34 2013(r254755)
+++ head/sys/sys/mount.hSat Aug 24 00:30:32 2013(r254756)
@@ -628,6 +628,7 @@ struct vfsops {
vfs_susp_clean_t*vfs_susp_clean;
vfs_notify_lowervp_t*vfs_reclaim_lowervp;
vfs_notify_lowervp_t*vfs_unlink_lowervp;
+   vfs_mount_t *vfs_spare[6];  /* spares for ABI compat */
 };
 
 vfs_statfs_t   __vfs_statfs;
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r254173 - head/usr.sbin/watchdogd

[Alfred Perlstein]

Author: alfred
Date: Sat Aug 10 01:48:15 2013
New Revision: 254173
URL: http://svnweb.freebsd.org/changeset/base/254173

Log:
  Fix bug in r253719: fix command line watchdog disable.
  
  r253719 disallowed watchdog(8) from disabling the watchdog
  by breaking the ability to pass 0 as a timeout arg.  Fix this.

Modified:
  head/usr.sbin/watchdogd/watchdogd.c

Modified: head/usr.sbin/watchdogd/watchdogd.c
==
--- head/usr.sbin/watchdogd/watchdogd.c Sat Aug 10 00:53:22 2013
(r254172)
+++ head/usr.sbin/watchdogd/watchdogd.c Sat Aug 10 01:48:15 2013
(r254173)
@@ -60,7 +60,8 @@ __FBSDID($FreeBSD$);
 
 #include getopt.h
 
-static longfetchtimeout(int opt, const char *longopt, const char 
*myoptarg);
+static longfetchtimeout(int opt,
+const char *longopt, const char *myoptarg, int zero_ok);
 static voidparseargs(int, char *[]);
 static int seconds_to_pow2ns(int);
 static voidsighandler(int);
@@ -219,7 +220,7 @@ parse_timeout_to_pow2ns(char opt, const 
if (!longopt)
shortopt[1] = opt;
 
-   a = fetchtimeout(opt, longopt, myoptarg);
+   a = fetchtimeout(opt, longopt, myoptarg, 1);
 
if (a == 0)
rv = WD_TO_NEVER;
@@ -487,7 +488,7 @@ usage(void)
 }
 
 static long
-fetchtimeout(int opt, const char *longopt, const char *myoptarg)
+fetchtimeout(int opt, const char *longopt, const char *myoptarg, int zero_ok)
 {
const char *errstr;
char *p;
@@ -499,7 +500,7 @@ fetchtimeout(int opt, const char *longop
rv = strtol(myoptarg, p, 0);
if ((p != NULL  *p != '\0') || errno != 0)
errstr = is not a number;
-   if (rv = 0)
+   if (rv  0 || (!zero_ok  rv == 0))
errstr = must be greater than zero;
if (errstr) {
if (longopt) 
@@ -721,7 +722,7 @@ parseargs(int argc, char *argv[])
break;
 #endif
case 's':
-   nap = fetchtimeout(c, NULL, optarg);
+   nap = fetchtimeout(c, NULL, optarg, 0);
break;
case 'S':
do_syslog = 0;
@@ -734,7 +735,8 @@ parseargs(int argc, char *argv[])
timeout);
break;
case 'T':
-   carp_thresh_seconds = fetchtimeout(c, NULL, optarg);
+   carp_thresh_seconds =
+   fetchtimeout(c, NULL, optarg, 0);
break;
case 'w':
do_timedog = 1;
@@ -742,7 +744,7 @@ parseargs(int argc, char *argv[])
case 0:
lopt = longopts[longindex].name;
if (!strcmp(lopt, pretimeout)) {
-   pretimeout = fetchtimeout(0, lopt, optarg);
+   pretimeout = fetchtimeout(0, lopt, optarg, 0);
} else if (!strcmp(lopt, pretimeout-action)) {
pretimeout_act = timeout_act_str2int(lopt,
optarg);
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r253719 - in head: sys/conf sys/dev/watchdog sys/libkern sys/sys usr.sbin/watchdogd

[Alfred Perlstein]

On 7/30/13 4:24 AM, Ulrich Spörlein wrote:

On Sat, 2013-07-27 at 20:47:02 +, Alfred Perlstein wrote:

Author: alfred
Date: Sat Jul 27 20:47:01 2013
New Revision: 253719
URL: http://svnweb.freebsd.org/changeset/base/253719

Log:
   Fix watchdog pretimeout.

Alfred,

this broken the build and hasn't been fixed for almost three days now.
What's up with that? How was this change tested before commit?

=== usr.sbin/watchdogd (all)
cc  -O2 -pipe  -std=gnu99 -Qunused-arguments -fstack-protector -Wsystem-headers 
-Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes 
-Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings 
-Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline 
-Wnested-externs -Wredundant-decls -Wold-style-definition 
-Wmissing-variable-declarations -Wno-pointer-sign -Wno-empty-body 
-Wno-string-plus-int -c /data/src/freebsd-head/usr.sbin/watchdogd/watchdogd.c
/data/src/freebsd-head/usr.sbin/watchdogd/watchdogd.c:777:18: error: comparison 
of integers of different signs: 'u_int' (aka 'unsigned int') and 'time_t' (aka 
'int') [-Werror,-Wsign-compare]
 if (pretimeout = ts.tv_sec) {
 ~~ ^  ~
1 error generated.
*** Error code 1


Yikes!  What?

I did test this a ton of times.  My apologies to all.

Let me see what I can do.

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r253719 - in head: sys/conf sys/dev/watchdog sys/libkern sys/sys usr.sbin/watchdogd

[Alfred Perlstein]

Author: alfred
Date: Sat Jul 27 20:47:01 2013
New Revision: 253719
URL: http://svnweb.freebsd.org/changeset/base/253719

Log:
  Fix watchdog pretimeout.
  
  The original API calls for pow2ns, however the new APIs from
  Linux call for seconds.
  
  We need to be able to convert to/from 2^Nns to seconds in both
  userland and kernel to fix this and properly compare units.

Added:
  head/sys/libkern/flsll.c   (contents, props changed)
Modified:
  head/sys/conf/files
  head/sys/dev/watchdog/watchdog.c
  head/sys/sys/libkern.h
  head/usr.sbin/watchdogd/watchdogd.c

Modified: head/sys/conf/files
==
--- head/sys/conf/files Sat Jul 27 20:15:18 2013(r253718)
+++ head/sys/conf/files Sat Jul 27 20:47:01 2013(r253719)
@@ -2976,6 +2976,7 @@ libkern/arc4random.c  standard
 libkern/bcd.c  standard
 libkern/bsearch.c  standard
 libkern/crc32.cstandard
+libkern/flsll.c standard
 libkern/fnmatch.c  standard
 libkern/iconv.coptional libiconv
 libkern/iconv_converter_if.m   optional libiconv

Modified: head/sys/dev/watchdog/watchdog.c
==
--- head/sys/dev/watchdog/watchdog.cSat Jul 27 20:15:18 2013
(r253718)
+++ head/sys/dev/watchdog/watchdog.cSat Jul 27 20:47:01 2013
(r253719)
@@ -39,6 +39,7 @@ __FBSDID($FreeBSD$);
 #include sys/kernel.h
 #include sys/malloc.h
 #include sys/module.h
+#include sys/sysctl.h
 #include sys/syslog.h
 #include sys/watchdog.h
 #include sys/bus.h
@@ -60,10 +61,56 @@ static int wd_softtimeout_act = WD_SOFT_
 
 static struct cdev *wd_dev;
 static volatile u_int wd_last_u;/* last timeout value set by kern_do_pat */
+static u_int wd_last_u_sysctl;/* last timeout value set by kern_do_pat */
+static u_int wd_last_u_sysctl_secs;/* wd_last_u in seconds */
+
+SYSCTL_NODE(_hw, OID_AUTO, watchdog, CTLFLAG_RD, 0, Main watchdog device);
+SYSCTL_UINT(_hw_watchdog, OID_AUTO, wd_last_u, CTLFLAG_RD,
+wd_last_u_sysctl, 0, Watchdog last update time);
+SYSCTL_UINT(_hw_watchdog, OID_AUTO, wd_last_u_secs, CTLFLAG_RD,
+wd_last_u_sysctl_secs, 0, Watchdog last update time);
 
 static int wd_lastpat_valid = 0;
 static time_t wd_lastpat = 0;  /* when the watchdog was last patted */
 
+static void
+pow2ns_to_ts(int pow2ns, struct timespec *ts)
+{
+   uint64_t ns;
+
+   ns = 1ULL  pow2ns;
+   ts-tv_sec = ns / 10ULL;
+   ts-tv_nsec = ns % 10ULL;
+}
+
+static int
+pow2ns_to_ticks(int pow2ns)
+{
+   struct timeval tv;
+   struct timespec ts;
+
+   pow2ns_to_ts(pow2ns, ts);
+   TIMESPEC_TO_TIMEVAL(tv, ts);
+   return (tvtohz(tv));
+}
+
+static int
+seconds_to_pow2ns(int seconds)
+{
+   uint64_t power;
+   uint64_t ns;
+   uint64_t shifted;
+
+   ns = ((uint64_t)seconds) * 10ULL;
+   power = flsll(ns);
+   shifted = 1ULL  power;
+   if (shifted = ns) {
+   power++;
+   }
+   return (power);
+}
+
+
 int
 wdog_kern_pat(u_int utim)
 {
@@ -86,6 +133,8 @@ wdog_kern_pat(u_int utim)
 * This can be zero (to disable the watchdog)
 */
wd_last_u = (utim  WD_INTERVAL);
+   wd_last_u_sysctl = wd_last_u;
+   wd_last_u_sysctl_secs = pow2ns_to_ticks(wd_last_u) / hz;
}
if ((utim  WD_INTERVAL) == WD_TO_NEVER) {
utim = 0;
@@ -101,7 +150,7 @@ wdog_kern_pat(u_int utim)
callout_stop(wd_softtimeo_handle);
} else {
(void) callout_reset(wd_softtimeo_handle,
-   hz*utim, wd_timeout_cb, soft);
+   pow2ns_to_ticks(utim), wd_timeout_cb, soft);
}
error = 0;
} else {
@@ -201,10 +250,13 @@ static int
 wd_set_pretimeout(int newtimeout, int disableiftoolong)
 {
u_int utime;
+   struct timespec utime_ts;
+   int timeout_ticks;
 
utime = wdog_kern_last_timeout();
+   pow2ns_to_ts(utime, utime_ts);
/* do not permit a pre-timeout = than the timeout. */
-   if (newtimeout = utime) {
+   if (newtimeout = utime_ts.tv_sec) {
/*
 * If 'disableiftoolong' then just fall through
 * so as to disable the pre-watchdog
@@ -222,8 +274,22 @@ wd_set_pretimeout(int newtimeout, int di
return 0;
}
 
+   timeout_ticks = pow2ns_to_ticks(utime) - (hz*newtimeout);
+#if 0
+   printf(wd_set_pretimeout: 
+   newtimeout: %d, 
+   utime: %d - utime_ticks: %d, 
+   hz*newtimeout: %d, 
+   timeout_ticks: %d - sec: %d\n,
+   newtimeout,
+   utime, pow2ns_to_ticks(utime),
+   hz*newtimeout,
+   timeout_ticks, timeout_ticks / hz);

svn commit: r253723 - head/usr.sbin/watchdogd

[Alfred Perlstein]

Author: alfred
Date: Sat Jul 27 22:23:32 2013
New Revision: 253723
URL: http://svnweb.freebsd.org/changeset/base/253723

Log:
  Provide some examples for watchdogd usage.

Modified:
  head/usr.sbin/watchdogd/watchdogd.8

Modified: head/usr.sbin/watchdogd/watchdogd.8
==
--- head/usr.sbin/watchdogd/watchdogd.8 Sat Jul 27 22:21:10 2013
(r253722)
+++ head/usr.sbin/watchdogd/watchdogd.8 Sat Jul 27 22:23:32 2013
(r253723)
@@ -27,7 +27,7 @@
 .\
 .\ $FreeBSD$
 .\
-.Dd March 5, 2013
+.Dd July 27, 2013
 .Dt WATCHDOGD 8
 .Os
 .Sh NAME
@@ -204,6 +204,81 @@ and the kernel
 .Xr log 4
 device for
 .Xr syslog 8 .
+.Sh EXAMPLES
+.Ss Debugging watchdogd and/or your watchdog script.
+.Pp
+This is a useful recipe for debugging watchdogd and your watchdog
+script.
+.Pp
+(Note that ^C works oddly because watchdogd calls system(3) so the
+first ^C will terminate the sleep command.)
+.Pp
+.Pp
+Explanation of options used:
+.Bl -enum -offset indent -compact
+.It
+Set Debug on (--debug)
+.It
+Set the watchdog to trip at 30 seconds. (-t 30)
+.It
+Use of a softtimeout:
+.Bl -enum -offset indent -compact -nested
+.It
+Use a softtimeout (don't arm the hardware watchdog) (--softtimeout)
+.It
+Set the softtimeout action to do both kernel printf(9) and log(9) when it 
trips. (--softtimeout-action log,printf)
+.El
+.It
+Use of a pre-timeout:
+.Bl -enum -offset indent -compact -nested
+.It
+Set a pre-timeout of 15 seconds (this will later trigger a panic/dump) 
(--pretimeout 15)
+.It
+Set the action to also kernel printf(9) and log(9) when it trips. 
(--pretimeout-action log,printf)
+.El
+.It
+Use of a script:
+.Bl -enum -offset indent -compact -nested
+.It
+Run sleep 60 as a shell command that acts as the watchdog (-e 'sleep 60')
+.It
+Warn us when the script takes longer than 1 second to run (-w)
+.El
+.El
+.Bd -literal
+watchdogd --debug -t 30 \\
+  --softtimeout --softtimeout-action log,printf \\
+  --pretimeout 15 --pretimeout-action log,printf \\
+  -e 'sleep 60' -w
+.Ed
+.Ss Production use of example
+.Bl -enum -offset indent -compact
+.It
+Set hard timeout to 120 seconds (-t 120)
+.It
+Set a panic to happen at 60 seconds (to trigger a
+.Xr crash 8
+for dump analysis):
+.Bl -enum -offset indent -compact -nested
+.It
+Use of pre-timeout (--pretimeout 60)
+.It
+Specify pre-timeout action (--pretimeout-action log,printf,panic )
+.El
+.It
+Use of a script:
+.Bl -enum -offset indent -compact -nested
+.It
+Run your script (-e '/path/to/your/script 60')
+.It
+Log if your script takes a longer than 15 seconds to run time. (-w -T 15)
+.El
+.El
+.Bd -literal
+watchdogd  -t 120 \\
+  --pretimeout 60 --pretimeout-action log,printf,panic \\
+  -e '/path/to/your/script 60' -w -T 15
+.Ed
 .Sh FILES
 .Bl -tag -width .Pa /var/run/watchdogd.pid -compact
 .It Pa /var/run/watchdogd.pid
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r253002 - head

[Alfred Perlstein]

On 7/8/13 4:24 PM, Garrett Cooper wrote:

On Mon, Jul 8, 2013 at 2:13 PM, John Baldwin j...@freebsd.org wrote:

On Monday, July 08, 2013 2:23:31 am Garrett Cooper wrote:

On Sun, Jul 7, 2013 at 7:25 PM, Garrett Cooper yaneurab...@gmail.com

wrote:

On Jul 7, 2013, at 2:15 PM, Alfred Perlstein alf...@freebsd.org wrote:


On 7/7/13 2:01 PM, Garrett Cooper wrote:

Why the magic number 12?

Numbers higher seem to result in worse performance as reported by some

members of my team.

The suggestion is good in spirit, but this doesn't justify the reasoning

for this recommendation for all cases.

Please revert this change and add a doc page or notes to the dev handbook

discussing what the empirical process and results were for determining this
value so people can come up with their own values that work best with their
hardware and software config. This recommendation is prone to bitrot like some
of the recommendations in tuning(7).

Misinformation is sometimes more harmful than no information.

I spoke with Alfred over the phone and did some more careful thought
about this and I'm rescinding this request.

Alfred did a good job at documenting how JFLAG works (it was
previously undocumented). My concern over -j12 was performance
related, and after giving things more careful thought it actually
makes sense why -j12 was chosen because Westmere and newer processors
have issues with NUMA and cache locality between multiple processor
packages as we've seen non-empirically and empirically at Isilon with
FreeBSD 7 and 10 (it's a known issue that jeffr@ and jhb@ are aware
of).

I'll come up with a concise patch that does what Alfred was trying to
achieve and have Alfred review it.

Thanks (and thank you Alfred for the contribution!!!)!

Westmere is fine, it's post-Westmere that is more troublesome.

Even the 6-core Westmeres (I'm being completely dumb here as you and
Jeff know a lot more about the NUMA issue than I do as I just caught
the tail end of the conversation at BSDCan)? I'm asking because they
(iX) are using build.ix as the primary build machine and it has 2
Westmere dies with (IIRC -- please correct me if I'm wrong
Alfred/Xin/etc) 6 cores each and are SMT enabled. It also has a
boatload of RAM and disks hooked up to an mfi(4) controller (which
could be a contributing factor in the performance degradation issue).


I think the comment is not super useful, but don't object enough to want
it to be removed.  I always use 'make tinderbox' instead of
'make universe' though as I want build failures to be obvious.  For the
described use case of checking if kernels build, 'tinderbox' certainly
seems to be the more appropriate target.

Changing it from universe to tinderbox seems like a better idea --
I'll put a short note in my proposed patch for that.

Thanks!
-Garrett

Just to clarify, the passing of jflag in the comments was there because 
it seems like most of the targets default to -j1 which out of the box 
is somewhat ludicrous these days.  It was only a guide such that someone 
who knows what -j does would be like oh that's absurd, I know a 
better value rather than just being oblivious to it (like me) or 
stupidly assume that some level of auto-tuning was done (also like me) 
and wind up wondering why make universe is taking as long as an actual 
real live universe to build.


;)

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r253002 - head

[Alfred Perlstein]

Author: alfred
Date: Sun Jul  7 20:39:11 2013
New Revision: 253002
URL: http://svnweb.freebsd.org/changeset/base/253002

Log:
  Document tip on how to build all kernels quickly.

Modified:
  head/Makefile

Modified: head/Makefile
==
--- head/Makefile   Sun Jul  7 19:58:14 2013(r253001)
+++ head/Makefile   Sun Jul  7 20:39:11 2013(r253002)
@@ -32,6 +32,12 @@
 # targets - Print a list of supported TARGET/TARGET_ARCH pairs
 #   for world and kernel targets.
 # toolchains  - Build a toolchain for all world and kernel targets.
+# 
+# quick way to test all kernel builds:
+#  _jflag=`sysctl -n hw.ncpu`
+#  _jflag=$(($_jflag * 2))
+#  [ $_jflag -gt 12 ]  _jflag=12
+#  make universe -DMAKE_JUST_KERNELS JFLAG=${jflag}
 #
 # This makefile is simple by design. The FreeBSD make automatically reads
 # the /usr/share/mk/sys.mk unless the -m argument is specified on the
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r253003 - head

[Alfred Perlstein]

Author: alfred
Date: Sun Jul  7 20:44:04 2013
New Revision: 253003
URL: http://svnweb.freebsd.org/changeset/base/253003

Log:
  Correct typo specifying jflags.

Modified:
  head/Makefile

Modified: head/Makefile
==
--- head/Makefile   Sun Jul  7 20:39:11 2013(r253002)
+++ head/Makefile   Sun Jul  7 20:44:04 2013(r253003)
@@ -37,7 +37,7 @@
 #  _jflag=`sysctl -n hw.ncpu`
 #  _jflag=$(($_jflag * 2))
 #  [ $_jflag -gt 12 ]  _jflag=12
-#  make universe -DMAKE_JUST_KERNELS JFLAG=${jflag}
+#  make universe -DMAKE_JUST_KERNELS JFLAG=-j${_jflag}
 #
 # This makefile is simple by design. The FreeBSD make automatically reads
 # the /usr/share/mk/sys.mk unless the -m argument is specified on the
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r253002 - head

[Alfred Perlstein]

On 7/7/13 2:40 PM, Andriy Gapon wrote:

on 08/07/2013 00:15 Alfred Perlstein said the following:

On 7/7/13 2:01 PM, Garrett Cooper wrote:

Why the magic number 12?

Numbers higher seem to result in worse performance as reported by some members
of my team.

Should we really commit all notes to self or my team's knowledge base to
FreeBSD source code like this?


I would hope so!  At least in comments, that way we can all help each other.

Otherwise each time another person comes along they are subject to 
having to discover the same things that I did.


What is the point of listing 10+ make targets if someone coming in 
doesn't know which ones to run and how to run them optimally in order to 
ensure a safe build?  Do you not agree that more of this sort of 
documentation should be in the code instead of having to ask on IRC?


For the record I checked multiple pages in developer primer and scoured 
makefiles for a while before giving up and asking on IRC.  I figured it 
would be helpful for the next clueless goon that came in and tried to 
make things better if I recorded my experience to guide them.


I hold no strong feelings towards the comments, other than they should 
be preserved in some form, or maybe even turned into a workable target 
for people that don't want or can not hold all that context in their brains.


-Alfred





On Jul 7, 2013, at 1:39 PM, Alfred Perlstein alf...@freebsd.org wrote:


Author: alfred
Date: Sun Jul  7 20:39:11 2013
New Revision: 253002
URL: http://svnweb.freebsd.org/changeset/base/253002

Log:
   Document tip on how to build all kernels quickly.

Modified:
   head/Makefile

Modified: head/Makefile
==
--- head/MakefileSun Jul  7 19:58:14 2013(r253001)
+++ head/MakefileSun Jul  7 20:39:11 2013(r253002)
@@ -32,6 +32,12 @@
# targets - Print a list of supported TARGET/TARGET_ARCH pairs
#   for world and kernel targets.
# toolchains  - Build a toolchain for all world and kernel targets.
+#
+# quick way to test all kernel builds:
+#_jflag=`sysctl -n hw.ncpu`
+#_jflag=$(($_jflag * 2))
+#[ $_jflag -gt 12 ]  _jflag=12
+#make universe -DMAKE_JUST_KERNELS JFLAG=${jflag}
#
# This makefile is simple by design. The FreeBSD make automatically reads
# the /usr/share/mk/sys.mk unless the -m argument is specified on the
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org




___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

svn commit: r252683 - head/sys/kern

[Alfred Perlstein]

Author: alfred
Date: Thu Jul  4 05:53:05 2013
New Revision: 252683
URL: http://svnweb.freebsd.org/changeset/base/252683

Log:
  The change in r236456 (atomic_store_rel not locked) exposed a bug
  in the ithread code where we could lose ithread interrupts if
  intr_event_schedule_thread() is called while the ithread is already
  running.  Effectively memory writes could be ordered incorrectly
  such that the unatomic write of 0 to ithd-it_need (in ithread_loop)
  could be delayed until after it was set to be triggered again by
  intr_event_schedule_thread().
  
  This was particularly a big problem for CAM because CAM optimizes
  scheduling of ithreads such that it only signals camisr() when it
  queues to an empty queue.  This means that additional completion
  events would not unstick CAM and quickly lead to a complete lockup
  of the CAM stack.
  
  To fix this use atomics in all places where ithd-it_need is accessed.
  
  Submitted by: delphij, mav
  Obtained from: TrueOS, iXsystems
  MFC After: 1 week

Modified:
  head/sys/kern/kern_intr.c

Modified: head/sys/kern/kern_intr.c
==
--- head/sys/kern/kern_intr.c   Thu Jul  4 05:35:56 2013(r252682)
+++ head/sys/kern/kern_intr.c   Thu Jul  4 05:53:05 2013(r252683)
@@ -841,7 +841,7 @@ ok:
 * again and remove this handler if it has already passed
 * it on the list.
 */
-   ie-ie_thread-it_need = 1;
+   atomic_store_rel_int(ie-ie_thread-it_need, 1);
} else
TAILQ_REMOVE(ie-ie_handlers, handler, ih_next);
thread_unlock(ie-ie_thread-it_thread);
@@ -912,7 +912,7 @@ intr_event_schedule_thread(struct intr_e
 * running.  Then, lock the thread and see if we actually need to
 * put it on the runqueue.
 */
-   it-it_need = 1;
+   atomic_store_rel_int(it-it_need, 1);
thread_lock(td);
if (TD_AWAITING_INTR(td)) {
CTR3(KTR_INTR, %s: schedule pid %d (%s), __func__, p-p_pid,
@@ -990,7 +990,7 @@ ok:
 * again and remove this handler if it has already passed
 * it on the list.
 */
-   it-it_need = 1;
+   atomic_store_rel_int(it-it_need, 1);
} else
TAILQ_REMOVE(ie-ie_handlers, handler, ih_next);
thread_unlock(it-it_thread);
@@ -1066,7 +1066,7 @@ intr_event_schedule_thread(struct intr_e
 * running.  Then, lock the thread and see if we actually need to
 * put it on the runqueue.
 */
-   it-it_need = 1;
+   atomic_store_rel_int(it-it_need, 1);
thread_lock(td);
if (TD_AWAITING_INTR(td)) {
CTR3(KTR_INTR, %s: schedule pid %d (%s), __func__, p-p_pid,
@@ -1247,7 +1247,7 @@ intr_event_execute_handlers(struct proc 
 * interrupt threads always invoke all of their handlers.
 */
if (ie-ie_flags  IE_SOFT) {
-   if (!ih-ih_need)
+   if (atomic_load_acq_int(ih-ih_need) == 0)
continue;
else
atomic_store_rel_int(ih-ih_need, 0);
@@ -1349,7 +1349,7 @@ ithread_loop(void *arg)
 * we are running, it will set it_need to note that we
 * should make another pass.
 */
-   while (ithd-it_need) {
+   while (atomic_load_acq_int(ithd-it_need) != 0) {
/*
 * This might need a full read and write barrier
 * to make sure that this write posts before any
@@ -1368,7 +1368,8 @@ ithread_loop(void *arg)
 * set again, so we have to check it again.
 */
thread_lock(td);
-   if (!ithd-it_need  !(ithd-it_flags  (IT_DEAD | IT_WAIT))) {
+   if ((atomic_load_acq_int(ithd-it_need) == 0) 
+   !(ithd-it_flags  (IT_DEAD | IT_WAIT))) {
TD_SET_IWAIT(td);
ie-ie_count = 0;
mi_switch(SW_VOL | SWT_IWAIT, NULL);
@@ -1529,7 +1530,7 @@ ithread_loop(void *arg)
 * we are running, it will set it_need to note that we
 * should make another pass.
 */
-   while (ithd-it_need) {
+   while (atomic_load_acq_int(ithd-it_need) != 0) {
/*
 * This might need a full read and write barrier
 * to make sure that this write posts before any
@@ -1551,7 +1552,8 @@ ithread_loop(void *arg)
 * set again, so we have to check it again.
 */
thread_lock(td);
-   if (!ithd-it_need  !(ithd-it_flags  (IT_DEAD | IT_WAIT))) {
+   if ((atomic_load_acq_int(ithd-it_need) 

Re: svn commit: r251894 - in head: lib/libmemstat sys/vm

[Alfred Perlstein]

On 6/18/13 4:37 AM, Gleb Smirnoff wrote:

On Tue, Jun 18, 2013 at 10:25:08AM +0200, Andre Oppermann wrote:
A There used to be a problem with per CPU caches accumulating large amounts
A of items without freeing back to the global (or socket) pool.
A
A Do these updates to UMA change this situation and/or do you have further
A improvements coming up?

This is especially a problem with ZFS, which utilizes UMA extensively.

IMHO, we need a flag for uma_zcreate() that would disable per CPU caches, so
that certain zones (ZFS at least) would have them off.

It might be a good idea to force this flag on every zone that has allocation =
then the page size.

What about people running with 256GB+ ram?  Do they also want the per 
cpu caches off?


--
Alfred Perlstein
VP Software Engineering, iXsystems

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r251894 - in head: lib/libmemstat sys/vm

[Alfred Perlstein]

On 6/18/13 5:21 PM, Jeff Roberson wrote:

On Tue, 18 Jun 2013, Alfred Perlstein wrote:


On 6/18/13 4:37 AM, Gleb Smirnoff wrote:

On Tue, Jun 18, 2013 at 10:25:08AM +0200, Andre Oppermann wrote:
A There used to be a problem with per CPU caches accumulating large 
amounts

A of items without freeing back to the global (or socket) pool.
A
A Do these updates to UMA change this situation and/or do you have 
further

A improvements coming up?

This is especially a problem with ZFS, which utilizes UMA extensively.

IMHO, we need a flag for uma_zcreate() that would disable per CPU 
caches, so

that certain zones (ZFS at least) would have them off.

It might be a good idea to force this flag on every zone that has 
allocation =

then the page size.

What about people running with 256GB+ ram?  Do they also want the per 
cpu caches off?


If you look at the new system there is a static threshold for the 
initial item size required for different sized per-cpu buckets. What 
might make sense is to tune this size based on available memory.  For 
what it's worth I looked at solaris settings and they cache roughly 4x 
as much on a per-cpu basis.


The new system should tend to cache less of large and infrequent 
allocations vs the old system.  I can't say yet whether it is still a 
problem.


I have an implementation of vmem to replace using vm_maps for 
kmem_map, buffer_map, etc. which may resolve the zfs allocation 
problems.  I hope to get this in over the next few weeks.




That looks really exciting Jeff.  Thank you.

I'm hoping we can give back some testing numbers when it goes in.

-Alfred

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r251507 - head/usr.sbin/portsnap/portsnap

[Alfred Perlstein]

On 6/17/13 11:59 AM, Dmitry Morozovsky wrote:

On Sun, 16 Jun 2013, Gavin Atkinson wrote:


   Make 'portsnap alfred' overwrite ports tree if it's not created by a
   portsnap.

FWIW, the 'alfred' command is poorly named and is used by other
projects (ISTR portshaker uses it).  It should also be documented.

I would love to see this renamed - unfortunately the most obvious name for
it update is already taken.  I wonder if it should be named auto?

Hmm, 'forceupdate' maybe, in brief mimic to /etc/rc.d/* ?



I've already told MANY people that it's easy to use when they just run 
portsnap alfred.


I think we need to leave it as this point.  An alias is fine.

--
Alfred Perlstein
VP Software Engineering, iXsystems

___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r251282 - head/sys/kern

[Alfred Perlstein]

On 6/5/13 10:55 AM, Adrian Chadd wrote:

... can people please boot an i386 kernel w/ this stuff in it, with
the RAM constrained to something small (say, 128mb), and verify that
you can at least start a buildworld (minus clang, of course) without
panicing?

There's a recent PR
(http://www.freebsd.org/cgi/query-pr.cgi?pr=179112) which shows
regressions on i386 + small RAM platforms.

I know it's a different area, but I'd really appreciate it if people
better tested this stuff out before they make i386 (and by extension,
any platform with limited KVA and small amounts of RAM) unusable
through carelessness.


Adrian, I'm pretty sure that kib's patch actually limits memory.

The concern I have is that it's a hard limit that appears not based on 
KVA, but rather 32GB.


I can be wrong, but Bruce's feedback looks like it backs up my concern.

-Alfred



Thanks,



Adrian

On 5 June 2013 05:11, Alfred Perlstein bri...@mu.org wrote:

Konstantin, is there any way to take some of Bruce's feedback into account
to get rid of the hard coded max?

-Alfred


On 6/4/13 1:14 AM, Bruce Evans wrote:

On Tue, 4 Jun 2013, Konstantin Belousov wrote:


On Mon, Jun 03, 2013 at 02:24:26AM -0700, Alfred Perlstein wrote:

On 6/3/13 12:55 AM, Konstantin Belousov wrote:

On Sun, Jun 02, 2013 at 09:27:53PM -0700, Alfred Perlstein wrote:

Hey Konstaintin, shouldn't this be scaled against the actual amount of
KVA we have instead of an arbitrary limit?

The commit changes the buffer cache to scale according to the available
KVA, making the scaling less dumb.

I do not understand what exactly do you want to do, please describe the
algorithm you propose to implement instead of my change.


Sure, how about deriving the hardcoded 32 from the maxkva a machine
can have?

Is that possible?

I do not see why this would be useful. Initially I thought about simply
capping nbuf at 10 without referencing any memory. Then I realized
that this would somewhat conflict with (unlikely) changes to the value
of BKVASIZE due to factor.


The presence of BKVASIZE in 'factor' is a bug.  My version never had this
bug (see below for a patch).  The scaling should be to maximize nbuf,
subject to non-arbitrary limits on physical memory and kva, and now an
arbirary limit of about 10 / (BKVASIZE / 16384) on nbuf.  Your new
limit is arbitrary so it shouldn't affect nbuf depending on BKVASIZE.

Expanding BKVASIZE should expand kva use, but on i386 this will soon
hit a non-arbitary kva limit so nbuf will not be as high as preferred.
nbuf needs to be very large mainly to support file systems with small
buffers.  Even 10 only gives 50MB of buffering if the fs block
size is 512.  This would shrink to only 12.5MB if BKVASIZE is expanded
by a factor of 4 and the bug is not fixed.  If 25000 buffers after
expanding BKVASIZE is enough, then that should be the arbitary limit
(independent of BKVASIZE) so as to save physical memory.

On i386 systems with 1GB RAM, nbuf defaults to about 7000.  With an
fs block size of 512, that can buffer 3.5MB.  Expanding BKVASIZE by a
factor of 4 shrinks this to 0.875MB in -current.  That is ridiculously
small.  VMIO limits the lossage from this.

BKVASIZE was originally 8KB.  I forget if nbuf was halved by not modifying
the scale factor when it was expanded to 16KB.  Probably not.  I used to
modify the scale factor to get twice as many as the default nbuf, but
once the default nbuf expanded to a few thousand it became large enough
for most purposes so I no longer do this.

Except on arches with extremely limit kva like i386, KVASIZE should be
MAXBSIZE, and all of the complications for expanding a buffer's kva
beyond BKVASIZE and handling the fragmentation problems caused by this
shouldn't exist.  Limits like vfs.maxbufspace should be scaled by
NOMBSIZE = current BKVASIZE instead of BKVASIZE.  Oops, my NOMBSIZE
has similar logical problems to BKVASIZE.  It needs to be precisely
16KB to preserve the defaults for nbuf and maxbufspace, but the nominal
(ffs) buffer size is now 32KB.  So the heuristic scale factors should
use the historical magic number 16K.  I changed sequential_heuristic()
to use a hard-coded 16K instead of BKVASIZE.  The correct number here
depends on disk technology.

The patch has many style fixes:

@ Index: vfs_bio.c
@ ===
@ RCS file: /home/ncvs/src/sys/kern/vfs_bio.c,v
@ retrieving revision 1.436
@ diff -u -2 -r1.436 vfs_bio.c
@ --- vfs_bio.c17 Jun 2004 17:16:49 -1.436
@ +++ vfs_bio.c3 Jun 2013 16:04:34 -
@ @@ -419,64 +493,54 @@
@ @  /*
@ - * Calculating buffer cache scaling values and reserve space for buffer
@ + * Calculate buffer cache scaling values and reserve space for buffer
@   * headers.  This is called during low level kernel initialization and
@   * may be called more then once.  We CANNOT write to the memory area
@   * being reserved at this time.
@   */
@ -caddr_t
@ -kern_vfs_bio_buffer_alloc(caddr_t v, long

Re: svn commit: r251507 - head/usr.sbin/portsnap/portsnap

[Alfred Perlstein]

Oh no, you made me destructive!! :)

-Alfred

On 6/7/13 1:21 PM, Xin LI wrote:

Author: delphij
Date: Fri Jun  7 20:21:30 2013
New Revision: 251507
URL: http://svnweb.freebsd.org/changeset/base/251507

Log:
   Make 'portsnap alfred' overwrite ports tree if it's not created by a
   portsnap.
   
   Discussed with:	alfred

   Reviewed by: cperciva

Modified:
   head/usr.sbin/portsnap/portsnap/portsnap.sh

Modified: head/usr.sbin/portsnap/portsnap/portsnap.sh
==
--- head/usr.sbin/portsnap/portsnap/portsnap.sh Fri Jun  7 19:45:04 2013
(r251506)
+++ head/usr.sbin/portsnap/portsnap/portsnap.sh Fri Jun  7 20:21:30 2013
(r251507)
@@ -1113,7 +1113,7 @@ cmd_alfred() {
else
cmd_cron
fi
-   if [ -d ${PORTSDIR} ]; then
+   if [ -r ${PORTSDIR}/.portsnap.INDEX ]; then
cmd_update
else
cmd_extract



___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r251282 - head/sys/kern

[Alfred Perlstein]

Konstantin, is there any way to take some of Bruce's feedback into 
account to get rid of the hard coded max?


-Alfred

On 6/4/13 1:14 AM, Bruce Evans wrote:

On Tue, 4 Jun 2013, Konstantin Belousov wrote:


On Mon, Jun 03, 2013 at 02:24:26AM -0700, Alfred Perlstein wrote:

On 6/3/13 12:55 AM, Konstantin Belousov wrote:

On Sun, Jun 02, 2013 at 09:27:53PM -0700, Alfred Perlstein wrote:
Hey Konstaintin, shouldn't this be scaled against the actual 
amount of

KVA we have instead of an arbitrary limit?
The commit changes the buffer cache to scale according to the 
available

KVA, making the scaling less dumb.

I do not understand what exactly do you want to do, please describe 
the

algorithm you propose to implement instead of my change.


Sure, how about deriving the hardcoded 32 from the maxkva a machine
can have?

Is that possible?

I do not see why this would be useful. Initially I thought about simply
capping nbuf at 10 without referencing any memory. Then I realized
that this would somewhat conflict with (unlikely) changes to the value
of BKVASIZE due to factor.


The presence of BKVASIZE in 'factor' is a bug.  My version never had this
bug (see below for a patch).  The scaling should be to maximize nbuf,
subject to non-arbitrary limits on physical memory and kva, and now an
arbirary limit of about 10 / (BKVASIZE / 16384) on nbuf.  Your new
limit is arbitrary so it shouldn't affect nbuf depending on BKVASIZE.

Expanding BKVASIZE should expand kva use, but on i386 this will soon
hit a non-arbitary kva limit so nbuf will not be as high as preferred.
nbuf needs to be very large mainly to support file systems with small
buffers.  Even 10 only gives 50MB of buffering if the fs block
size is 512.  This would shrink to only 12.5MB if BKVASIZE is expanded
by a factor of 4 and the bug is not fixed.  If 25000 buffers after
expanding BKVASIZE is enough, then that should be the arbitary limit
(independent of BKVASIZE) so as to save physical memory.

On i386 systems with 1GB RAM, nbuf defaults to about 7000.  With an
fs block size of 512, that can buffer 3.5MB.  Expanding BKVASIZE by a
factor of 4 shrinks this to 0.875MB in -current.  That is ridiculously
small.  VMIO limits the lossage from this.

BKVASIZE was originally 8KB.  I forget if nbuf was halved by not 
modifying

the scale factor when it was expanded to 16KB.  Probably not.  I used to
modify the scale factor to get twice as many as the default nbuf, but
once the default nbuf expanded to a few thousand it became large enough
for most purposes so I no longer do this.

Except on arches with extremely limit kva like i386, KVASIZE should be
MAXBSIZE, and all of the complications for expanding a buffer's kva
beyond BKVASIZE and handling the fragmentation problems caused by this
shouldn't exist.  Limits like vfs.maxbufspace should be scaled by
NOMBSIZE = current BKVASIZE instead of BKVASIZE.  Oops, my NOMBSIZE
has similar logical problems to BKVASIZE.  It needs to be precisely
16KB to preserve the defaults for nbuf and maxbufspace, but the nominal
(ffs) buffer size is now 32KB.  So the heuristic scale factors should
use the historical magic number 16K.  I changed sequential_heuristic()
to use a hard-coded 16K instead of BKVASIZE.  The correct number here
depends on disk technology.

The patch has many style fixes:

@ Index: vfs_bio.c
@ ===
@ RCS file: /home/ncvs/src/sys/kern/vfs_bio.c,v
@ retrieving revision 1.436
@ diff -u -2 -r1.436 vfs_bio.c
@ --- vfs_bio.c17 Jun 2004 17:16:49 -1.436
@ +++ vfs_bio.c3 Jun 2013 16:04:34 -
@ @@ -419,64 +493,54 @@
@ @  /*
@ - * Calculating buffer cache scaling values and reserve space for 
buffer

@ + * Calculate buffer cache scaling values and reserve space for buffer
@   * headers.  This is called during low level kernel initialization and
@   * may be called more then once.  We CANNOT write to the memory area
@   * being reserved at this time.
@   */
@ -caddr_t
@ -kern_vfs_bio_buffer_alloc(caddr_t v, long physmem_est)
@ +void *
@ +vfs_bio_alloc(void *va)

The API name was verbose and bogus.  The prefix for this subsystem is
vfs_bio_, not kern_vfs_bio_.  This is a generic allocation routine.
It always allocated swbufs and has expanded to do more allocations.

@  {
@ -/*
@ - * physmem_est is in pages.  Convert it to kilobytes (assumes
@ - * PAGE_SIZE is = 1K)
@ - */
@ -physmem_est = physmem_est * (PAGE_SIZE / 1024);

I use the global physmem.  This change may be too i386-specific. In
with 8-16 RAM, a significant amount of RAM may be eaten by the kernel
image or perhaps just unavailable to the kernel.  Now memories are
larger and the amount eaten is relatively insignificant (since we are
only using a small fraction of physmem, only the relative error matters).

@ +int factor, memsize;
@ @  /*
@ - * The nominal buffer size (and minimum KVA allocation) is 
BKVASIZE.
@ - * For the first 64MB of ram

Re: svn commit: r251282 - head/sys/kern

[Alfred Perlstein]

Hey Konstaintin, shouldn't this be scaled against the actual amount of 
KVA we have instead of an arbitrary limit?


-Alfred

On 6/2/13 9:16 PM, Konstantin Belousov wrote:

Author: kib
Date: Mon Jun  3 04:16:48 2013
New Revision: 251282
URL: http://svnweb.freebsd.org/changeset/base/251282

Log:
   When auto-sizing the buffer cache, limit the amount of physical memory
   used as the estimation of size, to 32GB.  This provides around 100K of
   buffer headers and corresponding KVA for buffer map at the peak.
   Sizing the cache larger is not useful, also resulting in the wasting
   and exhausting of KVA for large machines.
   
   Reported and tested by:	bdrewery

   Sponsored by:The FreeBSD Foundation

Modified:
   head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Mon Jun  3 04:11:42 2013(r251281)
+++ head/sys/kern/vfs_bio.c Mon Jun  3 04:16:48 2013(r251282)
@@ -560,7 +560,8 @@ kern_vfs_bio_buffer_alloc(caddr_t v, lon
nbuf += min((physmem_est - 4096) / factor,
65536 / factor);
if (physmem_est  65536)
-   nbuf += (physmem_est - 65536) * 2 / (factor * 5);
+   nbuf += min((physmem_est - 65536) * 2 / (factor * 5),
+   32 * 1024 * 1024 / (factor * 5));
  
  		if (maxbcache  nbuf  maxbcache / BKVASIZE)

nbuf = maxbcache / BKVASIZE;



___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r250411 - in head/sys: conf kern sys

[Alfred Perlstein]

Can we just admit to ourselves that tweaks to debugging macros/printing 
and WITNESS are our kernel developer's bikeshed zone and get over the 
fact that people's needs may diverge and changing non-default behavior 
in non-critical paths is not going to be the death of the kernel as we 
know it?


I could certainly believe that this sort of thing needs long and 
thorough discussion if it wasn't the equivalent of style tweaks to manpages.


Let's leave the long and lengthy discussions to things that matter such 
as standards compliance, ABI, API and really cool performance and 
stability stuff.


-Alfred




On 5/11/13 5:43 PM, Jeff Roberson wrote:

On Thu, 9 May 2013, Marcel Moolenaar wrote:


Author: marcel
Date: Thu May  9 16:28:18 2013
New Revision: 250411
URL: http://svnweb.freebsd.org/changeset/base/250411

Log:
 Add option WITNESS_NO_VNODE to suppress printing LORs between VNODE
 locks. To support this, VNODE locks are created with the LK_IS_VNODE
 flag. This flag is propagated down using the LO_IS_VNODE flag.

 Note that WITNESS still records the LOR. Only the printing and the
 optional entering into the kernel debugger is bypassed with the
 WITNESS_NO_VNODE option.


I'm replying to the original commit because the resulting thread got 
way out of hand.  We need to all take a deep breath and take a 
pragmatic approach to solving the problem at hand.


Let me first say I understand the utility here as this is also coming 
up in my organization.  Test, and users, do not want to see erroneous 
warning messages.  I understand that.  Let's find a solution.


Secondly, I think this project has grown too far for us to commit 
changes like this without some focused discussion.  We need to be more 
mindful of the size of the impact and the number of people who are 
interested in a particular area.  I'm not picking on you Marcel 
because this sort of thing has been coming up lately and we have all 
been guilty of it from time to time.  There are more companies and 
individuals than ever trying to push work into the repository and 
we're having some growing pains.


I am intimately familiar with the problems that lead to these 
erroneous witness messages as I have tracked down many of them and am 
even responsible for the code that generates them in some cases.  Let 
me first outline a handful of generic problems.  The root cause is 
that witness can not determine the real order between two locks due to 
relationships too complex to describe with a pair of strings.


One example, which has been brought up, is the hierarchical nature of 
vnode locks.  This impacts vnodes within one filesystem but it also 
involves vnodes between two different filesystems as you cross mount 
points.  We can construct perfectly valid and deadlock free chains of 
mount points that have two different filesystem types in different 
orders which will LOR at the boundaries.  We already skip duplicates 
to avoid this problem within each filesystem.  We need to skip 
cross-filesystem duplicates, most desirably at the few specific places 
where this happens. This problem comes up especially for devfs because 
we lock devvps while file vnodes are locked but we lock devfs 
directories after the rootfs lock when crossing mountpoints in lookup.


A second example, is locks of a fundamentally different type that have 
a complex ordering relationship.  For example, a vnode lock may be 
acquired after a buf lock belonging to the parent's directory block.  
A cg buf lock may be acquired after any file buf lock.  Here we want 
to ignore interactions between these two specific types at this 
particular location but not others as they may be unsafe.


The third example, is a complex locking pattern with shared locks as 
presented by dirhash.  We are seeing a similar pattern develop in the 
vm where we are going to use an exclusive object lock to protect pages 
or a shared object lock + a page lock.  The semantics only get more 
complex as we push for more scalability. I expect to see more of these 
patterns develop.


None of these problems can be solved with names alone.  So far we've 
just lived with the warnings and we're no longer willing to accept 
that. What we need is a solution that blesses the specific instances 
and the specific lock classes involved without silencing legitimate 
warnings that may only occur after new code is added. For example, it 
may be safe to add a sx lock around some vnode code but you may not 
notice that you LOR if you silence all witness warnings related to the 
vnode lock site.


I believe that the perfect solution would be a mechanism that could 
teach witness about and enforce these specific relationships.  
However, that may be computationally prohibitive and too complex to 
code.  A more reasonable option would be to bless the specific 
relationships at the specific call sites. Turning all witness off at 
particular sites or with particular types renders important 
infrastructure useless for very large 

Re: svn commit: r250411 - in head/sys: conf kern sys

[Alfred Perlstein]

On 5/10/13 8:46 AM, Marcel Moolenaar wrote:


And all I did is to allow someone (= Juniper) to not print the LOR
for this well-known and mostly ignored case that is impacting our
ability to keep witness enabled. And the reason I had to do that is
that this is a long-standing LOR that isn't being addressed. The
FreeBSD community apparently has settled on just ignoring it.



This whole issue about not allowing developers to mute warnings stems 
from some FreeBSD developers inability to imagine that they are not 
locus of architecture at an organization.


We really need to gain the ability to put ourselves in the shoes of 
someone that is just one of MANY people working on a product, a product 
that can choose its platform, FreeBSD, or if FreeBSD fails, then Linux 
or whatever works.


Allowing people to customize and/or mute these error messages, when they 
are often superfluous is good.  It allows the team to work on the parts 
that they need to work on and ignore the noise from other broken parts 
of FreeBSD that will eventually be fixed by the community.


If FreeBSD is supposed to be for the community, then why does it have 
portions (WITNESS/INVARIANTS/etc?) that are not for the community?


-Alfred


___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org

Re: svn commit: r250411 - in head/sys: conf kern sys

[Alfred Perlstein]

On 5/9/13 3:13 PM, Attilio Rao wrote:

On Thu, May 9, 2013 at 10:56 PM, Marcel Moolenaar mar...@xcllnt.net wrote:

On May 9, 2013, at 9:46 AM, Attilio Rao atti...@freebsd.org wrote:


On Thu, May 9, 2013 at 6:28 PM, Marcel Moolenaar mar...@freebsd.org wrote:

Author: marcel
Date: Thu May  9 16:28:18 2013
New Revision: 250411
URL: http://svnweb.freebsd.org/changeset/base/250411

Log:
  Add option WITNESS_NO_VNODE to suppress printing LORs between VNODE
  locks. To support this, VNODE locks are created with the LK_IS_VNODE
  flag. This flag is propagated down using the LO_IS_VNODE flag.

  Note that WITNESS still records the LOR. Only the printing and the
  optional entering into the kernel debugger is bypassed with the
  WITNESS_NO_VNODE option.

This is the wrong way to deal with such problem and I avoided to do
something like that on purpose.

I disagree. We have known LOR messages between VNODE locks that
pollute the console and so far we haven't fixed the root cause
in some form or shape. Silencing this known case is good to
maximize the attention LORs need to be given while still have
witness involved to catch locking problems with vnodes that are
of a different nature.


The way to fix this is to implement LK_NOWITNESS on a per-lock basis
into lockmgr, propagate the same concept to the vn_lock() (which
should be basically done automatically) and finally identify the
false-positive case and commit for them explicitely LK_NOWITNESS on a
per-call basis, explaining in detail why the single case reported is a
false-positive.

This is worse. You want witness involved.


Please revert this patch asap.

This change does not inhibit people from fixing the problem at the
root cause, and in the mean time maximize witness' effectiveness.
Calling for a backout is unwarranted and unnecessarily aggressive.

I completely disagree with the whole content of your e-mail.
Thanks for disrupting a useful tool along with other commits which
happened in the past by other people about invariants effectiveness.



This should be taken offline.  Marcel has some needs which without such 
a change are hard to manage I encourage you to assist him and meeting 
half-way on this as it will greatly help the project.


Please discuss this offline a bit so you can see where each are coming from.

If you would like to cc me about this I can help mediate and explain 
this pragmatic approach to assertions.


Will you both be at BSDCan?  That would be even better.

-Alfred
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org