Re: svn commit: r287606 - head/sys/kern

[Alfred Perlstein]

64k hard is too low a number for large memory machines.

-Alfred

On 9/10/15 9:18 AM, Adrian Chadd wrote:

On 10 September 2015 at 09:04, Warner Losh  wrote:


On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:

On 10 September 2015 at 04:05, Adrian Chadd  wrote:

Author: adrian
Date: Thu Sep 10 04:05:58 2015
New Revision: 287606
URL: https://svnweb.freebsd.org/changeset/base/287606

Log:
   Also make kern.maxfilesperproc a boot time tunable.
...
   TODO:

Also "we" should
* Submit patches upstream or to the ports tree to use closefrom


I thought the consensus was that we'd fix things to have fewer FDs
by default, but instead allow individual processes to raise it via the
usual methods.

I'm looking at how to do this in a somewhat sensible fashion. Right
now we just have openfiles=unlimited; in /etc/login.conf which seems a
little odd. I don't know yet if that affects the default set that
services started via /etc/rc get - init gets the whole default
maxfilesperproc and stuff seems to inherit from that unless told
otherwise.

I think the more sensible default would be:

* set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
* root can always override its settings up to kern.maxfilesperproc;
* modify /etc/rc to set some default rlimits as appropriate;
* introduce configuration options ({daemon_rlimit_XXX}?) in
/etc/rc.conf that lets someone override what the default rlimits
should be for a given process,, as (and I'm not making this up) if you
run 'service XXX restart' from a root login you get the rlimits from
the shell, which may differ from the system startup.

That way we can setup various services to have higher openfile limits
via /etc/rc.conf entries for those services rather than having to hack
each startup script. It also means that no matter what is running
'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.

Thoughts?


-adrian



___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Adrian Chadd]

On 11 September 2015 at 13:46, Alfred Perlstein  wrote:
> 64k hard is too low a number for large memory machines.

Root can always bump it up all the way to kern.maxfilesperproc.

I'm also a big fan of having the description of config of service
stuff be in /etc/rc.conf, rather than splattered around the place. So
I also like the idea of _rlimit_openfiles="" so it can be
clearly overridden for services that require it.

I'm open to other suggestions!



-adrian

> -Alfred
>
>
> On 9/10/15 9:18 AM, Adrian Chadd wrote:
>>
>> On 10 September 2015 at 09:04, Warner Losh  wrote:
>>>
>>>
>>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:

 On 10 September 2015 at 04:05, Adrian Chadd  wrote:
>
> Author: adrian
> Date: Thu Sep 10 04:05:58 2015
> New Revision: 287606
> URL: https://svnweb.freebsd.org/changeset/base/287606
>
> Log:
>Also make kern.maxfilesperproc a boot time tunable.
> ...
>TODO:

 Also "we" should
 * Submit patches upstream or to the ports tree to use closefrom
>>>
>>>
>>> I thought the consensus was that we'd fix things to have fewer FDs
>>> by default, but instead allow individual processes to raise it via the
>>> usual methods.
>>
>> I'm looking at how to do this in a somewhat sensible fashion. Right
>> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>> little odd. I don't know yet if that affects the default set that
>> services started via /etc/rc get - init gets the whole default
>> maxfilesperproc and stuff seems to inherit from that unless told
>> otherwise.
>>
>> I think the more sensible default would be:
>>
>> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
>> * root can always override its settings up to kern.maxfilesperproc;
>> * modify /etc/rc to set some default rlimits as appropriate;
>> * introduce configuration options ({daemon_rlimit_XXX}?) in
>> /etc/rc.conf that lets someone override what the default rlimits
>> should be for a given process,, as (and I'm not making this up) if you
>> run 'service XXX restart' from a root login you get the rlimits from
>> the shell, which may differ from the system startup.
>>
>> That way we can setup various services to have higher openfile limits
>> via /etc/rc.conf entries for those services rather than having to hack
>> each startup script. It also means that no matter what is running
>> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.
>>
>> Thoughts?
>>
>>
>> -adrian
>>
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Adrian Chadd]

Hi,

That would be nice. Maybe after this particular bug is fixed though.


-a


On 11 September 2015 at 09:47, Slawa Olhovchenkov  wrote:
> On Thu, Sep 10, 2015 at 03:11:39PM -0700, Adrian Chadd wrote:
>
>> Hi,
>>
>> Fixed a couple of bugs, and:
>>
>> https://reviews.freebsd.org/D3630
>
> May be you also can be sanitize and standartize enviroment for
> services started from init and from sheell?
>
> Currently some enviroment will be different if service started manual,
> from interactive shell and if all worked in this cases something may
> be broken after reboot. Rebooting for debug is not cute.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Alfred Perlstein]

The idea is sane defaults. Surely 256k makes sense for a machine with that much 
memory. 

Sent from my iPhone

> On Sep 11, 2015, at 2:02 PM, Adrian Chadd  wrote:
> 
>> On 11 September 2015 at 13:46, Alfred Perlstein  wrote:
>> 64k hard is too low a number for large memory machines.
> 
> Root can always bump it up all the way to kern.maxfilesperproc.
> 
> I'm also a big fan of having the description of config of service
> stuff be in /etc/rc.conf, rather than splattered around the place. So
> I also like the idea of _rlimit_openfiles="" so it can be
> clearly overridden for services that require it.
> 
> I'm open to other suggestions!
> 
> 
> 
> -adrian
> 
>> -Alfred
>> 
>> 
>>> On 9/10/15 9:18 AM, Adrian Chadd wrote:
>>> 
 On 10 September 2015 at 09:04, Warner Losh  wrote:
 
 
> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:
> 
>> On 10 September 2015 at 04:05, Adrian Chadd  wrote:
>> 
>> Author: adrian
>> Date: Thu Sep 10 04:05:58 2015
>> New Revision: 287606
>> URL: https://svnweb.freebsd.org/changeset/base/287606
>> 
>> Log:
>>   Also make kern.maxfilesperproc a boot time tunable.
>> ...
>>   TODO:
> 
> Also "we" should
> * Submit patches upstream or to the ports tree to use closefrom
 
 
 I thought the consensus was that we'd fix things to have fewer FDs
 by default, but instead allow individual processes to raise it via the
 usual methods.
>>> 
>>> I'm looking at how to do this in a somewhat sensible fashion. Right
>>> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>>> little odd. I don't know yet if that affects the default set that
>>> services started via /etc/rc get - init gets the whole default
>>> maxfilesperproc and stuff seems to inherit from that unless told
>>> otherwise.
>>> 
>>> I think the more sensible default would be:
>>> 
>>> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
>>> * root can always override its settings up to kern.maxfilesperproc;
>>> * modify /etc/rc to set some default rlimits as appropriate;
>>> * introduce configuration options ({daemon_rlimit_XXX}?) in
>>> /etc/rc.conf that lets someone override what the default rlimits
>>> should be for a given process,, as (and I'm not making this up) if you
>>> run 'service XXX restart' from a root login you get the rlimits from
>>> the shell, which may differ from the system startup.
>>> 
>>> That way we can setup various services to have higher openfile limits
>>> via /etc/rc.conf entries for those services rather than having to hack
>>> each startup script. It also means that no matter what is running
>>> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.
>>> 
>>> Thoughts?
>>> 
>>> 
>>> -adrian
> 
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Slawa Olhovchenkov]

On Thu, Sep 10, 2015 at 03:11:39PM -0700, Adrian Chadd wrote:

> Hi,
> 
> Fixed a couple of bugs, and:
> 
> https://reviews.freebsd.org/D3630

May be you also can be sanitize and standartize enviroment for
services started from init and from sheell?

Currently some enviroment will be different if service started manual,
from interactive shell and if all worked in this cases something may
be broken after reboot. Rebooting for debug is not cute.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Ed Maste]

On 10 September 2015 at 04:05, Adrian Chadd  wrote:
> Author: adrian
> Date: Thu Sep 10 04:05:58 2015
> New Revision: 287606
> URL: https://svnweb.freebsd.org/changeset/base/287606
>
> Log:
>   Also make kern.maxfilesperproc a boot time tunable.
> ...
>   TODO:

Also "we" should
* Submit patches upstream or to the ports tree to use closefrom.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Warner Losh]

On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:

> On 10 September 2015 at 04:05, Adrian Chadd  wrote:
> > Author: adrian
> > Date: Thu Sep 10 04:05:58 2015
> > New Revision: 287606
> > URL: https://svnweb.freebsd.org/changeset/base/287606
> >
> > Log:
> >   Also make kern.maxfilesperproc a boot time tunable.
> > ...
> >   TODO:
>
> Also "we" should
> * Submit patches upstream or to the ports tree to use closefrom
>

I thought the consensus was that we'd fix things to have fewer FDs
by default, but instead allow individual processes to raise it via the
usual methods.

Warner
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Adrian Chadd]

On 10 September 2015 at 09:04, Warner Losh  wrote:
>
>
> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:
>>
>> On 10 September 2015 at 04:05, Adrian Chadd  wrote:
>> > Author: adrian
>> > Date: Thu Sep 10 04:05:58 2015
>> > New Revision: 287606
>> > URL: https://svnweb.freebsd.org/changeset/base/287606
>> >
>> > Log:
>> >   Also make kern.maxfilesperproc a boot time tunable.
>> > ...
>> >   TODO:
>>
>> Also "we" should
>> * Submit patches upstream or to the ports tree to use closefrom
>
>
> I thought the consensus was that we'd fix things to have fewer FDs
> by default, but instead allow individual processes to raise it via the
> usual methods.

I'm looking at how to do this in a somewhat sensible fashion. Right
now we just have openfiles=unlimited; in /etc/login.conf which seems a
little odd. I don't know yet if that affects the default set that
services started via /etc/rc get - init gets the whole default
maxfilesperproc and stuff seems to inherit from that unless told
otherwise.

I think the more sensible default would be:

* set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
* root can always override its settings up to kern.maxfilesperproc;
* modify /etc/rc to set some default rlimits as appropriate;
* introduce configuration options ({daemon_rlimit_XXX}?) in
/etc/rc.conf that lets someone override what the default rlimits
should be for a given process,, as (and I'm not making this up) if you
run 'service XXX restart' from a root login you get the rlimits from
the shell, which may differ from the system startup.

That way we can setup various services to have higher openfile limits
via /etc/rc.conf entries for those services rather than having to hack
each startup script. It also means that no matter what is running
'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.

Thoughts?


-adrian
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[John-Mark Gurney]

Adrian Chadd wrote this message on Thu, Sep 10, 2015 at 09:18 -0700:
> On 10 September 2015 at 09:04, Warner Losh  wrote:
> >
> >
> > On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:
> >>
> >> On 10 September 2015 at 04:05, Adrian Chadd  wrote:
> >> > Author: adrian
> >> > Date: Thu Sep 10 04:05:58 2015
> >> > New Revision: 287606
> >> > URL: https://svnweb.freebsd.org/changeset/base/287606
> >> >
> >> > Log:
> >> >   Also make kern.maxfilesperproc a boot time tunable.
> >> > ...
> >> >   TODO:
> >>
> >> Also "we" should
> >> * Submit patches upstream or to the ports tree to use closefrom
> >
> >
> > I thought the consensus was that we'd fix things to have fewer FDs
> > by default, but instead allow individual processes to raise it via the
> > usual methods.
> 
> I'm looking at how to do this in a somewhat sensible fashion. Right
> now we just have openfiles=unlimited; in /etc/login.conf which seems a
> little odd. I don't know yet if that affects the default set that
> services started via /etc/rc get - init gets the whole default
> maxfilesperproc and stuff seems to inherit from that unless told
> otherwise.
> 
> I think the more sensible default would be:
> 
> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
> * root can always override its settings up to kern.maxfilesperproc;
> * modify /etc/rc to set some default rlimits as appropriate;

We should probably just use the daemon class from login.conf... Do we
have a program that will set the current limits to a specified class?

> * introduce configuration options ({daemon_rlimit_XXX}?) in
> /etc/rc.conf that lets someone override what the default rlimits
> should be for a given process,, as (and I'm not making this up) if you
> run 'service XXX restart' from a root login you get the rlimits from
> the shell, which may differ from the system startup.

Why not daemon_login_class w/ the above?

> That way we can setup various services to have higher openfile limits
> via /etc/rc.conf entries for those services rather than having to hack
> each startup script. It also means that no matter what is running
> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.

Then service would just use the above program to get sane defaults...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[Allan Jude]

On 2015-09-10 13:53, John-Mark Gurney wrote:
> Adrian Chadd wrote this message on Thu, Sep 10, 2015 at 09:18 -0700:
>> On 10 September 2015 at 09:04, Warner Losh  wrote:
>>>
>>>
>>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:

 On 10 September 2015 at 04:05, Adrian Chadd  wrote:
> Author: adrian
> Date: Thu Sep 10 04:05:58 2015
> New Revision: 287606
> URL: https://svnweb.freebsd.org/changeset/base/287606
>
> Log:
>   Also make kern.maxfilesperproc a boot time tunable.
> ...
>   TODO:

 Also "we" should
 * Submit patches upstream or to the ports tree to use closefrom
>>>
>>>
>>> I thought the consensus was that we'd fix things to have fewer FDs
>>> by default, but instead allow individual processes to raise it via the
>>> usual methods.
>>
>> I'm looking at how to do this in a somewhat sensible fashion. Right
>> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>> little odd. I don't know yet if that affects the default set that
>> services started via /etc/rc get - init gets the whole default
>> maxfilesperproc and stuff seems to inherit from that unless told
>> otherwise.
>>
>> I think the more sensible default would be:
>>
>> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
>> * root can always override its settings up to kern.maxfilesperproc;
>> * modify /etc/rc to set some default rlimits as appropriate;
> 
> We should probably just use the daemon class from login.conf... Do we
> have a program that will set the current limits to a specified class?
> 

/sbin/init applies the 'daemon' class to /etc/rc when it starts it

And so all rc.d scripts inherit it.

the issue is that it does not apply to services started manually, or via
the service(8) command.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=161401 (reported in
2011, working on getting this in this weekend)

>> * introduce configuration options ({daemon_rlimit_XXX}?) in
>> /etc/rc.conf that lets someone override what the default rlimits
>> should be for a given process,, as (and I'm not making this up) if you
>> run 'service XXX restart' from a root login you get the rlimits from
>> the shell, which may differ from the system startup.
> 
> Why not daemon_login_class w/ the above?

This is what I was thinking, as it also jives with my work to convert
login.conf to UCL and improve it overall. This would allow services to
install their own login class as part of the package, into
/usr/local/etc/login.conf.d/pkgname and use that by default, but the
user can always specify pkgname_login_class (or just _class maybe?) as
some other value.

> 
>> That way we can setup various services to have higher openfile limits
>> via /etc/rc.conf entries for those services rather than having to hack
>> each startup script. It also means that no matter what is running
>> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.
> 
> Then service would just use the above program to get sane defaults...
> 


-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: svn commit: r287606 - head/sys/kern

[Adrian Chadd]

Hi,

Fixed a couple of bugs, and:

https://reviews.freebsd.org/D3630




-adrian


On 10 September 2015 at 15:02, Adrian Chadd  wrote:
> I'd love for rc.subr to grow the ability to set per-daemon cpuset,
> class, environment, etc. We have some of that in the rc script
> already.
>
> What I have so far for local hacking is this, which at least gets the
> default login class bits and runs things as user daemon.
> Yes, there are issues with inheriting the environment and other things
> from the callee - I think that's a separate issue to solve.
>
> Thanks,
>
>
> -a
>
> adrian@hulk:~/work/freebsd/head/src % svn diff etc
>
> Index: etc/login.conf
> ===
> --- etc/login.conf (revision 28758)
> +++ etc/login.conf (working copy)
> @@ -36,7 +36,8 @@
>   :memoryuse=unlimited:\
>   :filesize=unlimited:\
>   :coredumpsize=unlimited:\
> - :openfiles=unlimited:\
> + :openfiles-cur=4096:\
> + :openfiles-max=65536:\
>   :maxproc=unlimited:\
>   :sbsize=unlimited:\
>   :vmemoryuse=unlimited:\
> @@ -61,6 +62,8 @@
>   :tc=default:
>  daemon:\
>   :memorylocked=128M:\
> + :openfiles-cur=32768:\
> + :openfiles-max=65536:\
>   :tc=default:
>  news:\
>   :tc=default:
> Index: etc/rc.subr
> ===
> --- etc/rc.subr (revision 287580)
> +++ etc/rc.subr (working copy)
> @@ -768,6 +768,8 @@
>  #
>  # ${name}_prepend n Command added before ${command}.
>  #
> +# ${name}_login_class n Login class to use, else "daemon".
> +#
>  # ${rc_arg}_cmd n If set, use this as the method when invoked;
>  # Otherwise, use default command (see below)
>  #
>
> @@ -942,8 +944,13 @@
>   _nice=\$${name}_nice _user=\$${name}_user \
>   _group=\$${name}_group _groups=\$${name}_groups \
>   _fib=\$${name}_fib _env=\$${name}_env \
> - _prepend=\$${name}_prepend
> + _prepend=\$${name}_prepend _login_class=\$${name}_login_class
>
> + # Default to 'daemon' if no login class is provided
> + if [ -n "$_login_class" ]; then
> + _login_class="daemon"
> + fi
> +
>   if [ -n "$_user" ]; then # unset $_user if running as that user
>   if [ "$_user" = "$(eval $IDCMD)" ]; then
>   unset _user
> @@ -1050,6 +1057,9 @@
>   fi
>   fi
>
> + # Prepend default limits
> + _doit="limits -C $_login_class $_doit"
> +
>   # run the full command
>   #
>   if ! _run_rc_doit "$_doit"; then
>
> On 10 September 2015 at 14:14, John-Mark Gurney  wrote:
>> Eric van Gyzen wrote this message on Thu, Sep 10, 2015 at 14:56 -0500:
>>> On 09/10/2015 12:53, John-Mark Gurney wrote:
>>> > Adrian Chadd wrote this message on Thu, Sep 10, 2015 at 09:18 -0700:
>>> >> On 10 September 2015 at 09:04, Warner Losh  wrote:
>>> >>>
>>> >>>
>>> >>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:
>>> 
>>>  On 10 September 2015 at 04:05, Adrian Chadd  wrote:
>>> > Author: adrian
>>> > Date: Thu Sep 10 04:05:58 2015
>>> > New Revision: 287606
>>> > URL: https://svnweb.freebsd.org/changeset/base/287606
>>> >
>>> > Log:
>>> >   Also make kern.maxfilesperproc a boot time tunable.
>>> > ...
>>> >   TODO:
>>> 
>>>  Also "we" should
>>>  * Submit patches upstream or to the ports tree to use closefrom
>>> >>>
>>> >>>
>>> >>> I thought the consensus was that we'd fix things to have fewer FDs
>>> >>> by default, but instead allow individual processes to raise it via the
>>> >>> usual methods.
>>>
>>> We could--and should--do both, because they're both good ideas.
>>>
>>> >> I'm looking at how to do this in a somewhat sensible fashion. Right
>>> >> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>>> >> little odd. I don't know yet if that affects the default set that
>>> >> services started via /etc/rc get - init gets the whole default
>>> >> maxfilesperproc and stuff seems to inherit from that unless told
>>> >> otherwise.
>>> >>
>>> >> I think the more sensible default would be:
>>> >>
>>> >> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k 
>>> >> hard;
>>> >> * root can always override its settings up to kern.maxfilesperproc;
>>> >> * modify /etc/rc to set some default rlimits as appropriate;
>>> >
>>> > We should probably just use the daemon class from login.conf... Do we
>>> > have a program that will set the current limits to a specified class?
>>>
>>> See limits(1).  The apache rc.d script uses it, along with some related
>>> rc.conf variables.
>>
>> So, one issue w/ limits is that it only does the limits side of
>> things, not environment or cpusets...  see:
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=161401
>>
>> limits doesn't address PATH and other environment variables...
>>
>> We should have rc.subr setup the environment completely when executing
>> the daemon/scripts instead of depending upon any of this..
>>
>> It turns out that init doesn't setup the environment 

Re: svn commit: r287606 - head/sys/kern

[Adrian Chadd]

I'd love for rc.subr to grow the ability to set per-daemon cpuset,
class, environment, etc. We have some of that in the rc script
already.

What I have so far for local hacking is this, which at least gets the
default login class bits and runs things as user daemon.
Yes, there are issues with inheriting the environment and other things
from the callee - I think that's a separate issue to solve.

Thanks,


-a

adrian@hulk:~/work/freebsd/head/src % svn diff etc

Index: etc/login.conf
===
--- etc/login.conf (revision 28758)
+++ etc/login.conf (working copy)
@@ -36,7 +36,8 @@
  :memoryuse=unlimited:\
  :filesize=unlimited:\
  :coredumpsize=unlimited:\
- :openfiles=unlimited:\
+ :openfiles-cur=4096:\
+ :openfiles-max=65536:\
  :maxproc=unlimited:\
  :sbsize=unlimited:\
  :vmemoryuse=unlimited:\
@@ -61,6 +62,8 @@
  :tc=default:
 daemon:\
  :memorylocked=128M:\
+ :openfiles-cur=32768:\
+ :openfiles-max=65536:\
  :tc=default:
 news:\
  :tc=default:
Index: etc/rc.subr
===
--- etc/rc.subr (revision 287580)
+++ etc/rc.subr (working copy)
@@ -768,6 +768,8 @@
 #
 # ${name}_prepend n Command added before ${command}.
 #
+# ${name}_login_class n Login class to use, else "daemon".
+#
 # ${rc_arg}_cmd n If set, use this as the method when invoked;
 # Otherwise, use default command (see below)
 #

@@ -942,8 +944,13 @@
  _nice=\$${name}_nice _user=\$${name}_user \
  _group=\$${name}_group _groups=\$${name}_groups \
  _fib=\$${name}_fib _env=\$${name}_env \
- _prepend=\$${name}_prepend
+ _prepend=\$${name}_prepend _login_class=\$${name}_login_class

+ # Default to 'daemon' if no login class is provided
+ if [ -n "$_login_class" ]; then
+ _login_class="daemon"
+ fi
+
  if [ -n "$_user" ]; then # unset $_user if running as that user
  if [ "$_user" = "$(eval $IDCMD)" ]; then
  unset _user
@@ -1050,6 +1057,9 @@
  fi
  fi

+ # Prepend default limits
+ _doit="limits -C $_login_class $_doit"
+
  # run the full command
  #
  if ! _run_rc_doit "$_doit"; then

On 10 September 2015 at 14:14, John-Mark Gurney  wrote:
> Eric van Gyzen wrote this message on Thu, Sep 10, 2015 at 14:56 -0500:
>> On 09/10/2015 12:53, John-Mark Gurney wrote:
>> > Adrian Chadd wrote this message on Thu, Sep 10, 2015 at 09:18 -0700:
>> >> On 10 September 2015 at 09:04, Warner Losh  wrote:
>> >>>
>> >>>
>> >>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:
>> 
>>  On 10 September 2015 at 04:05, Adrian Chadd  wrote:
>> > Author: adrian
>> > Date: Thu Sep 10 04:05:58 2015
>> > New Revision: 287606
>> > URL: https://svnweb.freebsd.org/changeset/base/287606
>> >
>> > Log:
>> >   Also make kern.maxfilesperproc a boot time tunable.
>> > ...
>> >   TODO:
>> 
>>  Also "we" should
>>  * Submit patches upstream or to the ports tree to use closefrom
>> >>>
>> >>>
>> >>> I thought the consensus was that we'd fix things to have fewer FDs
>> >>> by default, but instead allow individual processes to raise it via the
>> >>> usual methods.
>>
>> We could--and should--do both, because they're both good ideas.
>>
>> >> I'm looking at how to do this in a somewhat sensible fashion. Right
>> >> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>> >> little odd. I don't know yet if that affects the default set that
>> >> services started via /etc/rc get - init gets the whole default
>> >> maxfilesperproc and stuff seems to inherit from that unless told
>> >> otherwise.
>> >>
>> >> I think the more sensible default would be:
>> >>
>> >> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
>> >> * root can always override its settings up to kern.maxfilesperproc;
>> >> * modify /etc/rc to set some default rlimits as appropriate;
>> >
>> > We should probably just use the daemon class from login.conf... Do we
>> > have a program that will set the current limits to a specified class?
>>
>> See limits(1).  The apache rc.d script uses it, along with some related
>> rc.conf variables.
>
> So, one issue w/ limits is that it only does the limits side of
> things, not environment or cpusets...  see:
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=161401
>
> limits doesn't address PATH and other environment variables...
>
> We should have rc.subr setup the environment completely when executing
> the daemon/scripts instead of depending upon any of this..
>
> It turns out that init doesn't setup the environment vars provided by
> login.config either...
>
>> >> * introduce configuration options ({daemon_rlimit_XXX}?) in
>> >> /etc/rc.conf that lets someone override what the default rlimits
>> >> should be for a given process,, as (and I'm not making this up) if you
>> >> run 'service XXX restart' from a root login you get the rlimits from
>> >> the shell, which may differ from 

Re: svn commit: r287606 - head/sys/kern

[Eric van Gyzen]

On 09/10/2015 12:53, John-Mark Gurney wrote:
> Adrian Chadd wrote this message on Thu, Sep 10, 2015 at 09:18 -0700:
>> On 10 September 2015 at 09:04, Warner Losh  wrote:
>>>
>>>
>>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:

 On 10 September 2015 at 04:05, Adrian Chadd  wrote:
> Author: adrian
> Date: Thu Sep 10 04:05:58 2015
> New Revision: 287606
> URL: https://svnweb.freebsd.org/changeset/base/287606
>
> Log:
>   Also make kern.maxfilesperproc a boot time tunable.
> ...
>   TODO:

 Also "we" should
 * Submit patches upstream or to the ports tree to use closefrom
>>>
>>>
>>> I thought the consensus was that we'd fix things to have fewer FDs
>>> by default, but instead allow individual processes to raise it via the
>>> usual methods.

We could--and should--do both, because they're both good ideas.

>> I'm looking at how to do this in a somewhat sensible fashion. Right
>> now we just have openfiles=unlimited; in /etc/login.conf which seems a
>> little odd. I don't know yet if that affects the default set that
>> services started via /etc/rc get - init gets the whole default
>> maxfilesperproc and stuff seems to inherit from that unless told
>> otherwise.
>>
>> I think the more sensible default would be:
>>
>> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
>> * root can always override its settings up to kern.maxfilesperproc;
>> * modify /etc/rc to set some default rlimits as appropriate;
> 
> We should probably just use the daemon class from login.conf... Do we
> have a program that will set the current limits to a specified class?

See limits(1).  The apache rc.d script uses it, along with some related
rc.conf variables.

>> * introduce configuration options ({daemon_rlimit_XXX}?) in
>> /etc/rc.conf that lets someone override what the default rlimits
>> should be for a given process,, as (and I'm not making this up) if you
>> run 'service XXX restart' from a root login you get the rlimits from
>> the shell, which may differ from the system startup.
> 
> Why not daemon_login_class w/ the above?
> 
>> That way we can setup various services to have higher openfile limits
>> via /etc/rc.conf entries for those services rather than having to hack
>> each startup script. It also means that no matter what is running
>> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.
> 
> Then service would just use the above program to get sane defaults...
> 
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r287606 - head/sys/kern

[John-Mark Gurney]

Eric van Gyzen wrote this message on Thu, Sep 10, 2015 at 14:56 -0500:
> On 09/10/2015 12:53, John-Mark Gurney wrote:
> > Adrian Chadd wrote this message on Thu, Sep 10, 2015 at 09:18 -0700:
> >> On 10 September 2015 at 09:04, Warner Losh  wrote:
> >>>
> >>>
> >>> On Thu, Sep 10, 2015 at 9:53 AM, Ed Maste  wrote:
> 
>  On 10 September 2015 at 04:05, Adrian Chadd  wrote:
> > Author: adrian
> > Date: Thu Sep 10 04:05:58 2015
> > New Revision: 287606
> > URL: https://svnweb.freebsd.org/changeset/base/287606
> >
> > Log:
> >   Also make kern.maxfilesperproc a boot time tunable.
> > ...
> >   TODO:
> 
>  Also "we" should
>  * Submit patches upstream or to the ports tree to use closefrom
> >>>
> >>>
> >>> I thought the consensus was that we'd fix things to have fewer FDs
> >>> by default, but instead allow individual processes to raise it via the
> >>> usual methods.
> 
> We could--and should--do both, because they're both good ideas.
> 
> >> I'm looking at how to do this in a somewhat sensible fashion. Right
> >> now we just have openfiles=unlimited; in /etc/login.conf which seems a
> >> little odd. I don't know yet if that affects the default set that
> >> services started via /etc/rc get - init gets the whole default
> >> maxfilesperproc and stuff seems to inherit from that unless told
> >> otherwise.
> >>
> >> I think the more sensible default would be:
> >>
> >> * set  /etc/login.conf to some much lower values - say, 4k soft, 64k hard;
> >> * root can always override its settings up to kern.maxfilesperproc;
> >> * modify /etc/rc to set some default rlimits as appropriate;
> > 
> > We should probably just use the daemon class from login.conf... Do we
> > have a program that will set the current limits to a specified class?
> 
> See limits(1).  The apache rc.d script uses it, along with some related
> rc.conf variables.

So, one issue w/ limits is that it only does the limits side of
things, not environment or cpusets...  see:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=161401

limits doesn't address PATH and other environment variables...

We should have rc.subr setup the environment completely when executing
the daemon/scripts instead of depending upon any of this..

It turns out that init doesn't setup the environment vars provided by
login.config either...

> >> * introduce configuration options ({daemon_rlimit_XXX}?) in
> >> /etc/rc.conf that lets someone override what the default rlimits
> >> should be for a given process,, as (and I'm not making this up) if you
> >> run 'service XXX restart' from a root login you get the rlimits from
> >> the shell, which may differ from the system startup.
> > 
> > Why not daemon_login_class w/ the above?
> > 
> >> That way we can setup various services to have higher openfile limits
> >> via /etc/rc.conf entries for those services rather than having to hack
> >> each startup script. It also means that no matter what is running
> >> 'service XXX YYY' as root, you'll get the 'correct'(er) rlimits.
> > 
> > Then service would just use the above program to get sane defaults...

-- 
  John-Mark Gurney  Voice: +1 415 225 5579

 "All that I will do, has been done, All that I have, has not."
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r287606 - head/sys/kern

[Adrian Chadd]

Author: adrian
Date: Thu Sep 10 04:05:58 2015
New Revision: 287606
URL: https://svnweb.freebsd.org/changeset/base/287606

Log:
  Also make kern.maxfilesperproc a boot time tunable.
  
  Auto-tuning threshold discussions aside, it turns out that if you want
  to lower this on say, rather memory-packed machines, you either set maxusers
  or kern.maxfiles, or you set it in sysctl.  The former is a non-exact
  way to tune this; the latter doesn't actually affect anything in the
  startup scripts.
  
  This first occured because I wondered why the hell screen would take upwards
  of 10 seconds to spawn a new screen.  I then found python doing the same
  thing during fork/exec of child processes - it calls close() on each FD
  up to the current openfiles limit.  On a 1TB machine this is like, 26 million
  FDs per process.  Ugh.
  
  So:
  
  * This allows it to be set early in /boot/loader.conf;
  * It can be used to work around the ridiculous situation of
screen, python, etc doing a close() on potentially millions of FDs
even though you only have four open.
  
  Tested:
  
  * 4GB, 32GB, 64GB, 128GB, 384GB, 1TB systems with autotune, ensuring
screen and python forking doesn't result in some pretty hilariously
bad behaviour.
  
  TODO:
  
  * Note that the default login.conf sets openfiles-cur to unlimited,
effectively obeying kern.maxfilesperproc.  Perhaps we should fix
this.
  
  * .. and even if we do, we need to also ensure that daemons get
a soft limit of something reasonable and capped - they can request
more FDs themselves.
  
  MFC after:1 week
  Sponsored by: Norse Corp, Inc.

Modified:
  head/sys/kern/subr_param.c

Modified: head/sys/kern/subr_param.c
==
--- head/sys/kern/subr_param.c  Wed Sep  9 23:41:24 2015(r287605)
+++ head/sys/kern/subr_param.c  Thu Sep 10 04:05:58 2015(r287606)
@@ -265,7 +265,8 @@ init_param2(long physpages)
if (maxfiles > (physpages / 4))
maxfiles = physpages / 4;
maxfilesperproc = (maxfiles / 10) * 9;
-   
+   TUNABLE_INT_FETCH("kern.maxfilesperproc", );
+
/*
 * Cannot be changed after boot.
 */
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"