Re: Dansguardian not working after updating OBSD Current
On 2014-09-17, Kaya Saman kayasa...@gmail.com wrote: Ok, so this is just a quick follow up. Squid started dying too, checking the logs showed not enough file descriptors. After looking at both /etc/login.conf openfiles-cur and the sysctl kern.maxfiles limits which were set extremely high to begin with turns out that the: ulimit -n was only showing as 64. Changing that over to a value of 1 (overkill but better safe then sorry), Dansguardian managed to start and now both DG and Squid seem to be online and stable! I wonder if it has something to do with the: openfiles parameter in /etc/login.conf? What version were you running before? If it was pre-5.4, this is the most likely reason: http://www.openbsd.org/faq/upgrade54.html#login.conf It has bitten me on a number of occasions.
Re: Dansguardian not working after updating OBSD Current
Ok, so this is just a quick follow up. Squid started dying too, checking the logs showed not enough file descriptors. After looking at both /etc/login.conf openfiles-cur and the sysctl kern.maxfiles limits which were set extremely high to begin with turns out that the: ulimit -n was only showing as 64. Changing that over to a value of 1 (overkill but better safe then sorry), Dansguardian managed to start and now both DG and Squid seem to be online and stable! I wonder if it has something to do with the: openfiles parameter in /etc/login.conf? A response by Philip suggests this: http://openbsd.7691.n7.nabble.com/setting-resource-limits-login-conf-and-ulimit-td223656.html Just a quick observation, not sure if by design or other but ulimit doesn't seem to have a man page? man ulimit man: no entry for ulimit in the manual. man ulimit man: no entry for ulimit in the manual. Though I did find a copy here: http://ss64.com/bash/ulimit.html :-) So thanks Philip, looks like you saved my day twice! Regards, Kaya On 09/17/2014 03:55 AM, Philip Guenther wrote: On Tue, Sep 16, 2014 at 4:27 PM, Kaya Saman kayasa...@gmail.com wrote: I'm not sure what happened but after updating OpenBSD today, then updating the installed packages Dansguardian seems to not be working. ... The only error in the logs that I can see is: dansguardian[11832]: Error polling child process sockets: Invalid argument dansguardian[11832]: Error polling child process sockets: Invalid argument So the error means that poll() is being passed an nfds argument too large, larger than the process could have open as fds. It looks like the code, for some reason I cannot understand, passes poll() a pollfd structure for each child process...with fd=-1, so that it will be ignored. Uh, why? Why is it passing pollfd structures to the kernel that it *wants* the kernel to ignore? It seems that the code could simply skip over allocating and filling in those pollfd structures and have the exact same effect. As for what changed, well, something changed the number of child processes you're experiencing (load?), or the process fd limit (RLIMIT_NOFILE) for dansguardian changed. Philip Guenther
Re: Dansguardian not working after updating OBSD Current
On September 17, 2014 4:08:48 PM CEST, Kaya Saman kayasa...@gmail.com wrote: Ok, so this is just a quick follow up. Squid started dying too, checking the logs showed not enough file descriptors. After looking at both /etc/login.conf openfiles-cur and the sysctl kern.maxfiles limits which were set extremely high to begin with turns out that the: ulimit -n was only showing as 64. Changing that over to a value of 1 (overkill but better safe then For some definition of safe. The limits exists for a reason. ;-) But sure, one size does not fit all. sorry), Dansguardian managed to start and now both DG and Squid seem to be online and stable! I wonder if it has something to do with the: openfiles parameter in /etc/login.conf? A response by Philip suggests this: http://openbsd.7691.n7.nabble.com/setting-resource-limits-login-conf-and-ulimit-td223656.html Just a quick observation, not sure if by design or other but ulimit doesn't seem to have a man page? man ulimit man: no entry for ulimit in the manual. man ulimit man: no entry for ulimit in the manual. Though I did find a copy here: http://ss64.com/bash/ulimit.html Hint is in the url: bash. ulimit is a shell command, not a binary. man $SHELL /Alexander :-) So thanks Philip, looks like you saved my day twice! Regards, Kaya On 09/17/2014 03:55 AM, Philip Guenther wrote: On Tue, Sep 16, 2014 at 4:27 PM, Kaya Saman kayasa...@gmail.com wrote: I'm not sure what happened but after updating OpenBSD today, then updating the installed packages Dansguardian seems to not be working. ... The only error in the logs that I can see is: dansguardian[11832]: Error polling child process sockets: Invalid argument dansguardian[11832]: Error polling child process sockets: Invalid argument So the error means that poll() is being passed an nfds argument too large, larger than the process could have open as fds. It looks like the code, for some reason I cannot understand, passes poll() a pollfd structure for each child process...with fd=-1, so that it will be ignored. Uh, why? Why is it passing pollfd structures to the kernel that it *wants* the kernel to ignore? It seems that the code could simply skip over allocating and filling in those pollfd structures and have the exact same effect. As for what changed, well, something changed the number of child processes you're experiencing (load?), or the process fd limit (RLIMIT_NOFILE) for dansguardian changed. Philip Guenther
Re: Dansguardian not working after updating OBSD Current
On 09/17/2014 03:32 PM, Stefan Olsson wrote: Date: Wed, 17 Sep 2014 15:08:48 +0100 From: kayasa...@gmail.com Just a quick observation, not sure if by design or other but ulimit doesn't seem to have a man page? -It is a ksh builtin. man ksh -It would be nice to have it show up when doing apropos though... Thanks for the clarification :-)
Re: Dansguardian not working after updating OBSD Current
On 09/17/2014 03:34 PM, Alexander Hall wrote: On September 17, 2014 4:08:48 PM CEST, Kaya Saman kayasa...@gmail.com wrote: Ok, so this is just a quick follow up. Squid started dying too, checking the logs showed not enough file descriptors. After looking at both /etc/login.conf openfiles-cur and the sysctl kern.maxfiles limits which were set extremely high to begin with turns out that the: ulimit -n was only showing as 64. Changing that over to a value of 1 (overkill but better safe then For some definition of safe. The limits exists for a reason. ;-) But sure, one size does not fit all. sorry), Dansguardian managed to start and now both DG and Squid seem to be online and stable! I wonder if it has something to do with the: openfiles parameter in /etc/login.conf? A response by Philip suggests this: http://openbsd.7691.n7.nabble.com/setting-resource-limits-login-conf-and-ulimit-td223656.html Just a quick observation, not sure if by design or other but ulimit doesn't seem to have a man page? man ulimit man: no entry for ulimit in the manual. man ulimit man: no entry for ulimit in the manual. Though I did find a copy here: http://ss64.com/bash/ulimit.html Hint is in the url: bash. ulimit is a shell command, not a binary. man $SHELL /Alexander Yep I see that now :-) Next time will understand better! :-) So thanks Philip, looks like you saved my day twice! Regards, Kaya On 09/17/2014 03:55 AM, Philip Guenther wrote: On Tue, Sep 16, 2014 at 4:27 PM, Kaya Saman kayasa...@gmail.com wrote: I'm not sure what happened but after updating OpenBSD today, then updating the installed packages Dansguardian seems to not be working. ... The only error in the logs that I can see is: dansguardian[11832]: Error polling child process sockets: Invalid argument dansguardian[11832]: Error polling child process sockets: Invalid argument So the error means that poll() is being passed an nfds argument too large, larger than the process could have open as fds. It looks like the code, for some reason I cannot understand, passes poll() a pollfd structure for each child process...with fd=-1, so that it will be ignored. Uh, why? Why is it passing pollfd structures to the kernel that it *wants* the kernel to ignore? It seems that the code could simply skip over allocating and filling in those pollfd structures and have the exact same effect. As for what changed, well, something changed the number of child processes you're experiencing (load?), or the process fd limit (RLIMIT_NOFILE) for dansguardian changed. Philip Guenther
Re: Dansguardian not working after updating OBSD Current
Date: Wed, 17 Sep 2014 15:08:48 +0100 From: kayasa...@gmail.com Just a quick observation, not sure if by design or other but ulimit doesn't seem to have a man page? -It is a ksh builtin. man ksh -It would be nice to have it show up when doing apropos though...
Re: Dansguardian not working after updating OBSD Current
Hi,
Stefan Olsson wrote on Wed, Sep 17, 2014 at 10:32:19AM -0400:
From: kayasa...@gmail.com:
Just a quick observation, not sure if by design or other but ulimit
doesn't seem to have a man page?
-It is a ksh builtin.
man ksh
-It would be nice to have it show up when doing apropos though...
It does, it just isn't in the .Nm (name) and .Nd (one-line description)
macros that are searched by default, it's in .Ic (internal command)
macros, so you can find it by either searching across all keys with
any= or by specifically searching for internal commands:
$ apropos any=ulimit
ksh, rksh(1) - public domain Korn shell
sh(1) - public domain Bourne shell
getrlimit, setrlimit(2) - control maximum system resource consumption
$ apropos Ic=ulimit
ksh, rksh(1) - public domain Korn shell
sh(1) - public domain Bourne shell
getrlimit, setrlimit(2) - control maximum system resource consumption
That works on OpenBSD 5.6 or -current if you have set
MAKEWHATISARGS=' '
in weekly.local(8) or have run
sudo makewhatis
manually.
This may also be useful:
$ apropos -O Ic Nm=ksh
ksh, rksh(1) - until # privileged # list # + # - # / # noexec #
noclobber # G # nounset # T # +(foo|bar) # a # c # d # f # i # j
# p # vi-tabcomplete # t # # login # !(*) # ++ # () # builtin
# suspend # F # [[ foobar = f*r ]] # elif # \ # exec 3p; exec
3- # select i; { echo $i; } # $! # %- # $* # # _ # %+ # eval
`false` # # $_ # $@ # +x # unset # # cmd foo foo #
complete # readonly # y # mknod # ~ # print -u # [ # ] # errexit
# continue # +-A # down-history # $! # false # markdirs # posix
# monitor # break # ulimit # function # $$ # allexport # !(foo|bar)
# : # alias # notify # set - # . # times # set -o monitor #
autoload # yank # stdin # jobs # command # {} # $(cat foo) #
emacs-usemeta # sh # select # read # ignoreeof # emacs # {foo} #
hash # echo `echo \hi\` # cd # FOO=bar # vi-esccomplete #
exec # if # print -p # in # alias -d # verbose # +A # noglob #
case $foo { *) echo bar; } # nolog # bg # umask # typeset -ft #
fc # do # wait # trackall # getconf CS_PATH # vi-show8 # set --
`false`; echo $? # else # done # time # $(...) # whence # echo
$((2+3*4)) # fc -e ${VISUAL:-${EDITOR:-vi}} # *(foo|bar) # return
# restricted # +-o # stuff # shift # ^F # +-abCefhkmnpsuvXx #
@(foo|bar) # case # I#^J # viraw # physical # for i; { echo $i;
} # alias r='fc -e -' # set -o vi-tabcomplete # !(?)* # pwd #
echo # `..` # set +o # read -p # typeset +f # set # typeset -f #
set -m # set -o # set -h # esac # nohup # $( foo) # test #
up-history # set -A # # alias -t # [ ... ] # p # exit # ||
# read nfoo?'number of foos: ' # let # kill # p # typeset -fx
# while # typeset -fu # ! # export # vi # kill -l # ?(foo|bar) #
unalias # typeset # true # interactive # eot # csh-history #
keyword # eval # alias -d fac=/usr/local/facilities; cd ~fac/bin
# $? # bind # for # gmacs # . # getopts # xtrace # 10l #
search-history # braceexpand # trap # delete-char-forward # print
# set -o posix # bgnice # +-lprtUux # fg
Some noise in there, but also most built-in commands.
Oh, and if i ever manage to do the switch of man(1) from the
BSD to the mandoc implementation, this will work too - it already
works on -current if you do
sudo mv /usr/bin/man /usr/bin/oman
sudo ln -s /usr/bin/mandoc /usr/bin/man
at your own risk:
$ man -ak Ic=ulimit
That directly brings up the pager with the ksh(1), sh(1), and
getrlimit(2) manuals, so you can readily use the more(1) or
less(1) / search method across all three.
Yours,
Ingo
Re: Dansguardian not working after updating OBSD Current
On Tue, Sep 16, 2014 at 4:27 PM, Kaya Saman kayasa...@gmail.com wrote: I'm not sure what happened but after updating OpenBSD today, then updating the installed packages Dansguardian seems to not be working. ... The only error in the logs that I can see is: dansguardian[11832]: Error polling child process sockets: Invalid argument dansguardian[11832]: Error polling child process sockets: Invalid argument So the error means that poll() is being passed an nfds argument too large, larger than the process could have open as fds. It looks like the code, for some reason I cannot understand, passes poll() a pollfd structure for each child process...with fd=-1, so that it will be ignored. Uh, why? Why is it passing pollfd structures to the kernel that it *wants* the kernel to ignore? It seems that the code could simply skip over allocating and filling in those pollfd structures and have the exact same effect. As for what changed, well, something changed the number of child processes you're experiencing (load?), or the process fd limit (RLIMIT_NOFILE) for dansguardian changed. Philip Guenther
Re: Dansguardian not working after updating OBSD Current
On 09/17/2014 03:55 AM, Philip Guenther wrote: On Tue, Sep 16, 2014 at 4:27 PM, Kaya Saman kayasa...@gmail.com wrote: I'm not sure what happened but after updating OpenBSD today, then updating the installed packages Dansguardian seems to not be working. ... The only error in the logs that I can see is: dansguardian[11832]: Error polling child process sockets: Invalid argument dansguardian[11832]: Error polling child process sockets: Invalid argument So the error means that poll() is being passed an nfds argument too large, larger than the process could have open as fds. It looks like the code, for some reason I cannot understand, passes poll() a pollfd structure for each child process...with fd=-1, so that it will be ignored. Uh, why? Why is it passing pollfd structures to the kernel that it *wants* the kernel to ignore? It seems that the code could simply skip over allocating and filling in those pollfd structures and have the exact same effect. As for what changed, well, something changed the number of child processes you're experiencing (load?), or the process fd limit (RLIMIT_NOFILE) for dansguardian changed. Philip Guenther Thanks Philip for the response. It wouldn't be load that's causing this issue as I'm monitoring the machine using multiple monitoring tools. As to the process limit, I've got infinity set for the daemon under login.conf though root user is at defaults. Since Dansguardian is being run as it's own user the root portion shouldn't apply. The kernel maxproc value is set to 1310 (default??). Outside of modifying the source code is there anything I can try within the system to allow Dansgaurdian to run? I'm not sure if the upgrade changed any default values that may have effected this as the last update went through fine?? Weird! Regards, Kaya
