Re: Need info about FreeBSD and interrupted system calls for MySQL code
Dan, your info is very valuable - thanks: Dan Nelson wrote: > In the last episode (Apr 30), Joerg Bruehe said: >> Dan Nelson wrote: >>> In the last episode (Apr 29), Joerg Bruehe said: For some long, unknown time, the MySQL code contains a variable "net_retry_count" which is by default set to 10 (ten) for all platforms, but to 100 (1 million) for FreeBSD (during configure phase). The source code comment about this variable reads If a read on a communication port is interrupted, retry this many times before giving up. [[...]] >>> I'm pretty sure this is a holdover from when FreeBSD only had a user >>> pthreads package (libc_r). [[...]] >> Interesting information - thanks. I never heard that before, but it >> explains a lot. > > This may also have been due to a bug in the early libc_r code. Appropriate > use of sigwait() and pthread_sigmask() should let the pthreads library know > which read() calls it can silently retry on behalf of threads that are > ignoring signals (and thus shouldn't have their syscalls aborted with > EINTR). I have email records talking about libc_r problems with signal > masking from the FreeBSD 2.2.7 days (~1998). It's possible that later > libc_r versions had fixed the bug. I used to have copies of the ancient > mysql source code around (3.22 and 3.23 era), but have since deleted them, > so I don't know when the 100 workaround was added. The readily available revision control history of the MySQL source code goes back to the year 2000 only (the system used was changed back then, without history transfer), but a colleague checked that this workaround is documented in the manual of 3.22. All this seems to be a good indication we should get rid of this. Thanks for your help, Jörg -- Joerg Bruehe, MySQL Build Team, joerg.bru...@sun.com (+49 30) 417 01 487 Sun Microsystems GmbH, Komturstrasse 18a, D-12099 Berlin Geschaeftsfuehrer: Juergen Kunz Amtsgericht Muenchen: HRB161028 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Need info about FreeBSD and interrupted system calls for MySQL code
In the last episode (Apr 30), Joerg Bruehe said: > Dan Nelson wrote: > > In the last episode (Apr 29), Joerg Bruehe said: > >> For some long, unknown time, the MySQL code contains a variable > >> "net_retry_count" which is by default set to 10 (ten) for all platforms, > >> but to 100 (1 million) for FreeBSD (during configure phase). > >> > >> The source code comment about this variable reads > >>If a read on a communication port is interrupted, retry this many > >>times before giving up. > >> > >> [[...]] > > > > I'm pretty sure this is a holdover from when FreeBSD only had a user > > pthreads package (libc_r). libc calls that would normally block got > > converted into non-blocking versions and a select() loop would execute > > threads as the events they were waiting on occurred. Incoming signals > > would cause all threads waiting on read() to return EINTR. If you have > > other threads doing work and sending/receiving signals, this can add up > > to a lot of extra EINTR's. > > Interesting information - thanks. I never heard that before, but it > explains a lot. This may also have been due to a bug in the early libc_r code. Appropriate use of sigwait() and pthread_sigmask() should let the pthreads library know which read() calls it can silently retry on behalf of threads that are ignoring signals (and thus shouldn't have their syscalls aborted with EINTR). I have email records talking about libc_r problems with signal masking from the FreeBSD 2.2.7 days (~1998). It's possible that later libc_r versions had fixed the bug. I used to have copies of the ancient mysql source code around (3.22 and 3.23 era), but have since deleted them, so I don't know when the 100 workaround was added. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Need info about FreeBSD and interrupted system calls for MySQL code
Dan, thanks for your reply: Dan Nelson wrote: > In the last episode (Apr 29), Joerg Bruehe said: >> For some long, unknown time, the MySQL code contains a variable >> "net_retry_count" which is by default set to 10 (ten) for all platforms, >> but to 100 (1 million) for FreeBSD (during configure phase). >> >> The source code comment about this variable reads >>If a read on a communication port is interrupted, retry this many >>times before giving up. >> >> [[...]] > > I'm pretty sure this is a holdover from when FreeBSD only had a user > pthreads package (libc_r). libc calls that would normally block got > converted into non-blocking versions and a select() loop would execute > threads as the events they were waiting on occurred. Incoming signals would > cause all threads waiting on read() to return EINTR. If you have other > threads doing work and sending/receiving signals, this can add up to a lot > of extra EINTR's. Interesting information - thanks. I never heard that before, but it explains a lot. > > FreeBSD 5.0 (released in 2003) was the first version to have kernel-based > pthread support, so the original reason for raising net_retry_count has long > since disappeared. It is quite possible that nobody checked this: "If it ain't broken ..." > > A related question might be, though: Should that variable even exist? > EINTR isn't technically a failure, and most programs that read from sockets > simply wrap their read()s in a loop that retries when EINTR is received. > Only mysql actually counts the number of times through the loop. I know and agree that EINTR is no failure if a system call takes long, like read() or write() from/to a socket (or other slow device) on sufficiently large data. But my current action is not to change the code, rather it is a cleanup in the build system (you may have heard we are changing from the autotools to cmake), so currently I won't change that loop dealing with possible system call interruptions (by not counting). So you are saying it might all be obsolete, and current versions of FreeBSD don't need this special setting. This sounds like I should do a build without it and then run tests. Thanks! Regards, Jörg -- Joerg Bruehe, MySQL Build Team, joerg.bru...@sun.com Sun Microsystems GmbH, Komturstrasse 18a, D-12099 Berlin Geschaeftsfuehrer: Juergen Kunz Amtsgericht Muenchen: HRB161028 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Need info about FreeBSD and interrupted system calls for MySQL code
In the last episode (Apr 29), Joerg Bruehe said: > For some long, unknown time, the MySQL code contains a variable > "net_retry_count" which is by default set to 10 (ten) for all platforms, > but to 100 (1 million) for FreeBSD (during configure phase). > > The source code comment about this variable reads >If a read on a communication port is interrupted, retry this many >times before giving up. > > The documentation (manual) has this sentence in addition: >This value should be set quite high on FreeBSD because internal >interrupts are sent to all threads. > > I read that as > "On FreeBSD, a thread may receive many more interrupts than on other > platforms, so an operation which may take some time (like network I/O) > may be interrupted much more often than on other platforms, and hence > the retry count should be higher." > > I trust that this comment was valid at the time it was written - > is it still true for current versions of FreeBSD, or did things change? I'm pretty sure this is a holdover from when FreeBSD only had a user pthreads package (libc_r). libc calls that would normally block got converted into non-blocking versions and a select() loop would execute threads as the events they were waiting on occurred. Incoming signals would cause all threads waiting on read() to return EINTR. If you have other threads doing work and sending/receiving signals, this can add up to a lot of extra EINTR's. FreeBSD 5.0 (released in 2003) was the first version to have kernel-based pthread support, so the original reason for raising net_retry_count has long since disappeared. A related question might be, though: Should that variable even exist? EINTR isn't technically a failure, and most programs that read from sockets simply wrap their read()s in a loop that retries when EINTR is received. Only mysql actually counts the number of times through the loop. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Need info about FreeBSD and interrupted system calls for MySQL code
Hi Groggy (whom I didn't contact for too long a time), everybody, following the advice on your page, I include the FreeBSD list, even though I'm not subscribed there (hoping it will allow me to post) - so please, whoever replies, could you please cc: me directly? Of course, I tried Google, but I didn't find any answers to my question. For some long, unknown time, the MySQL code contains a variable "net_retry_count" which is by default set to 10 (ten) for all platforms, but to 100 (1 million) for FreeBSD (during configure phase). The source code comment about this variable reads If a read on a communication port is interrupted, retry this many times before giving up. The documentation (manual) has this sentence in addition: This value should be set quite high on FreeBSD because internal interrupts are sent to all threads. I read that as "On FreeBSD, a thread may receive many more interrupts than on other platforms, so an operation which may take some time (like network I/O) may be interrupted much more often than on other platforms, and hence the retry count should be higher." I trust that this comment was valid at the time it was written - is it still true for current versions of FreeBSD, or did things change? Thanks for all your hints, Jörg -- Joerg Bruehe, MySQL Build Team, joerg.bru...@sun.com (+49 30) 417 01 487 Sun Microsystems GmbH, Komturstrasse 18a, D-12099 Berlin Geschaeftsfuehrer: Juergen Kunz Amtsgericht Muenchen: HRB161028 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
