Chris,
thanks a lot for explaining what could be overflowing the FD_SETSIZE of
1024.
Just for reference, I use mpm_event mostly with SSL connections, so it
should behave like mpm_worker. This is my configuration:
StartServers 2
ServerLimit 16
MinSpareThreads 256
MaxSpareThreads 1280
ThreadLimit 1024
ThreadsPerChild 1024
MaxRequestWorkers 16384
MaxConnectionsPerChild 0
One more thing, poll() seems to be used in jk_connect.c in other places
already, just not at the spot that matters in my case.
Anyhow, I will submit a bug report later this week with all the information
and will post a link over here as well.
Thank you,
Michael
On 18 July 2016 at 16:56, Christopher Schultz <ch...@christopherschultz.net>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Michael,
>
> On 7/18/16 10:10 AM, Christopher Schultz wrote:
> > Michael,
> >
> > On 7/18/16 8:53 AM, Michael Diener wrote:
> >> On 6 July 2016 at 00:09, Christopher Schultz
> >> <ch...@christopherschultz.net> wrote:
> >
> >>>> From what I understand a buffer overflow would only happen
> >>>> for FD_SET if the fd_set gets over 1024 descriptors. I made
> >>>> sure that my ulimit for open files is set and applied large
> >>>> enough, so that's not it.
> >>>
> >>> There's nothing magic about the ulimit. An fd_set should size
> >>> appropriately for your OS. On my Linux system, FD_SETSIZE
> >>> happens to be set to 1024. Reading through the byzantine
> >>> labyrinth of includes, it appears that FD_SET has zero
> >>> boundary-checking, so it's therefore possible that overflow
> >>> will occur.
> >
> >
> >> Regarding the FD_SETSIZE, it is also set for me to 1024 although
> >> the ulimit is set higher.
> >
> > Well, the FD_SETSIZE is just the maximum size of the fd set that
> > can be passed-into select() specifically. That doesn't limit the
> > number of file descriptors your processes can open. The ulimit
> > settings are the limits on those.
> >
> >> I'm a bit lost now on what I should do now. What makes me wonder
> >> is, that nobody else seems to hit this limitation of FD_SET and
> >> this makes me think something on my Linux machine is not right.
> >
> > Well, not everyone is using the same setup. For example, using NIO
> > through Java is likely to get you the poll() call under the hood,
> > so supporting more than e.g. 1024 file descriptors is not an issue
> > there. That's just a guess, but I think it's a reasonable guess.
> >
> > I think tcnative+APR is not widely deployed. I have no numbers to
> > back that up, but we don't get a huge volume of questions in the
> > list about it.
>
> Of course, the above statement makes no sense whatsoever. This is
> about mod_jk and not tcnative. Sorry for the confusion.
>
> But I was thinking, the case above would require a single httpd thread
> to accumulate more than 1024 connections, and that would require some
> very specific circumstances.
>
> First, you'd have to be using an MPM that allowed more than one
> connection per process/thread, so that means no pre-fork, leaving
> basically event or worker available.
>
> Then, you'd have to have enough traffic to cause one thread to grow to
> more than 1024 connections. The default for httpd's worker MPM has 64
> threads per child and 16 server processes.
>
> For one process to exceed the 1024 fd limit, you'd have to be handling
> roughly 16 simultaneous connections per thread PER PROCESS. I'm not
> sure how httpd auto-scales-out its child processes, so you could
> conceivably get 1024 simultaneous connections in a burst of requests
> to a single child process, but it seems likely that load would be
> roughly-split between 16 child processes, keeping those 1024
> connections down to a mild 64-jk-connections per-process.
>
> Assuming you have 16 child processes and a perfectly uniform
> load-distribution between them, you'd have to get a burst of 16k
> simultaneous requests in order to max-out that fd array on a single
> server. That's probably why it's quite rare.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJXjO4tAAoJEBzwKT+lPKRYb9wP/j2i6FW7gOGf/G4/CTundEMY
> WcKKPfNXAOU270KyiBkuSLw06Cjw8mk3vsjpg5i5JkPC4TUNy1jyhaI5cs0RvisV
> pzqFgZCuyz4gPtg4TRCXw5RrmLwe7unBldTETjK54P9/Nd6Vuj34mUV8OLcnYZh6
> UMCe0ULbBV5IoGhLmGQ5yy7MfHGdq1vwmxR41i4A4rc4J9fOC1UI4pIOvJRM1cnT
> 5cfLEavT3tbqxaxLCs5V/pQkkuwQMKITSW+JGdDPxN3oXL7b1QCw8RGBqhpDgjE/
> FIIIBGfbMGHDXstmSBRXGlxkASKKdYlK9qoYU3f7G0PK053zx5TD+2vOfb0u0Vi/
> lwIkk3lhL7Azw9hYKFr1R+PW3ewUqXI7Nh05HldNWlJ48I91cTGLF2mC7uRo6uQ2
> M9pUCuyCtL1ajgG6eUmBlo2soIAaHPkorCmCdUAiv/zWfHKSSEGTwr3l81DtSapE
> iORRoCyLVIhxKQprvBlTHp2uDIa7lzXOI83RcMb6ZqdxiNe3LscTRsVl/Ll8cVHj
> Fw7klJgbPrRPmMn02hANdalDE96CvagPlEmgCGhp9h3TPD7fV28a3vY154IYxLBW
> C2ksoMNv12ha+kiTvrYLc7j85drtZg7V0C/5TfRdBRSxxJ0KF/ye7ed2SN/8RRKC
> QgyoIkZ2oSBIoHc/ss26
> =f33p
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
--
______________________________
NEW GAME! http://www.dig-pig.com
Michael Diener - Software e.K.
mdie...@mdiener.de
+49 178 501 601 8
www.mdiener.de
@mdienersoftware
Grünberger Str. 62,
10245 Berlin, Germany
Sitz Berlin, Amtsgericht Charlottenburg, HRA 46760 B
USt-IdNr. DE233968393
