Hi,
One of my routers got stuck today, not being able to answer http requests
(routing and API).
When trying to investigate the issue, I found catalina.log with a lot of
messages complaining on failure to open a socket due to too many open
files. See example below.
No issues were found in the log earlier to that point, beyond a periodic
warnings of pulling the certificates every 5 minutes.
When trying to understand "what are these open files", I found about 4k
open connections in "CLOSE_WAIT" towards the monitor.
Note: I'm running TC2.1 RC3 with golang traffic-monitor.
Have anyone encountered a similar issue?
Are the warnings for pulling the certificates a normal thing?
Thanks,
Nir
Feb 01, 2018 7:33:09 AM
com.comcast.cdn.traffic_control.traffic_router.secure.CertificateRegistry
importCertificateDataList
WARNING: Imported handshake data with alias my-ds.my-cdn.com
Feb 01, 2018 8:43:13 AM org.apache.tomcat.util.net.NioEndpoint$Acceptor run
SEVERE: Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:1309)
at java.lang.Thread.run(Thread.java:745)
Feb 01, 2018 8:43:14 AM org.apache.tomcat.util.net.NioEndpoint$Acceptor run
SEVERE: Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:1309)
at java.lang.Thread.run(Thread.java:745)
