> I believe that the essential problem, as explained in a previous email to Rob > Sargent, is that I'm getting several hundred database requests per day from > web crawlers. I just spent some time reading through my ngnix access.log and > found that the vast majority of these are from GoogleBot. My guess it that, > due to a time lag between opening and closing connections, many connections > will be opened simultaneously. This is why a small pool size won't work. > > Also, I'm advised to not block the web crawlers because this assists with > SEO. My understanding is that you just have to live with this.
I think that the search bots are just exposing the existing problem, they're not the cause of it. If you blocked them you would just be masking the underlying issue for now and when you receive an increase in traffic the problem will re-occur. Looking at the logs that you shared, there's only 1,026 requests (bots or otherwise) over about two days to /calendar (which I presume triggers the DB request) .. this is a very small amount. I'd suggest revisiting your code. Other than the connection handling, some other areas to look into would be: are DB requests only being triggered when necessary, how long do queries take - are there excessive joins and column/row returns. Clint --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
