On 6/17/2018 12:59 AM, Paul M Furley wrote: > Hi Pete, > > On 17/06/18 04:53, Pete Stephenson wrote: >> Thanks. >> >> I then have three more questions: >> >> 1. If this issue is affecting my server to the point of it being booted >> from the pool (since it's stalling near-continuously and can't respond >> toe queries), why are other servers not being similar affected? There's >> lots of servers still in the pool. > > I certainly should've been booted from the pool since my server has > filled up its disk and trashed its database (twice) so it was offline > all of yesterday. > > I'm bringing it back up with the `set_flags DB_LOG_AUTOREMOVE` setting > this time which will hopefully save it.
Yeah, I added the same line. There's now just two log files rather than dozens. Seems to work ok in controlling the disk space usage, but it doesn't seem to do anything about the spikes in CPU usage, non-responsiveness, etc. >> 2. Is there some countermeasure one can use to protect their server? I >> have LimitRequestBody set to 8000000 (8MB) to prevent blatant abuse, but >> clearly something is still annoying the server. > > It appears from Rob's previous email that our servers are failing to > synchronise a 22M key (because of settings like this) which is causing > sks to continually retry: > > https://lists.nongnu.org/archive/html/sks-devel/2018-06/msg00014.html : The server had been running with no limits on the request body size for several years without problems. I added that line in the hopes of controlling things from getting worse. I've since removed it, but it doesn't seem to have much of an effect. Is there some way of (a) resolving the problem with this key (e.g. locally adding it to the server, so it won't keep choking while retrying) and (b) preventing such issues from occurring in the future that I can take now? >> 3. Any suggestions on how to deal with the unreasonably high-speed >> queries from corporate mail systems? Ideally, they'd run their own >> server locally to handle their huge amount of queries, but I have no >> real way of communicating that with them. I'd love to slow down their >> queries (tarpitting, maybe?) to minimize excess resource consumption >> while still answering their queries as opposed to just cutting them off >> once they hit a rate limit. > > Are you sure these users are the cause of your troubles? Or is it this > constant-retry loop caused by this large key? I don't know. Regardless, I do think that the high-volume users are being a bit unreasonable: SKS queries are relatively "heavy" compared to lightweight queries like those to DNSbls, so making queries to the SKS pool for each email sent or received seems excessive, but that may just be me. Anyway, I've removed the rate limits since they didn't seem to have any effect on the constant-retry loop or stalling. > I'd suggest contacting them before rate limiting them, ask them to point > at the pool or slow down their queries. I think they already were querying the pool, and just happened to get my server as part of the rotation. I just sent them an email inquiring about the number of queries and encouraging them to run their own server and have it join the pool, but in general I don't have the time or motivation to contact every potentially abusive user. I'm just curious if there's any recommended practices for throttling abusive users. Cheers! -Pete -- Pete Stephenson _______________________________________________ Sks-devel mailing list Sks-devel@nongnu.org https://lists.nongnu.org/mailman/listinfo/sks-devel
