It really sounds like your site is under attack (specifically, a DDOS attack). Switching hosting providers won't help (they'll just attack the new site). Blocking based on the UA string (to reject curl requests only) is not a standard feature of any hosting service. And besides, spoofing the UA in curl is a trivial command-line argument, so it would be ineffective if this is really a hostile act.
File a production issue, to see if the folks at google can come up with a way to thwart the attack. -Joshua On Aug 1, 2012, at 5:13 PM, Kate <[email protected]> wrote: > I've posted about this problem before but I don't think I have explained it > adequately. > I am getting tens of thousands of CURL requests throwing HEAD method errors. > This is thousands of times more than normal browser requests. > > I can't block them as they come from different IPs and you are only allowed > to block 100 IPs. > > So - is there way of stopping them with code. If not, I can set up my site > somewhere else but it seems a pity to have to do this. > > First I was concerned that these requests were throwing errors and it was > suggested I enable the head method, but I can't see any point in doing this > as the CURL requests are no no use to me. I have stopped being a billable > site as I can't afford to support these requests. > > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/Kc8kRRoDKKkJ. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
