The main problem is the with examples/global/vars and book/default/ docstring They generate dynamically links to docstrings recursively. Web spider get stuck there and keep requesting pages over pages, thus consuming lots of memory and bandwidth. Humans do not do that. Humans would not more than 2-3 levels deep. So I monitor the apache log and if somebody requests
examples/global/vars/<a>/<b>/<c> or longer expression, I assume they are a spider and add them to iptable reject. I also block anybody who requests .php pages, .jsp pages and .asp pages since they are clear attempts to find vulnerablites. I block anybody who tries a directory traversal attack. On top of this I use psad. Not sure that is working well and I may need to tweak it. Massimo On Mar 12, 6:29 am, Christopher Steel <[email protected]> wrote: > Sounds like your new setup is working really well Massimo. Eventually > I will be bugging you for some details as well. > > Congrats! > > On Mar 11, 8:37 pm, "mr.freeze" <[email protected]> wrote: > > > Massimo has a script that blocks IPs of things it interprets as denial > > of service attacks, including crawling content without obeying > > robots.txt. I am building an experimental search engine at work and > > accidentally bypassed robots.txt on one of the debug runs. I've been > > blocked ever since. Works fine from home. DNS resolves correctly from > > work. > > > On Mar 9, 2:42 am, Yarko Tymciurak <[email protected]> > > wrote: > > > > I see web2py.com responding nicely... maybe something is blocking it > > > for you? > > > > have a try athttp://68.169.39.35/ > > > > Or try nslookup on web2py.com to see if your DNS server returns > > > something valid. > > > > On Mar 9, 2:18 am, Sky <[email protected]> wrote: > > > > > as I see web2py.com is down since 2 days ago. > > > > is there any body to inform the site administrator ??? -- You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/web2py?hl=en.
