Hi all, Excellent discussion! We have been experiencing with this use case since day one with the http://www.restlet.org web site. To block attacks at the network level, we rely on Linux's iptables firewall which can easily be fed with a blacklist of IP addresses. To determine those IP addresses, we regularly monitor our web log files to detect attacks, based on data volume and time intensity (scripts written by Thierry).
We also rely on the firewall to reroute the traffic from 80 to an internal port number (8080), this way we can dynamically change the routing to point to a new server running on a different internal port (8081). Right behind we have our Restlet/Jetty server running. In general this set up works for months without restarting anything. This is still running Restlet 1.1 at this point and is quite fast and stable. The problem with the Restlet filter approach is that when the call reaches you filter, it has already consumed some precious resources and a massive DoS attack could be hard to prevent. You can still limit the number of concurrent network connections at the firewall level though. >From a Restlet perspective, our goal with version 2.1 is to ship a fast and >scalable internal NIO/HTTP connector so that we take back full control of the >network layer. We will then easily block IP addresses by not even accepting >the offending TCP connections, without having to rely on the iptables >strategy. We could also share a common thread pool for the whole >Component/JVM, share a common NIO selector/controller thread for both client >and server connections, etc.. The connector could easily ask the Component#connectorService if the given IP address is acceptable before establishing the connection! This way, a blocked address wouldn't consume CPU/data traffic for HTTP parsing nor pollute the log files. We should provide a default configurable firewall strategy in the ConnectorService, such as a max number of calls per time period, a blacklist and a whilelist of IP addresses, etc. The goal of the Restlet edition for Java SE is to become a solid alternative to Apache HTTP server, popular load balancers, caches, proxies, etc. and live right next to your network layer. Easier said than done, but it doesn't seem too far ahead. I've added a requirement for this in the "NIO connectors" specifications page: http://wiki.restlet.org/developers/172-restlet/354-restlet.html Best regards, Jerome -- Restlet ~ Founder and Technical Lead ~ http://www.restlet.org Noelios Technologies ~ http://www.noelios.com -----Message d'origine----- De : Tal Liron [mailto:[email protected]] Envoyé : vendredi 24 septembre 2010 20:45 À : [email protected] Objet : Re: Best way to deal with bot attacks and vulnerability searches using Restlet for a public webs site. Exactly. You would still want two layers for everyday production applications, even if you have just one Restlet node. Upgrading your app, for example -- you have Varnish show a "under maintenance" screen while you swap out your Restlet nodes. Perhaps it looks appealing to have both layers in the JVM. Couldn't your your load balancer be written in Java? Perlbal is written in Perl, after all. Wouldn't a Java house want everything in Java? Well, fact is, for simple products like load balancers, that aim for maximum scalability, the JVM isn't quite there yet. Perhaps Java 7's NIO would be better. Using Perl directly over Linux async IO is far more efficient. Likewise, a caching tool like Varnish is a masterpiece of efficient use of direct memory paging under Linux. No way you could do the same with the JVM. By the way -- I don't necessarily recomend using Varnish, Squid, Pound, etc. I see upstream caching as a patchy solution from a previous LAMP era. Modern applications, especially on the JVM, should be able to handle caching on their own, and do it smartly. But, it's good to know that there are products there in case you need them. Since Restlet lets you so easily set the HTTP caching headers, Varnish, upstream, would know exactly what to do with your representations. I address this in more depth in my article about scaling: http://threecrickets.com/prudence/scaling/#toc-Subsubsection-135 My current favorite deployment is Perlbal+Prudence (which is Restlet over Grizzly). I find Perlbal deliciously minimal and reliable: all it does is forward to my application nodes, and handle SSL for me. This minimalism forces the application developers to be smart, and not rely on any upstream features that would compartmentalize your deployment and add dependencies to your application. (Restlet/Grizzly can do SSL, but there are some thorny issues with persistent connections that are easily avoided by just handling it at the front.) -Tal On 09/24/2010 10:53 AM, jrb wrote: > A 2 layer architecture for a public service is useful for a bunch of > reasons- load balancing, service availability during upgrades, > security, separation of concerns (e.g. segregate apps by workload, > offload gzip encoding, ssl endpoint, local static asset caching). > > Now, I haven't tried this with restlet yet, but it looks to me like > restlet could be used on both layers. > > That said, there is a point at which *running* a public service > requires its own expertise to manage the endpoint, and typically > people focused on that use those tools- HAProxy, nginx, apache, etc. > And they tend to think of a jvm as opaque and hard to manage. Try > telling an ops person that they have to tune GC on their load balancer > and they'll probably quit. > > But I definitely see the appeal of having a restlet uberjar, packaging > jetty and spring and a bunch of other connectors, to serve as a kind > of universal REST server that could do load balancing, reverse proxy, > template rendering, or whatever- expose a REST interface to a database > or to other proprietary back end services- all by just swapping in and > out spring bean configuration. There are still some areas of weakness- > I think e.g. the slowloris DoS attack is not defensible from a JVM. > But as a devops tool, it seems like it has potential. > > > On Fri, Sep 24, 2010 at 10:56 AM, Tim Peierls<[email protected]> wrote: >> I share David's frustration. One of the things that attracted me to >> Restlet in the first place was the hope that I could throw away all >> the other processes and do everything in the JVM. Why couldn't I put >> whatever throttling/defensive logic (that I would otherwise have to >> put in a separate >> process) right into the Restlet Component? Is a Restlet Component >> inherently "vulnerable" as Tal puts it, or is it just that no one has >> provided the right defensive Restlet Filter so far? >> --tim >> >> On Fri, Sep 24, 2010 at 2:24 AM, David Fogel<[email protected]> wrote: >>> Hi Tal, William- >>> >>> Thanks for your feedback! >>> >>> Regarding putting the JVM behind Apache or some other proxy: well, >>> we're likely to have to use a load-balancer soon, but aside from >>> that, I find it frustrating to put something in front of the JVM- >>> shouldn't java web servers be considered just as robust and fast as >>> ones written in C or something else? It's not like java web servers >>> are inefficient CGI scripts or something, yet everyone is always >>> saying to stick things in front of them. I'm not suggesting this is >>> wrong, just that it's frustrating :-) >>> >>> Wouldn't it be nice if products like Jetty, Simple, etc had >>> appropriate built-in features to combat various misuse? It seems >>> like if they're serious about being web servers, they should >>> consider these features to be important. >>> >>> Anyhow, I'll probably try to block a few things at the Simple or >>> Restlet Filter level, just to make myself feel better... >>> >>> thanks, >>> Dave Fogel >>> >>> ------------------------------------------------------ >>> >>> http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessage >>> Id=2663951 >> > ------------------------------------------------------ > http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId > =2664279 ------------------------------------------------------ http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2664312 ------------------------------------------------------ http://restlet.tigris.org/ds/viewMessage.do?dsForumId=4447&dsMessageId=2664324
