MrtnBalazs opened a new pull request, #668: URL: https://github.com/apache/knox/pull/668
## What changes were proposed in this pull request? The DoS security provider have been removed, and it's functionality have been moved in the Web App Security provider. This provider is now able to add a Jetty DoSFilter into to filterchain if configured. The configuration options are the same as jetty's DoSFilter and can be found in jetty's DoSFilter documentation (https://www.eclipse.org/jetty/documentation/jetty-9/index.html#dos-filter), EXCEPT that the parameters need a `rate.limiting.` prefix AND when using delayMs with 0 or higher value or using throttling the `gateway.servlet.async.supported` configuration must be set to `true` in `gateway-site.xml` configuration file (it defaults to false)! Example configuration: ``` <provider> <role>webappsec</role> <name>WebAppSec</name> <enabled>true</enabled> <param> <name>rate.limiting.enabled</name> <value>true</value> </param> <param> <name>rate.limiting.maxRequestsPerSec</name> <value>2</value> </param> <param> <name>rate.limiting.delayMs</name> <value>2000</value> </param> <param> <name>rate.limiting.maxWaitMs</name> <value>20000</value> </param> <param> <name>rate.limiting.throttledRequests</name> <value>3</value> </param> <param> <name>rate.limiting.throttleMs</name> <value>20000</value> </param> ``` ## How was this patch tested? I have written tests into the `WebAppSecContributorTest` class. These tests test whether the contributor sets the right filters and parameters or not. I have also tested the feature manually by sending 10 curl requests to a server which response time is 3 seconds and enabled DEBUG level logs for `org.eclipse.jetty.servlets`. When the feature is disabled: ``` <param> <name>rate.limiting.enabled</name> <value>false</value> </param> ``` There is no effect: ``` 2022-11-03 09:58:24,806 2ca3f959-da98-414a-81a3-fe8c9d053ba6 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:24,832 93b6bcd0-5af3-4f77-b27a-324cc9e052c6 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:24,842 4651eabd-df67-48a2-b3f1-27ab32d0dd5a INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:24,859 cb2e9b3b-2d78-4edc-b432-bdc145d52dff INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:24,879 c3ee6514-3618-4974-9362-594159216c69 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:24,892 cc1f31f4-046b-466e-b4a4-e057c4d06f00 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:24,997 a2ff2f2a-5b13-4ded-ae4b-8e959cd94ebd INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:25,014 85957a67-3d85-4087-beee-c9d1324efa32 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:25,030 0581df96-feab-4221-8050-ae59e6b431ef INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:58:25,049 0fabb544-1cbc-4413-8b99-e19d34ce516e INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom ``` When it is enabled with a normal configuration: ``` <provider> <role>webappsec</role> <name>WebAppSec</name> <enabled>true</enabled> <param> <name>rate.limiting.enabled</name> <value>true</value> </param> <param> <name>rate.limiting.maxRequestsPerSec</name> <value>2</value> </param> <param> <name>rate.limiting.delayMs</name> <value>2000</value> </param> <param> <name>rate.limiting.maxWaitMs</name> <value>20000</value> </param> <param> <name>rate.limiting.throttledRequests</name> <value>3</value> </param> <param> <name>rate.limiting.throttleMs</name> <value>20000</value> </param> <param> <name>rate.limiting.maxRequestMs</name> <value>30000</value> </param> <param> <name>rate.limiting.maxIdleTrackerMs</name> <value>30000</value> </param> <param> <name>rate.limiting.insertHeaders</name> <value>true</value> </param> <param> <name>rate.limiting.trackSessions</name> <value>false</value> </param> <param> <name>rate.limiting.remotePort</name> <value>false</value> </param> <param> <name>rate.limiting.ipWhitelist</name> <value></value> </param> <param> <name>rate.limiting.managedAttr</name> <value>false</value> </param> </provider> ``` The requests are delayed and throttled: ``` 2022-11-03 09:31:30,811 08b8316a-4190-4f16-8464-cb01edc834c7 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@6b2e4211 2022-11-03 09:31:30,811 08b8316a-4190-4f16-8464-cb01edc834c7 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(335)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@6b2e4211 2022-11-03 09:31:30,812 08b8316a-4190-4f16-8464-cb01edc834c7 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:30,820 d7943185-159b-450c-bb6c-8ab72c495cfc DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@39eab2ac 2022-11-03 09:31:30,821 d7943185-159b-450c-bb6c-8ab72c495cfc DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(335)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@39eab2ac 2022-11-03 09:31:30,821 d7943185-159b-450c-bb6c-8ab72c495cfc INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:30,830 44f679f8-941a-4c2b-b8fe-e7516dabcd8f DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@6d02dfae 2022-11-03 09:31:30,830 44f679f8-941a-4c2b-b8fe-e7516dabcd8f WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@208a8383[type=IP, id=127.0.0.1, duration=PT0.019S, count=2], session=null, user=null 2022-11-03 09:31:30,849 52bc8732-690a-4ec8-9daf-adfdfa199bd8 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@117d64af 2022-11-03 09:31:30,850 52bc8732-690a-4ec8-9daf-adfdfa199bd8 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@72ccc9f[type=IP, id=127.0.0.1, duration=PT0.03S, count=2], session=null, user=null 2022-11-03 09:31:30,872 16e1b7ec-9d02-4a18-b972-340402228a72 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@24ea7668 2022-11-03 09:31:30,872 16e1b7ec-9d02-4a18-b972-340402228a72 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@17dcce0f[type=IP, id=127.0.0.1, duration=PT0.042S, count=2], session=null, user=null 2022-11-03 09:31:30,877 44ef7557-d84f-466e-aeb6-b20db8c2d4d8 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@6c588cf5 2022-11-03 09:31:30,878 44ef7557-d84f-466e-aeb6-b20db8c2d4d8 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@183a9c66[type=IP, id=127.0.0.1, duration=PT0.027S, count=2], session=null, user=null 2022-11-03 09:31:30,980 4b3ba59d-fa0a-43b0-a783-746c52253dc8 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@472c85bb 2022-11-03 09:31:30,981 4b3ba59d-fa0a-43b0-a783-746c52253dc8 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@3c76baca[type=IP, id=127.0.0.1, duration=PT0.108S, count=2], session=null, user=null 2022-11-03 09:31:30,998 f99b7eba-b439-47a2-bec6-14ae8b2b9205 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@40197a1 2022-11-03 09:31:30,998 f99b7eba-b439-47a2-bec6-14ae8b2b9205 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@28327311[type=IP, id=127.0.0.1, duration=PT0.121S, count=2], session=null, user=null 2022-11-03 09:31:31,016 4fbf69d5-7f4b-45ec-b445-ee37ac33be17 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@3c93326f 2022-11-03 09:31:31,016 4fbf69d5-7f4b-45ec-b445-ee37ac33be17 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@63636127[type=IP, id=127.0.0.1, duration=PT0.036S, count=2], session=null, user=null 2022-11-03 09:31:31,027 d8f34a59-2cd2-40dd-9914-d15a62642a31 DEBUG servlets.DoSFilter (DoSFilter.java:doFilter(323)) - Filtering org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@7790a707 2022-11-03 09:31:31,027 d8f34a59-2cd2-40dd-9914-d15a62642a31 WARN servlets.DoSFilter (DoSFilter.java:onRequestOverLimit(1544)) - DOS ALERT: Request delayed=2000ms, ip=127.0.0.1, overlimit=OverLimit@2f8095fa[type=IP, id=127.0.0.1, duration=PT0.029S, count=2], session=null, user=null 2022-11-03 09:31:32,838 89e4d19e-b60e-404a-9aa7-6c394d8ff6b6 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@3037fda7 2022-11-03 09:31:32,838 89e4d19e-b60e-404a-9aa7-6c394d8ff6b6 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@3037fda7 2022-11-03 09:31:32,839 89e4d19e-b60e-404a-9aa7-6c394d8ff6b6 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:32,856 29e0b9f8-586f-44b9-a7a5-b24822f55a7e DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@538b9ae0 2022-11-03 09:31:32,856 29e0b9f8-586f-44b9-a7a5-b24822f55a7e DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@538b9ae0 2022-11-03 09:31:32,856 29e0b9f8-586f-44b9-a7a5-b24822f55a7e INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:32,875 a8fc1224-8ad2-417b-9643-ebebffd65aaa DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@244f6e4d 2022-11-03 09:31:32,875 a8fc1224-8ad2-417b-9643-ebebffd65aaa DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@244f6e4d 2022-11-03 09:31:32,876 a8fc1224-8ad2-417b-9643-ebebffd65aaa INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:32,880 f44a7823-8e7f-48bd-b185-c6577e26c264 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@1d925046 2022-11-03 09:31:32,986 98e36ba4-8ba8-4056-9754-31ca0e9f382f DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@3b9899be 2022-11-03 09:31:33,002 53fa94f9-f606-4647-b3aa-1a698f0ef0c1 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@480299bd 2022-11-03 09:31:33,019 c479ac57-8c39-41cf-bd22-e467d6cf05d0 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@62b847f4 2022-11-03 09:31:33,031 e4821957-0bec-42d9-aa47-ccd8b81a18e3 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(385)) - Throttling org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@f7b5325 2022-11-03 09:31:35,853 f44a7823-8e7f-48bd-b185-c6577e26c264 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@1d925046 2022-11-03 09:31:35,853 f44a7823-8e7f-48bd-b185-c6577e26c264 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:35,868 98e36ba4-8ba8-4056-9754-31ca0e9f382f DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@3b9899be 2022-11-03 09:31:35,869 98e36ba4-8ba8-4056-9754-31ca0e9f382f INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:35,892 53fa94f9-f606-4647-b3aa-1a698f0ef0c1 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@480299bd 2022-11-03 09:31:35,893 53fa94f9-f606-4647-b3aa-1a698f0ef0c1 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:38,868 c479ac57-8c39-41cf-bd22-e467d6cf05d0 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@62b847f4 2022-11-03 09:31:38,869 c479ac57-8c39-41cf-bd22-e467d6cf05d0 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:31:38,879 e4821957-0bec-42d9-aa47-ccd8b81a18e3 DEBUG servlets.DoSFilter (DoSFilter.java:throttleRequest(429)) - Allowing org.apache.knox.gateway.filter.XForwardedHeaderRequestWrapper@f7b5325 2022-11-03 09:31:38,880 e4821957-0bec-42d9-aa47-ccd8b81a18e3 INFO knox.gateway (KnoxLdapRealm.java:getUserDn(688)) - Computed userDn: uid=tom,ou=people,dc=hadoop,dc=apache,dc=org using dnTemplate for principal: tom 2022-11-03 09:32:00,813 DEBUG servlets.DoSFilter (DoSFilter.java:removeFromRateTrackers(1308)) - Tracker removed: 127.0.0.1 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
