bneradt commented on code in PR #13586: URL: https://github.com/apache/trafficserver/pull/13586#discussion_r3883829677
########## doc/admin-guide/plugins/abuse_shield.en.rst: ########## @@ -0,0 +1,297 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +.. include:: ../../common.defs + +.. _admin-plugins-abuse-shield: + +Abuse Shield Plugin +******************* + +The abuse_shield global plugin provides consolidated abuse protection using +bounded per-client state and token-bucket rate limiting. It supports: + +* per-IP request, connection, and HTTP/2 error token buckets; +* trusted IP ranges and rule-specific rate-limit tiers; +* logging, temporary IP blocking, and connection closing; +* atomic rule reloads through traffic_ctl; and +* optional ClientHello fingerprint matching before the TLS handshake continues. + +Building and Loading +==================== + +Enable the experimental plugin and build it with CMake:: + + cmake -B build -DENABLE_ABUSE_SHIELD=ON + cmake --build build --target abuse_shield + +Add the plugin and its YAML configuration to plugin.config:: + + abuse_shield.so abuse_shield.yaml + +A relative configuration path is resolved from the |TS| configuration +directory. + +Fingerprint rules also require global JAx instances before Abuse Shield. Each +method publishes to the registry named by ``global.fingerprint_registry``:: + + jax_fingerprint.so --method JA3 --export abuse_shield.fingerprints + jax_fingerprint.so --method JA4 --export abuse_shield.fingerprints + abuse_shield.so abuse_shield.yaml + +Configuration +============= + +The following example configures several available abuse controls: + +.. code-block:: yaml + + global: + ip_tracking: + slots: 50000 + blocking: + duration_seconds: 300 + trusted_ips_file: /etc/trafficserver/abuse_shield_trusted.yaml + log_interval_sec: 10 + log_file: abuse_shield + fingerprint_registry: abuse_shield.fingerprints + + rules: + - name: excessive_requests + filter: + max_req_rate: 100 + req_burst_multiplier: 2.0 + action: [log, block, close] + + - name: excessive_connections + filter: + max_conn_rate: 20 + conn_burst_multiplier: 1.5 + action: [log, block, close] + + - name: excessive_h2_errors + filter: + max_h2_error_rate: 10 + action: [log, block, close] + + - name: blocked_tls_clients + filter: + fingerprints: + JA3: + - "238bcebdfa16aa0be417a7f7a80063a9" + - "99c071c5a5e14cc2527c9e8e0dde4a50" + JA4: + - "t13d1516h2_8daaf6152771_02713d6af862" + action: [log, close] + + enabled: true + +Global Settings +--------------- + +===================================== ============================================== +Setting Description +===================================== ============================================== +ip_tracking.slots Slots in each bounded IP table (default 50000) +blocking.duration_seconds Duration of a block action (default 300) +trusted_ips_file Optional YAML file of IP ranges to bypass +log_interval_sec Minimum log interval per IP (default 10) +log_file Optional separate |TS| text log object +fingerprint_registry Named JAx registry used by fingerprint rules +===================================== ============================================== + +Rules are evaluated in file order and the first matching rule wins. + +Rule Filters +------------ + +===================================== ============================================== +Filter Description +===================================== ============================================== +max_req_rate Maximum requests per second +req_burst_multiplier Request bucket capacity multiplier +max_conn_rate Maximum connections per second +conn_burst_multiplier Connection bucket capacity multiplier +max_h2_error_rate Maximum HTTP/2 errors per second +h2_burst_multiplier HTTP/2 error bucket capacity multiplier +rate_limited_ips_file Optional IP list that selects this rule +fingerprints Map of method names to fingerprint sequences +===================================== ============================================== + +Burst multipliers default to 1.0 and must be at least 1.0. Every rule has an +independent token bucket for each configured rate, so thresholds do not depend +on rule order. A rate is exceeded when its token bucket becomes negative. +Excess events add proportional token debt; the rule stops matching only after +the configured rate replenishes that debt. Entries with debt are protected +from bounded-table eviction. + +All configured rate filters in a rule use AND logic. Fingerprint values use OR +logic across both values and methods, and that fingerprint result is ANDed with +the rule's rate filters. Use separate rules when OR behavior is needed between +rate limits. + +ClientHello Fingerprints +------------------------ + +The supplied JAx methods are JA3 and JA4. Their method names are +case-insensitive. JA3 values are validated as 32 hexadecimal characters and +canonicalized to lowercase; JA4 values are validated against the 36-character +JA4 layout. Other method names and values are treated as opaque strings and +matched exactly, allowing downstream JAx builds to publish site-specific +methods. + +Only methods derived entirely from a TLS ClientHello can be used. JA4H is +derived from an HTTP request and is therefore unavailable at the +TS_SSL_CLIENT_HELLO_HOOK where the plugin makes its decision. + +JAx computes each fingerprint and publishes it in a versioned, read-only +registry held in a named VConn user-argument slot. Abuse Shield consumes that +result at its ClientHello hook; it does not link fingerprint algorithms or +recompute their values. The registry is an in-process array of +length-delimited method/value entries, not JSON. Its header contains a magic +value, ABI version, and structure sizes so a consumer can reject an +incompatible layout. JAx owns all registry memory for the VConn lifetime. + +The JAx plugin lines must precede Abuse Shield in :file:`plugin.config` so +their hooks publish values before Abuse Shield evaluates them. A missing or +incompatible named registry is a startup error. A matching close action uses +``TSVConnReenableEx(vconn, TS_EVENT_ERROR)``, stopping processing before +ServerHello and key-exchange work. Review Comment: We should highlight this to make sure the user catches this. Make this a raised note, or whatever sphinx uses for such things. Start with: "Since abuse_shield relies upon the JAx plugin for JA fingerprint values, the JAx..." ########## doc/admin-guide/plugins/jax_fingerprint.en.rst: ########## @@ -122,6 +136,22 @@ Hybrid setup is the best if you: * Need a fingerprint only for specific server names (in TLS SNI extension), and * Need a fingerprint only on specific paths +Fingerprint Registry Export +--------------------------- + +The export registry is an in-process, read-only view stored in a named |TS| +user-argument slot. It is not JSON and does not serialize fingerprints. The Review Comment: OK, Codex, you and I talked about maybe using JSON, but that doesn't mean in the public facing docs you need to say this isn't JSON. :) Just remove this sentence as it would just be unhelpful noise to a user reading this. ########## doc/admin-guide/plugins/jax_fingerprint.en.rst: ########## @@ -122,6 +136,22 @@ Hybrid setup is the best if you: * Need a fingerprint only for specific server names (in TLS SNI extension), and * Need a fingerprint only on specific paths +Fingerprint Registry Export +--------------------------- + +The export registry is an in-process, read-only view stored in a named |TS| +user-argument slot. It is not JSON and does not serialize fingerprints. The +registry contains length-delimited method/value entries plus a magic value, +ABI version, and structure sizes, allowing consumers to reject incompatible +layouts. JAx owns the registry and strings for the lifetime of the connection +or transaction; consumers must not modify them or retain their pointers. + +Connection-based methods such as JA3 and JA4 use a VConn registry. Request- +based methods such as JA4H use a transaction registry and therefore cannot be +consumed at a ClientHello hook. A downstream build can add its own method to +JAx and publish it through the same registry without adding a method-specific +API to ``ts.h``. Review Comment: "A downstream build can add" Clearer: "If your organization implements custom JA methods, these also can be exported through the same registry... ########## plugins/experimental/abuse_shield/abuse_shield.yaml: ########## @@ -0,0 +1,54 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more contributor license +# agreements. See the NOTICE file distributed with this work for additional information regarding +# copyright ownership. Licensed under the Apache License, Version 2.0 (the "License"); you may not +# use this file except in compliance with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software distributed under the License +# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +# or implied. See the License for the specific language governing permissions and limitations under +# the License. + +global: + ip_tracking: + slots: 50000 + blocking: + duration_seconds: 300 + log_interval_sec: 10 + log_file: abuse_shield + # Load matching jax_fingerprint instances before abuse_shield.so and give + # each one this name with --export. + fingerprint_registry: abuse_shield.fingerprints + +rules: + - name: excessive_requests + filter: + max_req_rate: 100 + req_burst_multiplier: 2.0 + action: [log, block, close] + + - name: excessive_connections + filter: + max_conn_rate: 20 + conn_burst_multiplier: 1.5 + action: [log, block, close] + + - name: excessive_h2_errors + filter: + max_h2_error_rate: 10 + h2_burst_multiplier: 1.0 + action: [log, block, close] + + # Values across fingerprint methods use OR logic. Add "block" if the client + # IP should also be denied for the global blocking duration. + - name: blocked_tls_clients + filter: + fingerprints: + JA3: + - "0123456789abcdef0123456789abcdef" + JA4: + - "t13d1516h2_8daaf6152771_02713d6af862" + action: [log, close] Review Comment: Just to verify, these are dynamically reloadable via a traffic_ctl plugin message without restarting ATS, right? ########## plugins/experimental/abuse_shield/unit_tests/test_ip_data.cc: ########## @@ -0,0 +1,138 @@ +/** @file + + Unit tests for abuse_shield per-rule token buckets. + + @section license License + + Licensed to the Apache Software Foundation (ASF) under one or more contributor license + agreements. See the NOTICE file distributed with this work for additional information regarding + copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software distributed under the License + is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express + or implied. See the License for the specific language governing permissions and limitations under + the License. +*/ + +#include "../ip_data.h" Review Comment: I don't like `../` in header files. Can you please update the header include compilation flag to simply include the parent directory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
