Hello,
Thanks for sharing this incident with the community. On May 17, because of an
incorrect filter configuration, routers of AS202734 exported a number of
prefixes from its internal routing table to the route collector of the
Hurricane Electric BGP looking glass (bgp.he.net).
For BGP sessions with upstream transit providers (real routers), AS202734 uses
a tighter set of filters, and export only prefixes that are within a small
number of whitelisted "ANNOUNCED_v4/v6" subnets. Thus, none of the said
prefixes were ever exported to real routers on the internet, and this incident
did not affect any public routing infrastructure. However, this incident might
have triggered a number of automated emails from the HE.net looking glass to
those who subscribed to updates about the affected prefixes. These are truly
what one would describe as "false alarms". If you did receive one of these
emails and was alerted, you have my apologies.
For the benefit of the NANOG community, here is a detailed description of what
triggered this "false alarm" incident.
AS202734/AS402333 runs a custom fork of BIRD3 as the routing control plane for
its routers. The configuration files for these routers are rendered
automatically from a Jinja template- and the aforementioned export filter is no
exception. The same filter template controls both routes exported to transit
providers and public looking glass route collectors such as bgp.he.net.
When AS202734 receives a prefix from its upstreams or IX peers, the import
filter would automatically tag the route with a large community of the form ({{
cold_potato_asn }}, {{ router_ld2_id }}, {{ neighbor_asn }})). This community
identifies the source of the route, as well as the AS202734 router where the
prefix was learned.
BGP looking glasses- bgp.he.net, bgptools, etc.- typically require a full table
to be exported- not just routes whitelisted for announcement to upstreams.
Since the whitelist filter no longer applies, some alternative method would be
necessary to filter out routes that are in the iBGP table, but are not learned
from peers or upstreams- these include internal routing infrastructure
prefixes, as well as other types of routing optimizations that should not be
exported to the public internet. It seems like a natural option to build this
filter around the aforementioned BGP Large Community, which is supposedly
applied only to routes learned from other networks, and not from the internal
infrastructure. As seen in line 30 of the Jinja template below,
"session_export_non_local" evaluates to True for route collectors (but not for
upstream transit providers,) and routes would be accepted for export to the
collectors even if they are not in the "announced" whitelist- as long as they
are tagged
with this large community when learned from peers or upstream. In the filter
for regular upstream transit providers, the original whitelist still applies.
This setup worked just as expected for months, and routes observed at AS202734
were visible through various public looking glasses, helping other networks
understand the reachability of their prefixes. However, up to this point, there
is no clear way to determine from which session or geographic region was a
particular prefix exported- either to the route collector or to a real
upstream. To resolve this, it seems natural to reuse the same large community-
tagging exported routes with geographical router info and session info in the
export filter. See line 15 of the code snippet below:
# v6 version of the filter is shown; the filter for v4 follows the same
logic
1 filter ebgp_export_v6_{{ session_id }} {
2 {% for community in session_communities %}
3 {% set community_value = community if community is string else
(community | join(', ')) %}
4 bgp_community.add(({{ community_value }}));
5 {% endfor %}
6 {% for community in session_large_communities %}
7 {% set community_value = community if community is string else
(community | join(', ')) %}
8 bgp_large_community.add(({{ community_value }}));
9 {% endfor %}
10 {% for community in session_ext_communities %}
11 {% set community_value = community if community is string else
(community | join(', ')) %}
12 bgp_ext_community.add(({{ community_value }}));
13 {% endfor %}
14
15 # Added on the day of the incident:
# bgp_large_community.add(({{ cold_potato_asn }}, {{ router_ld2_id
}}, {{ neighbor_asn }}));
16
17 {% for prepend in session_prepend %}
18 bgp_path.prepend({{ prepend }});
19 {% endfor %}
20
21 if net = ::/0 then {
22 {% if session_export_default %}
23 accept;
24 {% else %}
25 reject;
26 {% endif %}
27 }
28
29 if (net ~ ANNOUNCED_v6) then accept;
30 {% if session_export_non_local %}
31 if (bgp_large_community ~ [ ({{ cold_potato_asn }}, *, *) ]) then
accept;
32 {% endif %}
33 reject;
As you might have anticipated, the issue is that this newly-added BGP large
community applies to all prefixes routes in the table, and not just those
directly learned from upstream/peers. As a result, a large number of
internal-only prefixes (4,622 to be exact) with missing BGP PATH info also
received this large community. While the whitelisting mechanism prevented these
prefixes from being announced to upstream transit providers, these prefixes
were announced to route collectors of BGP.he.net from two of the AS202734
routers.
The fix, as you might expect, is to move the filter "accept" logic into its own
function, and apply any information communities only after if the "accept"
function returned true and no "reject" filter was triggered. After deploying
this fix and testing extensively, AS202734 is now once again contributing data
to the public looking glasses- with the BGP large community included to help
engineers of other networks better understand the reachability of their
prefixes across geographies. To avoid similar incidents in the future, a
staging environment paired with extensive code reviews might be helpful.
I hereby thank the NANOG community and in particular Zhong Miao, the
Independent Security Researcher, for bringing this issue to my attention so
swiftly. I also appreciate comments from the community regarding best practices
to make the deployment process more reliable and secure.
Regards,
Jacob-Junqi Tian
May. 23, 2026.
_______________________________________________
NANOG mailing list
https://lists.nanog.org/archives/list/[email protected]/message/DOAA6GMU234E5XXMNYR53F7MMGBHHILD/
