Re: Networks ignoring prepends?

[James Jun]

On Wed, Jan 24, 2024 at 09:22:06AM -0800, William Herrin wrote:
> On Wed, Jan 24, 2024 at 8:39???AM James Jun  wrote:
> > On Wed, Jan 24, 2024 at 08:16:56AM -0800, William Herrin wrote:
> > > Sophistry. I buy IP transit from 3 providers, one of which has a 3 AS
> > > path to 3356.
> >
> > Again you omit context.
> 
> What you're calling context, I call deceptive.
> 
> For one thing, Centurylink's process is, like a spammer, opt-out
> rather than opt-in. 

Nope. Your allegation that Lumen (Centurylink)'s "process" is out-out like a 
spammer is factually and historically incorrect.  However, Lumen's practice is 
complaint with best common practices and experiences as documented on RFC 4277 
and provided by RFC 4271.

Lumen/Centurylink's alleged "opt-out spamming" practice predates their very 
existence and was established during the NSFNET, with an operational need at 
the time to differenciate commercial networks from R networks. Just as R 
networks needed to treat commercial network traffic differently during the 
needs of the NSFNET, commercial operators of the Internet are also expected and 
demanded to prioritize traffic by their paying customers, over non-paying 
customers.

> 3356 enables the local pref unless told through a
> BGP community not to. There's no evidence that 47787 even knows that
> Centurylink is preferring them despite shorter AS paths elsewhere, let
> alone desires that behavior. Indeed, given the prepends that 47787
> added, it's quite possible they desire the opposite.

The evidence is widely documented and is in best common practices of every 
major ASN exercising routing policy and subsequent RFCs and BCPs published 
concerning discussions herein.  Internet standards and documented widely 
accepted current practices exist for a good reason.  Your, or alleged 47787's 
possibility of failure, ignorance or act of ommission in being informed of how 
the current practices work does not make you any less responsible in 
identifying the problem at hand.  Your allegation and arguments that currently 
adopted and documented inter-AS traffic engineering practices are deceptive and 
"opt-out" in a bad-faith nature are simply too tenuous a connection and amount 
to reductio ad absurdum.  You are however welcome to participate in IETF 
process to propose to alter the way BGP practices work for the better, as you 
wish.  That's what's so great about community input-based policy development 
processes.

> 
> For another, a key implication in your "context" is that if one
> customer intentionally pays 3356 to intentionally send another
> customer's packets on a longer, slower trip than 3356 otherwise would,
> that's a legitimate above-board business transaction. Not obviously
> corrupt.

False.  None of the parties described herein, neither 47784, nor 3356 are 
liable in "intentionally" sending traffic of another customer on a longer, less 
efficient path.  What they are however likely liable for, are contractual 
obligations and commercial expectations of bilateral parties engaged in an 
ongoing transaction.  You fit into the chain of buying from 53356 without 
understanding the underlying infrastructure and connectivity relationships that 
53356 has toward 3356.  And you're now litigating that it's corrupt and is 
possibly some kind of a coordinated scheme or a racket without your consent.  
You gave your consent by agreeing to run BGP with 53356 as your vendor, which 
you awarded that business to, and began advertising your prefix.  It's not 
working the way you want, so engage with your vendor to fix it, or fire them.  
This is not hard.

James

Re: Networks ignoring prepends?

[James Jun]

On Wed, Jan 24, 2024 at 08:16:56AM -0800, William Herrin wrote:
> On Wed, Jan 24, 2024 at 8:11???AM James Jun  wrote:
> > You (AS11875) have an operational need for good connectivity
> > into 3356 but, you made a poor purchasing decision by buying
> > IP transit for 11875 from a provider who has 10-AS path into
> > 3356 instead of <=3 AS path. You've done a _bad_ job here
> > in selecting an inferior pathway into 3356, and what you
> > SHOULD have done is to select an IP transit provider who
> > has an optimal AS-path into 3356 to meet your operational
> > need of having good connectivity into 3356.
> 
> Sophistry. I buy IP transit from 3 providers, one of which has a 3 AS
> path to 3356.

Again you omit context.

We've already established as per the RFC, that calculation of degree of 
preference takes precedence over and overrides AS_PATH (Phase 1 decision).  

Therefore, let's rephrase what you've just said above:

You're buying IP transit from 3 providers, two of which are configured with the 
following known constraints:

- 20473 who buys from 1299, who has lower degree of preference into 3356, as 
1299 and 3356 are interconnection (could be settlement-free or paid-peer) 
peering partners.
- 53356 who buys from 47787 as a prioritized downstream customer, and then 
47787 too subsequently connects into 3356 as a prioritized downstream customer.

It's obviously clear that 53356 path you've bought has a priority ticket into 
3356 no matter how inferior or long its AS_PATH may be, and the solution is 
right in front of you.  Next.

James

Re: Networks ignoring prepends?

[James Jun]

On Wed, Jan 24, 2024 at 07:25:42AM -0800, William Herrin wrote:

[ snip ]

> or I chose my words poorly. What I did say, and stand behind, was that
> applying local prefs moves BGP's route selection off the _defaults_,
> and if Centurylink was routing to me based instead on the defaults
> they'd have made a _good_ route selection instead of a _bad_ one.

This cuts both ways Bill.  First, 3356 is making an intended route selection, 
their customer who interconnects directly into 3356 demands this.  That 
customer who connects into 3356 probably had no idea that you (AS11875) would 
someday decide to take IP transit from a downstream AS of them, and your 
situation was likely never in their minds of consideration in their network 
planning.

_You_ want better connectivity from 3356 to 11875 for the explicit benefit of 
11875, which _you_ operate and control.  That's good, so let's continue.


> 
> I do care whether you're routing packets in a reasonable way. When you
> pick the 10-AS path over the 3-AS path because the 10-AS path arrives
> from a customer, the odds that you're routing those packets in a
> _good_ way are very low. I get that a lot of you do that. I'm telling
> you that when you do, you're doing a _bad_ job. If you think you're
> justified, well, it's your business. But don't doubt for a second that
> you've served your customers poorly.

Conversely at the same time, the below is also equally true:

You (AS11875) have an operational need for good connectivity into 3356 but, you 
made a poor purchasing decision by buying IP transit for 11875 from a provider 
who has 10-AS path into 3356 instead of <=3 AS path.  You've done a _bad_ job 
here in selecting an inferior pathway into 3356, and what you SHOULD have done 
is to select an IP transit provider who has an optimal AS-path into 3356 to 
meet your operational need of having good connectivity into 3356.


> And before you suggest that I'm not your customer, let me point out
> what should be obvious: if none of your paying customers were trying
> to reach my network, I wouldn't notice which direction you routed my
> packets, let alone care. It's not about serving me, it's about serving
> your paying customers. My packets are their packets, and when you send
> _their_ packets along the scenic route, you have done a bad job.

We can do this all day long.  You (AS11875) also have the responsibility to 
yourself and your end-users to select and award business to an IP transit 
provider and make every reasonable efforts to ensuer that 11875 has good 
connectivity into 3356 as your operational needs require.  You've abrogated 
that responsibility in your own AS and decided to spew non-sense over the most 
critical and important knob that is more important than AS_PATH (LOCAL_PREF) in 
BGP-4 that was developed since NSFNET days and are telling us that we're doing 
a poor job.  Your argument fails.

The internet works upon the principle of "best-effort."  What you're describing 
is the net effect of that "best-effort", and you, as the operator and 
controller of AS11875 which is involved in the path are just as culpable and 
responsible.  Moreover, you, by being the operator of an AS in the problematic 
path, have the wherewithal and commercial ability to fix it, without involving 
the rest of us.  The answer right is in front of you.

James

Re: Networks ignoring prepends?

[James Jun]

On Tue, Jan 23, 2024 at 10:12:33PM -0800, William Herrin wrote:
> Respectfully Chris, you are mistaken.
> 
> https://datatracker.ietf.org/doc/html/rfc4271#section-9.1.2.2
> 
> "a) Remove from consideration all routes that are not tied for having
> the smallest number of AS numbers present in their AS_PATH
> attributes."
> 
> So literally, the first thing BGP does when picking the best next hop
> is to discard all but the routes with the shortest AS path.

Not true.  Read the whole RFC--you've ommitted Sections 9.1 and 9.1.1, which 
are very critical.

Discarding all but the routes with shortest AS path is _not_ literally the 
first thing BGP does as you stated above.

The first thing BGP does is to calculate the degree of preference whenever BGP 
receives a new route, withdrawn route or replacement route (See Section 9.1.1). 
 The determination of the degree of preference is considered to be a local 
matter for each Autonomous System exercising route policy, typically expressed 
using LOCAL_PREF, to execute upon the configured administrative policy to class 
the incoming routes.

After completion of 9.1.1, section 9.1.2 and 9.1.2.2 which you cited begins 
(Phase 2: Route Selection).  Route selection under 9.1.2 is only invoked after 
degree of preference is determined (called 'Phase 1' decision) as clearly 
described in Section 9.1.

In fact, even in 9.1.2.2 that you cited above, it clearly states:

   In its Adj-RIBs-In, a BGP speaker may have several routes to the same
   destination that have the same degree of preference. 

   [ snip ]

   The following tie-breaking procedure assumes that, for each candidate
   route, all the BGP speakers within an autonomous system can ascertain
   the cost of a path (interior distance) to the address depicted by the
   NEXT_HOP attribute of the route, and follow the same route selection
   algorithm.

   The tie-breaking algorithm begins by considering all equally
   preferable routes to the same destination, and then selects routes to
   be removed from consideration.  The algorithm terminates as soon as
   only one route remains in consideration.  The criteria MUST be
   applied in the order specified.

   [ snip ]

  a) Remove from consideration all routes that are not tied for
 having the smallest number of AS numbers present in their
 AS_PATH attributes.  Note that when counting this number, an
 AS_SET counts as 1, no matter how many ASes are in the set.



So you see, the comparison of AS_PATH and therefore the route selection process 
could only begin after routes are first resolved by their degree of preference, 
often typically exercised by LOCAL_PREF across the AS (or other similar import, 
such as Cisco's "weight" parameter which is applied before LOCAL_PREF locally 
significant to the router itself where its been configured).  The route 
selection process, including the elimination of routes with inferior AS paths, 
is a tie-breaker algorithm after degree of preference is first calculated, 
which is what we've been trying to tell you.  So no, AS_PATH comparison is not 
literally the first thing BGP does.

You're ignoring Section 9.1.1 in its entirety, which chronologically begins 
before Section 9.1.2.2 (the section you cited), which also clearly specifies 
that route selection process described in it (including AS_PATH comparison) is 
a tie-breaking procedure. 


> 
> It also says that BGP implementations are -allowed- to use other
> selection criteria.


Further followed by the following clause immediately afterwards: 
  "BGP implementations MAY use any algorithm that produces the __same results__ 
as those described here."

And restricted by the following clause in the preceding paragraph:
  "The criteria MUST be applied in the order specified."

And clarified by Section 9.1:
  "as long as the implementations support the described functionality and they 
exhibit the same externally visible behavior."


> And there are many situations where doing so is
> well advised and improves the result. But AS path length is
> unambiguously the default, off which a user has to move it.


So, when a BGP implementation is written in a router software, how does the 
manufacturer know whether your network is going to need to be applying lot of 
degrees of preference, or none?  The vendors have no idea, and RFC also 
clarifies that degree of preference is a local policy matter.  Therefore, the 
default behavior is to assume a universally same LOCAL_PREF until a policy is 
configured, which typically has been '100' across many vendor implementations.  
In this instance, since all routes have the same degree of preference of 100, 
Section 9.1.2.2 you cited then begins to tie-break the routes of same 
preference, starting with the AS_PATH comparison, but it is absolutely by no 
means, the first thing BGP does, at all.  The first thing BGP does as clearly 
specified in the RFC is to determine the degree of preference to meet local 
routing 

Re: Networks ignoring prepends?

[James Jun]

William Herrin wrote:
> Nevertheless, in the protocol's design, the one expressed in the RFC's, AS 
> path length = distance.

Since we're opening RFCs now, and somehow it is being opined that LOCAL_PREF is 
a profit-driven conspiracy and a coordinated scheme concocted by commercial 
networks to tamper with, or "override" AS_PATH desires of the majority, let us 
review factually about what LOCAL_PREF actually does and why it was implemented 
into BGP in the first place:

RFC 4277 entitled "Experience with the BGP-4 Protocol", Section 20:

   The NSFNET program used EGP, and then BGP, to provide external
   routing information.  It was the NSF policy of offering different
   prices and providing different levels of support to the Research and
   Education (RE) and the Commercial (CO) networks that led to BGP's
   initial policy requirements.  In addition to being charged more, CO
   networks were not able to use the NSFNET backbone to reach other CO
   networks.  The rationale for higher prices was that commercial users
   of the NSFNET within the business and research entities should
   subsidize the RE community.  Recognition that the Internet was
   evolving away from a hierarchical network to a mesh of peers led to
   changes away from EGP and BGP-1 that eliminated any assumptions of
   hierarchy.

   Enforcement of NSF policy was accomplished through maintenance of the
   NSF Policy Routing Database (PRDB).  The PRDB not only contained each
   networks designation as CO or RE, but also contained a list of the
   preferred exit points to the NSFNET to reach each network.  This was
   the basis for setting what would later be called BGP LOCAL_PREF on
   the NSFNET.  Tools provided with the PRDB generated complete router
   configurations for the NSFNET.


RFC 4271 entitled "A Border Gateway Protocol 4 (BGP-4)" (supersedes RFC 1771), 
Section 5.1.5:

   A BGP speaker SHALL calculate the degree of preference for
   each external route based on the locally-configured policy, and
   include the degree of preference when advertising a route to its
   internal peers.  The higher degree of preference MUST be preferred.
   A BGP speaker uses the degree of preference learned via LOCAL_PREF in
   its Decision Process (see Section 9.1.1).



It is clear by the experiences of NSFnet and early days of the Internet, that 
AS_PATH alone is insufficient to meet interconnection policy objectives.  In 
fact, this LOCAL_PREF "conspiracy" was actually concocted by Research and 
Education (R) networks to make evil commercial networks pay--but in reality, 
NSFnet and early R networks had actual operational and demonstrated reasons 
for this, and a path vector routing protocol where cross-border interconnection 
policies must be applied cannot simply rely on AS_PATH for decision mechanism.  
Otherwise, it'd have been easier to just scale up RIP into a global routing 
protocol instead of using BGP.  

This is where your argument and basis of your claim fails-- a parameter to 
express administrative policy preference was required even in early days of 
NSFnet, and that is why LOCAL_PREF was put in there in the first place, despite 
your assertions claiming it is broken and being used to "override" AS_PATH to 
small guys for bad faith reasons.  This was not some later "add-on" for 
conspiracy by commercial networks; LOCAL_PREF in fact, was one of the principal 
features and reasons for developing BGP-4.  You're 29 years late to this 
conversation buddy.



> > 4. Get yourself connected to 3356 directly.
> 
> I am, just not as a BGP customer. And I won't be as a BGP customer.
> Opening a ticket with them has not yielded results. Or any response
> from network engineering at all. Just the frontline support who wants
> me to reboot my modem. :(

I get that you are not in the position to buy from 3356, and to that extent, 
that is a completely respectable and reasonable position (commercial reasons, 
personal experience/preference or otherwise, you are the customer here).  But 
you have a voice as a customer on which BGP transit provider you're purchasing 
on the other end (the far-end location or data center where your ASN is 
operating and taking transit from) -- take it as a lesson learned going 
forward:  when choosing a smaller/nimble or blended bandwidth IP provider, make 
sure you to ask, what can the provider do to help you achieve better 
connectivity into 3356 or any other network you're trying to get to?  It's your 
transit provider's business to make sure your ASN's connectivity works to your 
expectations.  Otherwise why would you, the customer, choose to do business 
with a middle-man when you could just buy direct from 3356 at the data center 
for your ASN instead?  It is incumbent upon your IP transit provider to help 
you better meet your connectivity requirements (especially for retail and small 
traffic customers in data centers like yourself who are not subject to capacity 
or comercial interconnection disputes), 

Re: Networks ignoring prepends?

[James Jun]

On Mon, Jan 22, 2024 at 02:03:48PM -0800, William Herrin wrote:
> 
> It offends my pride to handle it this way, but -you- shoulder the cost.
>

You're misdiagnosing the issue at hand.

CL is choosing 3356 47787[x3] 53356 11875[x3] over better path via 1299:

What you need to be doing is reaching out to AS53356 (your upstream provider 
supposedly) to assist with traffic engineering.  Given the # of prepends that 
53356 added themselves, it looks like you're using their communities to prepend 
on top of your own prepends (wasted effort), or they've attempted to help you 
by prepending manually, but to no avail (see our prior discussion).  The next 
level of escalation is for 53356 to now work with 47787 to implement the 
correct traffic engineering policy facing 3356.

This is really something your IP transit providers should be assisting you 
with.  You're misdiagnosing and complaining about something which BGP is 
supposed to be doing, instead of escalating with the right parties who are in 
the best position to be assisting you.

Believe it or not, there are small-medium IP transit providers who are _very 
good_ at assisting their BGP customers in traffic engineering efforts, 
especially with extensive BGP community options, competent network engineers, 
automation and the likes.  Your upstream providers need to step up their game 
to help you out here.  This is not a Lumen/CenturyLink/Level 3 problem.

HTH,
James

Re: Networks ignoring prepends?

[James Jun]

On Mon, Jan 22, 2024 at 06:02:53AM -0800, William Herrin wrote:
> On Mon, Jan 22, 2024 at 5:24???AM Patrick W. Gilmore  
> wrote:
> > Standard practice is to localpref your customers up, which makes prepends 
> > irrelevant. Why would anyone expect different behavior?
> 
> It gives me, your paying customer, less control over my routing
> through your network than if I wasn't your paying customer. That
> seems... backwards.
>

Nope, that is not at all backwards.

Have you actually wondered what would happen, if every major ISP stopped 
classifying routes with localpref, and treated every route received by them 
(including customers and external peers) on same local-pref, so your AS 
prepending can work easily?

Some 21 years ago, there was this little known story during early stages of the 
IPv6 development, called 6bone.  Aside from the lack of native IPv6 (where 
everything had to be tunneled), the #1 issue that guaranteed IPv6 sucked many 
times worse than IPv4 back in the day was the lack of BGP clue by most of IPv6 
DFZ participants at that time, where nobody classified any of their routes 
accordingly with localpref and communities.

Not classifying your routes with local-pref leads to complete operational 
chaos, including world-tour hair-pin sightseeing becoming very common with IPv6 
during 6bone days (which resulted in rise of as30071/occaid to dominate the 
IPv6 DFZ for several years for many to transition out of 6bone).  Not 
classifying routes with local-pref means you do not care whether a particular 
peer is a settlement-free peer or a customer-- this lack of relationship 
classifiction leads to operational harm:  A customer may be paying you $/bits 
expecting you to deliver your on-net traffic onto them over their paid peering 
(or transit) link they bought from you, except, only to find you preferring an 
IX peer (e.g. Hurricane Electric, etc. over IX) as best-path, even without any 
AS Path prepending involved. 

Further, not classifying routes with local-pref and ident communities means you 
are entirely at the mercy of prefix-lists applied on your export policy.  A 
very common occurrence is often a rookie ISP appeared to be giving "transit" to 
a major Tier-1 backbone on a route that was supposed to be customer-originated 
route, but this network selected AS-Path via its uptream provider as best-path, 
instead of direct connection into the said customer.  This happens a lot on a 
route that is "downstream of a downstream" customer, who is also multi-homed 
with the said rookie ISP's upstream Tier-1 provider, thereby resulting in 
equidisant AS-Paths to what is supposed to be a customer-originated route.  
Scale this up to many routes and you have complete chaos and breakdown of your 
BGP routing table.



So, as a customer, you actually SHOULD be demanding your ISPs to positively 
identify and categorize their routes using local-pref and communities.  In 
fact, I will never purchase IP transit with BGP from a provider who doesn't 
categorize routes with local-pref.  As a customer, if you want more control 
over your network's incoming traffic, you need to instead ask your upstream 
providers about their BGP routing policy and how well they support BGP 
communities to let you steer traffic, and use those communities to make 
absolute traffic decisions.



Always remember this #1 rule of BGP decision process:  AS Path is a 
**tie-breaker** to local-pref classification.  When you prepend AS Path, your 
goal is to try to steer traffic from routes that are in the same category (i.e. 
customer or peer) as you.  When your goal is absolute steering (i.e. absolute 
as in, do not advertise to a particular peer, or make your connection standby 
backup where no traffic ever comes until there is complete outage on the other 
path, etc), you absolutely SHOULD be using BGP communities provided by your 
upstream IP provider.  If your IP transit provider does not provide extensive 
BGP communities to meet your requirements, cancel their service and give your 
business to someone else.

A rookie BGP mistake that is commonly made made by those without real-world 
experience, is the assumption that AS Path prepending should deliver absolute 
traffic steering -- it does not, and should NOT, by design.  The BGP Best-Path 
Selection Algorithm is taught very well in the CCIE curriculum, but last I 
looked, they don't teach you on the _why_, only on on the how.  So it's common 
to see enterprise CCIE's working for VARs often falling into the false 
assumption of AS Path.  See 
https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13753-25.html#toc-hId-1778347102

Hope this clarifies.

James

Re: 165 Halsey recurring power issues

[James Jun]

On Mon, Oct 23, 2023 at 03:31:21PM -0400, Babak Pasdar wrote:
> 
> Is the UPS the battery or the battery and controller combined?

"N+1" nominally means you're connected to the same UPS system/complex, but each 
of your feed is on a different module.  Your other leg will be diverse from a 
failure in that module, or downstream PDU/panel work fed by that module.  A 
common failure mode in the UPS system itself hosting the different modules can 
knock out both of your circuits.  It sounds like this is how you are configured 
presently.

"2N" generally means you're connected to completely different UPS 
system/complex and corresponding distribution systems for each of your circuit. 
 This is ideal configuration for most critical loads.

James

Re: 165 Halsey recurring power issues

[James Jun]

On Mon, Oct 23, 2023 at 10:38:09AM -0400, Babak Pasdar wrote:
> I wanted to get some feedback as to what is considered standard A/B 
> power setup when data centers sell redundant power.?? It has always been 
> my understanding that A/B power means individually unique and preferably 
> alternate path connections to disparate UPS units.

Generally speaking, the definition of A/B has become muddied in recent decades. 
 It has almost become an inaccurate marketing term. 

Most sane people have the opinion (myself included) that when "A/B" power is 
offered, it is at minimum offererd as 2N UPS (different building entrance and 
MSBs and even physically separate UPS rooms are also desired on a true 2N A/B, 
but may not always be available).  Some data center operators go even further 
and architect load switching within their distribution, thereby preventing 
single-side/one-leg power outages for customers during most of their power 
maintenance activities

Some data center operators treat "A/B" as convenience for them to undertake 
maintenance and offload uptime responsibilities to their own customers, and 
require them to either undertake their own transfer switching and/or dual-cord 
every equipment, so that they can keep taking one side of the power system down 
for repeated maintenance.  This does not scale well for retail colo, as not 
every customer is going to be good at maintaining two PSUs for every single 
piece of equipment.

Some data centers also view "N+1" system deployment at the UPS as an acceptable 
form of A/B protection, as long as customer circuits are on different PDUs.

Long story short, whether you're receiving N+1 or 2N or 1N, it's important to 
inquire about how your power circuits will be architected and delivered by the 
data center, and either have that codified in the contract or reflected 
appropriately in SLA offering.  There is nothing wrong with the data center 
providing N+1 or 1N power, as long as they're transparent about it and that it 
is what you're willing to accept for the right terms.  However, simply 
accepting "we are providing you A/B power" or "we've never had primary power 
failure" are not sufficient to meet proper due diligence during a site 
selection process, unless you can accept the site outage occurring from time to 
time, or you're deploying your own power plant (i.e. DC power and batteries) to 
supplant data center's own power protection scheme.

James

Re: Conduit Lease/IRU Pricing

[James Jun]

On Mon, Feb 06, 2023 at 06:57:27AM -0500, Fletcher Kittredge wrote:
> A big issue you don't mention is the easement, the legal right to place
> conduit. What does it mean to buy conduit if you don't have an easement on
> the property to use the conduit?

Typically, in large telecom installs like this (esp. for joint trench builds), 
the lead company obtains an Easement Agreement which allows "other 
telecommunications providers", "licensees" or "other designated agents" of the 
lead company to access and use the full enjoyment of the easement areas.  If 
you dig up registry of deeds for some large telecom joint trench builds (I can 
think of at least two examples), you'll find that these come pretty standard.

Further, conduit lease and license agreements of these sort for the buyer 
typically include Underlying Rights clause that also requires the trench system 
owner (the seller) to maintain underlying rights for the purchaser of rights to 
the conduit.  Seller is required to ensure that the buyer has proper legal 
rights to enter and make full enjoyment of the conduit capacity it purchased or 
otherwise licensed from the seller.

For constructions occuring outside of private property, the lead company is 
responsible for engaging local authorities owning the public right-of-way to 
propose the system installation in a multi-tenancy nature (i.e. the system 
meets and exceeds Dig Once and joint trench requirements set out by the 
municipality and so forth); as such, the right-of-way siting permits are 
developed to allow construction of the entire system and with the understanding 
of access by all users, in principle and procedures as provided under 
respective state laws.


James

Re: Conduit Lease/IRU Pricing

[James Jun]

On Sun, Feb 05, 2023 at 01:15:11PM -0600, Mike Hammett wrote:
> I've been following your work on LinkedIn. Great stuff. 
> 
> 
> I'm actually in a situation where I am on both sides of the transaction. I've 
> got a network I built that I've been asked pricing on and interested in 
> growth opportunities. One of the opportunities I have for growth quoted me at 
> roughly the cost of construction (or at least what I would budget for it, 
> anyway) for a 20-year term with a reasonably annual maintenance fee. When I 
> saw that, I kinda figured that if I was going to spend that kind of money, 
> I'd choose a permanent cost as opposed to 20-year terms and the opportunity 
> to place however many conduits I wanted as opposed to just getting one. 
> 
>

Thanks!

Without getting into specifics of your potential project, I can only comment on 
what I've seen and can cite examples of.

You mentioned 'opportunity to place however many conduits I wanted' -- are you 
talking about ability to pull your own innerducts inside an empty outer conduit 
you purchase, or are you talking about a joint trench partaking, where you have 
the opportunity to pay pro-rata share of trench construction to install as many 
conduits you want to have in the ground (subject to local authority approval 
ofcourse)?  


If it is the latter (joint trench), this is very straight forward in the 
utility industry.  It often goes like this:

- Say it costs the lead company (company who is doing the project) a figurative 
(just for example of this conversation) cost of $1 million to install 500 feet 
of 24 - 4" conduits in a large boulevard.
- Your company proposes to jump into the trench and you want 6 - 4" conduits 
for your own backbone.
- The most common and simple cost for you is straight-up pro-rata share:  25% 
of the trench costs for 6 ducts out of 24, so you need to pay up $250K to get 
your 6 - 4" conduits.
- If the lead company is installing smaller pull box manholes for cable pulls, 
in most cases, you will have the right of transit to use those manholes so you 
can use the very conduits you own.
- If the lead company is installing large underground vaults, don't be 
surprised if they don't let you in it -- they'll likely require you to pay 
additional costs to install your own separate manhole, where your 6 - 4" 
conduits will break off from the main trench, and lead into your own dedicated 
4'x4' manhole.  If this is not possible (i.e. road is full, local authority 
couldn't permit it due to conflict & heavy congestion with other utility lines 
in the area), then the lead company may also charge you a reasonable manhole 
license fee for you to use their vaults beyond the basic right of transit 
('beyond' as in, if you need to install a splice case or slack coil, as opposed 
to your cable simply transiting thru the said manhole).  For example, Empire 
City Subawy (ECS) duct system run by Verizon in NYC charges a publicized rate 
of $314/year for each splice case in an ECS manhole.


The legal definitions of what you're exactly getting for paying that $250K 
above is largely up to the lead company and the defined contract terms of the 
Joint Trench Agreement.  I've seen following cases: (a) you outright own the 
title to those 6 - 4" conduits in perpetuity; (b) you don't own the title, but 
you get an IRU or lease of 99 years to those conduits; or (c) you only get 
short-medium (5-25 years) IRU, but then it would probably have to be at a lower 
price that is more commercially reasonable to both parties.

Case (a) can be common in joint trench projects that are organized by local 
authorities (i.e. b/c municipality required the street dig to be a joint 
trench), and lead company has no interest in maintaining any manholes or 
conduits, beyond the bare minimum required for their own cable.  In these 
situations, manholes (municipalities often call them 'joint manholes') become 
effectively unmanaged chaotic no-man's land, where nobody owns the manholes, 
much less maintain them.  I've seen situations where municipality had to step 
in to fix a broken manhole cover/frame, because nobody in the joint trench 
would step in to take responsibility.

Cases (b) and (c) are often done by more larger telecom installations, where 
lead company is building a true joint-use infrastructure and would like to 
maintain it over long term.  These are usually part of large duct systems, and 
the lead company would typically take charge in maintaining the entirety of the 
trench and its manholes over their llifetime; as such, members of such joint 
trench systems will be separately charged maintenance fee as previously 
discussed.  


Outside of straight-forward joint trench projects, what determines "just and 
reasonable" costs in construction projects to outright own a conduit is largely 
a function of how much you (and the other party) are willing to bend, and how 
many lawyers and hardballers are going to be involved.  In many situations, 
sellers may 

Re: Conduit Lease/IRU Pricing

[James Jun]

On Sun, Feb 05, 2023 at 12:36:27PM -0800, William Herrin wrote:
> 
> One quick note: conduit on private property, such as the tail of a
> telco conduit serving a small office complex, is almost always a
> fixture of the property belonging to whoever owns the land. Even
> though the telco installed it for free. It's just too expensive for
> them to dot the i's and cross the t's legally when installing the
> conduit tail, so they generally just do it realizing there's low odds
> of anyone hijacking it from them.
> 

This is an important point that does come up.  In commercial property 
installations, telcos often choose to look over this just like you 
stated--since they control the mainline system out in the street anyway, they 
feel that provides enough of security and not enough of a concern to lose sleep 
over it.

However, in valuable multi-tenant facilities (carrier hotels, data centers, and 
large commercial/urban developer projects), you will also find that telecom 
carriers can, and often will, absolutely enforce their rights to their conduit 
systems (including tails) being placed upon private property.  Access and 
Easement Agreements are frequently used to enforce the purpose of the wayleave 
for large telecom installations occuring upon private property.  Therefore, it 
is important to contact the property owner and the owner of the wayleave (i.e. 
carrier owning the conduit system on private property) for permission/license 
to enter, and never assume that just because a conduit is in private property 
and the landlord thinks you're ok, yo're good to go.  More often than not, it 
is not ok, and you will likely end up getting nasty surprises from the said 
carrier's legal department, when they decide to enforce their rights.

This happened to a large Tier-1 carrier who was pulling fiber into a new 
developer project here.  The property owner told them it was ok to use the 
conduit, and all of the manholes were simply labeled 'FIBER OPTIC' engraved on 
them, so the carrier thought they had permission to pull fiber.  They never 
realized that the ILEC had an easement agreement and those conuits and manholes 
were paid for by the ILEC for installation.  Their fiber cable was discovered 
about a week later when ILEC crews were working to pull their own cable in-- 
the rest is hitory, they had to make amends to obtain proper authorization and 
pay fees, etc. after being discovered.  After this happened, the ILEC also 
replaced all manhole covers in the property with their name and logo engraved 
on them.

James

Re: Conduit Lease/IRU Pricing

[James Jun]

On Sun, Feb 05, 2023 at 11:21:09AM -0600, Mike Hammett wrote:
> I know that location matters, but I hope to be location agnostic.
> 
> How have you seen empty conduits sold? Entire route only, or is a partial 
> route okay? Twenty years only or less? Price compared to cost of 
> construction? Ongoing maintenance costs?

Yes and yes.  Yes on both partial and full route conduits.

Generally speaking, most telcos and utility owners will not sell you conduits.  
The commonly opportune time for you to "buy" a conduit or buy rights to use a 
conduit in perpetuity or for long periods is to join into a common trench when 
a joint trench construction is proposed in the area.  In this instance, in most 
cases, you don't even have to do any construction yourself, you simply specify 
the size/number of ducts you want as a customer and you pay pro-rata share of 
the joint trench construction costs.

Where you can buy an existing conduit, from published projects, I've seen them 
going for sale usually between $100 to $290 per linear foot, this is in Boston. 
 If you do the fine prints and math, when purchasing an empty conduit (even 
buying a municipal conduit), you may find that post-construction conduit sales 
are generally higher in price than what it costed to build it in the first 
place.  This is because most sellers will consider the current market rates and 
field conditions (i.e., they will approximate how painful and costly will be 
for you to re-open the street and build a new duct system for yourself, and 
whether local authorities would even allow that to happen based on how full the 
street is, and based on that difficulty, price will often commensurate 
accordingly).  

One thing to note here is that federal law stipulates (47 USC Sec. 224) that 
attachment rates shall be 'just and reasonable' (Note:  I am not a lawyer, seek 
legal advice from an actual attorney).  As such, this is one point of 
consideration that sellers do often consider, when developing pricing for 
selling a conduit-- attempting to make high-return profits off of an empty 
conduit that is in multiples of construction costs (e.g. many PE or investor 
backed companies will demand high returns in short term after construction), 
could potentially subject them to a regulatory complaint by a telecom attacher 
to the state PUC.  This is one of the reasons that was cited by one 
investor-backed utility in my area, as to why they will refuse to 'sell' empty 
conduits-- however, they will lease conduit space to you for annual recurring 
fee at reasonable rates, so they're compliant with the law.


So, often more common approach to acquiring access to ducts is leasing at a 
recurring fee, as outright purchasing them post-construction is not something 
many utility owners do.  Here, most conduit leases from publicized figures go 
from $0.05/ft/year to $3.5/ft/year for leases by privately held or investor 
owned utilities, and $1-$100/ft/year from state transit agencies.

If you lease a conduit, usually ongoing maintenance costs are baked into the 
cost of your annual lease.  If you purchase an empty conduit outright, or buy 
long-term rights to use it in lump sum, you are often charged pro-rata share of 
O costs, similar to that of condominium fee, to cover ongoing expenses, such 
as utility costs to invest in crew safety training programs, plant protection 
(Dig Safe/USA/one-call locate responses to mark the trench, etc), weekly trench 
patrols, manhole inspections, etc.  You will also pay an inspection fee every 
time you enter and work in a utility-owned manhole, generally priced similar to 
that of hiring a police detail.  These are all reasonable costs and you should 
expect to pay them accordingly when working in a utility conduit system.


You will also find that leasing conduit is a difficult topic in itself.  
Usually it may be easier to engage a heavily regulated incumbent LEC or another 
public utility who is regulated to provide telecom duct space (electric 
transmission owners providing UG duct space for telecom, etc).  Process will 
take long, but these guys are regulated and required by law to provide you 
conduit license at affordable, "just and reasonable" rates.

Another aspect that cannot be ignored when it comes to obtaining duct space, is 
asset trading/exchange agreements.  Telcos (both ILECs and CLECs) love this, as 
much as we network operators in NANOG love peering--you can propose to give 
them duct space in conduits you already have that they don't have (or even 
fiber optic cable capacity in some cases), in exchange for them giving you 
their conduit space to you, in a trade.  Just like IP peering, many asset 
exchange/trade agreements are often settlement-free, but commercially settled 
exchange arrangements are also very popular, specific terms of these agreements 
and negotiations are often confidential.

James

Re: private 5G networks?

[James Jun]

On Wed, Dec 01, 2021 at 12:23:46PM +0100, Baldur Norddahl wrote:

[ snip ]

> 
> And yes these are low bandwidth but on the other hand often stretch wifi to
> the very limits on the distance between bases. I am not claiming this is
> the same use case as a warehouse. I am pointing out that the argument that
> a system critical implementation _must_ be based on licensed frequencies
> does not hold as nothing could be more critical than a system that prevents
> trains from colliding.


The public transit market of rail industry has been in discussions for a while 
re:
mitigation measures (such as licensed band) against possible interference on 
CBTC
signalling data links.  It is however a standardization issue (much like we here
in internet infrastructure continue to discuss improvements to BGP and its 
lingering
security issues, nothing is perfect in every industry I suppose..).

> 
> I do claim that the reason these metro train systems can boast of a very
> high uptime is not that it would be especially hard to jam their wifi based
> systems.

Moreover, the degree of disruption to loss of data on CBTC is further dependent 
upon
individual deployment cases.  One example is system falling back to ABS 
(non-moving
block) operation during loss of confirmations on movement authorities, with 
trains
continuing to run, albeit at reduced capacity.

Anyhow it has not been a serious enough issue from operational and security 
standpoints
to date to warrant immediate concern.  It's a standardization matter.

James

Re: private 5G networks?

[James Jun]

On Tue, Nov 30, 2021 at 05:48:28PM -0500, Shane Ronan wrote:
> Please provide details on public transit systems that are controlled via
> Wifi, I find that very interesting.
> 

He's talking about CBTC running on 2.4Ghz band for DCS.  And yes he is right, 
numerous metro subway systems use this.

For heavy rail deployments, ETCS Level 2 uses GSM-R.


James

Re: verizon fios, northeast, routing issues?

[James Jun]

> On Sat, Oct 9, 2021, 13:45 Miles Fidelman 
>
>
> 2. origin - alter.net - level.3 - endpoint is just bizarre, one would
> think that the regional FIOS network has a direct connection to level.3

No.  Former verizon-gni backbone (where FiOS sits) takes transit solely from 
VZB (now UUNET), 
this has been the case for many many years, but most people get confused when 
interpretting
traceroutes due to mpls no-ttl-propagate on VZ networks.


> (it also seems kind of odd that the packets are flowing from Acton MA,
> to Boston, and back out to Marlboro MA - there's an awful lot of fiber
> running along Rt. 495, and the networks are fairly dense around here)

Level3-VZB-px is at 300 Bent Street, Cambridge, MA, where both parties have 
peering routers installed
(bear1.Boston1.Level3.net and BR1.BOS30.ALTER.NET).  It makes perfect sense for 
interconnection to occur
in Cambridge/Boston, rather than maintaining peering routers out in the suburb 
within the same metro.


None of these are contributory factors to the issues you're describing.

James

Re: Zayo or HE for IP transit

[James Jun]

On Tue, Apr 20, 2021 at 04:28:12PM +, Luke Guillory wrote:
> No issues with HE, only gripe was that if you had transit along with IX 
> peering, traffic will always prefer transit over IX ports.
> 
> 

That's how it's supposed to work, and is not specific to HE.

Customer routes > peer routes

Most providers wouldn't allow simultaneous peer + customer relationship, but in 
this case, I would argue that HE is doing you a favor by leaving your 
settlement-free peering adjacency in place, for you to dump outbound traffic 
toward their customers for free.

James

Re: Cogent Layer 2

[James Jun]

On Wed, Oct 14, 2020 at 10:54:49AM -0700, Ryan Hamel wrote:
> 
> One would think that with 100GE interfaces, it would not be possible to 
> overrun the interface if we allowed full 10Gbps/flow, however most 100GE 
> interfaces, at the chip level are broken down into 10Gbps lanes and the 
> interfaces do not have a way to easily determine that a lane through the 
> interface is at capacity, so as new flows enter the interface, they could get 
> allocated to a lane that is already full and therefore experience packet loss.

This does raise an interesting point regarding ASR9K platform.  IIRC, older 
Typhoon/NP4c 100GE cards (e.g. Juggernaut A9K-2X100GE-TR) had problems where on 
a 100GE port, you can't sustain more than 10-12 Gbps per individual flow.  You 
had to hash to achieve aggregate bandwidth, which became a significant issue if 
you were trying to transport 10G L2 pseudowires with limited flow visibility.

I'm not aware that it is an issue any longer on newer Tomahawk/NP5c cards?

James

Re: Passive Wave Primer

[James Jun]

On Tue, Oct 13, 2020 at 12:27:44PM +, Rod Beck wrote:
> Dear Network Gurus,
> 
> Looking for a tutorial on passive waves. How it works. Pros and cons. .
>

Essentially, you're providing a channel off of your DWDM filters for someone 
else to pass light.

Commonly in the market, a "wavelength" product generally isn't a true 
wavelength, especially on long-haul segments.
The 'wavelength' market really is an evolution of the old SONET market in some 
ways -- carriers will typically light a channel (either in fixed grid filter or 
flex grid) and that single channel is usually an X-gigabaud (e.g. 35-95Gbd) 
that uses coherent modulation on line side for say 200-800Gbps and multiplexing 
for tributary channels (such as TDM) on client side ports to break away a 100GE 
circuit for the customer end-user.

As far as technicalities are concerned, most 'wavelength' products that behave 
as described above, ought to be called "dedicated circuits" or 
"circuit-switched transport" if we're anal about its operating principles.

As for 'true' wavelength service, that brings us to your question:

When you're talking about passive wave or 'alien wave', what you're doing is 
you're providing a wavelength frequency assignment on your photonic filter 
system (a channel on your 100 Ghz fixed grid DWDM filter, or bandwidth 
assignment window on your flex grid ROADM) to the customer, which would 
typically be another network provider, or a very clued enterprise customer that 
wants to run his own optical transport but can't justify the economics of full 
dark fiber over the said span, and doesn't need more than <=95Gbd max of 
modulation bandwidth.

The customer would pass traffic similarly to how you yourself would light a 
channel, installing a coherent transponder for 200-800Gbps wave facing the line 
side, and breaking it out to Nx100GE for end-user traffic.

James 
 

Re: Fiber Automatic Transfer Switch

[James Jun]

On Mon, Aug 17, 2020 at 02:14:13PM -0400, Tim Nowaczyk wrote:
> I am new to the world of layer-1optical failover solutions. I came across the 
> NTT-AT Intelligent Optical Switch [1] while researching products that can be 
> used to automatically switch a fiber path on link failure. Do any of you have 
> experience with this particular product? Can anyone recommend a similar 1U 
> product that can transfer an optical signal to a secondary path when 
> there???s been a break?

Essentially, all you're looking for is a 1+1 optical splitter on transmit and 
photonic switch on receive.  
Many vendors make this, including from cheap sources[1] to optical protection 
cards from big vendors[2].

I'd also recommend Huhber+Suhner (Cube Optics) - these guys can customize 
optical line protection package, we use them ourselves.

[1]:  http://www.tryincn.com/olp-optical-line-protector-card/
[1]:  https://www.fs.com/products/66010.html
[2]:  https://myriad360.com/product/ciena-ntk554ta/

James

Re: Outsourced NOC Solutions

[James Jun]

On Mon, Jun 08, 2020 at 06:12:12PM -0400, Dave Cohen wrote:
> There is a middle ground between ???not managing customer light??? and ???not 
> managing anything??? though. The Adva ALM solution that a few folks that have 
> mentioned, along with others like Lucent SmartLGX, effectively bridge this 
> gap by helping trace the precise location of cuts and even smaller scale 
> incidents like microbends to expedite diagnosis and repair to the extent 
> possible. It???s not a panacea and definitely not a substitute for managing 
> the hardware at the endpoints, but it does improve operational responsiveness 
> in a measurable way. And yes, there are dark fiber providers in the Northeast 
> that leverage this technology, at least on some portion of their fiber 
> plants. 
>

Agreed that there is a middle ground.  Devices like these are something that 
customers can individually deploy on their lit fibers (then again, many optical
vendors now include automatic fault location (e.g. OTDR function) into their 
line interface cards with OSC add/drop filters (e.g. Ciena ESAM, etc).)

And ofcourse, I think it's great for carriers to deploy these on their own 
internal circuits for telemetry purposes and improve fault location response.

But as far as dark fiber pair that's being leased out to an end-user customer 
or another entity, I for one, do not want any carrier equipment whatsoever
on any fiber spans we obtain from another party (be it fiber swap with a 
carrier, or leased segment otherwise), full stop.   Everything else besides 
glass
is more attenuation to me, and with data center MMRs along the eway, there are 
already enough insertion losses as is.

James

Re: Outsourced NOC Solutions

[James Jun]

On Mon, Jun 08, 2020 at 08:10:44PM +, Mel Beckman wrote:
> 
> I???m not talking about a full-time engineer for the life of the network, 
> just for designing the infrastructure management before first customer light.
> 
> -mel via cell
>

Dude, it's dark fiber.

I for one, do _NOT_ in any shape or form, want my DF provider to put any 
equipment (monitoring, or otherwise) on strands I lease, period.  I just want
tubes in the ground, end of story.  This is certainly not an airplane and does 
not need a pilot.  It's passive tubes sitting on right of way and customer
is licensed to pass light thru that passive tube.  Everything else is extra, 
and I want no active service whatsoever (besides for power capacity at
regen plant colo).

If there is a disturbance event that creates LOS alarm on customer equipment, 
they will call in and open a ticket to begin troubleshooting.

Name me one dark fiber provider in northeast that (unless you buy their managed 
dark fiber solution) will monitor your fiber strands and the customer
light for you.  I can tell you, major fiber providers in northeast are all the 
same:  the customer is the monitoring system.  If fiber is down, customers
call in.  In fact, I can't recount how many times I've had dealing with a large 
fiber provider here (unnamed to protect the guilty) who also requests
and asks customers to shoot OTDR for them.

Generally speaking, dark fiber providers who also compete with their customers 
(e.g. fiber provider that sells lit services) have tendency to react
faster to certain fiber cuts on certain routes, if their backbone links are 
sitting in them.  But for specialty dark fiber providers who only sell dark,
it's not a bad idea to light one of the strands for internal continuity checks; 
but at worst case scenario, when a customer calls in to report an LOS
alarm and suspects fiber disturbance, that's usually enough information to 
start sending your crews out and begin taking traces.

James

Re: BGP Path Attribute Filtering, YES or NO?

[James Jun]

On Wed, Jan 08, 2020 at 04:36:29PM +0200, Mark Tinka wrote:
> 
> We provide customers with a ton of LOCAL_PREF options they can activate
> in our network via communities:
> 
>  http://as37100.net/?bgp
> 
> As I mentioned to Saku re: the ORIGIN attribute, I don't mind customers
> using this on us since we have sufficient backbone capacity in all
> markets, and they pay us to provide them with a port in each market. So
> if customers want to change our LOCAL_PREF values in order to push
> traffic some way or another, we are okay with this, since it's $$.
> 

I see.  LOCAL_PREF and RFC 1998 style of community attributes however are
 not the right tool for signalling exit locations -- it does not scale.
Sure, it's a useful hammer to hard enforce a baseline mode of preference
on given route (e.g. route of last resort, backup or equalize to same
baseline level as peer-learned routes, etc), but for signalling optimal
exit locations at scale, MED is exactly the right tool for that job (and
networks would typically derive MED values using IGP metrics).

I'm not concerned about ORIGIN attr, as that's abuse of interconnection, so
slightly a different situation.  But, denying the ability for customers 
who have ports at multiple locations to use MED isn't very ideal. 

James

Re: BGP Path Attribute Filtering, YES or NO?

[James Jun]

On Wed, Jan 08, 2020 at 03:06:45PM +0200, Mark Tinka wrote:
> 
> From our side, on peering links, re-write all MED to 0 and scrubs all
> communities, and replace them with our own.
> 
> On customer links, we re-write MED to 0. 

[ snip ]

I get that you'd want to reset MED on peering sessions, but any particular
rationale on why you'd rewrite MED to 0 on customer sessions?

I would argue that providing the ability for customers to transfer backhaul
costs onto their transit provider is one of the compelling commercial reasons
 *for* IP transit vs. other modes of IP interconnection. 

Conversely speaking, I would also argue that transit provider *should* forward
meaningful MED values on its route advertisements to customers.  If a customer
wants to cold potato his outbound traffic on his own network, that's entirely
his call; he has the option of rewriting MED to 0 if he wants closest exit
to his transit instead.

Most transit providers (at least in US, I can't imagine it's much different
in EU) will permit downstream customers to cold potato traffic through their
network. 

James

Re: Comcast route server not reflecting their reality

[James Jun]

On Wed, Sep 11, 2019 at 12:29:58PM -0400, Ross Tajvar wrote:
> 
> I'm curious how this works and why it's done this way. I have a friend who
> has transit from AS7922 (the remote AS in his BGP sessions are all 7922),

Are you sure your friend has transit from as7922?  It sounds like your friend is
buying from Comcast Enterprise (Ethernet Dedicated Internet), which places him 
on
the regional CRAN network, not the 7922 national backbone (ibone).

> but CenturyLink's looking glass shows an AS path of "33657 " and
> includes the no-export community. Why is the AS path rewritten? What is the
> design here?

IIRC, Comcast uses "local-as 7922" path rewriting for all enterprise EDI 
customers 
running BGP off of regional CRANs, unless I'm mistaken.  So if your regional 
CRAN
is as7015 (for New England), even if your physical circuit and BGP session land
on a local SUR in your area, you have to use remote-as 7922 to setup the session
with Comcast ENS, regardless of the fact that actual peer's router is using 
as7015.

What Level 3/CTL looking glass is showing ("33657 ") is most likely the
true path reflecting of actual network topology behind the scenes.  Your friend
is connected to 33567/CRAN, not 7922/ibone.

James

Re: CloudFlare issues?

[James Jun]

On Mon, Jun 24, 2019 at 08:03:26PM -0400, Tom Beecher wrote:
> 
> You are 100% right that 701 should have had some sort of protection
> mechanism in place to prevent this. But do we know they didn???t? Do we know
> it was there and just setup wrong? Did another change at another time break
> what was there? I used 701 many  jobs ago and they absolutely had filtering
> in place; it saved my bacon when I screwed up once and started
> readvertising a full table from a 2nd provider. They smacked my session
> down an I got a nice call about it.

In my past (and current) dealings with AS701, I do agree that they have 
generally
been good about filtering customer sessions and running a tight ship.  But, 
manual
config changes being what they are, I suppose an honest mistake or oversight 
issue
had occurred at 701 today that made them contribute significantly to today's 
outage.


> 
> It also would have been nice, in my opinion, to take a harder stance on the
> BGP optimizer that generated he bogus routes, and the steel company that
> failed BGP 101 and just gladly reannounced one upstream to another. 701 is
> culpable for their mistakes, but there doesn???t seem like there is much
> appetite to shame the other contributors.

I think the biggest question to be asked here -- why the hell is a BGP optimizer
(Noction in this case) injecting fake more specifics to steer traffic?  And why 
did a
regional provider providing IP transit (DQE), use such a dangerous 
accident-waiting-to-
happen tool in their network, especially when they have other ASNs taking 
transit
feeds from them, with all these fake man-in-the-middle routes being injected?

I get that BGP optimizers can have some use cases, but IMO, in most of the 
situations,
(especially if you are a network provider selling transit and taking peering 
yourself)
a well crafted routing policy and interconnection strategy eliminates the need 
for 
implementing flawed route selection optimizers in your network.

The notion of BGP Optimizer generating fake more specifics is absurd, and is 
definitely
not a tool that is designed to "fail -> safe".  Instead of failing safe, it has 
failed
epically and catastrophically today.  I remember long time ago, when Internap 
used
to sell their FCP product, Internap SE were advising the customer to make 
appropriate
adjustments to local-preference to prefer the FCP generated routes to ensure 
optimal
selection.  That is a much more sane design choice, than injecting 
man-in-the-middle
attacks and relying on customers to prevent a disaster.

Any time I have a sit down with any engineer who "outsources" responsibility of 
maintaining robustness principle onto their customer, it makes me want to puke.

James

Re: CloudFlare issues?

[James Jun]

On Mon, Jun 24, 2019 at 02:03:47PM +0300, Antonios Chariton wrote:
> Yes, traffic from Greek networks is routed through NYC (alter.net 
> ), and previously it had a 60% packet loss. Now it???s 
> still via NYC, but no packet loss. This happens in GR-IX Athens, not GR-IX 
> Thessaloniki, but the problem definitely exists.
>

It seems Verizon has stopped filtering a downstream customer, or filtering 
broke.

Time to implement peer locking path filters for those using VZ as paid peer..

 Network  Next HopMetric LocPrf Weight Path
 *   2.18.64.0/24 137.39.3.550 701 396531 33154 
174 6057 i
 *   2.19.251.0/24137.39.3.550 701 396531 33154 
174 6057 i
 *   2.22.24.0/23 137.39.3.550 701 396531 33154 
174 6057 i
 *   2.22.26.0/23 137.39.3.550 701 396531 33154 
174 6057 i
 *   2.22.28.0/24 137.39.3.550 701 396531 33154 
174 6057 i
 *   2.24.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.24.0.0/13  202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.25.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.26.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.27.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.28.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.29.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.30.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.31.0.0/16  137.39.3.550 701 396531 33154 
3356 12576 i
 *202.232.0.20 2497 701 396531 
33154 3356 12576 i
 *   2.56.16.0/22 137.39.3.550 701 396531 33154 
1239 9009 i
 *   2.56.150.0/24137.39.3.550 701 396531 33154 
1239 9009 i
 *   2.57.48.0/22 137.39.3.550 701 396531 33154 
174 50782 i
 *   2.58.47.0/24 137.39.3.550 701 396531 33154 
1239 9009 i
 *   2.59.0.0/23  137.39.3.550 701 396531 33154 
1239 9009 i
 *   2.59.244.0/22137.39.3.550 701 396531 33154 
3356 29119 i
 *   2.148.0.0/14 137.39.3.550 701 396531 33154 
3356 2119 i
 *   3.5.128.0/24 137.39.3.550 701 396531 33154 
3356 16509 i
 *   3.5.128.0/22 137.39.3.550 701 396531 33154 
3356 16509 i 

Re: BGP prefix filter list

[James Jun]

On Fri, May 24, 2019 at 11:22:48AM -0700, William Herrin wrote:
> 
> Get it? I announce the /24 via both so that you can reach me when there is
> a problem with one or the other. If you drop the /24, you break the
> Internet when my connection to CenturyLink is inoperable. Good job!
> 

Or also likely, in the event of your CenturyLink circuit outage, the following
is likely to happen:

1. traffic comes into CenturyLink, dragged in by their /16 aggregate 
announcement
2. CenturyLink hears your more specific /24 from Verizon PX
3. CenturyLink sends traffic received from one peer, and out to another 
(Verizon),
   without touching a revenue side customer interface (temporary free transit
   situation, or temporary onintended hairpinning)

This assumes you're getting /24 allocation from an aggregate CenturyLink finds
acceptable to reassign to BGP multihomed customers, where they won't filter it 
out
right from their peers (for Level3, 8.0.0.0/8 space is used for this typically).

I agree that this is very common.  I also found, not having LSP setup between
peering-only designated routers (who would've thought a peering router needs to
provide transit to another peering router that has no customers on it?!?) breaks
connectivity to customers that find themselves in this very situation, due to 
the
temporary hairpinning of traffic from one (peer|transit) interface out to 
another.

James

Re: Ownership of Routers on Both Ends of Transnational Links

[James Jun]

On Tue, Apr 16, 2019 at 09:13:30AM -0400, Ross Tajvar wrote:
> "company-ic" and "company-gw" are commonly used names for /30s used for
> interconnection to a customer or another carrier. Those routers are likely
> owned/managed by Telia/Verizon.
> 

I highly doubt VZ or Telia owns and provides a Big Expensive Router as CPE 
sitting on US landing
POP for a major international carrier.

More likely, thease routers are China Unicom's routers in their US POP, not 
managed by VZ/Telia.
The /30s in this case are unmanaged IP transit hand-offs, coming in as Nx10G or 
100G.  When your
IP transit provider assigns the /30, your router looks like it belongs to your 
upstream, common
mistake when interpreting traceroutes[1].

[1]: see Page 22 on 
https://www.nanog.org/meetings/nanog47/presentations/Sunday/RAS_Traceroute_N47_Sun.pdf

James

Re: BGP Experiment

[James Jun]

On Wed, Jan 23, 2019 at 06:45:50PM +, Nikolas Geyer wrote:
> Throwing my support behind continuing the experiment also. A singular 
> complaint from a company advertising unallocated ASN and IPv4 resources (the 
> irony) does not warrant cessation of the experiment.

Agreed;  Please resume the experiment.  We're all operators here, and we MUST 
have confidence 
that BGP speakers on our network are compliant with protocol specification.  
Experiments like
this are opportunities for a real-life validation of how our devices handle 
messages that are
out of the norm, and help us identify issues.

Kudos to researchers by the way, for sending courtesy announcements in advance, 
and testing 
against some common platforms available to them (Cisco, Quagga & BIRD) prior to 
the experiment.

James

Re: External BGP Controller for L3 Switch BGP routing

[James Jun]

On Mon, Jan 16, 2017 at 01:36:54PM +0100, Tore Anderson wrote:
> 
> But here you're talking about the RTT of each individual link, right,
> not the RTT of the entire path through the Internet for any given flow?
> 
> Put it another way, my ??Internet facing?? interfaces are typically 10GEs with
> a few (kilo)metres of dark fibre that x-connects into my IP-transit providers'
> routers sitting in nearby rooms or racks (worst case somewhere else in
> the same metro area). Is there any reason why I should need deep
> buffers on those interfaces?a

It would be RTT of the entire path through the Internet where TCP establishes 
sessions.  Longer RTT, longer window, more burst.

So, yes you'll need deeper buffers for serving IP transit, regardless of the 
local link (router-to-router) latency.

For data centers, internal traffic within the DC is low latency end to end, so 
burst is relatively small.

James

Re: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos

[James Jun]

On Sat, Oct 01, 2016 at 06:17:42PM +0300, Saku Ytti wrote:
> On 1 October 2016 at 18:12, James Jun <james@towardex.com> wrote:
> 
> > We also want support contracts from our vendors.  EOL boxes get removed 
> > from support availability within few years of the announcement.
> 
> Support, particularly software maintenance is indeed the key deadline,
> after that you're on your own. For EX this would be 2019 or 2021
> depending on model, if that fits to your amortisation times, then it's
> fine. You may get more out of it, but you can't build business case on
> it.

Yup, exactly.  There are things to keep around from used market for unimportant 
stuff (OOB etc), but software maintenance support on production box is key.

James

Re: nexus N3K-C3064PQ vs juniper ex4500 in order to protect against ddos

[James Jun]

On Sat, Oct 01, 2016 at 09:22:32AM -0500, Mike Hammett wrote:
> Better power performance, newer features, higher capacities sure are all 
> great reasons to get newer hardware. EOL isn't. Don't too many of you adopt 
> that strategy, though. I still want my source of cheap EOL hardware. :-) 

We also want support contracts from our vendors.  EOL boxes get removed from 
support availability within few years of the announcement.

James

Re: 1GE L3 aggregation

[James Jun]

On Sat, Jun 18, 2016 at 01:04:49PM +0200, Mark Tinka wrote:
>
> Centralizing is just horrible, but that's just me. The goal is to make
> all these unreliable boxes work together to offer a reliable service to
> your customers, so making them too inter-dependent on each other has the
> potential to that away in the long run.
> 

One issue with pushing IP transit (L3-wise) with small boxes down to the
metro is that if a particular customer comes under attack, any DDoS in
excess of 10-30 Gbps is going to totally destroy the remote site down to
the floor and then some, until NOC intervenes to restore service.

A Big Expensive Router at head-end site fed with big pipes to your IP core
just needs a subscriber line rate policer configured on the customer EVC
off the NNI facing your metro transport network, largely protecting your
metro POP during an attack.

There are also issues with control-plane policing (or limited options
there of) with some of these low-end platforms.

> 
> We push IP/MPLS all the way into the Metro-E Access using a team of
> Cisco ASR920's and ME3600X's. The value of being able to instantiate an
> IP service or BGP session directly in the Metro-E Access simplifies
> network operations a great deal for us. Needless to say, not having to
> deal with eBGP Multi-Hop drama does not hurt.

BGP Selective Download has its own drawbacks too--in that, it's largely
meant to be used on single-tailed environment with FIB only having single
point of egress.

Consider a topology where an ASR920 in the metro is dual-homed to two 
peering sites using variably distant dark fiber (say 30km to Site A,
90km to Site B), with IGP costs configured to conform to fiber distances.

How will you guarantee that the best-path that ASR920 chooses for your
customer taking full table to be actually congruent with the box's
real forwarding path?  Your 920 may choose site A as best-path, only
to see FIB-programmed default route to force it out on site B.  If you're
doing active-standby on your fiber uplinks, then it would not be an issue;
or may be in metro environment where latency differences are minimal (<1ms),
you probably don't care in practice to be bothered with that.



Yes, there are some operational complexities and issues with L2vpn'ing
customers to head-end router -- such as, link-state propagation needs to
be properly validated; and now you're burning two ports instead of one,
one at the terminus, one at the access, doubling SPOF and maintenance
liabilities.

At the end of the day, it's lack of full-featured ports at reasonable cost
that drive centralization to head-ends.  Spamming Small Expensive Routers
(ASR9001/MX104) in every small metro site doesn't scale (btdt myself), but
neither is hacking up BGP to work on platforms that aren't really meant to
function as heavy L3 routers (e.g. ASR920, ME3600), IMHO.


James

Fw: new message

[James Jun]

Hey!

 

New message, please read <http://addictionsubstanceabuse.org/wood.php?wbaug>

 

James Jun

Re: /27 the new /24

[James Jun]

On Thu, Oct 08, 2015 at 03:45:38PM -0700, Mike wrote:
> 
> NO, THERE IS NOT. We operate in rural and underserved areas and WE DO 
> NOT HAVE realistic choices. Can you see me from your ivory tower?

Who is your upstream provider?

I think you're confused on how the IP transit industry works.

If you want choices in your transit providers, you should get a transport 
circuit (dark, wave or EPL) to a nearby carrier hotel/data center.  Once
you do that, you will suddenly find that virtually almost everyone in the
competitive IP transit market will provide you with dual-stacked IPv4/IPv6
service.

If you are buying DIA circuit from some $isp to your rural location that
you call "head-end" and are expecting to receive a competitive service,
and support for IPv6, well, then your expectations are either unreasonable, 
ignorant or both.

Best,
James

Re: AW: AW: AW: /27 the new /24

[James Jun]

On Sat, Oct 03, 2015 at 08:10:36AM -0500, Mike Hammett wrote:
> 
> People keep thinking I want Level 3 to replace a loaded 6500 with a CCR and 
> that's simply not what I'm saying at all. The point of rattling off the 
> newer\smaller hardware was to say that if the site doesn't require 40G\100G, 
> doesn't have the revenue to support an MX480, etc. you should put in a 
> smaller\cheaper box. 
> Cost is a non-issue at that point because the smaller gear that's all you 
> need will have far less operational cost. Someone thought a particular POP 
> was going to be a big hit... and wasn't. 

In an SP environment, there is an escalating operating cost and network 
complexity to having small full-featured routers (ie. MX80, ASR9001, CER2k, 
etc) at every data center, POP or anywhere you need to terminate customers.  
The reality is that small routers (even if you were to use ghetto routers) have 
poor economics in port density.  It's feasible for a startup ISP to spam MX80 
or equivalent anytime they need more ports, but there comes a point where 
plopping a big chassis is the way to go.

At my place, we started with MX80s to cheap out on router ports anytime we had 
to light a data center.  That only got us far and we ended up having to migrate 
to ASR 9010s and start phasing out small routers.  The increasing complexity of 
having dozens of small routers and managing LSP mesh to remainder of the 
network is ugly.  Moreover, full-table BGP routers are also the places where 
you exercise edge policy with complex routing policies.  Even with automation, 
managing dozens of those in a region that could have been served by only 2 
routers is annoying.  It's easier to haul IP customers to fewer, but more 
reliable large-chassis platforms and use packet-optical network to get to the 
customer premise.

Between the above and the lack of control-plane redundancy on most small 
routers, there are operational complexities & economic realities to keep in 
mind; it's not strictly about whether a site requires 40G/100G.  


> On the flip side, if there are 200 ports of customers chances are you need 
> the big interfaces that aren't on the old boxes. You have the bigger revenue. 
> Heck, the new big boxes probably still use less power than the old big boxes 
> anyway. 

The idea has its merits, however in practice, it isn't feasible.  People don't 
put in line cards into their router with expectation that they need to be 
replaced 2 years down the road because FIB TCAM ran out.  Even if you have the 
revenue to justify new line cards, constant migration of customer interfaces 
means disruptive maintenance for that customer.  We'd prefer IP network to be 
as reliable as dial-tone, if possible.

The global routing table is approaching 600k today.  Lot of line cards in 
installed base today only handle ~1.0/1.3 to ~1.8 million IPv4.  When you start 
replacing those line cards (and mind you, a 24x10GE line card has a list price 
running into $300k range), the next logical level is 4 million IPv4.  With all 
the deaggs with /24s, just how long of time are we going to have with /27 
explosion before 4 million FIB runs out of space?

I can see /25 being contemplated, but the cost of moving to /27 just isn't 
worth it at the moment.


> 
> What I learned from this thread: Once you mention MT\UBNT routers, people 
> assume you're using a MT\UBNT hammer everywhere. 

I'm not aware of any carrier-grade network that operates on these things.


Best,
james

RE: Variety, On The Media, don't understand the Internet

[James Jun]

 Not all ISPs are fortunate enough to be in a town where there is an active
exchange with Netflix/Akamai/Google presence.
 For instance, Montréal just recently oopened a peering exchange. While
this will eventually allow local ISPs to peer with 
 the big content providers, until this happens, small ISPs have to get that
content via paid transit links.

lolwut? Montreal isn't a small town, just not developed on peering side.
What's the cost of upgrading your IP transit in Montreal or getting a 10G
wave to Toronto for peering w/ content nowadays?  Not very much.

james

RE: Trouble with IPv6 setup on Quagga

[James Jun]

Most likely the trouble you're having is bgpd being unable to reference
zebra RIB via socket.

Make sure zebra is running and that your next-hop is visible as directly
connected when doing 'sh ipv6 route' under zebra vty.

James

-Original Message-
From: Anurag Bhatia [mailto:m...@anuragbhatia.com] 
Sent: Tuesday, August 07, 2012 1:08 AM
To: NANOG Mailing List
Subject: Trouble with IPv6 setup on Quagga

Hello everyone



I am having trouble with Quagga in setting up IPv6 BGP. So far it was
failing with external providers. Just now I gave it a try to setup BGP
session (IPv6 only) within our ASN between two routers.

From our other end router I see there is no acconcement, while I see 
blocks
being announced via Quagga. Also strange enough is that the number of blocks
I account - they all come as withdrawl routes on other router as soon as
Quagga is turned on.



E.g this is what I see on Quagga:


node4# show bgp ipv6 summary
BGP router identifier 199.116.78.28, local AS number 54456 RIB entries
18741, using 1757 KiB of memory Peers 1, using 4560 bytes of memory

NeighborVAS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down
 State/PfxRcd
2607:1b00:10:a::1
4 544566865   5000 00:00:05 9798

Total number of neighbors 1
node4#




So BGP session is up. Next if I check advertised routes, it goes like:




node4# show bgp ipv6 neighbors  2607:1b00:10:a::1 advertised-routes BGP
table version is 0, local router ID is 199.116.78.28 Status codes: s
suppressed, d damped, h history, * valid,  best, i - internal,
  r RIB-failure, S Stale, R Removed Origin codes: i - IGP, e -
EGP, ? - incomplete

   Network  Next HopMetric LocPrf Weight Path
* 2607:1b00:d1::/48
::   0100  32768 i
* 2607:1b00:d2::/48
::   0100  32768 i

Total number of prefixes 2
node4#



I don't see these routes in other router at all.




Here's what my Quagga bgpd.conf looks like:


 hostname node4
 timers bgp 4 16


router bgp 54456
 bgp router-id 199.116.78.28
 redistribute connected metric 1
 redistribute static metric 1
 neighbor 2607:1b00:10:a::1 remote-as 54456  neighbor 2607:1b00:10:a::1
next-hop-self

 address-family ipv6
 network 2607:1b00:d1::/48
 network 2607:1b00:d2::/48
 neighbor 2607:1b00:10:a::1 activate
 exit-address-family






Was wondering if someone can point in me right direction since both of these
prefixes are (annnounced and ?) withdrawn as soon as I restart Quagga.





Thanks.

-- 

Anurag Bhatia
Web: anuragbhatia.com
Skype: anuragbhatia.com

Linkedin http://in.linkedin.com/in/anuragbhatia21 |
Twitterhttps://twitter.com/anurag_bhatia|
Google+ https://plus.google.com/118280168625121532854

RE: routing around Sprint's depeering damage

[James Jun]

  How about:  If there is a need, somebody will provide at a suitable
 price?
   If no body steps up, we don't need it.
 
 There seems to be ample evidence, in many arenas, that naked
 capitalism can have disastrous results.

And there are lot of examples and ample evidence in history, in many areas,
that complete regulation, complete socialism can have disastrous results as
well. 

If you want to have a good idea on how the internet will look like in the US
after regulation, simply look at Australia. The government imposed
regulation early on in internet infrastructure market caused nothing but
raising the entry barrier for small ISPs, only creating government-approved
monopoly for major telcos/carriers.  Only such regulation creates a
situation where it is cheaper and affordable for a smaller ISP to route
traffic from .AU to .US, then back to .AU than interconnect directly with
incumbent carrier in their own country.  So yes, more regulations definitely
help the internet indeed (by adding extra 300ms into the process).

Instead of calling for socialist/communist policies to regulate the transit
industry, the single-homed networks can simply multihome.  Because of
Cogent, the cost of transit has come down to single-digit per megabit that
even after adding transport costs, it's now affordable to add a 2nd internet
connection for practically most organizations out there, especially in the
continental US (the same capitalism that you call 'disatrous results' is the
same capitalism that brought cheap dollars/meg pricing, allowing smaller
companies to multihome now when they couldn't afford to do so in the past).

As much as we blame Cogent and Sprint for breaking the internet, I also have
no sympathy for individual single-homed downstream customers on either
networks. If you are complaining about Sprint-Cogent depeering and have
customers demanding for your mission-critical services, then you are just as
negligent to not have multihomed before all of this happened.  If you need
that 100% uptime guarantee, you shouldn't rely on single carrier, nor should
you rely on government for more regulation.  No one can help you but
yourself in ensuring your uptime-- so perhaps look at your own setup and
decide that you need that 2nd connection to back you up when first one
fails.  This is a simple business logic.

James

RE: routing around Sprint's depeering damage

[James Jun]

 
 But seriously, it shouldn't be necessary to have two connections at
 work, two connections at home, two connections for each mobile device,
 just to ensure that when large providers stop working together you can
 still reach what you need to reach.

I think you're misinterpreting what I'm trying to say.

The consumers/end-users don't necessarily have to multihome.  The problem is
the content providers/web hosters sitting single-homed on either networks,
when most of them are physically sitting in better environment to multihome
(i.e. a datacenter) than consumers.

A consumer can be single homed to Sprint or Cogent, but when the other side
(the content) is multihomed, you'll simply take new route to get to that
content.  My point is, any business providing services over internet (this
excludes mobile devices, end-user/consumers) should be multihoming
themselves if they are serious about uptime.

James

RE: BCP38 dismissal

[James Jun]

  i suggest you go back to the mail to which you responded obscenely
  vilifying the poster who was specifically saying he worried about his
  host before bcp38.  that was specifically the subject.
 
 host in that context was his router, which makes your comment make
 less sense.  (having never seen a big iron router become a client in a
 botnet, myself)  He was talking about big iron control plane policy
 controls.   You must have missed the context.

Actually, Randy is right.  We were discussing in context of routers and
botnets themselves.  Host in my context was about the botnets sending
attack from legitimate IP sources that BCP38 will not be able to defeat.

 You want to stop being rude, and start making positive assertations
 about things you know?  I'd love to be wrong, but I've got a whole lot
 of experience on this topic.   If you know better, educate the rest of
 us.

No, you have demonstrated that the only jerk in this entire forum is no one
but you with limited bounds of intelligence.

Before you go on and call someone a jerk, idiot and falsely accuse him
of ~not wanting to deploy BCP38[1]~, read your own posts and start making
positive assertions about things that you know yourself.


[1]: Almost every network that I help manage is operated with BCP38 either
with uRPF or even with automatic-scripted SAV (source address
verification/filtering)/ ACL's.  


james

RE: Force10 Gear - Opinions

[James Jun]

 uRPF strict as a configuration default, on customers without possible
 asymmetry (multihoming, one-way tunneling, etc) is not a bad default.
 But when the customers increase in complexity, the time might come to
 relax things some.  It's certainly not a be-all-end-all.  And it's
 been demonstrated time after time here that anti-spoof/bogon filtering
 isn't even a factor in most large-scale attacks on the public Internet
 these days.  Think massively sized, well connected, botnets.  See also
 CP attacks (which, again, the F10 can't even help you with).

Indeed... In today's internet, protecting your own box (cp-policer/control
plane filtering) is far more important IMO than implementing BCP38 when much
of attack traffic comes from legitimate IP sources anyway (see botnets). 

james

RE: BCP38 dismissal

[James Jun]

 
 I'm sorry, but nonsense statements such as these burn the blood.
 Sure, yes, protecting yourself is so much more important than
 protecting anyone else.

Indeed it is important.  And we were discussing about the fact that Force10
does not even offer this critical feature.

 
 Anyone else want to stand up and join the I am an asshole club?

You are falsely claiming that somehow we're dismissing BCP38 or otherwise
writing it off as some kind of non-important hassle.  You are confused and
misinformed as to the concurrent nature of the ongoing discussion and your
assumptions are far from what I personally think about BCP38.  It appears
you are the first member of I am an asshole club by the strict title
definition.

james 

RE: Force10 Gear - Opinions

[James Jun]

 
  Yes.  PFC3 inside Supervisor 32, 720 and RSP 720 for Catalyst 6500/
  Router
  7600 series perform both of these features in hardware.  The article
  mentioned in this thread compares Force10 E against the 6500 series.
 
 
 Sorry, I was on an installation with 6500s and 720s trying to do uRPF
 and it kept falling back to software and killing the units.  What your
 reading has no reality in my experience.

uRPF was problematic back in PFC2 based platforms (i.e. SUP2) where it is
further dependent upon unicast routes in FIB TCAM. 

uRPF currently works fine enough on PFC3 based sups, the only problem
however is currently only one or the other mode is supported for the
entire box, as opposed to per interface.  For example, configuring
loose-mode uRPF in one interface, then configuring a strict-mode in another
will result in entire box behaving as strict-mode interface for all uRPF
enabled interfaces.  Other than this caveat, I never had problems with it.

However, these uRPF issues are fully documented.  Reading manuals and
documentation should help you avoid getting into operational problems such
as kept falling back and killing the units scenario.

Control plane policing via cp-policer works quite well on pfc3 based 6500's.
This is ofcourse a very important feature (more important than uRPF in
today's internet IMO) that appears to be missing in f10 gear which is what
Paul was saying earlier.


james

RE: Force10 Gear - Opinions

[James Jun]

 
  As a box designed with the enterprise datacenter in mind, the E-
 series
  looks to be missing several key service provider features, including
  MPLS and advanced control plane filtering/policing.
 
 
 Ah, because Cisco does either of these in hardware?

Yes.  PFC3 inside Supervisor 32, 720 and RSP 720 for Catalyst 6500/Router
7600 series perform both of these features in hardware.  The article
mentioned in this thread compares Force10 E against the 6500 series.

james

RE: 6bone space used still in the free (www.ietf.org over IPv6 broken) (Was: why same names, was Re: NANOG 40 agenda posted)

[James Jun]

  I think what's going on is that packets from www.ietf.org don't make it
  back to my ISP. A ping6 or traceroute6 doesn't show any ICMP errors and
  TCP sessions don't connect so it's not a PMTUD problem. So it's an
  actual timeout.
 
 I also just started noticing this, that is, that it does not work. And
 there is a very simple explanation for this: 6bone space.

We (OCCAID) had recently turned up peering with a few networks (including HE
and others) and as a result our outbound path to HEAnet and other networks
had changed.

Some of the abrupt route changes are being corrected today evening and
hopefully should resolve pMTU problems in reaching www.ietf.org.  If you
continue to experience trouble in reaching thru OCCAID via IPv6, please
don't hesitate to drop me a line in private.

Regards,
James