Re: U.S. test of national alerts on Oct. 4 at 2:20pm EDT (1820 UTC)

[Aaron de Bruyn via NANOG]

I was kinda surprised that none of my NOAA weather radios went off. I sorta 
assumed they'd be tied into the whole "national" alert setup.

Why interrupt cell phones, AM/FM radio stations, and TV stations, but exclude 
NOAA weather radios?

-A

On Sun Oct 1, 2023, 10:24 PM GMT, Sean Donelan  wrote:
>
> This year's test of the U.S. national emergency alert includes something
> for ISPs and network operators.
>
> The wireless portion of the national test is scheduled 2 minutes (2:18pm
> EDT or 1818 UTC) before the main broadcast test at 2:20. Mobile phones
> usually receive the alert about a minute later. Radio and TV will receive
> the national alert a few minutes after 2:20pm.
>
> iPhone iOS 17 added a new feature for Wireless Emergency Alerts. When iOS
> 17 iPhones get a wireless emergency alert (WEA), it will trigger a data
> network query for additional information. Its a small query and
> response, but there are a lot of iPhones making the query at the same
> time (I'm assuming Apple engineer's have built in some time skew).
>
> Apple has assured FEMA that Apple's CDN and servers will be able to handle
> the triggered load.
>
> The iOS 17 triggered query will either be a tiny blip in the network
> graphs around 2:18pm to 2:22pm which no one will notice, or some CDNs and
> ISP operators will be wondering what that heck that spike was.
>
> If your phone is configured with Spanish, it will display the alert in
> both English and Spanish.
>
> “THIS IS A TEST of the National Wireless Emergency Alert System. No action is
> needed.”
>
> “ESTA ES UNA PRUEBA del Sistema Nacional de Alerta de Emergencia. No se
> necesita acción.”
>
> You'll know your iOS17 device did an extra data query, if it displays a
> longer message (extra sentences) in addition to the messages above.
>
> "This is only a test. No action is required by the public."
>
>
> https://www.fema.gov/press-release/20230803/fema-and-fcc-plan-nationwide-emergency-alert-test-oct-4-2023

Re: Comcast contact sought

[Aaron de Bruyn via NANOG]

We get around the brain-damage by having our router grab all DNS requests and 
convert them to DoT or DoH using dnsdist. That probably won't work if you're 
hosting a DNS server on your cable connection though.

Call the normal support number and have them disable the "Security Edge" 
service. The "best" they can apparently offer is that it'll stay disabled until 
your modem gets a firmware upgrade or is factory reset. Then you'll have to 
call back in and disable it again.

Just be prepared that they're going to tell you it'll cost more for providing 
less service. Security Edge is horrible? Disabling it costs more. Don't need a 
phone number so Comcast can pad their numbers to the FCC? It'll cost you more. 
Same with not needing cable TV for your business. It costs you more because 
Comcast can't use you as a bargaining chip when negotiating with other media 
companies.

-A

On Sun Sep 24, 2023, 05:05 AM GMT, Al Whaley  
wrote:
> I am looking for a senior contact at Comcast.
>
> I have been trying to assist someone with a business connection that runs a 
> server farm. Recently the business cable modem started to short-stop port 53 
> for UDP and TCP. Apparently, a transparent DNS proxy somehow got activated 
> and all outbound traffic to any IPv4 or IPv6 address is intercepted and 
> handled by the modem – or not handled. Sadly, the proxy is stupid and a) 
> ignores the intended destination address, and b) drops things it doesn’t know 
> about, including any AXFR / IXFR and other more esoteric traffic, normal for 
> DNS server installations, but not used by the public. The DNS servers are not 
> able to do work, e.g. act as secondaries.
>
> I know others in the same configuration with servers that have been lucky and 
> not had this ‘feature’ activated, but I have found several references on 
> forums where people have been caught by this and unsuccessful in reaching 
> anyone in management, so it is a known problem.
>
> Comcast doesn’t allow customer supplied DOCSIS modems with multiple fixed 
> IPs. Other avenues exhausted as well.
>
> I’m hoping someone at Comcast can disable this. Attempts to go through 
> customer service… well we all know where that ends up. Escalations just don’t 
> go to anyone technical or interested.
>
> regards
> Al Whaley
> Sunnyside Computing, Inc.

Re: So what do you think about the scuttlebutt of Musk interfering in Ukraine?

[Aaron de Bruyn via NANOG]

> Starlink isn't a monopoly. Ukraine could have guided their munitions with 
> Iridium or another satellite Internet system.

Don't forget GLONASS. 

On Thu Sep 14, 2023, 03:10 AM GMT, William Herrin  wrote:
> On Wed, Sep 13, 2023 at 5:47 PM Michael Thomas  wrote:
>> Doesn't this bump up against common carrier protections?
>
> Hi Michael,
>
> Internet providers aren't common carriers. If they were, it'd be
> unlawful to stop your customers from sending email spam that was
> merely offensive rather than illegal. It's also why Internet providers
> aren't required to follow network neutrality. Internet providers gain
> their immunity through section 230 and the DMCA instead.
>
> Common carrier status typically applies to shipping companies and
> basic telephone service. Part of the mess with unwanted phone calls is
> that the caller has to break the law (e.g. by calling a number on the
> do-not-call list) before the phone company is allowed to act against
> them.
>> I sure don't
>> want my utilities weaponizing their monopoly status to the whims of any
>> random narcissist billionaire.
>
> Starlink isn't a monopoly. Ukraine could have guided their munitions
> with Iridium or another satellite Internet system.
>
> That said, volunteering services to the military of a nation at war
> and then pulling the rug out from under them is so classless, one
> wonders if Musk isn't trying to build a communist utopia.
>
> Regards,
> Bill Herrin
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/

Re: it's mailman time again

[Aaron de Bruyn via NANOG]

I donno Rich...a couple of decades ago I lost my Slashdot account because 
someone was able to access it.
I used the password in two places...Slashdot and all the blasted mailman 
instances I was signed up with.

To this day, I still use the same password on all my mailman subscriptions 
because I consider mailman insecure for emailing out passwords. I just 
obviously don't use the password anywhere else. So you're right that all anyone 
can do is unsubscribe me from something...which isn't a big deal, but it makes 
me wonder just how many people have terrible mailman passwords and maybe use 
them elsewhere...and wouldn't report a compromise because...well...it'd make me 
look stupid. 

Ignoring all of that—it's just a horrible practice to not encrypt passwords and 
to email them out. You don't really even need a mailman password. You just put 
in your email address and hit 'unsubscribe'...and it'll send you a link to 
click as authorization...so why not drop passwords altogether and just reply on 
click-to-authorize? Or just encrypt the passwords and have a "forgot password" 
click-to-reset like every other app on the planet?

-A

On Sat Sep 2, 2023, 07:57 AM GMT, Rich Kulawiec  wrote:
> On Fri, Sep 01, 2023 at 10:16:05AM -0700, Randy Bush wrote:
>> and i just have to wonder about sending passords over the net in
>> cleartext in 2023. really?
>
> This is a non-issue.
>
> Given that pretty much every SMTP connection is encrypted and that
> the worst thing that an attacker in possession of one of your Mailman
> passwords can do is unsubscribe you (in which case you and the list
> manager will be notified, and you can solve the problem quite rapidly),
> no, this isn't a problem that anyone needs to worry about.
>
> I've run (and am running) a lot of mailing lists with Mailman including
> some large-ish ones for what's now approaching 20 years. The scenario
> above has never happened. Nobody's even tried, which isn't surprising
> given that such an attack is increasingly difficult and yields little,
> if any, benefit to the attacker. Moreover, any hypothetical attacker
> possessing the resources and expertise required to pull this off could
> certainly find far more effective things to do.
>
> ---rsk
>

Re: Comcast Business Account Website Broken

[Aaron de Bruyn via NANOG]

Someone else here gave me a pointer when I was running into this on the USPS 
site.

Clear your cookies for that site. (In Chrome/Edge, go to the site, open up the 
dev tools, go to the "Application" tab, find cookies, delete them all).

Something probably went a little nuts with the site and ended up creating too 
much data in one or more cookies.

-A

On Wed Jun 14, 2023, 06:20 PM GMT, Matt Hoppes 
 wrote:
> For the last two weeks we have been unable to pay any bills on the
> business Comcast website.
>
> Clicking on any billing link results in:
>
> 400 Bad Request
> Request Header Or Cookie Too Large
>
> This is going to the URL of:
> https://business.comcast.com/oauth/oauth2/authorize?client_id=comcast-business-myaccount-prod_type=code_uri=https%3A%2F%2Fbusiness.comcast.com%2Faccount%2Fbilling
>
>
> Even trying to log out from the customer portal results in:
>
> Sorry
>
> Something went wrong. Please check back later.
>
> This is going to this URL:
> https://business.comcast.com/account/logout
>
> Hoping someone on here can get this to the right people to fix. I'm
> sure Comcast would love to get payments from their commercial customer base.

Re: Scheduled outage -- Nationwide no driver license updates this weekend

[Aaron de Bruyn via NANOG]

If we have downtime, we lose revenue, customers, sleep, etc...

If the government does it, what are you going to do? Get your license somewhere 
else?

-A

On Sat Feb 25, 2023, 11:39 PM GMT, Christopher Morrow 
 wrote:
> On Sat, Feb 25, 2023 at 6:12 PM Sean Donelan  wrote:
>>
>> Verizon network maintenance will impact access to the “National Driver
>> Register,” a system that motor vehicle offices around the country need to
>> check before handing out a license.
>
> Wait, what year is it?
> how is a network maintenance on what seems like a fairly critical system going
> to cause a total outage of said system?
>
> I think we time traveled back to 1990 here...
>>
>> All 50 states and D.C. participate in the National Driver Register, a
>> database maintained by the National Highway Traffic Safety Administration.
>> The register contains information about drivers who have had their driving
>> privileges revoked, suspended or denied due to serious traffic violations,
>> such as driving under the influence of alcohol or drugs, reckless driving
>> or excessive speeding.
>>
>>
>> The scheduled maintenance should be finished by Monday, in case you needed
>> to update your driver's license or planned to do some reckless driving
>> this weekend.

Re: txt.att.net outage?

[Aaron de Bruyn via NANOG]

txt.att.net  is returning MX records and those machines 
don't have port 444 open...

Wouldn't you want to be sending something like a SNPP message instead? It's a 
much less convoluted delivery process and is almost real-time (no queuing).

I guess it's been a decade or so since I've dealt with emergency services and 
paging...is SNPP even a thing anymore?

I looked at some old code I wrote 
(https://github.com/darkpixel/snppsend/blob/master/more-providers 
), and it 
doesn't look like snpp.attws.net  exists.

-A

On Fri Jan 20, 2023, 02:12 PM GMT, William Herrin  wrote:
> On Thu, Jan 19, 2023 at 8:09 PM Dan Walters via NANOG  wrote:
>> Know this is a longshot, any chance anyone from the txt.att.net domain might 
>> be able to help us with what we believe is a blacklist block or possibly an 
>> outage?
>> We deal with 911 cad dispatching and is affecting first responders so 
>> looking to see if there is a faster way to resolution.
>
> Hi Dan,
>
> As I understand it, txt.att.net is a low-volume courtesy service not
> intended for important communications. A paid service like Twilio can
> handle production-grade SMS delivery.
>
> Regards,
> Bill Herrin
>
> --
> For hire. https://bill.herrin.us/resume/

Re: Reporting Comcast outside plant issues?

[Aaron de Bruyn via NANOG]

I had that during the 2020 storm that swept through the US. I called PUD a few 
months before about a tree hanging at a 45 degree angle above the primaries. I 
called again a month later when I noticed the tree had been slowly shifting. No 
sense or urgency from the PUD. Then the storm hit and I watched from my car as 
it smashed into a pole, snapped the primaries, destroyed a transformer, snapped 
the secondaries, snapped the pole, and then hung bits of itself from the cable 
space. It was pretty spectacular—I wish I had gotten it on video.

Ignoring it for ~2 months turned a $500 tree removal into something that cost 
tens of thousands of dollars—not to mention the teams that had to do all the 
work in sub-freezing temps instead of cool with intermittent showers.

Everyone on in a ~1 mile stretch went without power for ~17 hours in 13 degree 
weather. Fortunately I have two generators that are worth more than my car and 
I had the ability to fail over to a Starlink connection. Internet was back up 
about about 38 hours later.

-A

On Mon Jun 27, 2022, 05:14 PM GMT, Mike Hammett  wrote:
> Maybe.
>
> I saw multiple reports of a town this past week end that didn't respond to 
> multiple calls for a transformer and pole CURRENTLY on fire. I guess they had 
> better things to do.
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions 
[image] [image] 
[image] 
[image] 

> Midwest Internet Exchange 
[image] [image] 
[image] 

> The Brothers WISP 
[image] [image] 

--
> From: "Jay Hennigan" 
> To: nanog@nanog.org
> Sent: Monday, June 27, 2022 12:07:16 PM
> Subject: Re: Reporting Comcast outside plant issues?
>
> On 6/26/22 19:27, Justin Streiner wrote:
> > Does anyone here have a contact at Comcast for reporting outside plant
> > issues that are not (at the moment) service-affecting? I am not a
> > Comcast customer, and they make it nearly impossible for non-customers
> > to reach them unless you're signing up for service.
>
> Call the non-emergency number for your local PSAP (police or fire
> department) and report wires down. They'll know how to get it handled.
>
> --
> Jay Hennigan - j...@west.net
> Network Engineering - CCIE #7880
> 503 897-8550 - WB6RDV
>

Comcast: "Reloading Statics" today

[Aaron de Bruyn via NANOG]

Just a heads-up for the Comcast crew lurking here...

I've had 3 different cable connections (Oregon and Washington State) go
down in the last ~1.5 hours.
Staff on-site have tried rebooting the modem with no success.
When we call support, they say something along the lines of "huh, that's
odd...your modem is online, but the statics aren't loaded".
They then proceed to reload the statics and everything comes back up.

The third time I commented to the support rep that this was my third call
today and she replied that she had taken "several" calls about the same
issue from other customers.

Not sure if there's a new firmware update going our or what, but the issue
appears to be on the Comcast end.

-A

Re: PoE, Comcast Modems, and Service Outages

[Aaron de Bruyn via NANOG]

Thanks Jason—they are all are business connections. I know they can be
restarted through the business portal, but honestly the business portal is
terrible for large clients.
Not all our connections are listed under the same account due to something
with Comcast and the way "regions" and various services are split. i.e.
there doesn't appear to be any way of seeing EDI circuts, only cable
connections. We were basically told for all the cable connections that we
had to have multiple email addresses for the various groups (i.e.
i...@example.tld could add accounts in one region, i...@example.tld could
handle a different region, etc...)

In some cases a reboot will trigger a pull of the latest firmware, which
might include security fixes, performance improvements, and other changes.

Good to know...but there generally seems to be a culture in both the "home"
side and the premier side that Comcast modems *will* start having problems
if they are left up and running for more than ~1 month. Some people say
they need to be rebooted every 21 days, others say "three to four weeks",
etc...

My routers run FreeBSD and they get rebooted maybe once a year when there's
a security update that affects us. Granted, the Comcast modems are probably
some embedded Linux variant, but it seems odd to me that there's this
generally accepted idea that they just need to be rebooted every few weeks
to "clear stuff out". They aren't my grandmother's Windows PC.

High packet loss typically suggests an RF impairment of some type. I don’t
know how to explain the PoE comment but am happy to look at your connection
if you want to email me off-list.

I would suspect RF issues on the cable side as well. There's a very large
cell phone tower about 2 blocks away with about half a million antennas on
it.

As for the PoE issue, I'm not trying to get anyone canned. The tech was
professional and polite—just wrong. He was insistent enough that I started
to question my understanding of the PoE standard, my networking knowledge
and possibly my own sanity. 

Thanks for reaching out though. I really do appreciate how responsive
Comcast is—not just you and others on NANOG, but generally as a large
corporation as a whole. For ~40 connections I usually find myself reaching
out to the premier group once a week and they're polite, knowledgeable and
99% of the time 'hassle free'. 

-A

On Wed Mar 30, 2022, 05:53 PM GMT, Livingood, Jason
 wrote:

> I asked him to remotely reboot the modem because there was high packet
loss.



FWIW, as a customer (assuming residential), you can login to the website
and check for area outages/impairments at
https://www.xfinity.com/support/status-map. You can also use the Xfinity
app to remotely reboot your cable modem, run diagnostics/check for outages,
etc. See https://www.xfinity.com/support/articles/check-service-outage



> Both times I've talked with him, he noted the high packet loss, started
to reboot the modem, and then asked me point-blank if we had any PoE
switches on our network.



High packet loss typically suggests an RF impairment of some type. I don’t
know how to explain the PoE comment but am happy to look at your connection
if you want to email me off-list.



> I said "it's up and working fine, why would I reboot it?".



In some cases a reboot will trigger a pull of the latest firmware, which
might include security fixes, performance improvements, and other changes.



Jason

Re: PoE, Comcast Modems, and Service Outages

[Aaron de Bruyn via NANOG]

Thanks Blake,

As I understand it all that stuff is on the "cable provider" side of the
CPE and (within reason) it's up to the provider to deal with the signals
arriving on the cable side of the modem.
i.e. if it was a blower or something in our suite that was causing RF
interference, the provider might work with us to move the modem or the
cable run.

-A

On Tue Mar 29, 2022, 09:59 PM GMT, Blake Hudson  wrote:


On 3/29/2022 3:24 PM, Joe Greco wrote:

He's got graphs showing it every 24 hours? Liar, liar, pants on fire,
lazy SOB is looking for an excuse to clear you off the line. Where the
heck does this "24 hour" cycle even come from? What SNMP OID is there
for "ghostly PoE build-up"? What crontab is there that would clear out
such buildups in the router's daily run? What capacitor would store up
juice for precisely 24 hours? What's the mechanism here? CURIOUS MINDS
WANT TO KNOW!


Taken at face value, I assume the tech would be looking at historical
signal graphs (we keep them for cable networks for each CM) that record
stats like FEC, SNR, and signal strength. For aerial runs it's common to
see some change throughout the day due to warming and cooling. These
look like waves with peaks and valleys around 4PM/4AM and generally
affect all customers in a service area equally. Sometimes there will be
a device at a customer premise that causes interference with a CM,
something like a motor or tool. These could absolutely be on a 24hr
cycle (think of a programmable thermostat kicking on the blower fan in
your HVAC at the same time every day).

As Joe said, there's no SNMP MIB for PoE buildup. There are well
documented MIBs for DOCSIS to cover standard signal level, quality, or
similar. The cause of that signal strength or quality can be myriad.
This Comcast tech has likely climbed the ladder of inference several
steps too far.

Re: PoE, Comcast Modems, and Service Outages

[Aaron de Bruyn via NANOG]

Just to be clear Josh, I'm not insulting him.

I find the situation extremely difficult to believe based on my (possibly
incorrect) understanding of how PoE works and very (very!) basic knowledge
of things like RF interference—especially when it comes to Cable networks.

I mean, the call literally went like this:
"Thank you for calling Comcast this is , how can I help you?"
"Hey, can you remotely reboot the modem on account 12345? We're seeing high
packet loss and latency starting about 10 minutes ago."
"Yeah...uh...do you have a PoE switch at that location?"

When you hear hoof beats, look for horses, not zebras. As a first
troubleshooting step, I certainly wouldn't jump to "it's PoE". Granted, I
have no idea if Comcast has "PoE Buildup" graphs in their internal tools,
but based on my conversations with tons of other Comcast reps about tons of
other Comcast connections and never hearing one of them mention those
graphs, I'm leaning towards him lying through his teeth.

Lastly, the reboot of the Comcast modem "fixed" the issue.

I saw one of the IT guys from another office in the complex a few minutes
ago and he said their internet had problems at the same time. Comcast has
been out to the equipment room in the facility ~5 times over the last few
years to "adjust" things...so I'm still leaning towards this being
something more common like faulty equipment, bad signal levels, etc...and
not "It's because you have a PoE switch".

-A

On Tue Mar 29, 2022, 07:42 PM GMT, Josh Luthman
 wrote:

There's a certain manufacturer of TDD radio where the CPU clock is at the
same frequency as what Verizon's enodeB will transmit.  Even at miles away,
it can and will cause PIM issues.  Again, don't rule it out.

Maybe he's just looking for a simple answer that 99% of callers will accept
and it makes them happy.  When a customer of mine tells me they think it's
something and I know it's off, I just let them believe in their statement.
There's no reason to go after this tech and insult him, all that's doing is
making everyone miserable.

On Tue, Mar 29, 2022 at 3:26 PM Joe Greco  wrote:

> On Tue, Mar 29, 2022 at 03:07:47PM -0400, Josh Luthman wrote:
> > We've routinely seen where lines not even connected to the same circuit
> in
> > any way (ie an OTA antenna coax line and cat5 POE) cause issues with one
> > another.  As much as we would all love to have a perfect line in the
> sand,
> > there isn't.  Don't rule anything out until the issue is resolved.
> >
> > As someone that sees this in the field and watches people simply hate on
> > someone because there's a frustrating situation, it's worth taking a
> breath
> > before too upset.
>
> You can run cable lines next to A/C wiring and get problems too.  Or
> ethernet lines next to A/C wiring.  That does not justify wild claims
> about PoE such as what this tech was making, and until someone shows
> me a graph of "PoE buildups" observable via SNMP or whatever the
> cable company is using to graph trends, it seems pretty clear that
> this is a bogus answer.
>
> There's a lot of difference between "we observed this very specific kind
> of interference related to PoE in a particular circumstance" and the
> crazy generalizations being made by the tech.  Asking to please make sure
> your switch is grounded properly?  That'd be good.  Asking for PoE to be
> disabled on the port?  Yeah fine.  Suggesting separation of cables?
> Sure.  Checking for proper grounding of the ground block (on the cable
> inlet)?  Sure.  There's room for things to happen.
>
> I'm all for investigating with an open mind, but I draw the line at crazy.
>
> Given that so much of the world works on PoE, it seems like the other
> potential resolution would be to note that there's an implication here
> by the tech that Comcast's hardware is standards noncompliant and ask
> them what they plan to replace their cheap CPE with.
>
> ... JG
> --
> Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
> "The strain of anti-intellectualism has been a constant thread winding its
> way
> through our political and cultural life, nurtured by the false notion that
> democracy means that 'my ignorance is just as good as your
> knowledge.'"-Asimov
>

PoE, Comcast Modems, and Service Outages

[Aaron de Bruyn via NANOG]

I just got off the phone with a Comcast tech, and wanted to double-check my
sanity.

Somehow in the last 6 months I've managed to reach the exact same rep twice
when dealing with an outage or a degraded service event.

I asked him to remotely reboot the modem because there was high packet loss.

Both times I've talked with him, he noted the high packet loss, started to
reboot the modem, and then asked me point-blank if we had any PoE switches
on our network.

When I said "yes", he said I needed to disable PoE because it messes with
the Comcast modems and he can see "buildups" in his graphs that show power
is "leaking" to the Comcast modem every 24 hours.

For reference, our setup is:

Internal Network ←→ PoE Switch ←→ My Router (FreeBSD Box) ←→ Comcast Modem

I told him the Comcast modem isn't plugged into the PoE Switch, it's
plugged into My Router (FreeBSD box) and My Router does not negotiate PoE+
and the switch shows PoE isn't being send to My Router's LAN port. While
the switch is capable of outputting old-school 24v PoE, it must be
specifically turned on for a port, and it's not enabled or used anywhere on
the networks I manage.

When provided with that information, the Comcast tech still insisted that
the switch was sending PoE to My Router and it was "leaking through" to the
Comcast modem and that's why every 4-6 weeks the Comcast modem needs to be
reset. The tech insisted that switches that *are* PoE-capable *always* send
PoE even if the device doesn't request it or negotiate it. Attempts to
explain the difference between the old 24-volt PoE and PoE+/++ were met
with arguing that he's been in the industry for decades and I don't know
what I'm talking about...and that all my problems would go away if I just
disabled PoE everywhere on the switch.

Again, I double-checked the port and said "It's not sending PoE to my
router, but even if I were, I highly doubt PoE would leak through a PCI
card to the opposite side of the chassis to the on-board NIC and out to
your modem".

He insisted it happened "all the time" and he had previously fried
equipment by plugging it into a PoE switch. He insisted that he's also
handled quite a few calls relating to this magic PoE problem over the years
and Comcast has internal tools that show graphs of how much PoE power
"builds up" inside their modems and he "can see a buildup in my router that
resets every 24 hours".

I didn't have the heart to tell him that I manage about 40 networks that
have Comcast connections...and they *all* have identical FreeBSD boxes
acting as their router, and they are *all* using the exact same PoE
switches at every location with all ports set to PoE+...and we only have
degraded service or outages after ~30 days at ~3 locations.

Slightly off-topic, but if I call Comcast about outages or degraded service
and any *other* tech but this guy answers, they all say "you need to unplug
your Comcast modem and plug it back in once every 3-4 weeks" and they act
like it's normal to reboot the modems every few weeks. In fact, last week I
wanted Comcast to check on a modem setting at one location and they said
the modem had been up for over 127 days and it should be rebooted. I said
"it's up and working fine, why would I reboot it?".

Anyways, am I insane for thinking the tech was flat-out wrong? I
mean...occasionally some really bizarre stuff happens in IT...but this
seems extremely far-fetched and contrary to everything I know about the PoE
standard.

-A

Re: Is soliciting money/rewards for 'responsible' security disclosures when none is stated a thing now?

[Aaron de Bruyn via NANOG]

I had a situation like that a few years ago.

Someone accidentally included the .git directory in a docker image that was
deployed to a customer's website.
Unfortunately early checkins of the .git directory included a copy of the
WordPress (yuck!) config file with hard-coded passwords. Those were moved
to environment variables, but never changed. And for some reason the
"developer" left indexing turned on. So the person was able to download the
git directory and walk back through the history and found the
passwordsand then connected to the database which had some mild PHI
(first names and phone numbers).

Since the tech contact for the domain came back to my company and not the
developer, they reached out to me. After a few pleasant emails back and
forth he told me exactly where he found the passwords. I rotated passwords
and yelled at the developer, and thanked the guy who found it. He kindly
asked if I would "donate" to him by buying something from his Amazon
wishlist. I should note that he asked *after* he told us exactly what the
problem was.

I discussed it with the client and they picked some ~$400 item from the
list and sent it to him.

It could have been worse, but everyone involved agreed that it would be
nice to reward the guy for pointing out the blunder.

$400 was a small price to pay for the client since they do something like
$10 million USD per month. After that the client paid for a full security
audit of their web presence by a 3rd party company and everything came back
clean.

Do what you think is appropriate, but I'm all for encouraging responsible
and positive disclosure as well as being kind. If the guy had started the
email with "send me money or else I'll disclose" the entire process would
have been very different.

-A

On Wed Mar 2, 2022, 10:30 PM GMT, Brie  wrote:

I just got this in my e-mail...

--
From: xxx 
Date: Thu, 3 Mar 2022 03:14:03 +0500
Message-ID: 
Subject: Found Security Vulnerability
To: undisclosed-recipients:;
Bcc: sxx...@ahbl.org

Hi Team

I am a web app security hunter. I spent some time on your website and found
some vulnerabilities. I see on your website you take security very
passionately.

Tell me will you give me rewards for my finding and responsible
disclosure? if Yes, So tell me where I send those vulnerability reports?
share email address.

Thank you

Good day, I truly hope it treats you awesomely on your side of the screen :)

x Security
--


Is soliciting for money/rewards when the site makes no indication they
offer them a common thing now?

If you want to see a copy of the original message, let me know off list
and I'll send it to you.


-- 
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org / http://www.ahbl.org