Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-10 Thread Marlon K. Schafer (509) 982-2181
Except it's not effecting the ap's further out.  Even if they were on a 
different segment they should still pick up the interference from the ap's 
right?


Marlon
(509) 982-2181   Equipment sales
(408) 907-6910 (Vonage)Consulting services
42846865 (icq)And I run my own wisp!
64.146.146.12 (net meeting)
www.odessaoffice.com/wireless
www.odessaoffice.com/marlon/cam



- Original Message - 
From: Lonnie Nunweiler [EMAIL PROTECTED]

To: WISPA General List wireless@wispa.org
Sent: Tuesday, May 09, 2006 5:55 PM
Subject: Re: [WISPA] Weird problem - 20 seconds latency and other oddness


I agree it could be noise but a bridge runaway will give you the 10+
second pings and with that much traffic being echoed ALL of your AP
and Clients are spewing.  It would look like a massive RF flood on the
Spectrum Analyzer.  Think about what the air wave look like when you
have full radio usage.  To nearby units and competitors it would be a
massive increase in the noise floor.

Lonnie

On 5/9/06, David E. Smith [EMAIL PROTECTED] wrote:

Lonnie Nunweiler wrote:
 Any confirmation on this?  A customer router plugged in with LAN to
 the WAN or not getting a DHCP entry or even a DNS entry has caused
 many bridges to collapse and appear as if it is noise, simply because
 the bridges are all echoing the massive broadcast traffic.
There's no DHCP anywhere on the network, and the DHCP UDP ports are
filtered out at every POP, so that specifically is a bad example. :)

Sorry for not getting back to this, we've had massive weirdness on our
dialup gear too (mostly related to moving it).

Yes, our network is part bridged/part routed.

I'm pretty sure it's a real RF problem, because we pulled out the
Bumblebee and my field guy said he saw crazy mad noise all across the
2.4 spectrum a couple days ago, when we were having this weird hiccup.
(And another local WISP operator reportedly has had similar issues,
though I didn't talk to him personally; that's the boss' department.)

We'll probably just have to use some old-school triangulation and such
to find out where it is, if it's something we even can find. For
instance, today was a cool and cloudy day, and this problem didn't show
up all day. Thus, I blame sunspots. :D

(Honestly, I'm stumped, but at least we're now reasonably certain it's a
real RF issue.)

If/when we sort it out, I'll report back.

David Smith
MVN.net
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/




--
Lonnie Nunweiler
Valemount Networks Corporation
http://www.star-os.com/
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/

--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-10 Thread David E. Smith
Marlon K. Schafer (509) 982-2181 wrote:
 Except it's not effecting the ap's further out.  Even if they were on a
 different segment they should still pick up the interference from the
 ap's right?

Basically, yeah. I think. :)

(Good thing I never claimed to know much about RF, innit.)

A different picture, so to speak, at
http://thedave.us/pics/mvn/wispamap2.gif, which includes some of the
relevant bits of the network's physical design.

If it were a problem with, say, one of my Trango backhaul links, why
does it only affect one of the two SUs on Trango AP1? And why does it
affect customers on the far end of my licensed 39GHz link?

For that matter, note the two overlapping dots. (Green is good towers,
red is bad, and the one yellow dot on the left is a tower that we've
seen a couple issues with at the same time, but not nearly as bad. Note:
not to scale, and please don't make fun of me just because I can't draw
straight lines with MS Paint.) The points with overlapping dots show
where there are two 2.4GHz APs on the same water tower - the ones facing
north are having issues, the ones facing south rarely do. And when the
south-facing antennas' customers are having problems, the noise levels
and network latency are at their worst (a couple days ago I saw one ping
packet that took almost thirty seconds to complete a round-trip, a new
record).

The backhaul links appear to be solid, because all the while, I can ping
the APs themselves, and the managed switches at the key tower locations,
with the expected 2-3ms latency and zero packet loss.

David Smith
MVN.net
-- 
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-09 Thread Marlon K. Schafer (509) 982-2181
Yes.  It could also be caused by a bad microwave oven.  One that's really 
bad.  But to put out 30dB of always on signal wouldn't be that hard to do. 
Sure screw up a c to i ratio though :-).


This could also be a plastics plant that uses 2.4 to heat pellets.

Could be a competitor with a screwed up device/customer that's generating 
far more than normal traffic etc.


After talking to John I believe the cause is probably pretty close to the 
ground though.  He's got omni based systems only 10 miles away and they 
don't see nearly as much of a change.  We'd expect nearly as much noise at 
those as the close in systems if they source were above the trees.


It's a fun one.  Probably come down to having a guy in the area with a high 
gain antenna ready to locate the interference when it happens.  Two or three 
reference points at lines drawn on a map should pin point the interference 
easily enough.  If that's what it really turns out to be ;-)


Marlon
(509) 982-2181   Equipment sales
(408) 907-6910 (Vonage)Consulting services
42846865 (icq)And I run my own wisp!
64.146.146.12 (net meeting)
www.odessaoffice.com/wireless
www.odessaoffice.com/marlon/cam



- Original Message - 
From: Michael Watson [EMAIL PROTECTED]

To: WISPA General List wireless@wispa.org
Sent: Monday, May 08, 2006 1:10 PM
Subject: Re: [WISPA] Weird problem - 20 seconds latency and other oddness


If this was rf noise, Arent hamm operators allowed in 2.4 with higher 
power limits? Could this account for the 5- 10 mile affected area?


-Michael


David E. Smith wrote:

Okay, Scriv and I are stumped on this one.

Over the last couple of weeks, we've started seeing some very odd
oddness on a few of our 2.4GHz POPs. Not all, just some. Here's what
appears to be happening:

A couple times a day, usually during business hours, something somewhere
generates a massive amount of noise. Connections which report an RF
noise of -90 start showing noise levels of -60 (or worse in some cases),
as reported by our StarOS access point. If it really is RF noise, it's
very broad, as it's simultaneously hitting five or six POPs, some
several miles away, but all at the same time.

The towers are all running StarOS on Mikrotik RouterBoard hardware, with
a mix of Orinoco and Prism cards, some with amps, some not. Some have
sectored antennas (180 degrees), some have omnis. Between them, the
towers cover just about the entire 2.4 spectrum (obviously, one channel
per access point, but we're using at least channels 1, 4, 6, 8, and 11).

Those towers are basically identical to several other towers that aren't
affected.

The other really really weird part is the crazy latency. Pings to the
APs themselves are reliable, and our backhaul links (5.3 and 5.8 GHz)
don't seem to be affected. And pings to our end-customers don't seem to
get lost, they just take their sweet time getting there. While the
event is happening, I've seen pings that take in excess of twenty
seconds to complete their round trip.

64 bytes from 10.232.175.130: icmp_seq=7 ttl=62 time=27239 ms

(I think that's my record. In that particular test, there were no
packets lost, they just took a very long time to get there.)

I've checked or replaced just about everything I can think of in our
network that might cause something like this, and frankly, I'm stumped.
I don't think it's a network problem (traffic bursts or similar) because
of the weird bursts of RF noise. But that'd have to be one helluva burst
of noise to do what it's doing - affecting every channel across ten
miles at once.

I can go into more detail on any part of the network if you like, though
I think all the likely-relevant details are covered here.

Help!

David Smith
MVN.net



--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/



--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-09 Thread Marlon K. Schafer (509) 982-2181
He's running Star OS and MT as per the original post.  Some of the effected 
ap's are routers.  That's one of the first questions I double-checked too!


Marlon
(509) 982-2181   Equipment sales
(408) 907-6910 (Vonage)Consulting services
42846865 (icq)And I run my own wisp!
64.146.146.12 (net meeting)
www.odessaoffice.com/wireless
www.odessaoffice.com/marlon/cam



- Original Message - 
From: Lonnie Nunweiler [EMAIL PROTECTED]

To: WISPA General List wireless@wispa.org
Sent: Monday, May 08, 2006 2:32 PM
Subject: Re: [WISPA] Weird problem - 20 seconds latency and other oddness


I suspect your system is bridged.  Can you confirm that?

Lonnie

On 5/8/06, David E. Smith [EMAIL PROTECTED] wrote:

Okay, Scriv and I are stumped on this one.

Over the last couple of weeks, we've started seeing some very odd
oddness on a few of our 2.4GHz POPs. Not all, just some. Here's what
appears to be happening:

A couple times a day, usually during business hours, something somewhere
generates a massive amount of noise. Connections which report an RF
noise of -90 start showing noise levels of -60 (or worse in some cases),
as reported by our StarOS access point. If it really is RF noise, it's
very broad, as it's simultaneously hitting five or six POPs, some
several miles away, but all at the same time.

The towers are all running StarOS on Mikrotik RouterBoard hardware, with
a mix of Orinoco and Prism cards, some with amps, some not. Some have
sectored antennas (180 degrees), some have omnis. Between them, the
towers cover just about the entire 2.4 spectrum (obviously, one channel
per access point, but we're using at least channels 1, 4, 6, 8, and 11).

Those towers are basically identical to several other towers that aren't
affected.

The other really really weird part is the crazy latency. Pings to the
APs themselves are reliable, and our backhaul links (5.3 and 5.8 GHz)
don't seem to be affected. And pings to our end-customers don't seem to
get lost, they just take their sweet time getting there. While the
event is happening, I've seen pings that take in excess of twenty
seconds to complete their round trip.

64 bytes from 10.232.175.130: icmp_seq=7 ttl=62 time=27239 ms

(I think that's my record. In that particular test, there were no
packets lost, they just took a very long time to get there.)

I've checked or replaced just about everything I can think of in our
network that might cause something like this, and frankly, I'm stumped.
I don't think it's a network problem (traffic bursts or similar) because
of the weird bursts of RF noise. But that'd have to be one helluva burst
of noise to do what it's doing - affecting every channel across ten
miles at once.

I can go into more detail on any part of the network if you like, though
I think all the likely-relevant details are covered here.

Help!

David Smith
MVN.net
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/




--
Lonnie Nunweiler
Valemount Networks Corporation
http://www.star-os.com/
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/ 


--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-09 Thread Lonnie Nunweiler

Any confirmation on this?  A customer router plugged in with LAN to
the WAN or not getting a DHCP entry or even a DNS entry has caused
many bridges to collapse and appear as if it is noise, simply because
the bridges are all echoing the massive broadcast traffic.

Lonnie

On 5/8/06, Lonnie Nunweiler [EMAIL PROTECTED] wrote:

I suspect your system is bridged.  Can you confirm that?

Lonnie

On 5/8/06, David E. Smith [EMAIL PROTECTED] wrote:
 Okay, Scriv and I are stumped on this one.

 Over the last couple of weeks, we've started seeing some very odd
 oddness on a few of our 2.4GHz POPs. Not all, just some. Here's what
 appears to be happening:

 A couple times a day, usually during business hours, something somewhere
 generates a massive amount of noise. Connections which report an RF
 noise of -90 start showing noise levels of -60 (or worse in some cases),
 as reported by our StarOS access point. If it really is RF noise, it's
 very broad, as it's simultaneously hitting five or six POPs, some
 several miles away, but all at the same time.

 The towers are all running StarOS on Mikrotik RouterBoard hardware, with
 a mix of Orinoco and Prism cards, some with amps, some not. Some have
 sectored antennas (180 degrees), some have omnis. Between them, the
 towers cover just about the entire 2.4 spectrum (obviously, one channel
 per access point, but we're using at least channels 1, 4, 6, 8, and 11).

 Those towers are basically identical to several other towers that aren't
 affected.

 The other really really weird part is the crazy latency. Pings to the
 APs themselves are reliable, and our backhaul links (5.3 and 5.8 GHz)
 don't seem to be affected. And pings to our end-customers don't seem to
 get lost, they just take their sweet time getting there. While the
 event is happening, I've seen pings that take in excess of twenty
 seconds to complete their round trip.

 64 bytes from 10.232.175.130: icmp_seq=7 ttl=62 time=27239 ms

 (I think that's my record. In that particular test, there were no
 packets lost, they just took a very long time to get there.)

 I've checked or replaced just about everything I can think of in our
 network that might cause something like this, and frankly, I'm stumped.
 I don't think it's a network problem (traffic bursts or similar) because
 of the weird bursts of RF noise. But that'd have to be one helluva burst
 of noise to do what it's doing - affecting every channel across ten
 miles at once.

 I can go into more detail on any part of the network if you like, though
 I think all the likely-relevant details are covered here.

 Help!

 David Smith
 MVN.net
 --
 WISPA Wireless List: wireless@wispa.org

 Subscribe/Unsubscribe:
 http://lists.wispa.org/mailman/listinfo/wireless

 Archives: http://lists.wispa.org/pipermail/wireless/



--
Lonnie Nunweiler
Valemount Networks Corporation
http://www.star-os.com/




--
Lonnie Nunweiler
Valemount Networks Corporation
http://www.star-os.com/
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-09 Thread Lonnie Nunweiler

I agree it could be noise but a bridge runaway will give you the 10+
second pings and with that much traffic being echoed ALL of your AP
and Clients are spewing.  It would look like a massive RF flood on the
Spectrum Analyzer.  Think about what the air wave look like when you
have full radio usage.  To nearby units and competitors it would be a
massive increase in the noise floor.

Lonnie

On 5/9/06, David E. Smith [EMAIL PROTECTED] wrote:

Lonnie Nunweiler wrote:
 Any confirmation on this?  A customer router plugged in with LAN to
 the WAN or not getting a DHCP entry or even a DNS entry has caused
 many bridges to collapse and appear as if it is noise, simply because
 the bridges are all echoing the massive broadcast traffic.
There's no DHCP anywhere on the network, and the DHCP UDP ports are
filtered out at every POP, so that specifically is a bad example. :)

Sorry for not getting back to this, we've had massive weirdness on our
dialup gear too (mostly related to moving it).

Yes, our network is part bridged/part routed.

I'm pretty sure it's a real RF problem, because we pulled out the
Bumblebee and my field guy said he saw crazy mad noise all across the
2.4 spectrum a couple days ago, when we were having this weird hiccup.
(And another local WISP operator reportedly has had similar issues,
though I didn't talk to him personally; that's the boss' department.)

We'll probably just have to use some old-school triangulation and such
to find out where it is, if it's something we even can find. For
instance, today was a cool and cloudy day, and this problem didn't show
up all day. Thus, I blame sunspots. :D

(Honestly, I'm stumped, but at least we're now reasonably certain it's a
real RF issue.)

If/when we sort it out, I'll report back.

David Smith
MVN.net
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/




--
Lonnie Nunweiler
Valemount Networks Corporation
http://www.star-os.com/
--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-09 Thread David E. Smith
Lonnie Nunweiler wrote:
 I agree it could be noise but a bridge runaway will give you the 10+
 second pings and with that much traffic being echoed ALL of your AP
 and Clients are spewing.  It would look like a massive RF flood on the
 Spectrum Analyzer.  Think about what the air wave look like when you
 have full radio usage.  To nearby units and competitors it would be a
 massive increase in the noise floor.
That's certainly possible, but then how do I track down which specific
piece of hardware is responsible? As much as you'd enjoy the extra
StarOS software sales, I'm not sure it's feasible to wholesale-replace
twenty APs. :)

If it were, say, a specific radio running amok, I expect that the
problem would temporarily disappear for a few seconds while the AP in
question is being rebooted. This hasn't (so far) proven to be the case.

Example: If the problem were in AP3, then I'd expect the noise problem
in AP4 (almost ten miles away, running ten channels away) to disappear
for at least a few seconds while AP3 reboots. I've tested this with
virtually every combination of APs, rebooting all the affected ones (and
even a few others) in turn, and watching customers on other APs for a
change, and haven't seen one.

If it were some kind of network flood, why does it only affect certain
tower locations, all of which are at least in vague geographical
proximity? (And not other towers twenty miles away?)

http://www.thedave.us/pics/mvn/wispamap.jpg is a real quick map of the
affected areas. The big green dots are towers that are doing alright,
the big red dots are towers where weird stuff is happening. (The green
dot that's right in the center of the three red dots is a 900MHz tower
that I probably shouldn't have put on the map, as the Waverider stuff
there has been humming right along all the while.)

The next-nearest couple of towers (the green dots near Dix at the far
north, and Woodlawn to the east) have occasionally exhibited the same
behaviour, but not nearly as often as the three I marked in red.

Just about the only explanation that makes much sense to me is,
basically, someone on the north edge of town, or a bit further north,
is intermittently spamming RF. Maybe it's something on my network, or
someone else's. As always, I'm open to suggestions, ideally ones that
come with meaningful ways to test.

If/when we find the source, I'll try to follow up with everyone, just to
put the whole thing to rest.

David Smith
MVN.net
-- 
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/


Re: [WISPA] Weird problem - 20 seconds latency and other oddness

2006-05-08 Thread Michael Watson
If this was rf noise, Arent hamm operators allowed in 2.4 with higher 
power limits? Could this account for the 5- 10 mile affected area?


-Michael


David E. Smith wrote:

Okay, Scriv and I are stumped on this one.

Over the last couple of weeks, we've started seeing some very odd
oddness on a few of our 2.4GHz POPs. Not all, just some. Here's what
appears to be happening:

A couple times a day, usually during business hours, something somewhere
generates a massive amount of noise. Connections which report an RF
noise of -90 start showing noise levels of -60 (or worse in some cases),
as reported by our StarOS access point. If it really is RF noise, it's
very broad, as it's simultaneously hitting five or six POPs, some
several miles away, but all at the same time.

The towers are all running StarOS on Mikrotik RouterBoard hardware, with
a mix of Orinoco and Prism cards, some with amps, some not. Some have
sectored antennas (180 degrees), some have omnis. Between them, the
towers cover just about the entire 2.4 spectrum (obviously, one channel
per access point, but we're using at least channels 1, 4, 6, 8, and 11).

Those towers are basically identical to several other towers that aren't
affected.

The other really really weird part is the crazy latency. Pings to the
APs themselves are reliable, and our backhaul links (5.3 and 5.8 GHz)
don't seem to be affected. And pings to our end-customers don't seem to
get lost, they just take their sweet time getting there. While the
event is happening, I've seen pings that take in excess of twenty
seconds to complete their round trip.

64 bytes from 10.232.175.130: icmp_seq=7 ttl=62 time=27239 ms

(I think that's my record. In that particular test, there were no
packets lost, they just took a very long time to get there.)

I've checked or replaced just about everything I can think of in our
network that might cause something like this, and frankly, I'm stumped.
I don't think it's a network problem (traffic bursts or similar) because
of the weird bursts of RF noise. But that'd have to be one helluva burst
of noise to do what it's doing - affecting every channel across ten
miles at once.

I can go into more detail on any part of the network if you like, though
I think all the likely-relevant details are covered here.

Help!

David Smith
MVN.net
  


--
WISPA Wireless List: wireless@wispa.org

Subscribe/Unsubscribe:
http://lists.wispa.org/mailman/listinfo/wireless

Archives: http://lists.wispa.org/pipermail/wireless/