This ended up being a router issue on our side, changed to a mikrotik and jesus gave me a thumbs up
On Wed, Aug 5, 2015 at 11:31 AM, Josh Luthman <[email protected]> wrote: > Probe tab? Really doubt it. > > Josh Luthman > Office: 937-552-2340 > Direct: 937-552-2343 > 1100 Wayne St > Suite 1337 > Troy, OH 45373 > On Aug 5, 2015 12:24 PM, "That One Guy /sarcasm" < > [email protected]> wrote: > >> We did the 5.6 beta and went through the probe fixes some time back. >> Haven't added any probes. >> >> These situations require liberal applications of swear words and liquor >> to troubleshoot when home at night and would rather be doing something >> other than troubleshooting. >> >> I have an open ticket with them, they were looking at it, I saw in the >> log they poked some ubnt probes. >> >> I see in the billing server logs they were looking at time. >> >> Would rrd and time issue affect the probes tab updates >> On Aug 5, 2015 8:57 AM, "Josh Luthman" <[email protected]> >> wrote: >> >>> If you have one bad OID for a price none of them work. Removing the bad >>> one makes the probes work instantly. >>> >>> Status check isn't effected by this. >>> >>> Josh Luthman >>> Office: 937-552-2340 >>> Direct: 937-552-2343 >>> 1100 Wayne St >>> Suite 1337 >>> Troy, OH 45373 >>> On Aug 5, 2015 9:46 AM, "Jason Pond" <[email protected]> wrote: >>> >>>> I had this happen last week when I started upgrading my ubnt network to >>>> 5.6.2 it is the result of a bad SNMP probe causing all SNMP to stop >>>> functioning from the powercode server. I opened a ticket with powercode >>>> and they were able to resolve it. >>>> >>>> If you made any changes to the monitoring you might want to try to >>>> remove bad probes. Probes that were working on 5.5.x are not the same for >>>> 5.6.x. Once the bad probes were removed it took about 3 hours for things >>>> to really start populating graphs again. >>>> >>>> Sincerely, >>>> >>>> Jason Pond >>>> >>>> On Tue, Aug 4, 2015 at 10:53 PM, That One Guy /sarcasm < >>>> [email protected]> wrote: >>>> >>>>> Rebooted this evening, pushed the pending update this morning hoping >>>>> it was that (two revisions pending) no joy.Im at 20gb of 91gb space on the >>>>> VHD. This is a massive virtual server terabyte raid 10 array powercode >>>>> datastore 16 core 64gb ram host. Only thing on it is powercode and two >>>>> lightly hit DNS servers behind a firewall that shows little traffic, so >>>>> not >>>>> likely an external issue >>>>> >>>>> On Tue, Aug 4, 2015 at 11:48 PM, Josh Luthman < >>>>> [email protected]> wrote: >>>>> >>>>>> Disk full? Are you monitoring the servers health? >>>>>> >>>>>> Might want to try rebooting the billing server, do a snapshot before >>>>>> though. >>>>>> >>>>>> Josh Luthman >>>>>> Office: 937-552-2340 >>>>>> Direct: 937-552-2343 >>>>>> 1100 Wayne St >>>>>> Suite 1337 >>>>>> Troy, OH 45373 >>>>>> On Aug 5, 2015 12:45 AM, "That One Guy /sarcasm" < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> bandwidth gathering seems to be functional, probes are busted, about >>>>>>> every tenth click on reprobe gives me a green box, but it never >>>>>>> populates >>>>>>> the probes. I rebooted the bmu when the mirror was still present and >>>>>>> probes >>>>>>> worked during the resync, but failed again at some point, subsequent >>>>>>> reboots would not replicate the temporary functionality. >>>>>>> >>>>>>> I cannot find any devices, customer or infrastructure that are >>>>>>> working, so it seems the entire module is failing, not just hanging at >>>>>>> some >>>>>>> point in the polling as best I can tell >>>>>>> >>>>>>> On Tue, Aug 4, 2015 at 11:35 PM, Josh Luthman < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> So probes are what's busted? What about bandwidth collection? >>>>>>>> >>>>>>>> Josh Luthman >>>>>>>> Office: 937-552-2340 >>>>>>>> Direct: 937-552-2343 >>>>>>>> 1100 Wayne St >>>>>>>> Suite 1337 >>>>>>>> Troy, OH 45373 >>>>>>>> On Aug 5, 2015 12:31 AM, "That One Guy /sarcasm" < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> anybody using powercode and your monitoring is dead? Hopefully its >>>>>>>>> just us and not systemic. Apparently they tested and they can kick >>>>>>>>> probes >>>>>>>>> off on the back end, its just not logging. real time tools work, so >>>>>>>>> its not >>>>>>>>> a communication issue, we didnt make any system changes near the >>>>>>>>> failure. >>>>>>>>> The entire network monitoring component seems to be croaked. Ive >>>>>>>>> removed mirrors, minimized down to a single bmu, synchronized, >>>>>>>>> restarted >>>>>>>>> BMUs, restarted the billing server, cussed and got drunk, nothing >>>>>>>>> seems to >>>>>>>>> work. >>>>>>>>> I cant say Im all that excited with responsiveness at bertram >>>>>>>>> since the coup so Im reaching out to the community hoping somebody >>>>>>>>> has this >>>>>>>>> happen occasionally and its something like running some archaic linux >>>>>>>>> command and burning sage. >>>>>>>>> Our customer facing staff is bright as a brick when it comes to >>>>>>>>> using the statuses as it is and this is going to be a disaster, no >>>>>>>>> matter >>>>>>>>> how much i try to show them. We have 3/4 technical staff tied up all >>>>>>>>> day >>>>>>>>> tomorrow migrating an office domain so that coupled with the >>>>>>>>> inevitable >>>>>>>>> hangover is going to make this situation salty >>>>>>>>> >>>>>>>>> -- >>>>>>>>> If you only see yourself as part of the team but you don't see >>>>>>>>> your team as part of yourself you have already failed as part of the >>>>>>>>> team. >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> If you only see yourself as part of the team but you don't see your >>>>>>> team as part of yourself you have already failed as part of the team. >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> If you only see yourself as part of the team but you don't see your >>>>> team as part of yourself you have already failed as part of the team. >>>>> >>>> >>>> -- If you only see yourself as part of the team but you don't see your team as part of yourself you have already failed as part of the team.
