Yea, that's for sure, it sucks when it turns out you're THAT customer On Aug 6, 2015 5:31 PM, "Josh Luthman" <j...@imaginenetworksllc.com> wrote:
> Thank you Powercode for figuring out a problem on my network? ;) > > Josh Luthman > Office: 937-552-2340 > Direct: 937-552-2343 > 1100 Wayne St > Suite 1337 > Troy, OH 45373 > On Aug 6, 2015 6:28 PM, "That One Guy /sarcasm" <thatoneguyst...@gmail.com> > wrote: > >> This ended up being a router issue on our side, changed to a mikrotik and >> jesus gave me a thumbs up >> >> On Wed, Aug 5, 2015 at 11:31 AM, Josh Luthman < >> j...@imaginenetworksllc.com> wrote: >> >>> Probe tab? Really doubt it. >>> >>> Josh Luthman >>> Office: 937-552-2340 >>> Direct: 937-552-2343 >>> 1100 Wayne St >>> Suite 1337 >>> Troy, OH 45373 >>> On Aug 5, 2015 12:24 PM, "That One Guy /sarcasm" < >>> thatoneguyst...@gmail.com> wrote: >>> >>>> We did the 5.6 beta and went through the probe fixes some time back. >>>> Haven't added any probes. >>>> >>>> These situations require liberal applications of swear words and liquor >>>> to troubleshoot when home at night and would rather be doing something >>>> other than troubleshooting. >>>> >>>> I have an open ticket with them, they were looking at it, I saw in the >>>> log they poked some ubnt probes. >>>> >>>> I see in the billing server logs they were looking at time. >>>> >>>> Would rrd and time issue affect the probes tab updates >>>> On Aug 5, 2015 8:57 AM, "Josh Luthman" <j...@imaginenetworksllc.com> >>>> wrote: >>>> >>>>> If you have one bad OID for a price none of them work. Removing the >>>>> bad one makes the probes work instantly. >>>>> >>>>> Status check isn't effected by this. >>>>> >>>>> Josh Luthman >>>>> Office: 937-552-2340 >>>>> Direct: 937-552-2343 >>>>> 1100 Wayne St >>>>> Suite 1337 >>>>> Troy, OH 45373 >>>>> On Aug 5, 2015 9:46 AM, "Jason Pond" <p...@grizzlyinternet.com> wrote: >>>>> >>>>>> I had this happen last week when I started upgrading my ubnt network >>>>>> to 5.6.2 it is the result of a bad SNMP probe causing all SNMP to stop >>>>>> functioning from the powercode server. I opened a ticket with powercode >>>>>> and they were able to resolve it. >>>>>> >>>>>> If you made any changes to the monitoring you might want to try to >>>>>> remove bad probes. Probes that were working on 5.5.x are not the same >>>>>> for >>>>>> 5.6.x. Once the bad probes were removed it took about 3 hours for things >>>>>> to really start populating graphs again. >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Jason Pond >>>>>> >>>>>> On Tue, Aug 4, 2015 at 10:53 PM, That One Guy /sarcasm < >>>>>> thatoneguyst...@gmail.com> wrote: >>>>>> >>>>>>> Rebooted this evening, pushed the pending update this morning hoping >>>>>>> it was that (two revisions pending) no joy.Im at 20gb of 91gb space on >>>>>>> the >>>>>>> VHD. This is a massive virtual server terabyte raid 10 array powercode >>>>>>> datastore 16 core 64gb ram host. Only thing on it is powercode and two >>>>>>> lightly hit DNS servers behind a firewall that shows little traffic, so >>>>>>> not >>>>>>> likely an external issue >>>>>>> >>>>>>> On Tue, Aug 4, 2015 at 11:48 PM, Josh Luthman < >>>>>>> j...@imaginenetworksllc.com> wrote: >>>>>>> >>>>>>>> Disk full? Are you monitoring the servers health? >>>>>>>> >>>>>>>> Might want to try rebooting the billing server, do a snapshot >>>>>>>> before though. >>>>>>>> >>>>>>>> Josh Luthman >>>>>>>> Office: 937-552-2340 >>>>>>>> Direct: 937-552-2343 >>>>>>>> 1100 Wayne St >>>>>>>> Suite 1337 >>>>>>>> Troy, OH 45373 >>>>>>>> On Aug 5, 2015 12:45 AM, "That One Guy /sarcasm" < >>>>>>>> thatoneguyst...@gmail.com> wrote: >>>>>>>> >>>>>>>>> bandwidth gathering seems to be functional, probes are busted, >>>>>>>>> about every tenth click on reprobe gives me a green box, but it never >>>>>>>>> populates the probes. I rebooted the bmu when the mirror was still >>>>>>>>> present >>>>>>>>> and probes worked during the resync, but failed again at some point, >>>>>>>>> subsequent reboots would not replicate the temporary functionality. >>>>>>>>> >>>>>>>>> I cannot find any devices, customer or infrastructure that are >>>>>>>>> working, so it seems the entire module is failing, not just hanging >>>>>>>>> at some >>>>>>>>> point in the polling as best I can tell >>>>>>>>> >>>>>>>>> On Tue, Aug 4, 2015 at 11:35 PM, Josh Luthman < >>>>>>>>> j...@imaginenetworksllc.com> wrote: >>>>>>>>> >>>>>>>>>> So probes are what's busted? What about bandwidth collection? >>>>>>>>>> >>>>>>>>>> Josh Luthman >>>>>>>>>> Office: 937-552-2340 >>>>>>>>>> Direct: 937-552-2343 >>>>>>>>>> 1100 Wayne St >>>>>>>>>> Suite 1337 >>>>>>>>>> Troy, OH 45373 >>>>>>>>>> On Aug 5, 2015 12:31 AM, "That One Guy /sarcasm" < >>>>>>>>>> thatoneguyst...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> anybody using powercode and your monitoring is dead? Hopefully >>>>>>>>>>> its just us and not systemic. Apparently they tested and they can >>>>>>>>>>> kick >>>>>>>>>>> probes off on the back end, its just not logging. real time tools >>>>>>>>>>> work, so >>>>>>>>>>> its not a communication issue, we didnt make any system changes >>>>>>>>>>> near the >>>>>>>>>>> failure. >>>>>>>>>>> The entire network monitoring component seems to be croaked. Ive >>>>>>>>>>> removed mirrors, minimized down to a single bmu, synchronized, >>>>>>>>>>> restarted >>>>>>>>>>> BMUs, restarted the billing server, cussed and got drunk, nothing >>>>>>>>>>> seems to >>>>>>>>>>> work. >>>>>>>>>>> I cant say Im all that excited with responsiveness at bertram >>>>>>>>>>> since the coup so Im reaching out to the community hoping somebody >>>>>>>>>>> has this >>>>>>>>>>> happen occasionally and its something like running some archaic >>>>>>>>>>> linux >>>>>>>>>>> command and burning sage. >>>>>>>>>>> Our customer facing staff is bright as a brick when it comes to >>>>>>>>>>> using the statuses as it is and this is going to be a disaster, no >>>>>>>>>>> matter >>>>>>>>>>> how much i try to show them. We have 3/4 technical staff tied up >>>>>>>>>>> all day >>>>>>>>>>> tomorrow migrating an office domain so that coupled with the >>>>>>>>>>> inevitable >>>>>>>>>>> hangover is going to make this situation salty >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> If you only see yourself as part of the team but you don't see >>>>>>>>>>> your team as part of yourself you have already failed as part of >>>>>>>>>>> the team. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> If you only see yourself as part of the team but you don't see >>>>>>>>> your team as part of yourself you have already failed as part of the >>>>>>>>> team. >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> If you only see yourself as part of the team but you don't see your >>>>>>> team as part of yourself you have already failed as part of the team. >>>>>>> >>>>>> >>>>>> >> >> >> -- >> If you only see yourself as part of the team but you don't see your team >> as part of yourself you have already failed as part of the team. >> >
