My no-issue APU2's BIOS is somewhat older :-) # dmesg | grep DMI: [ 0.000000] DMI: PC Engines apu2/apu2, BIOS 88a4f96 03/07/2016
Lonnie > On Sep 17, 2020, at 7:20 AM, Daryl Richards via Astlinux-users > <astlinux-users@lists.sourceforge.net> wrote: > > I don't run PPPoE, just DHCP. I do have a few systems at client locations, > also running APU2 and Astlinux 1.3.10, and either DHCP or static IP. None of > them are having the problem. One used to run PPPoE and didn't crash then > either but has since moved to static so can't test things there. > > I do like the question about BIOS though, as I do remember updating the BIOS > on this system recently. I think I may have done this in response to the > crashing but perhaps I'm mixing up the cause and effect on that.. My system > here: > > DMI: PC Engines apu2/apu2, BIOS v4.12.0.3 07/30/2020 > > On 2020-09-16 8:29 p.m., Michael Knill wrote: >> That's correct. >> Regards >> Michael Knill >> On 17/9/20, 9:34 am, "Lonnie Abelbeck" <li...@lonnie.abelbeck.com> wrote: >> > I have had EXACTLY the same problem on a Qotom box. >> But no UPS, correct ? >> Lonnie >> > On Sep 16, 2020, at 5:54 PM, Michael Knill >> <michael.kn...@ipcsolutions.com.au> wrote: >> > >> > Hi sorry all but this is not an APU problem. I have had EXACTLY the >> same problem on a Qotom box. >> > >> > At all three sites the common denominators are: >> > -- Astlinux 1.3.7.1 >> > -- PPPoE >> > >> > My next tests will be (in this order): >> > 1) Install another system onsite connected to the same environment >> e.g. power to see if it happens to both systems >> > 2) Put a firewall in front of the Astlinux box to do PPPoE >> > 3) OS upgrade >> > >> > Regards >> > Michael Knill >> > >> > On 17/9/20, 8:12 am, "Lonnie Abelbeck" <li...@lonnie.abelbeck.com> >> wrote: >> > >> > Yes, this is very strange. >> > >> > I have had an APU2 for years, never locked-up once ... and it is >> currently running a UniFi controller in a Linux container. >> > >> > I also ran my APU2 without a UPS for a year or two, though it is >> using one now. But we have good quality power where I live outside of >> thunder-storms. >> > >> > Possibly the Coreboot BIOS version also needs to be tracked to find >> some common denominator. I have never updated my APU2 coreboot BIOS. >> > >> > Also my APU2 sits inside my network, without a public interface. >> > >> > A common denominator list for APU2 lock-up reports: >> > >> > -- AstLinux version: >> > -- Asterisk: 13se, 13 or 16 >> > -- RAM: 2GB or 4GB >> > -- Coreboot BIOS version: >> > -- Public interface yes/no, if 'yes' public SSH, HTTPS, etc. access >> yes/no >> > -- PPPoE: yes/no >> > -- UPS Power: yes/no >> > >> > >> > Lonnie >> > >> > >> > >> > >> >> On Sep 16, 2020, at 4:09 PM, Michael Knill >> <michael.kn...@ipcsolutions.com.au> wrote: >> >> >> >> I do have Zabbix running on these boxes which checks memory and will >> notify me if its getting too low and this has not happened. This is what >> alerted me to the Zabbix client memory leak. >> >> Also the lockups are very sporadic. Can happen after a couple of >> weeks or the next day. >> >> >> >> Regards >> >> Michael Knill >> >> >> >> On 16/9/20, 11:21 pm, "Michael Keuter" <li...@mksolutions.info> wrote: >> >> >> >> Hi all, >> >> >> >> it might be completely unrelated, but it reminds me at an issue >> that only occured many years ago only on 32bit Geode boxes (Alix/net5501) >> after we integrated Monit into AstLinux. >> >> On these boxes all RAM was eaten up slowly and then these boxes >> locked up after a few days. That's why we didn't installed Monit for these >> build types. >> >> >> >> To figure it out I then created a cronjob that logged uptime, RAM >> and used Asterisk channels into a logfile every 10 minutes. >> >> >> >>> Am 16.09.2020 um 15:07 schrieb Lonnie Abelbeck >> <li...@lonnie.abelbeck.com>: >> >>> >> >>> Hi Daryl, >> >>> >> >>> Has the AstLinux version changed in the last couple weeks ? >> >>> >> >>> What version are you running ? >> >>> >> >>> Lonnie >> >>> >> >>> >> >>> >> >>>> On Sep 16, 2020, at 8:01 AM, Daryl Richards via Astlinux-users >> <astlinux-users@lists.sourceforge.net> wrote: >> >>>> >> >>>> This is good timing (in a bad way..) Time for a "Me too!" >> >>>> >> >>>> Recently, over the last couple weeks my APU2 has started locking up >> exactly the same way, just hard lock. I have a serial console cable hooked >> up and there are no messages printed out before, there's just nothing. When >> the problem first started I didn't have a console cable hooked up but I put >> it in to see if any messages were appearing.. >> >>>> >> >>>> My system is on a UPS. Nothing else in the rack glitches. >> >>>> >> >>>> On 2020-09-15 8:36 p.m., Michael Knill wrote: >> >>>>> Thanks Chris >> >>>>> Yep I have plenty out there too with no issues. It also happened >> on a Qotom box so not hardware related. >> >>>>> Regards >> >>>>> Michael Knill >> >>>>> *From: *AstLinux List <astlinux-users@lists.sourceforge.net> >> >>>>> *Reply to: *AstLinux List <astlinux-users@lists.sourceforge.net> >> >>>>> *Date: *Wednesday, 16 September 2020 at 8:53 am >> >>>>> *To: *AstLinux List <astlinux-users@lists.sourceforge.net> >> >>>>> *Cc: *The Cadillac Kid <eldorado...@yahoo.com> >> >>>>> *Subject: *Re: [Astlinux-users] APU2 keeps locking up >> >>>>> for whatever its worth.. from a hardware standpoint I have >> probably 200 APU2s in the field and dont have then just freeze like that.. >> granted they arent running astlinux, but just mentioning it from a power / >> hardware point of view. >> >>>>> mine are all running Centos 6.X and asterisk 11 or 13 >> >>>>> I have had a few chan_sip freezes.. (we wrote a watchdog to catch >> those and restart asterisk).. we get maybe 1 every 3 or 4 months .. (not >> each site but collectively) >> >>>>> I have kenrel panicked an APU before by upping and downing the >> ethernet port too much (or so it seems).. it doesnt happen very often and >> is hard to repeat in the lab. but have done it esp on an install.. where >> one is prone to plug and unplug cables multiple times in succession for >> dressing in.. >> >>>>> I have had a few power bricks go bad.. in all cases there was just >> no output.. no lights on the board at all. its not been that many 3 or 4.. >> considering we have sites in some pretty lightning-prone areas i dont feel >> too bad about a few power bricks. >> >>>>> we run UPSs on all of our sites >> >>>>> we run the US power supply that is sold on the PC-Engines store. >> >>>>> On Tuesday, September 15, 2020, 6:04:44 PM EDT, Michael Knill >> <michael.kn...@ipcsolutions.com.au> wrote: >> >>>>> Yep I would say environmental for 1, maybe 2 sites but for 3 sites >> and all only recently I cant see how it could be. >> >>>>> No serial or long ethernet cables are connected. >> >>>>> Yes completely different power adaptors when I changed from APU2 >> -> Qotom. >> >>>>> Yes they are all DSL however ALL different modems and service >> types. >> >>>>> I'm thinking I will upgrade all Runnix versions at these sites and >> if still happening upgrade to 1.3.10 and if still happening then I have no >> idea what to do. >> >>>>> Regards >> >>>>> Michael Knill >> >>>>> On 16/9/20, 7:36 am, "Lonnie Abelbeck" <li...@lonnie.abelbeck.com >> <mailto:li...@lonnie.abelbeck.com>> wrote: >> >>>>> So a UPS may solve the issue as with Site 1 ? >> >>>>> Do you have any serial cables connected ? >> >>>>> Any long ethernet cables connected ? >> >>>>> Is this with a variety of power adapters ? ie. the APU2 -> Qotom >> switch did the power adapter change as well ? >> >>>>> Sure sounds environmental to me. >> >>>>> I've seen DC-to-DC UPSs for about $40 USD, but never tried one. >> >>>>> https://protectli.com/product/uninterruptible-power-supply/ >> >>>>> Lonnie >> >>>>>> On Sep 15, 2020, at 3:58 PM, Michael Knill >> <michael.kn...@ipcsolutions.com.au<mailto:michael.kn...@ipcsolutions.com.au>> >> wrote: >> >>>>>> >> >>>>>> Ok I'm reviving this thread as I have now had my third site with >> this issue. >> >>>>>> >> >>>>>> Symptom: >> >>>>>> Astlinux completely locks up and requires a power reset. The log >> shows NOTHING in all cases. >> >>>>>> >> >>>>>> Troubleshooting conducted: >> >>>>>> Site 1 - The problem has not occurred since both a UPS AND Power >> Filter have been added >> >>>>>> Site 2 - A power filter has been added and the problem has >> reoccurred. I have completely changed the system from an APU2 to Qotom and >> it did the same thing again >> >>>>>> Site 3 - A known working APU2 was installed and it locked up >> yesterday. >> >>>>>> >> >>>>>> I have checked mSATA cards and they were different across systems >> at these sites. >> >>>>>> All systems are running 1.3.7.1 but I am running this at many >> other sites on the same hardware with no issues. >> >>>>>> >> >>>>>> Power quality testing is extremely expensive and surely I cant be >> having a power issue at 3 sites! >> >>>>>> The whole thing just doesn't make any sense and I don't know >> where to go from here. >> >>>>>> Any ideas? >> >>>>>> >> >>>>>> Regards >> >>>>>> Michael Knill >> >>>>>> >> >>>>>> On 22/4/20, 9:24 pm, "Michael Knill" >> <michael.kn...@ipcsolutions.com.au<mailto:michael.kn...@ipcsolutions.com.au>> >> wrote: >> >>>>>> >> >>>>>> Looks like this problem was bad power. I was told by the local IT >> Guy that there are regular brownouts so I installed a UPS. No more problems >> since doing so. >> >>>>>> Seems like APU's don't like low voltage scenarios. >> >>>>>> >> >>>>>> Thanks for your help. >> >>>>>> >> >>>>>> Regards >> >>>>>> Michael Knill >> >>>>>> >> >>>>>> On 13/4/20, 1:05 pm, "Michael Knill" >> <michael.kn...@ipcsolutions.com.au<mailto:michael.kn...@ipcsolutions.com.au>> >> wrote: >> >>>>>> >> >>>>>> Yes could be but very unusual. I have never had this problem >> with any other APU. >> >>>>>> Maybe I will look for a good surge suppressor. >> >>>>>> >> >>>>>> Regards >> >>>>>> Michael Knill >> >>>>>> >> >>>>>> On 13/4/20, 12:47 pm, "Lonnie Abelbeck" >> <li...@lonnie.abelbeck.com <mailto:li...@lonnie.abelbeck.com>> wrote: >> >>>>>> >> >>>>>> Interesting, maybe bad power (spikes, noise, etc.) >> >>>>>> >> >>>>>> Test with a UPS attached or good surge suppresser. >> >>>>>> >> >>>>>> Lonnie >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>>> On Apr 12, 2020, at 9:17 PM, Michael Knill >> <michael.kn...@ipcsolutions.com.au<mailto:michael.kn...@ipcsolutions.com.au>> >> wrote: >> >>>>>>> >> >>>>>>> Hi Lonnie >> >>>>>>> >> >>>>>>> I have replaced the hardware already and it did EXACTLY the same >> thing ☹ >> >>>>>>> >> >>>>>>> Regards >> >>>>>>> Michael Knill >> >>>>>>> >> >>>>>>> On 13/4/20, 12:15 pm, "Lonnie Abelbeck" >> <li...@lonnie.abelbeck.com <mailto:li...@lonnie.abelbeck.com>> wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>>> On Apr 12, 2020, at 7:03 PM, Michael Knill >> <michael.kn...@ipcsolutions.com.au<mailto:michael.kn...@ipcsolutions.com.au>> >> wrote: >> >>>>>>>> >> >>>>>>>> May not have anything to do with Astlinux but I have a site >> which completely locks up e.g. cannot even communicate using serial port. >> >>>>>>>> On reboot its fine but there is nothing in the logs but the >> bootup messages. >> >>>>>>>> So far I have replaced the hardware with new storage as well >> and power supply and it is still doing it. >> >>>>>>>> >> >>>>>>>> It is currently running Astlinux 1.3.7.1 which has been running >> fine on another APU2 but its only been 8 days. >> >>>>>>>> >> >>>>>>>> Any ideas what I can do next? >> >>>>>>>> >> >>>>>>>> Regards >> >>>>>>>> Michael Knill >> >>>>>>> >> >>>>>>> Sounds like an APU2 hardware issue. >> >>>>>>> >> >>>>>>> Maybe Pascal will give you a replacement. >> >>>>>>> >> >>>>>>> Lonnie >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> _______________________________________________ >> >>>>>>> Astlinux-users mailing list >> >>>>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>>>> >> >>>>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> _______________________________________________ >> >>>>>>> Astlinux-users mailing list >> >>>>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>>>> >> >>>>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> Astlinux-users mailing list >> >>>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>>> >> >>>>>> Donations to support AstLinux are graciously accepted via >> PayPal to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>>> >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> Astlinux-users mailing list >> >>>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>>> >> >>>>>> Donations to support AstLinux are graciously accepted via >> PayPal to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>>> >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> Astlinux-users mailing list >> >>>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>>> >> >>>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>>> >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> Astlinux-users mailing list >> >>>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>>> >> >>>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>> _______________________________________________ >> >>>>> Astlinux-users mailing list >> >>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>> _______________________________________________ >> >>>>> Astlinux-users mailing list >> >>>>> Astlinux-users@lists.sourceforge.net >> <mailto:Astlinux-users@lists.sourceforge.net> >> >>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. <mailto:pay...@krisk.org.> >> >>>>> _______________________________________________ >> >>>>> Astlinux-users mailing list >> >>>>> Astlinux-users@lists.sourceforge.net >> >>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>>> Donations to support AstLinux are graciously accepted via PayPal >> to pay...@krisk.org. >> >>>> >> >>>> -- >> >>>> Daryl Richards >> >>>> Isle Technical Services Inc. >> >>>> >> >>>> >> >>>> _______________________________________________ >> >>>> Astlinux-users mailing list >> >>>> Astlinux-users@lists.sourceforge.net >> >>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>>> >> >>>> Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> >>> >> >>> >> >>> >> >>> _______________________________________________ >> >>> Astlinux-users mailing list >> >>> Astlinux-users@lists.sourceforge.net >> >>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >>> >> >>> Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> >> >> >> >> >> Michael >> >> >> >> http://www.mksolutions.info >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> Astlinux-users mailing list >> >> Astlinux-users@lists.sourceforge.net >> >> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >> >> >> Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> >> >> >> >> >> _______________________________________________ >> >> Astlinux-users mailing list >> >> Astlinux-users@lists.sourceforge.net >> >> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >> >> >> Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> > >> > >> > >> > _______________________________________________ >> > Astlinux-users mailing list >> > Astlinux-users@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/astlinux-users >> > >> > Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> > >> > >> > _______________________________________________ >> > Astlinux-users mailing list >> > Astlinux-users@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/astlinux-users >> > >> > Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> _______________________________________________ >> Astlinux-users mailing list >> Astlinux-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. >> _______________________________________________ >> Astlinux-users mailing list >> Astlinux-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> Donations to support AstLinux are graciously accepted via PayPal to >> pay...@krisk.org. > > -- > Daryl Richards > Isle Technical Services Inc. > > > _______________________________________________ > Astlinux-users mailing list > Astlinux-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to > pay...@krisk.org. _______________________________________________ Astlinux-users mailing list Astlinux-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/astlinux-users Donations to support AstLinux are graciously accepted via PayPal to pay...@krisk.org.