We did a round of updates on our network Saturday. I had 4 of them fail the update, but the units were still online responding to pings, only I could not get to the Reset page to reboot them remotely. 3 of them, after a hard reboot, were able to come back up and get updated. 1 was "bricked" by the failed update and had to be replaced, the on site tech was unable to get a login page with static set, and wasn't getting a DHCP lease from it.
I have started rebooting the CPE each time *before *doing the firmware update remotely. I am not sure if it helps or not though. Interestingly, I had a tech on site where I tried to update remotely for him. I was unable to, but he was able to just fine on the LAN side. It seems to indicate the file is corrupting in transit? We are having the installers update the latest version when they install. I attempted another one today remotely that was detaching (signal strength 25). It didn't come back up. An hour later I had them unplug, wait for 10 minutes and reboot it. It still wouldn't reconnect, and is now a scheduled service call. We have just started deploying the CPE8000 series and so only have ~50 in the field. 10% firmware failure rate seems about right. I'm nervous to upgrade the firmwares now '~' On Wed, Feb 8, 2017 at 7:50 AM, <[email protected]> wrote: > Send Telrad mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.wispa.org/mailman/listinfo/telrad > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Telrad digest..." > > > Today's Topics: > > 1. UE upgrade failure rate (Shayne Lebrun) > 2. Re: UE upgrade failure rate (Adam Moffett) > 3. Re: UE upgrade failure rate (Matthew Carpenter) > 4. Re: UE upgrade failure rate (Gabriel Pike) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 8 Feb 2017 09:14:36 -0500 > From: "Shayne Lebrun" <[email protected]> > Subject: [Telrad] UE upgrade failure rate > To: <[email protected]> > Message-ID: <[email protected]> > Content-Type: text/plain; charset="us-ascii" > > Does anybody else experience a ten to fifteen percent failure rate when > upgrading UEs? The behavior is, you upgrade the firmware, reboot, and the > device doesn't come back. Logging into the UE's management from LAN, > you'll > see it's stuck in 'device init.' Defaulting the unit and rebooting allows > it to boot and attach. > > > > We're not using the residential gateway device or anything, and the only > config we put in is device name, SNMP and ACS settings. Sometimes we > hardcode the client's device in the DHCP server, to turn on DMZ to allow > port forwarding, but that doesn't seem to be a causal factor. > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.wispa.org/mailman/private/telrad/ > attachments/20170208/be5eb21f/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Wed, 08 Feb 2017 14:23:47 +0000 > From: "Adam Moffett" <[email protected]> > Subject: Re: [Telrad] UE upgrade failure rate > To: [email protected] > Message-ID: <eme5dcb657-ec2e-4e6f-b53c-e69a87202ef5@techlaptop1> > Content-Type: text/plain; charset="utf-8" > > We've had a helluva time upgrading UE firmware over the air. It was > worse with Wimax. On Wimax it was more like 75% of the time we would > lose the channel scan table and have to go on site to add it back in. > It became SOP to leave the operator password at default so we had the > option of having the customer log in and fix the scan table for us. > > I think we've had more success since going to LTE. However, failed > firmware updates was one of the incentives to set up a dedicated > management bearer. I was hoping it would help with these things. We > haven't pushed out an update recently enough to say whether it helped. > > -Adam > > > > ------ Original Message ------ > From: "Shayne Lebrun" <[email protected]> > To: [email protected] > Sent: 2/8/2017 9:14:36 AM > Subject: [Telrad] UE upgrade failure rate > > >Does anybody else experience a ten to fifteen percent failure rate when > >upgrading UEs? The behavior is, you upgrade the firmware, reboot, and > >the device doesn?t come back. Logging into the UE?s management from > >LAN, you?ll see it?s stuck in ?device init.? Defaulting the unit and > >rebooting allows it to boot and attach. > > > > > > > >We?re not using the residential gateway device or anything, and the > >only config we put in is device name, SNMP and ACS settings. Sometimes > >we hardcode the client?s device in the DHCP server, to turn on DMZ to > >allow port forwarding, but that doesn?t seem to be a causal factor. > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.wispa.org/mailman/private/telrad/ > attachments/20170208/9a328967/attachment-0001.html > > ------------------------------ > > Message: 3 > Date: Wed, 8 Feb 2017 08:37:43 -0600 > From: Matthew Carpenter <[email protected]> > Subject: Re: [Telrad] UE upgrade failure rate > To: Adam Moffett <[email protected]>, Telrad List > <[email protected]> > Message-ID: > <CAEBu8BjM4yH3dtLWrgot0QfuC-f24OJkc3dpDDtzUtvKtBfMOA@mail. > gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, > > So far only 1 CPE8000 UE that did not come back after a firmware update. > Normally a hard reboot would fix it, but in this case we had to replace it. > I have that CPE8000 on my desk and need to see what the status is from the > LAN side. Thanks for the info on defaulting it, will try it. > > Matt C. > > > > On Wed, Feb 8, 2017 at 8:23 AM, Adam Moffett <[email protected]> > wrote: > > > We've had a helluva time upgrading UE firmware over the air. It was > worse > > with Wimax. On Wimax it was more like 75% of the time we would lose the > > channel scan table and have to go on site to add it back in. It became > SOP > > to leave the operator password at default so we had the option of having > > the customer log in and fix the scan table for us. > > > > I think we've had more success since going to LTE. However, failed > > firmware updates was one of the incentives to set up a dedicated > management > > bearer. I was hoping it would help with these things. We haven't pushed > > out an update recently enough to say whether it helped. > > > > -Adam > > > > > > > > ------ Original Message ------ > > From: "Shayne Lebrun" <[email protected]> > > To: [email protected] > > Sent: 2/8/2017 9:14:36 AM > > Subject: [Telrad] UE upgrade failure rate > > > > Does anybody else experience a ten to fifteen percent failure rate when > > upgrading UEs? The behavior is, you upgrade the firmware, reboot, and > the > > device doesn?t come back. Logging into the UE?s management from LAN, > > you?ll see it?s stuck in ?device init.? Defaulting the unit and > rebooting > > allows it to boot and attach. > > > > > > > > We?re not using the residential gateway device or anything, and the only > > config we put in is device name, SNMP and ACS settings. Sometimes we > > hardcode the client?s device in the DHCP server, to turn on DMZ to allow > > port forwarding, but that doesn?t seem to be a causal factor. > > > > > > _______________________________________________ > > Telrad mailing list > > [email protected] > > http://lists.wispa.org/mailman/listinfo/telrad > > > > > > > -- > *Matthew Carpenter* > *806-316-5071 office* > *806-236-9558 cell* > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.wispa.org/mailman/private/telrad/ > attachments/20170208/fd708321/attachment-0001.html > > ------------------------------ > > Message: 4 > Date: Wed, 8 Feb 2017 09:50:03 -0500 > From: "Gabriel Pike" <[email protected]> > Subject: Re: [Telrad] UE upgrade failure rate > To: <[email protected]> > Message-ID: <[email protected]> > Content-Type: text/plain; charset="utf-8" > > I have had 3 out of 20 8000 UE?s lock up when remote upgrade is used for > pac 0. No problems with 7000 UE?s at all. On the 8000 UE?s it is like their > flash got wiped but never loaded correctly. They light up on boot but never > establish an Ethernet link. When I try to use the fallback IP or the > default IP I never get a connection. The 3 that I have had problems with > are on their way back to Telrad engineering to see what happened. > > > > Regards, > > > > Gabriel Pike > > Network Engineering > > MTCNA > > <http://dmcibb.net/> DMCI Broadband, LLC > > [email protected] > > > > 877.936.2422 > > Ext. 103 > > > > > > From: [email protected] [mailto:[email protected]] On > Behalf Of Matthew Carpenter > Sent: Wednesday, February 08, 2017 9:38 AM > To: Adam Moffett; Telrad List > Subject: Re: [Telrad] UE upgrade failure rate > > > > Hi, > > > > So far only 1 CPE8000 UE that did not come back after a firmware update. > Normally a hard reboot would fix it, but in this case we had to replace it. > > I have that CPE8000 on my desk and need to see what the status is from the > LAN side. Thanks for the info on defaulting it, will try it. > > > > Matt C. > > > > > > > > On Wed, Feb 8, 2017 at 8:23 AM, Adam Moffett <[email protected]> > wrote: > > We've had a helluva time upgrading UE firmware over the air. It was worse > with Wimax. On Wimax it was more like 75% of the time we would lose the > channel scan table and have to go on site to add it back in. It became SOP > to leave the operator password at default so we had the option of having > the customer log in and fix the scan table for us. > > > > I think we've had more success since going to LTE. However, failed > firmware updates was one of the incentives to set up a dedicated management > bearer. I was hoping it would help with these things. We haven't pushed > out an update recently enough to say whether it helped. > > > > -Adam > > > > > > > > ------ Original Message ------ > > From: "Shayne Lebrun" <[email protected]> > > To: [email protected] > > Sent: 2/8/2017 9:14:36 AM > > Subject: [Telrad] UE upgrade failure rate > > > > Does anybody else experience a ten to fifteen percent failure rate when > upgrading UEs? The behavior is, you upgrade the firmware, reboot, and the > device doesn?t come back. Logging into the UE?s management from LAN, > you?ll see it?s stuck in ?device init.? Defaulting the unit and rebooting > allows it to boot and attach. > > > > We?re not using the residential gateway device or anything, and the only > config we put in is device name, SNMP and ACS settings. Sometimes we > hardcode the client?s device in the DHCP server, to turn on DMZ to allow > port forwarding, but that doesn?t seem to be a causal factor. > > > _______________________________________________ > Telrad mailing list > [email protected] > http://lists.wispa.org/mailman/listinfo/telrad > > > > > > > > -- > > Matthew Carpenter > > 806-316-5071 office > > 806-236-9558 cell > > > > <https://docs.google.com/uc?export=download&id= > 0BxDRq5UV7HPOaEM4LXVaVnk5cWM&revid=0BxDRq5UV7HPOTDdiVjM0TXRIc3ZzM > XVUVDdDVjBiaFU0bHJNPQ> > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: http://lists.wispa.org/mailman/private/telrad/ > attachments/20170208/e4c6639f/attachment.html > > ------------------------------ > > _______________________________________________ > Telrad mailing list > [email protected] > http://lists.wispa.org/mailman/listinfo/telrad > > > End of Telrad Digest, Vol 28, Issue 17 > ************************************** > -- -Jack Ward, Staff Colorado Central Telecom
_______________________________________________ Telrad mailing list [email protected] http://lists.wispa.org/mailman/listinfo/telrad
