On Thu, May 30, 2019 at 08:49:15PM +0200, Dario Nieuwenhuis wrote:
> Hello,
>
> We have some embedded devices deployed in the field using a huawei E173s-1
> modem. We're having some issues where connectivity stops working randomly,
> after 1-3 weeks of uptime.
>
> First, there's a PPP disconnection. Then there's a few loops of this error
> for 1-2 seconds, rather fast:
>
> failed to connect modem: Couldn't connect: cannot keep data port
> open.Could not open serial device ttyUSB0: reopen operation in progress
>
> Afterwards, there are 10 attempts of reconnecting, with "at port timed out
> X consecutive times" errors. After 10 attempts, ModemManager gives up
> permanently, and never tries again to bring the device back up.
>
> (tty/ttyUSB0) at port timed out 10 consecutive times, marking modem
> '/org/freedesktop/ModemManager1/Modem/0' as invalid
>
> A few hours later, when we could get onsite, restarting ModemManager and
> NetworkManager brought the modem back online with no issues (with no
> unplug/replug/powercycle of the modem or the Linux board.) This means the
> modem is not irreversibly crashed/failed, so I think this is a software
> issue that should be fixable.
>
> Unfortunately, ModemManager was set to INFO log level because we though
> DEBUG was too verbose. We have set DEBUG log level, so the next time it
> happens we will have more logs.
>
> I would really appreciate any input you may have on how to solve this.
> - I thought of patching out the "max 10 timeouts" limit, so ModemManager
> keeps retrying indefinitely. Is this a good idea?
> - What can I try so next time this happens we can get more info on the
> issue? (besides debug log level)
> - Any recommendations in general on how to ensure the device is always
> connected? Any config knobs to tweak? We've been thinking of adding a "if
> no internet during 10 minutes, powercycle everything" watchdog, but that
> feels like giving up on getting this working properly.
I'm using MM&NM restart and if that doesn't help, then modem powercycle and
then daemon restart. Boards do have modem power supply controlled by gpio,
so that's easy to do, but rather hackish. Is there any plan to add some
"modem hw reset" infrastructure here?
ladis
_______________________________________________
ModemManager-devel mailing list
ModemManager-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/modemmanager-devel
