Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Aleksander Morgado writes ("Re: Bug#877024: modemmanager should ask before 
messing with serial ports"):
> Ian, any comment about this 1.8-rc1 version with the filter policies
> implemented?

Thanks for the ping.  I haven't had a chance to test it, but if it
behaves as described earlier here then (with the appropriate policy) I
think it would DTWT.

I will try to get around to teting it.

Regards,
Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

>> Also, is there any new maintainer I should talk to regarding
>> integration tasks for this new version?
>
> I've been in touch with Mathieu Trudel-Lapierre (Cc:ed) and he's still
> happy to act as maintainer.  He's currently not a DD/DM so needs
> sponsorship for his packages, but that should not be problem.
>
> The @ubuntu email address that was on the package turns out to be
> outdated, and was bouncing, which is why he's been out of the loop until
> now.

Ah! Superb :)

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Philip Hands]

On Mon, 26 Feb 2018, Aleksander Morgado  wrote:
>>>
>>> As soon as we get this merged to git master, I can tag a 1.8 beta
>>> release (e.g. 1.7.990). What do you think?
>>>
>>
>> FYI, 1.8-rc1 has been tagged with the optional filter policy support 
>> included:
>> https://lists.freedesktop.org/archives/modemmanager-devel/2018-January/006157.html
>>
>> By default it uses the DEFAULT policy (equivalent to what we were
>> doing until now). You can enable the STRICT policy passing
>> --filter-policy=STRICT on the ModemManager startup (e.g. as a patch to
>> the systemd service file).
>>
>
> Ian, any comment about this 1.8-rc1 version with the filter policies
> implemented?
>
> Also, is there any new maintainer I should talk to regarding
> integration tasks for this new version?

I've been in touch with Mathieu Trudel-Lapierre (Cc:ed) and he's still
happy to act as maintainer.  He's currently not a DD/DM so needs
sponsorship for his packages, but that should not be problem.

The @ubuntu email address that was on the package turns out to be
outdated, and was bouncing, which is why he's been out of the loop until
now.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

>>
>> As soon as we get this merged to git master, I can tag a 1.8 beta
>> release (e.g. 1.7.990). What do you think?
>>
>
> FYI, 1.8-rc1 has been tagged with the optional filter policy support included:
> https://lists.freedesktop.org/archives/modemmanager-devel/2018-January/006157.html
>
> By default it uses the DEFAULT policy (equivalent to what we were
> doing until now). You can enable the STRICT policy passing
> --filter-policy=STRICT on the ModemManager startup (e.g. as a patch to
> the systemd service file).
>

Ian, any comment about this 1.8-rc1 version with the filter policies
implemented?

Also, is there any new maintainer I should talk to regarding
integration tasks for this new version?

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

>
> As soon as we get this merged to git master, I can tag a 1.8 beta
> release (e.g. 1.7.990). What do you think?
>

FYI, 1.8-rc1 has been tagged with the optional filter policy support included:
https://lists.freedesktop.org/archives/modemmanager-devel/2018-January/006157.html

By default it uses the DEFAULT policy (equivalent to what we were
doing until now). You can enable the STRICT policy passing
--filter-policy=STRICT on the ModemManager startup (e.g. as a patch to
the systemd service file).

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Wouter Verhelst writes ("Re: Bug#877024: modemmanager should ask before messing 
with serial ports"):
> On Wed, Oct 18, 2017 at 09:18:37PM +0200, Aleksander Morgado wrote:
> > How about waiting a bit until patches reviewed and tested upstream?
> 
> The whole point of experimental is "to test and review stuff". It's fine
> if things break there; that's what it's for.

Precisely.

> Having packages in experimental allows for more widespread testing. It
> would seem that in this kind of situation, you would actually want
> that...?

That was my thinking.

For upstream: things in "experimental" are not automatically installed
on user systems (well, unless the user has been exceptionally
foolish), and do not automatically propagate to places where they
might be so installed (let alone, be released).

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#877024: modemmanager should ask before messing with serial ports

[Wouter Verhelst]

On Wed, Oct 18, 2017 at 09:18:37PM +0200, Aleksander Morgado wrote:
> On Tue, Oct 17, 2017 at 6:48 PM, Ian Jackson
> > I will do that upload: to DELAYED, picking some suitably cautious
> > version number, unless I hear to the contrary.  (And I'll wait at
> > least a week for a reply to this question.)
> >
> 
> How about waiting a bit until patches reviewed and tested upstream?

The whole point of experimental is "to test and review stuff". It's fine
if things break there; that's what it's for.

Having packages in experimental allows for more widespread testing. It
would seem that in this kind of situation, you would actually want
that...?

-- 
Could you people please use IRC like normal people?!?

  -- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008
 Hacklab

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

On Tue, Oct 17, 2017 at 6:48 PM, Ian Jackson
<ijack...@chiark.greenend.org.uk> wrote:
> Aleksander Morgado writes ("Re: Bug#877024: modemmanager should ask before 
> messing with serial ports"):
>> FYI, already implemented an initial approach, ready for testing:
>> https://lists.freedesktop.org/archives/modemmanager-devel/2017-October/005969.html
>>
>> In the Debian case, you would want to use "--filter-policy=STRICT"
>> when starting ModemManager.
>
> Thanks.  This is very encouraging.  I would like to test this.
>
> My testing approach would probably be to prepare a Debian package of
> this version, based on the existing Debian package.
>
> It might be useful for me to upload such a thing to experimental so
> others can try it too ?  Would the Debian modemmanager maintainers
> object to that ?
>
> I will do that upload: to DELAYED, picking some suitably cautious
> version number, unless I hear to the contrary.  (And I'll wait at
> least a week for a reply to this question.)
>

How about waiting a bit until patches reviewed and tested upstream?
I'm worried about some devices in particular, like Bluetooth DUN based
ones, as I rarely test those and I'm almost certain that I broke them
in strict mode.

As soon as we get this merged to git master, I can tag a 1.8 beta
release (e.g. 1.7.990). What do you think?

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Aleksander Morgado writes ("Re: Bug#877024: modemmanager should ask before 
messing with serial ports"):
> FYI, already implemented an initial approach, ready for testing:
> https://lists.freedesktop.org/archives/modemmanager-devel/2017-October/005969.html
> 
> In the Debian case, you would want to use "--filter-policy=STRICT"
> when starting ModemManager.

Thanks.  This is very encouraging.  I would like to test this.

My testing approach would probably be to prepare a Debian package of
this version, based on the existing Debian package.

It might be useful for me to upload such a thing to experimental so
others can try it too ?  Would the Debian modemmanager maintainers
object to that ?

I will do that upload: to DELAYED, picking some suitably cautious
version number, unless I hear to the contrary.  (And I'll wait at
least a week for a reply to this question.)

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

FYI, already implemented an initial approach, ready for testing:
https://lists.freedesktop.org/archives/modemmanager-devel/2017-October/005969.html

In the Debian case, you would want to use "--filter-policy=STRICT"
when starting ModemManager.

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Philip Hands]

Aleksander Morgado <aleksan...@aleksander.es> writes:

> Hey Ian,
>
> On Thu, Oct 12, 2017 at 2:39 PM, Ian Jackson
> <ijack...@chiark.greenend.org.uk> wrote:
>> Aleksander Morgado writes ("Bug#877024: modemmanager should ask before 
>> messing with serial ports"):
>>> This is part of the discussion we had in the MM mailing list for such
>>> a solution:
>>> https://lists.freedesktop.org/archives/modemmanager-devel/2017-September/005917.html
>>
>> Thanks, this looks constructive.
>>
>> Of the heuristics in that mail, most seem to me to be very sound
>> justifications for thinking that the device is safe to probe.
>>
>> The one big exception is this:
>>
>>  | * If vid is a known modem vendor (e.g. huawei, zte, sierra, u-blox,
>>  |  telit), it's a modem and we probe the tty.
>>
>> This is a hostage to the future, since of course we don't know what
>> devices might be manufactured by a particular vendor in the many-years
>> life of a Debian release.
>>
>
> Yes, this one is probably the weakest rule of all. Still not sure at
> which point to apply the rule, though. E.g. should it be applied after
> having applied all the previous rules (in that case it would be a very
> safe rule, maybe totally unneeded) or should it be applied as an OR to
> some other rule (e.g. driver is option/sierra/qcserial OR vendor is
> huawei/zte..., in this case it would be a weaker rule). Will need to
> decide this based on testing with real devices.

It's nice to see that a workable solution seems to be emerging here.

My opinion on the final wrinkle is that for Debian, in the case of
uncertainty, modemmanager should not be probing, and that guessing based
on manufacturer seems insufficiently certain.

Optimistic probing hides the fact that one doesn't _really_ know the
device is a modem.  The result being that the bug report mentioning the
features of the device that might allow an improved heuristic to be
developed is never going to be submitted.

That being the case, I agree with Ian that if such behaviour is
implemented, it would be best if it can be disabled at run-time, and that
the Debian package should then default to disabling it.

Having the option to enable it at run-time would still be useful, so
that people that know that they really do have a modem, can:

  read the package's README to discover why it doesn't work

  report a bug saying what sort of modem they have, and that it's not
  being found by the Debian default configuration of MM

  and then flick the switch to make modemmanager optimistic enough to
  find their modem (on the understanding that it might break other
  stuff they could plug in, but at least they'll know why)

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,GERMANY


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

Hey Ian,

On Thu, Oct 12, 2017 at 2:39 PM, Ian Jackson
<ijack...@chiark.greenend.org.uk> wrote:
> Aleksander Morgado writes ("Bug#877024: modemmanager should ask before 
> messing with serial ports"):
>> This is part of the discussion we had in the MM mailing list for such
>> a solution:
>> https://lists.freedesktop.org/archives/modemmanager-devel/2017-September/005917.html
>
> Thanks, this looks constructive.
>
> Of the heuristics in that mail, most seem to me to be very sound
> justifications for thinking that the device is safe to probe.
>
> The one big exception is this:
>
>  | * If vid is a known modem vendor (e.g. huawei, zte, sierra, u-blox,
>  |  telit), it's a modem and we probe the tty.
>
> This is a hostage to the future, since of course we don't know what
> devices might be manufactured by a particular vendor in the many-years
> life of a Debian release.
>

Yes, this one is probably the weakest rule of all. Still not sure at
which point to apply the rule, though. E.g. should it be applied after
having applied all the previous rules (in that case it would be a very
safe rule, maybe totally unneeded) or should it be applied as an OR to
some other rule (e.g. driver is option/sierra/qcserial OR vendor is
huawei/zte..., in this case it would be a weaker rule). Will need to
decide this based on testing with real devices.

> I guess that if we in Debian don't like that particular heuristic it
> would be possible for us to disble just that one ?
>
> I don't understand this one:
>
>  | * If kernel driver of the TTY is option/sierra/qcserial,
>
> but maybe it is OK.
>

The "option", "sierra" and "qcserial" drivers are all TTY kernel
drivers for modems. Most TTYs from USB modems would fall into one of
these. No non-modem device should bind to these drivers.

I'm planning to have a patch ready to play with in the following week,
will let you know.

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Sam Hartman writes ("Bug#877024: modemmanager should ask before messing with 
serial ports"):
> I'd be happy with guidelines that  said something along the lines of
> by default, programs in Debian should not access a  serial device unless
> they have high confidence that such a device is the kind of device they
> are trying to use.

Such a general statement would definitely satisfy me.

> I know that sounds horrible and we'd have to word-smith it into
> something that people could understand without being part of this
> conversation.

One way to do that would be to be to state the general but rather
abstract principle, and then to explicitly state the specialisation
to modemmanager and modems.

> I'd expect that if the guideline were along those lines, the Debian
> modemmanager maintainer would conclude that the new approach was
> sufficient.  I suspect if someone brought it back to us later we'd
> support such a conclusion from the maintainer.

I hope that that such a supposition could be rebutted - for example,
by examples of misprobing (either predicted by looking at existing
devices, or actually experienced) despite the new heuristic.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Aleksander Morgado writes ("Bug#877024: modemmanager should ask before messing 
with serial ports"):
> This is part of the discussion we had in the MM mailing list for such
> a solution:
> https://lists.freedesktop.org/archives/modemmanager-devel/2017-September/005917.html

Thanks, this looks constructive.

Of the heuristics in that mail, most seem to me to be very sound
justifications for thinking that the device is safe to probe.

The one big exception is this:

 | * If vid is a known modem vendor (e.g. huawei, zte, sierra, u-blox,
 |  telit), it's a modem and we probe the tty.

This is a hostage to the future, since of course we don't know what
devices might be manufactured by a particular vendor in the many-years
life of a Debian release.

I guess that if we in Debian don't like that particular heuristic it
would be possible for us to disble just that one ?

I don't understand this one:

 | * If kernel driver of the TTY is option/sierra/qcserial,

but maybe it is OK.

Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Bdale Garbee]

Aleksander Morgado  writes:

I'm no longer a TC member, but have been bitten by this problem, so will
chime in.

> Would the new approach satisfy Debian's needs?

It seems so to me.  The key thing I care about is that we *not* assume
any arbitrary cdc-acm device is a modem.  I can't say for sure, but my
guess is that checking class=2/subclass=2/protocol=1 as you propose
would be a workable approach to differentiating modems from non-modems.

Bdale


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Sam Hartman]

> "Aleksander" == Aleksander Morgado  writes:

>> In going forward, I think it is important to consider that
>> Modemmanager's needs and Debian's needs may be different here.
>> In the case of Modemmanager as an upstream project, it may be
>> desirable to give the best experience for users who do have
>> modems.  However, for Debian and for the Technical committee, we
>> need to consider what experience we want to give all our users
>> and as a result value damage caused by false positives more
>> highly than the upstream project might.

Aleksander> Would the new approach satisfy Debian's needs?

So, I'm only one member of the Debian technical committee.

I suspect that since the matter has been referred to us, we're likely to
rule.
I haven't seen other TC members   give opinions.  Myf suspicion is that
we're likely to try and give general guidelines to the Debian
modemmanager maintainers and let them choose a specific solution.

I'd be happy with guidelines that  said something along the lines of
by default, programs in Debian should not access a  serial device unless
they have high confidence that such a device is the kind of device they
are trying to use.
I know that sounds horrible and we'd have to word-smith it into
something that people could understand without being part of this
conversation.

I'd expect that if the guideline were along those lines, the Debian
modemmanager maintainer would conclude that the new approach was
sufficient.  I suspect if someone brought it back to us later we'd
support such a conclusion from the maintainer.

If you don't see action before then, I expect we'll have some internal
discussion in our next meeting near the end of October.

--Sam

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

>
> What I'd love to see is a patch that makes modemmanger behave better for
> our users while remaining installed and operational. If it's useful
> enough, it should be acceptable upstream as Debian is not the only
> community affected by this particular problem.
>

This is part of the discussion we had in the MM mailing list for such
a solution:
https://lists.freedesktop.org/archives/modemmanager-devel/2017-September/005917.html

> (If only the USB standard had a class definition for 'serial device' that
> was separate from 'modem device'.)
>

Yeah :)

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

Hey Sam,

> It looks like there hasn't been much traffic on this issue in the last
> couple of weeks.
>

Well, I moved the discussion to the ModemManager mailing list and it's
already been discussed there among MM developers.

> My analysis is that the key technical point here is whether it's
> acceptable to treat unknown devices as possible modems.
>
> It sounds like:
>
> 1) We don't have a good white list of modems.
>
> 2)  We don't have an adequate blacklist of modems.  Ian argues we never
> will have an adequate blacklist of modems.
>

There's a third approach, which is what we've been discussing in the
mailing list as a possible good compromise. We are going to update
ModemManager with a new command line option that will enable some
heuristics to try to guess whether a given set of ports correspond to
a modem or not (e.g. based on kernel driver, usb layout and so on).
Ideally, this means no blacklist would be needed, and ModemManager
would only probe TTYs that are definitely modems. The drawback is that
ModemManager won't automatically probe modems that would fall out of
the heuristics, but given that most new modems come with network
devices, the amount of devices left out should be minimal.

The new command line option will enable the new heuristics, and if the
command line option not given, the old behavior with the blacklist
will be kept. I'm assuming Debian could make use of the new command
line option by default, and that would solve this issue.

> In going forward, I think it is important to consider that
> Modemmanager's needs and Debian's needs may be different here.
> In the case of Modemmanager as an upstream project, it may be desirable
> to give the best experience for users who do have modems.
> However, for Debian and for the Technical committee, we need to consider
> what experience we want to give all our users and as a result value
> damage caused by false positives more highly than the upstream project
> might.

Would the new approach satisfy Debian's needs?

-- 
Aleksander
https://aleksander.es

Bug#877024: modemmanager should ask before messing with serial ports

[Keith Packard]

Sam Hartman  writes:

> Have I got that right?

Yes, I think you have summarized the issue accurately.

> However, for Debian and for the Technical committee, we need to consider
> what experience we want to give all our users and as a result value
> damage caused by false positives more highly than the upstream project
> might.

This does seem reasonable, but I haven't seen a concrete technical
proposal that would meet this goal.

Ian suggests asking the user, which seems like a fine goal, but I don't
think there's any plan for how to make this work in practice?

I made a suggestion (without my TC hat on) to not install modemmanager
by default. This is obviously easy to implement, however it also clearly
breaks the user experience for anyone who needs to use a modem to
download additional packages (e.g. modemmanager).

What I'd love to see is a patch that makes modemmanger behave better for
our users while remaining installed and operational. If it's useful
enough, it should be acceptable upstream as Debian is not the only
community affected by this particular problem.

(If only the USB standard had a class definition for 'serial device' that
was separate from 'modem device'.)

-- 
-keith


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Sam Hartman]

Hi.
It looks like there hasn't been much traffic on this issue in the last
couple of weeks.

My analysis is that the key technical point here is whether it's
acceptable to treat unknown devices as possible modems.

It sounds like:

1) We don't have a good white list of modems.

2)  We don't have an adequate blacklist of modems.  Ian argues we never
will have an adequate blacklist of modems.

It sounds like Aleksander is saying that from Modemmanager's standpoint
assuming unknown devices aren't modems would produce an unacceptable
user experience.

Ian is saying that even if we do everything we can to reduce the false
positive rate, treating some non-modems as modems has unacceptable
consequences.

Have I got that right?

In going forward, I think it is important to consider that
Modemmanager's needs and Debian's needs may be different here.
In the case of Modemmanager as an upstream project, it may be desirable
to give the best experience for users who do have modems.
However, for Debian and for the Technical committee, we need to consider
what experience we want to give all our users and as a result value
damage caused by false positives more highly than the upstream project
might.

--Sam

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Aleksander Morgado writes ("Re: Bug#877024: modemmanager should ask before 
messing with serial ports"):
> The solution you're suggesting involves changes not only in
> ModemManager, but in the whole stack... I personally don't like that,
> I don't want to ask users "is this a modem" especially when they're
> not plugging in one.

Well, there are two scenarios if a usb tty device appears.

A. It's a modem (or a 3G/4G stick that pretends to be a modem)

  Current behaviour: it gets autoprobed, and hopefully is useable
  right away with modemmanager.

  Possible safe behaviour: message pops up somehow inviting user
  to say whether it is a modem; user has to say "yes it is"
  and then the modem works.

  Raw intended behaviour of my patch, without further improvement:
  user has to explicitly tell modemmanager to probe this port.

B. It's a braille display, milling machine, rpi, gps, telescope,
   hadron collider, ...

  Current behaviour: modemmanager opens it the serial port and
  sends gibberish to the device, causing lossage.

  Possible safe behaviour: message pops up somehow inviting user
  to say whether it is a modem.  User curses "daft desktop stuff"
  and clicks "no".

  Raw intended behaviour of my patch, without further improvement:
  User is not bothered at all.

Even "possible safe behaviour" is IMO clearly an improvement
in case B.  It is much worse to interfere with a device without
asking, than to erroneously pester the user about it.

Obviously ideally the machine would remember the answer so that if the
same user plugs in the same device later, it all Just Works.  So
hopefully the user only has to answer the question once.

> If I had to suggest how to do this, probably removing the
> "ID_MM_CANDIDATE" from "tty" subsystems devices would do it:
> https://cgit.freedesktop.org/ModemManager/ModemManager/tree/src/80-mm-candidate.rules

Thanks very much for the pointer.  I will investigate.

> But not advocating for that option! :)

Understood.  I really appreciate your constructive approach.

Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

>
> However, I have to say that that upstream bug log does not really
> suggest to me that upstream thinks that the current probing behaviour
> is unacceptable.  It has been open since 2014 and if the bug is to be
> believed the modemmanager default behaviour in upstream has not
> changed.
>

We just didn't find a better solution, that's it. Keeping the
blacklist updated has been the fallback we've used all these years.

> The key part of your response in #683839 is as follows:
>
>   Asking the user... well, what would we ask? Should we ask "is this a
>   modem" for every TTY we find? Is it better to annoy thousands of
>   users each time a TTY is found instead of blacklist for all of them?
>   We try to keep the blacklist in stable releases updated and stables
>   happening each 2-3 months.
>
> Your concern here is precisely the user convenience which I am saying
> needs to be traded off for safety and reliability.
>
> "Every TTY" is rather hyperbolic, of course.  We are only speaking
> here of tty devices which might be modems.  modemmanager already
> refrains from autoprobing builtin serial devices ttyS* and I assume it
> ignores things like multiport serial cards.  So we are speaking mostly
> of USB-attached serial ports.  And we only need confirmation the first
> time one is plugged in.
>

The solution you're suggesting involves changes not only in
ModemManager, but in the whole stack... I personally don't like that,
I don't want to ask users "is this a modem" especially when they're
not plugging in one. I think the solution should involve keeping
ModemManager automatically detect modems, maybe with some heuristics
to try to ignore devices that don't look like modems. For TTY-only
devices (most of the ones that we end up blacklisting), though, the
solution isn't easy, which is why we're still here after all these
years...

> As I have explained, a blocklist is not sufficiently safe or reliable.
>

Yes, I agree.

> A passlist of things known definitely to be modems (not generic USB to
> serial chips) would help reduce the user query burden.
>

If only we had that whitelist...

>
>> >  * Say that, in the absence of a better solution, my patch should
>> >be applied to modemmanager.
>>
>> I commented about that patch in bug 683839, please don't apply it,
>> it would break all modems not only those with TTYs.
>
> Sorry about that.  I wasn't able to get modemmanager working with my
> MBIM device (which I'm now using in Hilink mode, hence without
> modemmanager, anyway) so I wasn't able to test that path.
>
> Please can you point me to the right place to make this change ?
> I can try revising my patch by myself but it will be more likely
> to be right if you help.
>

If I had to suggest how to do this, probably removing the
"ID_MM_CANDIDATE" from "tty" subsystems devices would do it:
https://cgit.freedesktop.org/ModemManager/ModemManager/tree/src/80-mm-candidate.rules

But not advocating for that option! :)

>> Again, I'd have suggested to bring this patch to the ModemManager
>> mailing list before just saying that the patch should be applied in
>> absence of a better solution...
>
> As a Debian user, experiencing a problem with the default
> configuration of my Debian system, and trying to use Debian's
> governance processes to escalate the bug, it is appropriate for me to
> use a Debian channel for this.
>

I disagree with that reasoning, but well nevermind :)

The discussion of how to best solving the automatic TTY probing is
definitely something that has to be done in the ModemManager mailing
list. I've already started a new thread to bump the discussion again:
https://lists.freedesktop.org/archives/modemmanager-devel/2017-September/005912.html

-- 
Aleksander

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Aleksander Morgado writes ("Bug#877024: modemmanager should ask before messing 
with serial ports"):
> [Ian Jackson:]
> > Many such modems present as USB serial devices, eg ttyACM or ttyUSB.
> > Consequently, modemmanager has the ability to open serial ports and
> > probe them to see if they respond to Hayes-style AT commands.  That
> > functionality is currently triggered automatically by default, even
> > for USB serial ports whose USB device IDs are unknown to modemmanager,
> > or whose device IDs correspond to generic USB-to-serial adapters.
...
> A clarification here, ModemManager doesn't automatically probe
> usb-to-serial converters, those are "greylisted" so that they're
> only probed on "manual scans". Of course, the vid:pid needs to be
> known to MM and in the greylist, for this to happen.

Thanks for the clarification.

The greylist is nevertheless never going to be complete.  My initial
message in #877024 lists some of the consequences, many of which
actually occurred with generic USB-to-serial adapters (or generic
USB-to-serial chips).

Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Aleksander Morgado writes ("Bug#877024: modemmanager should ask before messing 
with serial ports"):
> Hey Ian!
> > Since the maintainers and upstream evidently disagree with this
> > tradeoff, it has been necessary for me to ask the TC to step in.
> 
> The maintainers don't actually disagree. Why didn't you bring this
> issue to the ModemManager mailing list in the first place? I
> personally don't follow any downstream ModemManager bug, and we even
> have a bug open upstream:
> https://bugs.freedesktop.org/show_bug.cgi?id=85007

Thanks for your response.  It's excellent to see someone taking an
interest in this bug.

However, I have to say that that upstream bug log does not really
suggest to me that upstream thinks that the current probing behaviour
is unacceptable.  It has been open since 2014 and if the bug is to be
believed the modemmanager default behaviour in upstream has not
changed.

The key part of your response in #683839 is as follows:

  Asking the user... well, what would we ask? Should we ask "is this a
  modem" for every TTY we find? Is it better to annoy thousands of
  users each time a TTY is found instead of blacklist for all of them?
  We try to keep the blacklist in stable releases updated and stables
  happening each 2-3 months.

Your concern here is precisely the user convenience which I am saying
needs to be traded off for safety and reliability.

"Every TTY" is rather hyperbolic, of course.  We are only speaking
here of tty devices which might be modems.  modemmanager already
refrains from autoprobing builtin serial devices ttyS* and I assume it
ignores things like multiport serial cards.  So we are speaking mostly
of USB-attached serial ports.  And we only need confirmation the first
time one is plugged in.

As I have explained, a blocklist is not sufficiently safe or reliable.

A passlist of things known definitely to be modems (not generic USB to
serial chips) would help reduce the user query burden.

  Plus, there isn't always a "user" to ask
  when a modem is plugged in, ModemManager (as NetworkManager) don't
  really require a GUI to work and lots of headless systems out there
  use it.

In those situations, obviously, I would expect use of a command-line
tool or configuration file.


> >  * Say that, in the absence of a better solution, my patch should
> >be applied to modemmanager.
> 
> I commented about that patch in bug 683839, please don't apply it,
> it would break all modems not only those with TTYs.

Sorry about that.  I wasn't able to get modemmanager working with my
MBIM device (which I'm now using in Hilink mode, hence without
modemmanager, anyway) so I wasn't able to test that path.

Please can you point me to the right place to make this change ?
I can try revising my patch by myself but it will be more likely
to be right if you help.

> Again, I'd have suggested to bring this patch to the ModemManager
> mailing list before just saying that the patch should be applied in
> absence of a better solution...

As a Debian user, experiencing a problem with the default
configuration of my Debian system, and trying to use Debian's
governance processes to escalate the bug, it is appropriate for me to
use a Debian channel for this.

But I really appreciate you taking an interest.

> An easier thing would be to allow just "dpkg -R
> modemmanager", there should be no other package depending on the
> daemon itself.

That's no help because someone might have (for example) both a modem
and a braille display, or whatever.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

> Many such modems present as USB serial devices, eg ttyACM or ttyUSB.
> Consequently, modemmanager has the ability to open serial ports and
> probe them to see if they respond to Hayes-style AT commands.  That
> functionality is currently triggered automatically by default, even
> for USB serial ports whose USB device IDs are unknown to modemmanager,
> or whose device IDs correspond to generic USB-to-serial adapters.
> 
> I.e., if one is running a normal Debian installation and plugs in a
> usb-to-serial converter, modemmanager will open the device and send AT
> commands to it, to see if it is a modem.
> 
> This behaviour is IMO unaccaptable, as a default.
> 

A clarification here, ModemManager doesn't automatically probe usb-to-serial 
converters, those are "greylisted" so that they're only probed on "manual 
scans". Of course, the vid:pid needs to be known to MM and in the greylist, for 
this to happen.

ModemManager also doesn't automatically probe platform TTYs, like physical 
RS232 ports in the host.

-- 
Aleksander

Bug#877024: modemmanager should ask before messing with serial ports

[Aleksander Morgado]

Hey Ian!

> Since the maintainers and upstream evidently disagree with this
> tradeoff, it has been necessary for me to ask the TC to step in.
> 

The maintainers don't actually disagree. Why didn't you bring this issue to the 
ModemManager mailing list in the first place? I personally don't follow any 
downstream ModemManager bug, and we even have a bug open upstream:
https://bugs.freedesktop.org/show_bug.cgi?id=85007

> 
>  * Say that, in the absence of a better solution, my patch should
>be applied to modemmanager.
> 

I commented about that patch in bug 683839, please don't apply it, it would 
break all modems not only those with TTYs. Again, I'd have suggested to bring 
this patch to the ModemManager mailing list before just saying that the patch 
should be applied in absence of a better solution... An easier thing would be 
to allow just "dpkg -R modemmanager", there should be no other package 
depending on the daemon itself.

-- 
Aleksander

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Keith Packard writes ("Re: Bug#877024: modemmanager should ask before messing 
with serial ports"):
> Ian Jackson <ijack...@chiark.greenend.org.uk> writes:
> > This has gone far enough.  I would like to remind you of Constitution
> > 6.3(5)
> 
> Here's what I prefaced my first remark with:
>  (speaking as a Debian user, not as a TC member)
> Perhaps I should have added this to each message I sent?

You snipped the following part from my email:

  Note the point about forums.  If you want to engage in "design of new
  proposals", for example your suggestion to drop modemmanager from
  default installs, you should do that on the relevant mailing lists for
  modemmanager, or network-manager (which normally pulls it in), or
  debian-devel.

  It is not appropriate to use the TC list to advocate a novel and
  radical proposal in this way.

Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Keith Packard]

Ian Jackson  writes:

> This has gone far enough.  I would like to remind you of Constitution
> 6.3(5)

Here's what I prefaced my first remark with:

 (speaking as a Debian user, not as a TC member)

Perhaps I should have added this to each message I sent?

-- 
-keith


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Keith Packard writes ("Bug#877024: modemmanager should ask before messing with 
serial ports"):
> That requires fixing the package instead of just getting it out of the
> way, a significantly harder thing to manage.

This has gone far enough.  I would like to remind you of Constitution
6.3(5)

 | The Technical Committee does not engage in design of new proposals
 | and policies. Such design work should be carried out by individuals
 | privately or together and discussed in ordinary technical policy
 | and design forums.
 |
 | The Technical Committee restricts itself to choosing from or
 | adopting compromises between solutions and decisions which have
 | been proposed and reasonably thoroughly discussed elsewhere.

Note the point about forums.  If you want to engage in "design of new
proposals", for example your suggestion to drop modemmanager from
default installs, you should do that on the relevant mailing lists for
modemmanager, or network-manager (which normally pulls it in), or
debian-devel.

It is not appropriate to use the TC list to advocate a novel and
radical proposal in this way.


I can see that a naive reading of this dispute is "Ian hates desktoppy
stuff and that's why he hates modemmanagaer and that's why he hates
this modemmanager behaviour".  But it is not accurate.  The
functionality in modemmanager is important to lots of people.  The
fact that I'm not using modemmanager right now is more to do with the
exact vagaries of hardware support than anything else.

All I am saying is that modemmanager must not, even if it seems
convenient, take these IMO unacceptable risks with people's serial
ports.  That is, the safety, security and software reliability of the
users with "stuff" connected to their serial ports should take
precedence over the convenience of the users who need modemmanager.

And this is true even if they are the same people: someone who has
"stuff" connected to their serial ports, and also needs modemmanager,
should find that Debian has prioritised not putting them at risk and
not breaking their stuff, over making their telephony experience more
convenient.

Since the maintainers and upstream evidently disagree with this
tradeoff, it has been necessary for me to ask the TC to step in.


I would like the TC to, overruling the maintainers:

 * Confirm the principle that modemmanager (and indeed other software
   in Debian) should not probe serial devices unless:
  - the user has given explicit permission; or
  - the device is known, for whatever reason, to definitely
 be a modem (or whatever kind of thing the program wants)

 * Say that, in the absence of a better solution, my patch should
   be applied to modemmanager.

 * Explicitly say that the TC expects the decision to be implemented
   in a way that the maintainers approve of, if possible, so long as
   that doesn't involve a large amount of additional work.

Thanks,
Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Keith Packard]

Ian Jackson  writes:

> But I should be able to use the same laptop to (1) control my machine
> tools or talk to my rpi or whatever (2) go online with a usb mobile
> modem when I'm out of the house.  Possibly even simultaneously.

That requires fixing the package instead of just getting it out of the
way, a significantly harder thing to manage.

I'm not saying your wrong. The simpler fix would make things better for
some people (those who use no USB modems, but do use other USB serial
devices), worse for others (those who use USB modems but not other USB
serial devices), and the same for a few (those who use both). The
question is whether the simpler fix would be a net positive for Debian
users, or a net negative.

Obviously, a "real" fix that asked the user whether a particular
device was actually a modem would be best for the third class of
people.

Of course, the simpler fix has the side effect of not installing
software or running I don't ever need and which serves only to annoy me.
So, for me, the simpler fix is even better...

> And, people shouldn't have to install support software to get their
> internet to work.

It's all a question of what support we install by default; there are
certainly plenty of network configurations which are not supported in a
default install. Are modems still common enough that supporting them by
default is worth the cost of wasting space and power on the remaining
machines which will never use one?

If it were just software that got installed and sat idle, I'd complain a
lot less. As it is, modemmanager does "stuff" by default, even if I
haven't asked it to. Software which runs on every machine by default
should be held to a higher standard than software which users explicitly
request. So, if we didn't install it by default, I'd be happy for it to
continue to suck.

-- 
-keith


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Keith Packard writes ("Re: Bug#877024: modemmanager should ask before messing 
with serial ports"):
> Yeah, I was just thinking that it would be easier to stop installing
> support for modems by default than to actually fix modemmanager to
> behave itself. It's certainly how I work -- apt remove modemmanager
> solves a world of problems for me.

But I should be able to use the same laptop to (1) control my machine
tools or talk to my rpi or whatever (2) go online with a usb mobile
modem when I'm out of the house.  Possibly even simultaneously.

And, people shouldn't have to install support software to get their
internet to work.

Ian.

Bug#877024: modemmanager should ask before messing with serial ports

[Keith Packard]

Ian Jackson  writes:

> Also, AIUI modemmanager contains code to do things with the new-style
> MBIM dongles (which can also be done with the cli tools in
> mbim-utils).  But I definitely wouldn't suggest disabling its ability
> to work with AT-command modems, as they are still being sold.[1]

Yeah, I was just thinking that it would be easier to stop installing
support for modems by default than to actually fix modemmanager to
behave itself. It's certainly how I work -- apt remove modemmanager
solves a world of problems for me.

-- 
-keith


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Wouter Verhelst]

On Wed, Sep 27, 2017 at 02:13:50PM -0700, Keith Packard wrote:
> Ian Jackson  writes:
> 
> (speaking as a Debian user, not as a TC member)
> 
> > I'm afraid don't really want to do the work of writing a better UI.
> > But I have provided a simple patch which at least makes the behaviour
> > safe.
> 
> Would it be sufficient to simply stop installing this largely-useless
> package at this point? In what environments do users typically have
> modems which work with AT-style commands?

ModemManager supports more than just those modems that work with AT
command sets; it supports the modern UMTS/3G/4G modems which don't even
support AT commands anymore, too.

> It would be far safer to not install this package than try to hack it up
> to keep it from breaking systems. Simply removing it from 'depends' on
> the handful of packages which currently list it would 'fix' this problem
> with a minimum of fuss.

Except that then for people who use NetworkManager, configuring their
mobile internet will stop functioning. While I agree with Ian that the
current behaviour of ModemManager is more than just highly annoying, it
cannot be said that ModemManager does not function as designed, nor that
it does not provide any useful functionality of which the loss would be
felt by those users who need it.

While dropping dependencies might make the issue somewhat less of a
problem, I don't think it's the right course of action.

-- 
Could you people please use IRC like normal people?!?

  -- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008
 Hacklab

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Ansgar Burchardt writes ("Bug#877024: modemmanager should ask before messing 
with serial ports"):
> It's not useless.  At least not when using UMTS via USB dongles which I
> would guess more people use than ham radio.  (Some of these USB dongles
> also emulate network cards and provide a DHCP server instead IIRC.)

Right.  These kind of dongles are very common.  My last one (which
died recently) was one.

Also, AIUI modemmanager contains code to do things with the new-style
MBIM dongles (which can also be done with the cli tools in
mbim-utils).  But I definitely wouldn't suggest disabling its ability
to work with AT-command modems, as they are still being sold.[1]

Ian.

[1] I think.  I tried to buy one and ended up with one which is
switchable between MBIM and Hilink instead.

Bug#877024: modemmanager should ask before messing with serial ports

[Ansgar Burchardt]

"Keith Packard" writes:
> Ian Jackson  writes:
>
> (speaking as a Debian user, not as a TC member)
>
>> I'm afraid don't really want to do the work of writing a better UI.
>> But I have provided a simple patch which at least makes the behaviour
>> safe.
>
> Would it be sufficient to simply stop installing this largely-useless
> package at this point? In what environments do users typically have
> modems which work with AT-style commands?

It's not useless.  At least not when using UMTS via USB dongles which I
would guess more people use than ham radio.  (Some of these USB dongles
also emulate network cards and provide a DHCP server instead IIRC.)

Ansgar

Bug#877024: modemmanager should ask before messing with serial ports

[Keith Packard]

Ian Jackson  writes:

(speaking as a Debian user, not as a TC member)

> I'm afraid don't really want to do the work of writing a better UI.
> But I have provided a simple patch which at least makes the behaviour
> safe.

Would it be sufficient to simply stop installing this largely-useless
package at this point? In what environments do users typically have
modems which work with AT-style commands?

It would be far safer to not install this package than try to hack it up
to keep it from breaking systems. Simply removing it from 'depends' on
the handful of packages which currently list it would 'fix' this problem
with a minimum of fuss.

-- 
-keith


signature.asc
Description: PGP signature

Bug#877024: modemmanager should ask before messing with serial ports

[Ian Jackson]

Package: tech-ctte
Control: block 683839 by -1

modemmanager is a program for handling modems, particularly
usb-connected mobile-telephony modems (aka "3G/4G sticks" etc.)

Many such modems present as USB serial devices, eg ttyACM or ttyUSB.
Consequently, modemmanager has the ability to open serial ports and
probe them to see if they respond to Hayes-style AT commands.  That
functionality is currently triggered automatically by default, even
for USB serial ports whose USB device IDs are unknown to modemmanager,
or whose device IDs correspond to generic USB-to-serial adapters.

I.e., if one is running a normal Debian installation and plugs in a
usb-to-serial converter, modemmanager will open the device and send AT
commands to it, to see if it is a modem.

This behaviour is IMO unaccaptable, as a default.

Serial connections are used to connect a wide variety of equipment
including industrial robots, electronic test equipment capable of
generating hazardous voltages, accessibility hardware, scientific
instruments, and embedded computers.

In the worst case (luckily, as far as I know, hypothetical):

 * This behaviour could cause someone personal injury, if a real-world
   device connected to the serial port misinteprets modemmanager's
   probe (or if its control firmware crashes) and makes hazardous
   movements or some such

Symptoms which have been observed include:

 * User's braille display stopped working during system boot
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621851
   With a less resourceful user, that might make the computer
   completely unuseable.

 * Random number generator "interfered with"
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840697
   Ultimate consequences not clear from the bug report, but if the RNG
   driver software has poor error handling it might include poor
   quality random numbers and therefore weak cryptographic keys.

 * GPS no longer recognised at boot
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798193
   (Not as trivial a problem as it sounds, as it could prevent the
   system being used as a navigation aid.)

 * "Breaks" apcupsd
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683839#32

 * User's Palm Pilot PDA crashes
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683839#52

 * User's 1-wire DS9097 interface is messed up, requiring a
   reboot to become functional again
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683839#72

 * My 3D printer reported protocol violations on startup
   and host software could sometimes not connect to printer
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683839#5

Other possible consequences which have been hypothesised are:

 * Simply opening the port and enabling RTS might enable radio
   transmissions in a ham radio context
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683839#42
   (Resulting in possibly-illegal radio interference.)

 * Simply opening the port and enabling RTS switches some scientific
   equipment into remote control mode, disabling the front panel
   and perhaps leading to hazardous situations.  (pers. comm)

 * Some 3D printers will reset when RTS is toggled, interrupting
   any print job (causing real world lossage in the form of wasted
   feedstock and printer time)
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683839#32

In August 2012 I experienced this bug and filed #683839 with severity
`important'.  The maintainers apparently do not agree with my analysis
and there has been no action by them since then other than efforts to
maintain the blocklist.

In the intervening time many users have reported manifestations of
this problem to #683839 and to other bug reports.  The reactions from
the maintainers have consistently been to try to get individual
devices blocklisted, even though as has been explained repeatedly,
this is not really sufficient.  The maintainers have not engaged with
the arguments against blanket probing.

Note that safe behaviour does not necessarily mean that every time the
user plugs in their modem, they have to confirm its use.  Other
solutions are possible, for example asking for explicit permission
from the user, the first time any particular device is seen, that it
is OK to probe that device.  (Similar behaviour is already implemented
for USB HID devices - keyboards etc. - because of the security
implications.)

I'm afraid don't really want to do the work of writing a better UI.
But I have provided a simple patch which at least makes the behaviour
safe.

IMO at the very least, for buster, we should apply that patch - or an
equivalent - and then the maintainers can consider how to improve the
UI/UX to explicitly ask permission, if they think that is desirable.

We should also consider whether to backport any of the resulting
changes, or include them in stable updates of some kind.

Thanks for your attention.

Ian.

-- 
Ian Jackson    These opinions are my own.

If I