Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Tue, 12 Sep 2017 11:52:11 -0700, Dmitry Torokhov wrote:
> On Tue, Sep 12, 2017 at 02:00:41PM +0200, Jakub Kicinski wrote:
> > On Mon, 11 Sep 2017 11:29:26 -0700, Dmitry Torokhov wrote:  
> > > 
> > > That is an option, but maybe we could have the patch below for a year or
> > > 2 instead?
> > > 
> > > Jakub, can you try and see if that works for you?  
> > 
> > Unfortunately this doesn't seem to solve it :(
> > 
> > # modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
> > nfp  1101824  0 
> > nfp  1101824  1   
> 
> Well, I should have tested it before sending out. How about this one?

This one is better!  Consider it:

Tested-by: Jakub Kicinski 

Thanks!

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Tue, 12 Sep 2017 11:52:11 -0700, Dmitry Torokhov wrote:
> On Tue, Sep 12, 2017 at 02:00:41PM +0200, Jakub Kicinski wrote:
> > On Mon, 11 Sep 2017 11:29:26 -0700, Dmitry Torokhov wrote:  
> > > 
> > > That is an option, but maybe we could have the patch below for a year or
> > > 2 instead?
> > > 
> > > Jakub, can you try and see if that works for you?  
> > 
> > Unfortunately this doesn't seem to solve it :(
> > 
> > # modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
> > nfp  1101824  0 
> > nfp  1101824  1   
> 
> Well, I should have tested it before sending out. How about this one?

This one is better!  Consider it:

Tested-by: Jakub Kicinski 

Thanks!

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Tue, Sep 12, 2017 at 02:00:41PM +0200, Jakub Kicinski wrote:
> On Mon, 11 Sep 2017 11:29:26 -0700, Dmitry Torokhov wrote:
> > 
> > That is an option, but maybe we could have the patch below for a year or
> > 2 instead?
> > 
> > Jakub, can you try and see if that works for you?
> 
> Unfortunately this doesn't seem to solve it :(
> 
> # modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
> nfp  1101824  0 
> nfp  1101824  1 

Well, I should have tested it before sending out. How about this one?

-- 
Dmitry


driver core: suppress sending MODALIAS in UNBIND uevents

From: Dmitry Torokhov 

The current udev rules cause modules to be loaded on all device events save
for "remove". With the introduction of KOBJ_BIND/KOBJ_UNBIND this causes
issues, as driver modules that have devices bound to their drivers get
immediately reloaded, and it appears to the user that module unloading doe
snot work.

The standard udev matching rule is foillowing:

ENV{MODALIAS}=="?*", RUN{builtin}+="kmod load $env{MODALIAS}"

Given that MODALIAS data is not terribly useful for UNBIND event, let's zap
it from the generated uevent environment until we get userspace updated
with the correct udev rule that only loads modules on "add" event.

Reported-by: Jakub Kicinski 
Signed-off-by: Dmitry Torokhov 
---
 lib/kobject_uevent.c |   49 +
 1 file changed, 41 insertions(+), 8 deletions(-)

diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c
index e590523ea476..f237a09a5862 100644
--- a/lib/kobject_uevent.c
+++ b/lib/kobject_uevent.c
@@ -294,6 +294,26 @@ static void cleanup_uevent_env(struct subprocess_info 
*info)
 }
 #endif
 
+static void zap_modalias_env(struct kobj_uevent_env *env)
+{
+   static const char modalias_prefix[] = "MODALIAS=";
+   int i;
+
+   for (i = 0; i < env->envp_idx;) {
+   if (strncmp(env->envp[i], modalias_prefix,
+   sizeof(modalias_prefix) - 1)) {
+   i++;
+   continue;
+   }
+
+   if (i != env->envp_idx - 1)
+   memmove(>envp[i], >envp[i + 1],
+   sizeof(env->envp[i]) * env->envp_idx - 1);
+
+   env->envp_idx--;
+   }
+}
+
 /**
  * kobject_uevent_env - send an uevent with environmental data
  *
@@ -409,16 +429,29 @@ int kobject_uevent_env(struct kobject *kobj, enum 
kobject_action action,
}
}
 
-   /*
-* Mark "add" and "remove" events in the object to ensure proper
-* events to userspace during automatic cleanup. If the object did
-* send an "add" event, "remove" will automatically generated by
-* the core, if not already done by the caller.
-*/
-   if (action == KOBJ_ADD)
+   switch (action) {
+   case KOBJ_ADD:
+   /*
+* Mark "add" event so we can make sure we deliver "remove"
+* event to userspace during automatic cleanup. If
+* the object did send an "add" event, "remove" will
+* automatically generated by the core, if not already done
+* by the caller.
+*/
kobj->state_add_uevent_sent = 1;
-   else if (action == KOBJ_REMOVE)
+   break;
+
+   case KOBJ_REMOVE:
kobj->state_remove_uevent_sent = 1;
+   break;
+
+   case KOBJ_UNBIND:
+   zap_modalias_env(env);
+   break;
+
+   default:
+   break;
+   }
 
mutex_lock(_sock_mutex);
/* we will send an event, so request a new sequence number */

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Tue, Sep 12, 2017 at 02:00:41PM +0200, Jakub Kicinski wrote:
> On Mon, 11 Sep 2017 11:29:26 -0700, Dmitry Torokhov wrote:
> > 
> > That is an option, but maybe we could have the patch below for a year or
> > 2 instead?
> > 
> > Jakub, can you try and see if that works for you?
> 
> Unfortunately this doesn't seem to solve it :(
> 
> # modprobe nfp; lsmod | grep nfp; modprobe -r nfp; lsmod | grep nfp
> nfp  1101824  0 
> nfp  1101824  1 

Well, I should have tested it before sending out. How about this one?

-- 
Dmitry


driver core: suppress sending MODALIAS in UNBIND uevents

From: Dmitry Torokhov 

The current udev rules cause modules to be loaded on all device events save
for "remove". With the introduction of KOBJ_BIND/KOBJ_UNBIND this causes
issues, as driver modules that have devices bound to their drivers get
immediately reloaded, and it appears to the user that module unloading doe
snot work.

The standard udev matching rule is foillowing:

ENV{MODALIAS}=="?*", RUN{builtin}+="kmod load $env{MODALIAS}"

Given that MODALIAS data is not terribly useful for UNBIND event, let's zap
it from the generated uevent environment until we get userspace updated
with the correct udev rule that only loads modules on "add" event.

Reported-by: Jakub Kicinski 
Signed-off-by: Dmitry Torokhov 
---
 lib/kobject_uevent.c |   49 +
 1 file changed, 41 insertions(+), 8 deletions(-)

diff --git a/lib/kobject_uevent.c b/lib/kobject_uevent.c
index e590523ea476..f237a09a5862 100644
--- a/lib/kobject_uevent.c
+++ b/lib/kobject_uevent.c
@@ -294,6 +294,26 @@ static void cleanup_uevent_env(struct subprocess_info 
*info)
 }
 #endif
 
+static void zap_modalias_env(struct kobj_uevent_env *env)
+{
+   static const char modalias_prefix[] = "MODALIAS=";
+   int i;
+
+   for (i = 0; i < env->envp_idx;) {
+   if (strncmp(env->envp[i], modalias_prefix,
+   sizeof(modalias_prefix) - 1)) {
+   i++;
+   continue;
+   }
+
+   if (i != env->envp_idx - 1)
+   memmove(>envp[i], >envp[i + 1],
+   sizeof(env->envp[i]) * env->envp_idx - 1);
+
+   env->envp_idx--;
+   }
+}
+
 /**
  * kobject_uevent_env - send an uevent with environmental data
  *
@@ -409,16 +429,29 @@ int kobject_uevent_env(struct kobject *kobj, enum 
kobject_action action,
}
}
 
-   /*
-* Mark "add" and "remove" events in the object to ensure proper
-* events to userspace during automatic cleanup. If the object did
-* send an "add" event, "remove" will automatically generated by
-* the core, if not already done by the caller.
-*/
-   if (action == KOBJ_ADD)
+   switch (action) {
+   case KOBJ_ADD:
+   /*
+* Mark "add" event so we can make sure we deliver "remove"
+* event to userspace during automatic cleanup. If
+* the object did send an "add" event, "remove" will
+* automatically generated by the core, if not already done
+* by the caller.
+*/
kobj->state_add_uevent_sent = 1;
-   else if (action == KOBJ_REMOVE)
+   break;
+
+   case KOBJ_REMOVE:
kobj->state_remove_uevent_sent = 1;
+   break;
+
+   case KOBJ_UNBIND:
+   zap_modalias_env(env);
+   break;
+
+   default:
+   break;
+   }
 
mutex_lock(_sock_mutex);
/* we will send an event, so request a new sequence number */

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Mon, 11 Sep 2017 11:29:26 -0700, Dmitry Torokhov wrote:
> On Mon, Sep 11, 2017 at 08:23:32AM -0700, Greg Kroah-Hartman wrote:
> > On Sun, Sep 10, 2017 at 02:22:22PM -0700, Dmitry Torokhov wrote:  
> > > On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:  
> > > > On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:  
> > > >> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:  
> > > >> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski 
> > > >> >  wrote:  
> > > >> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:  
> > > >> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:  
> > > >> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
> > > >> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > > >> > > wrote:  
> > > >> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> > > >> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  
> > > >> > >  
> > > >> > >> > > >wrote:  
> > > >> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote: 
> > > >> > >> > > >> >  
> > > >> > >  
> > > >> > >> > > >> >> Hi!
> > > >> > >> > > >> >>
> > > >> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod 
> > > >> > >> > > >> >>  
> > > >> > >succeeds  
> > > >> > >> > > >but the  
> > > >> > >> > > >> >> module is still loaded and the refcount goes to 1:
> > > >> > >> > > >> >>
> > > >> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > > >> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  
> > > >> > >> > > >> >> assembly.partno \
> > > >> > >> > > >> >>   lsmod | grep nfp; \
> > > >> > >> > > >> >>   rmmod nfp; \
> > > >> > >> > > >> >>   lsmod | grep nfp
> > > >> > >> > > >> >> nfp   249856  0
> > > >> > >> > > >> >> nfp   200704  1
> > > >> > >> > > >> >>
> > > >> > >> > > >> >> If I rmmod again the module will be actually unloaded.  
> > > >> > >> > > >> >> The  
> > > >> > >user  
> > > >> > >> > > >space  
> > > >> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm 
> > > >> > >> > > >> >> trying  
> > > >> > >to  
> > > >> > >> > > >bisect  
> > > >> > >> > > >> >> now...  
> > > >> > >> > > >> >
> > > >> > >> > > >> > Got 'em!
> > > >> > >> > > >> >
> > > >> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> > > >> > >> > > >refs/bisect/bad)  
> > > >> > >> > > >> > Author: Dmitry Torokhov 
> > > >> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > > >> > >> > > >> >
> > > >> > >> > > >> > driver core: emit uevents when device is bound to a  
> > > >> > >driver  
> > > >> > >> > > >>
> > > >> > >> > > >> Does it happen with all modules or only nfp one?
> > > >> > >> > > >>
> > > >> > >> > > >> It seems to work here:
> > > >> > >> > > >>
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> > >> > > >> psmouse   135168  0
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> > > >> > >> > > >
> > > >> > >> > > >It looks like the driver is actually reloaded.  The driver 
> > > >> > >> > > >used  
> > > >> > >to  
> > > >> > >> > > >return EPROBE_DEFER, but I think it doesn't any more 
> > > >> > >> > > >(rebuilding  
> > > >> > >the  
> > > >> > >> > > >kernel to test that right now).
> > > >> > >> > > >
> > > >> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or 
> > > >> > >> > > >somehow
> > > >> > >> > > >else cause the driver to be loaded again?  
> > > >> > >> > >
> > > >> > >> > > It depends on how silly the udev rules are, but yes, this can 
> > > >> > >> > >  
> > > >> > >definitely happen.  
> > > >> > >> >
> > > >> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > > >> > >> >
> > > >> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > > >> > >> > $  
> > > >> > >>
> > > >> > >> Not sure why you bring the deferrals here, they have nothing to 
> > > >> > >> do  
> > > >> > >with  
> > > >> > >> module removal. Also, deferrals are rarely issued by the leaf 
> > > >> > >> driver,  
> > > >> > >and  
> > > >> > >> more often by providers of resources (GPIO, regulator, interrupt, 
> > > >> > >>  
> > > >> > >etc).
> > > >> > >
> > > >> > >Yes, it's unusual, but this driver used to do it.  Which is exactly 
> > > >> > >why
> > > >> > >I brought it up.  Turns out it was irrelevant :)
> > > >> > >  
> > > >> > >> > I tested without any udev rules in /etc/udev/, just the 
> > > >> > >> > standard  
> > > >> > >distro  
> > > >> > >> > ones.  Same thing.  
> > > >> > >>
> > > >> > >> Right, so this is the default udev rule:
> > > >> > >>
> > > >> > >> /lib/udev/rules.d/80-drivers.rules:
> > > >> > >>
> > > >> > >> # do 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Mon, 11 Sep 2017 11:29:26 -0700, Dmitry Torokhov wrote:
> On Mon, Sep 11, 2017 at 08:23:32AM -0700, Greg Kroah-Hartman wrote:
> > On Sun, Sep 10, 2017 at 02:22:22PM -0700, Dmitry Torokhov wrote:  
> > > On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:  
> > > > On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:  
> > > >> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:  
> > > >> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski 
> > > >> >  wrote:  
> > > >> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:  
> > > >> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:  
> > > >> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
> > > >> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > > >> > > wrote:  
> > > >> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> > > >> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  
> > > >> > >  
> > > >> > >> > > >wrote:  
> > > >> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote: 
> > > >> > >> > > >> >  
> > > >> > >  
> > > >> > >> > > >> >> Hi!
> > > >> > >> > > >> >>
> > > >> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod 
> > > >> > >> > > >> >>  
> > > >> > >succeeds  
> > > >> > >> > > >but the  
> > > >> > >> > > >> >> module is still loaded and the refcount goes to 1:
> > > >> > >> > > >> >>
> > > >> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > > >> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  
> > > >> > >> > > >> >> assembly.partno \
> > > >> > >> > > >> >>   lsmod | grep nfp; \
> > > >> > >> > > >> >>   rmmod nfp; \
> > > >> > >> > > >> >>   lsmod | grep nfp
> > > >> > >> > > >> >> nfp   249856  0
> > > >> > >> > > >> >> nfp   200704  1
> > > >> > >> > > >> >>
> > > >> > >> > > >> >> If I rmmod again the module will be actually unloaded.  
> > > >> > >> > > >> >> The  
> > > >> > >user  
> > > >> > >> > > >space  
> > > >> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm 
> > > >> > >> > > >> >> trying  
> > > >> > >to  
> > > >> > >> > > >bisect  
> > > >> > >> > > >> >> now...  
> > > >> > >> > > >> >
> > > >> > >> > > >> > Got 'em!
> > > >> > >> > > >> >
> > > >> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> > > >> > >> > > >refs/bisect/bad)  
> > > >> > >> > > >> > Author: Dmitry Torokhov 
> > > >> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > > >> > >> > > >> >
> > > >> > >> > > >> > driver core: emit uevents when device is bound to a  
> > > >> > >driver  
> > > >> > >> > > >>
> > > >> > >> > > >> Does it happen with all modules or only nfp one?
> > > >> > >> > > >>
> > > >> > >> > > >> It seems to work here:
> > > >> > >> > > >>
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> > >> > > >> psmouse   135168  0
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> > > >> > >> > > >
> > > >> > >> > > >It looks like the driver is actually reloaded.  The driver 
> > > >> > >> > > >used  
> > > >> > >to  
> > > >> > >> > > >return EPROBE_DEFER, but I think it doesn't any more 
> > > >> > >> > > >(rebuilding  
> > > >> > >the  
> > > >> > >> > > >kernel to test that right now).
> > > >> > >> > > >
> > > >> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or 
> > > >> > >> > > >somehow
> > > >> > >> > > >else cause the driver to be loaded again?  
> > > >> > >> > >
> > > >> > >> > > It depends on how silly the udev rules are, but yes, this can 
> > > >> > >> > >  
> > > >> > >definitely happen.  
> > > >> > >> >
> > > >> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > > >> > >> >
> > > >> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > > >> > >> > $  
> > > >> > >>
> > > >> > >> Not sure why you bring the deferrals here, they have nothing to 
> > > >> > >> do  
> > > >> > >with  
> > > >> > >> module removal. Also, deferrals are rarely issued by the leaf 
> > > >> > >> driver,  
> > > >> > >and  
> > > >> > >> more often by providers of resources (GPIO, regulator, interrupt, 
> > > >> > >>  
> > > >> > >etc).
> > > >> > >
> > > >> > >Yes, it's unusual, but this driver used to do it.  Which is exactly 
> > > >> > >why
> > > >> > >I brought it up.  Turns out it was irrelevant :)
> > > >> > >  
> > > >> > >> > I tested without any udev rules in /etc/udev/, just the 
> > > >> > >> > standard  
> > > >> > >distro  
> > > >> > >> > ones.  Same thing.  
> > > >> > >>
> > > >> > >> Right, so this is the default udev rule:
> > > >> > >>
> > > >> > >> /lib/udev/rules.d/80-drivers.rules:
> > > >> > >>
> > > >> > >> # do not edit this file, it will be overwritten on update
> > > >> > >>
> > > >> > >> 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Mon, Sep 11, 2017 at 08:23:32AM -0700, Greg Kroah-Hartman wrote:
> On Sun, Sep 10, 2017 at 02:22:22PM -0700, Dmitry Torokhov wrote:
> > On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:
> > > On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
> > >> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> > >> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
> > >> > wrote:
> > >> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> > >> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> > >> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> > >> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> > >> > > wrote:
> > >> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> > >> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
> > >> > >
> > >> > >> > > >wrote:
> > >> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> > >> > >
> > >> > >> > > >> >> Hi!
> > >> > >> > > >> >>
> > >> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
> > >> > >succeeds
> > >> > >> > > >but the
> > >> > >> > > >> >> module is still loaded and the refcount goes to 1:
> > >> > >> > > >> >>
> > >> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > >> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno 
> > >> > >> > > >> >> \
> > >> > >> > > >> >>   lsmod | grep nfp; \
> > >> > >> > > >> >>   rmmod nfp; \
> > >> > >> > > >> >>   lsmod | grep nfp
> > >> > >> > > >> >> nfp   249856  0
> > >> > >> > > >> >> nfp   200704  1
> > >> > >> > > >> >>
> > >> > >> > > >> >> If I rmmod again the module will be actually unloaded.  
> > >> > >> > > >> >> The
> > >> > >user
> > >> > >> > > >space
> > >> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
> > >> > >to
> > >> > >> > > >bisect
> > >> > >> > > >> >> now...
> > >> > >> > > >> >
> > >> > >> > > >> > Got 'em!
> > >> > >> > > >> >
> > >> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> > >> > >> > > >refs/bisect/bad)
> > >> > >> > > >> > Author: Dmitry Torokhov 
> > >> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > >> > >> > > >> >
> > >> > >> > > >> > driver core: emit uevents when device is bound to a
> > >> > >driver
> > >> > >> > > >>
> > >> > >> > > >> Does it happen with all modules or only nfp one?
> > >> > >> > > >>
> > >> > >> > > >> It seems to work here:
> > >> > >> > > >>
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > >> > > >> psmouse   135168  0
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> > >> > >> > > >
> > >> > >> > > >It looks like the driver is actually reloaded.  The driver used
> > >> > >to
> > >> > >> > > >return EPROBE_DEFER, but I think it doesn't any more 
> > >> > >> > > >(rebuilding
> > >> > >the
> > >> > >> > > >kernel to test that right now).
> > >> > >> > > >
> > >> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or 
> > >> > >> > > >somehow
> > >> > >> > > >else cause the driver to be loaded again?
> > >> > >> > >
> > >> > >> > > It depends on how silly the udev rules are, but yes, this can
> > >> > >definitely happen.
> > >> > >> >
> > >> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > >> > >> >
> > >> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > >> > >> > $
> > >> > >>
> > >> > >> Not sure why you bring the deferrals here, they have nothing to do
> > >> > >with
> > >> > >> module removal. Also, deferrals are rarely issued by the leaf 
> > >> > >> driver,
> > >> > >and
> > >> > >> more often by providers of resources (GPIO, regulator, interrupt,
> > >> > >etc).
> > >> > >
> > >> > >Yes, it's unusual, but this driver used to do it.  Which is exactly 
> > >> > >why
> > >> > >I brought it up.  Turns out it was irrelevant :)
> > >> > >
> > >> > >> > I tested without any udev rules in /etc/udev/, just the standard
> > >> > >distro
> > >> > >> > ones.  Same thing.
> > >> > >>
> > >> > >> Right, so this is the default udev rule:
> > >> > >>
> > >> > >> /lib/udev/rules.d/80-drivers.rules:
> > >> > >>
> > >> > >> # do not edit this file, it will be overwritten on update
> > >> > >>
> > >> > >> ACTION=="remove", GOTO="drivers_end"
> > >> > >>
> > >> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> > >>
> > >> So if the new uevents do not have the MODALIAS line in them, then they
> > >> will not trigger this?  Dmitry, can you see if that would fix this
> > >> problem without having to fix everyone's old versions of udev/systemd?
> > 
> > Unfortunately MODALIAS= is being added by individual subsystems having
> > 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Mon, Sep 11, 2017 at 08:23:32AM -0700, Greg Kroah-Hartman wrote:
> On Sun, Sep 10, 2017 at 02:22:22PM -0700, Dmitry Torokhov wrote:
> > On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:
> > > On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
> > >> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> > >> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
> > >> > wrote:
> > >> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> > >> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> > >> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> > >> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> > >> > > wrote:
> > >> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> > >> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
> > >> > >
> > >> > >> > > >wrote:
> > >> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> > >> > >
> > >> > >> > > >> >> Hi!
> > >> > >> > > >> >>
> > >> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
> > >> > >succeeds
> > >> > >> > > >but the
> > >> > >> > > >> >> module is still loaded and the refcount goes to 1:
> > >> > >> > > >> >>
> > >> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > >> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno 
> > >> > >> > > >> >> \
> > >> > >> > > >> >>   lsmod | grep nfp; \
> > >> > >> > > >> >>   rmmod nfp; \
> > >> > >> > > >> >>   lsmod | grep nfp
> > >> > >> > > >> >> nfp   249856  0
> > >> > >> > > >> >> nfp   200704  1
> > >> > >> > > >> >>
> > >> > >> > > >> >> If I rmmod again the module will be actually unloaded.  
> > >> > >> > > >> >> The
> > >> > >user
> > >> > >> > > >space
> > >> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
> > >> > >to
> > >> > >> > > >bisect
> > >> > >> > > >> >> now...
> > >> > >> > > >> >
> > >> > >> > > >> > Got 'em!
> > >> > >> > > >> >
> > >> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> > >> > >> > > >refs/bisect/bad)
> > >> > >> > > >> > Author: Dmitry Torokhov 
> > >> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > >> > >> > > >> >
> > >> > >> > > >> > driver core: emit uevents when device is bound to a
> > >> > >driver
> > >> > >> > > >>
> > >> > >> > > >> Does it happen with all modules or only nfp one?
> > >> > >> > > >>
> > >> > >> > > >> It seems to work here:
> > >> > >> > > >>
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > >> > > >> psmouse   135168  0
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> > >> > >> > > >
> > >> > >> > > >It looks like the driver is actually reloaded.  The driver used
> > >> > >to
> > >> > >> > > >return EPROBE_DEFER, but I think it doesn't any more 
> > >> > >> > > >(rebuilding
> > >> > >the
> > >> > >> > > >kernel to test that right now).
> > >> > >> > > >
> > >> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or 
> > >> > >> > > >somehow
> > >> > >> > > >else cause the driver to be loaded again?
> > >> > >> > >
> > >> > >> > > It depends on how silly the udev rules are, but yes, this can
> > >> > >definitely happen.
> > >> > >> >
> > >> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > >> > >> >
> > >> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > >> > >> > $
> > >> > >>
> > >> > >> Not sure why you bring the deferrals here, they have nothing to do
> > >> > >with
> > >> > >> module removal. Also, deferrals are rarely issued by the leaf 
> > >> > >> driver,
> > >> > >and
> > >> > >> more often by providers of resources (GPIO, regulator, interrupt,
> > >> > >etc).
> > >> > >
> > >> > >Yes, it's unusual, but this driver used to do it.  Which is exactly 
> > >> > >why
> > >> > >I brought it up.  Turns out it was irrelevant :)
> > >> > >
> > >> > >> > I tested without any udev rules in /etc/udev/, just the standard
> > >> > >distro
> > >> > >> > ones.  Same thing.
> > >> > >>
> > >> > >> Right, so this is the default udev rule:
> > >> > >>
> > >> > >> /lib/udev/rules.d/80-drivers.rules:
> > >> > >>
> > >> > >> # do not edit this file, it will be overwritten on update
> > >> > >>
> > >> > >> ACTION=="remove", GOTO="drivers_end"
> > >> > >>
> > >> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> > >>
> > >> So if the new uevents do not have the MODALIAS line in them, then they
> > >> will not trigger this?  Dmitry, can you see if that would fix this
> > >> problem without having to fix everyone's old versions of udev/systemd?
> > 
> > Unfortunately MODALIAS= is being added by individual subsystems having
> > their subsystem specific format. Unless you'd be OK with
> > kobject_uevent_env() poking into 

Re: [bisected] Re: Module removal-related regression?

[Greg Kroah-Hartman]

On Sun, Sep 10, 2017 at 02:22:22PM -0700, Dmitry Torokhov wrote:
> On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:
> > On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
> >> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> >> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
> >> > wrote:
> >> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> >> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> >> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> >> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> >> > > wrote:
> >> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> >> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
> >> > >
> >> > >> > > >wrote:
> >> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> >> > >
> >> > >> > > >> >> Hi!
> >> > >> > > >> >>
> >> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
> >> > >succeeds
> >> > >> > > >but the
> >> > >> > > >> >> module is still loaded and the refcount goes to 1:
> >> > >> > > >> >>
> >> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> > >> > > >> >>   lsmod | grep nfp; \
> >> > >> > > >> >>   rmmod nfp; \
> >> > >> > > >> >>   lsmod | grep nfp
> >> > >> > > >> >> nfp   249856  0
> >> > >> > > >> >> nfp   200704  1
> >> > >> > > >> >>
> >> > >> > > >> >> If I rmmod again the module will be actually unloaded.  The
> >> > >user
> >> > >> > > >space
> >> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
> >> > >to
> >> > >> > > >bisect
> >> > >> > > >> >> now...
> >> > >> > > >> >
> >> > >> > > >> > Got 'em!
> >> > >> > > >> >
> >> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> >> > >> > > >refs/bisect/bad)
> >> > >> > > >> > Author: Dmitry Torokhov 
> >> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> > >> > > >> >
> >> > >> > > >> > driver core: emit uevents when device is bound to a
> >> > >driver
> >> > >> > > >>
> >> > >> > > >> Does it happen with all modules or only nfp one?
> >> > >> > > >>
> >> > >> > > >> It seems to work here:
> >> > >> > > >>
> >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > >> > > >> psmouse   135168  0
> >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> >> > >> > > >
> >> > >> > > >It looks like the driver is actually reloaded.  The driver used
> >> > >to
> >> > >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
> >> > >the
> >> > >> > > >kernel to test that right now).
> >> > >> > > >
> >> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >> > >> > > >else cause the driver to be loaded again?
> >> > >> > >
> >> > >> > > It depends on how silly the udev rules are, but yes, this can
> >> > >definitely happen.
> >> > >> >
> >> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> >> > >> >
> >> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> >> > >> > $
> >> > >>
> >> > >> Not sure why you bring the deferrals here, they have nothing to do
> >> > >with
> >> > >> module removal. Also, deferrals are rarely issued by the leaf driver,
> >> > >and
> >> > >> more often by providers of resources (GPIO, regulator, interrupt,
> >> > >etc).
> >> > >
> >> > >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> >> > >I brought it up.  Turns out it was irrelevant :)
> >> > >
> >> > >> > I tested without any udev rules in /etc/udev/, just the standard
> >> > >distro
> >> > >> > ones.  Same thing.
> >> > >>
> >> > >> Right, so this is the default udev rule:
> >> > >>
> >> > >> /lib/udev/rules.d/80-drivers.rules:
> >> > >>
> >> > >> # do not edit this file, it will be overwritten on update
> >> > >>
> >> > >> ACTION=="remove", GOTO="drivers_end"
> >> > >>
> >> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> >>
> >> So if the new uevents do not have the MODALIAS line in them, then they
> >> will not trigger this?  Dmitry, can you see if that would fix this
> >> problem without having to fix everyone's old versions of udev/systemd?
> 
> Unfortunately MODALIAS= is being added by individual subsystems having
> their subsystem specific format. Unless you'd be OK with
> kobject_uevent_env() poking into the generated environment and zapping
> MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Hm, any reason why it should be sending these values for those uevents?
I guess it's not worth hacking around in the lower levels just for this,
to work around crazy userspace stuff.

> I'm 

Re: [bisected] Re: Module removal-related regression?

[Greg Kroah-Hartman]

On Sun, Sep 10, 2017 at 02:22:22PM -0700, Dmitry Torokhov wrote:
> On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:
> > On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
> >> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> >> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
> >> > wrote:
> >> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> >> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> >> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> >> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> >> > > wrote:
> >> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> >> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
> >> > >
> >> > >> > > >wrote:
> >> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> >> > >
> >> > >> > > >> >> Hi!
> >> > >> > > >> >>
> >> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
> >> > >succeeds
> >> > >> > > >but the
> >> > >> > > >> >> module is still loaded and the refcount goes to 1:
> >> > >> > > >> >>
> >> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> > >> > > >> >>   lsmod | grep nfp; \
> >> > >> > > >> >>   rmmod nfp; \
> >> > >> > > >> >>   lsmod | grep nfp
> >> > >> > > >> >> nfp   249856  0
> >> > >> > > >> >> nfp   200704  1
> >> > >> > > >> >>
> >> > >> > > >> >> If I rmmod again the module will be actually unloaded.  The
> >> > >user
> >> > >> > > >space
> >> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
> >> > >to
> >> > >> > > >bisect
> >> > >> > > >> >> now...
> >> > >> > > >> >
> >> > >> > > >> > Got 'em!
> >> > >> > > >> >
> >> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> >> > >> > > >refs/bisect/bad)
> >> > >> > > >> > Author: Dmitry Torokhov 
> >> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> > >> > > >> >
> >> > >> > > >> > driver core: emit uevents when device is bound to a
> >> > >driver
> >> > >> > > >>
> >> > >> > > >> Does it happen with all modules or only nfp one?
> >> > >> > > >>
> >> > >> > > >> It seems to work here:
> >> > >> > > >>
> >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > >> > > >> psmouse   135168  0
> >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> >> > >> > > >
> >> > >> > > >It looks like the driver is actually reloaded.  The driver used
> >> > >to
> >> > >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
> >> > >the
> >> > >> > > >kernel to test that right now).
> >> > >> > > >
> >> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >> > >> > > >else cause the driver to be loaded again?
> >> > >> > >
> >> > >> > > It depends on how silly the udev rules are, but yes, this can
> >> > >definitely happen.
> >> > >> >
> >> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> >> > >> >
> >> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> >> > >> > $
> >> > >>
> >> > >> Not sure why you bring the deferrals here, they have nothing to do
> >> > >with
> >> > >> module removal. Also, deferrals are rarely issued by the leaf driver,
> >> > >and
> >> > >> more often by providers of resources (GPIO, regulator, interrupt,
> >> > >etc).
> >> > >
> >> > >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> >> > >I brought it up.  Turns out it was irrelevant :)
> >> > >
> >> > >> > I tested without any udev rules in /etc/udev/, just the standard
> >> > >distro
> >> > >> > ones.  Same thing.
> >> > >>
> >> > >> Right, so this is the default udev rule:
> >> > >>
> >> > >> /lib/udev/rules.d/80-drivers.rules:
> >> > >>
> >> > >> # do not edit this file, it will be overwritten on update
> >> > >>
> >> > >> ACTION=="remove", GOTO="drivers_end"
> >> > >>
> >> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> >>
> >> So if the new uevents do not have the MODALIAS line in them, then they
> >> will not trigger this?  Dmitry, can you see if that would fix this
> >> problem without having to fix everyone's old versions of udev/systemd?
> 
> Unfortunately MODALIAS= is being added by individual subsystems having
> their subsystem specific format. Unless you'd be OK with
> kobject_uevent_env() poking into the generated environment and zapping
> MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Hm, any reason why it should be sending these values for those uevents?
I guess it's not worth hacking around in the lower levels just for this,
to work around crazy userspace stuff.

> I'm still going to submit correction for the rule to systemd folks.

Yes please.

> > Perhaps 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:
> On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
>> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
>> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
>> > wrote:
>> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
>> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
>> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
>> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
>> > > wrote:
>> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
>> > >
>> > >> > > >wrote:
>> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
>> > >
>> > >> > > >> >> Hi!
>> > >> > > >> >>
>> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
>> > >succeeds
>> > >> > > >but the
>> > >> > > >> >> module is still loaded and the refcount goes to 1:
>> > >> > > >> >>
>> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> > >> > > >> >>   lsmod | grep nfp; \
>> > >> > > >> >>   rmmod nfp; \
>> > >> > > >> >>   lsmod | grep nfp
>> > >> > > >> >> nfp   249856  0
>> > >> > > >> >> nfp   200704  1
>> > >> > > >> >>
>> > >> > > >> >> If I rmmod again the module will be actually unloaded.  The
>> > >user
>> > >> > > >space
>> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
>> > >to
>> > >> > > >bisect
>> > >> > > >> >> now...
>> > >> > > >> >
>> > >> > > >> > Got 'em!
>> > >> > > >> >
>> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>> > >> > > >refs/bisect/bad)
>> > >> > > >> > Author: Dmitry Torokhov 
>> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> > >> > > >> >
>> > >> > > >> > driver core: emit uevents when device is bound to a
>> > >driver
>> > >> > > >>
>> > >> > > >> Does it happen with all modules or only nfp one?
>> > >> > > >>
>> > >> > > >> It seems to work here:
>> > >> > > >>
>> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > >> > > >> psmouse   135168  0
>> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>> > >> > > >
>> > >> > > >It looks like the driver is actually reloaded.  The driver used
>> > >to
>> > >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
>> > >the
>> > >> > > >kernel to test that right now).
>> > >> > > >
>> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>> > >> > > >else cause the driver to be loaded again?
>> > >> > >
>> > >> > > It depends on how silly the udev rules are, but yes, this can
>> > >definitely happen.
>> > >> >
>> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
>> > >> >
>> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
>> > >> > $
>> > >>
>> > >> Not sure why you bring the deferrals here, they have nothing to do
>> > >with
>> > >> module removal. Also, deferrals are rarely issued by the leaf driver,
>> > >and
>> > >> more often by providers of resources (GPIO, regulator, interrupt,
>> > >etc).
>> > >
>> > >Yes, it's unusual, but this driver used to do it.  Which is exactly why
>> > >I brought it up.  Turns out it was irrelevant :)
>> > >
>> > >> > I tested without any udev rules in /etc/udev/, just the standard
>> > >distro
>> > >> > ones.  Same thing.
>> > >>
>> > >> Right, so this is the default udev rule:
>> > >>
>> > >> /lib/udev/rules.d/80-drivers.rules:
>> > >>
>> > >> # do not edit this file, it will be overwritten on update
>> > >>
>> > >> ACTION=="remove", GOTO="drivers_end"
>> > >>
>> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
>>
>> So if the new uevents do not have the MODALIAS line in them, then they
>> will not trigger this?  Dmitry, can you see if that would fix this
>> problem without having to fix everyone's old versions of udev/systemd?

Unfortunately MODALIAS= is being added by individual subsystems having
their subsystem specific format. Unless you'd be OK with
kobject_uevent_env() poking into the generated environment and zapping
MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Let me know and I can try to come up with a patch.

I'm still going to submit correction for the rule to systemd folks.

>
> Perhaps another option is dropping the unbind event?  From the commit
> message it seems like only bind is really needed ATM.  Do events have
> to be symmetrical?

While you are absolutely right that bind is the most important one,
I'd be hesitant removing unbind even though we do not have concrete
use case for it yet. The bind operation 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sun, Sep 10, 2017 at 12:13 PM, Jakub Kicinski  wrote:
> On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
>> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
>> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
>> > wrote:
>> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
>> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
>> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
>> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
>> > > wrote:
>> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
>> > >
>> > >> > > >wrote:
>> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
>> > >
>> > >> > > >> >> Hi!
>> > >> > > >> >>
>> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
>> > >succeeds
>> > >> > > >but the
>> > >> > > >> >> module is still loaded and the refcount goes to 1:
>> > >> > > >> >>
>> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> > >> > > >> >>   lsmod | grep nfp; \
>> > >> > > >> >>   rmmod nfp; \
>> > >> > > >> >>   lsmod | grep nfp
>> > >> > > >> >> nfp   249856  0
>> > >> > > >> >> nfp   200704  1
>> > >> > > >> >>
>> > >> > > >> >> If I rmmod again the module will be actually unloaded.  The
>> > >user
>> > >> > > >space
>> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
>> > >to
>> > >> > > >bisect
>> > >> > > >> >> now...
>> > >> > > >> >
>> > >> > > >> > Got 'em!
>> > >> > > >> >
>> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>> > >> > > >refs/bisect/bad)
>> > >> > > >> > Author: Dmitry Torokhov 
>> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> > >> > > >> >
>> > >> > > >> > driver core: emit uevents when device is bound to a
>> > >driver
>> > >> > > >>
>> > >> > > >> Does it happen with all modules or only nfp one?
>> > >> > > >>
>> > >> > > >> It seems to work here:
>> > >> > > >>
>> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > >> > > >> psmouse   135168  0
>> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>> > >> > > >
>> > >> > > >It looks like the driver is actually reloaded.  The driver used
>> > >to
>> > >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
>> > >the
>> > >> > > >kernel to test that right now).
>> > >> > > >
>> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>> > >> > > >else cause the driver to be loaded again?
>> > >> > >
>> > >> > > It depends on how silly the udev rules are, but yes, this can
>> > >definitely happen.
>> > >> >
>> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
>> > >> >
>> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
>> > >> > $
>> > >>
>> > >> Not sure why you bring the deferrals here, they have nothing to do
>> > >with
>> > >> module removal. Also, deferrals are rarely issued by the leaf driver,
>> > >and
>> > >> more often by providers of resources (GPIO, regulator, interrupt,
>> > >etc).
>> > >
>> > >Yes, it's unusual, but this driver used to do it.  Which is exactly why
>> > >I brought it up.  Turns out it was irrelevant :)
>> > >
>> > >> > I tested without any udev rules in /etc/udev/, just the standard
>> > >distro
>> > >> > ones.  Same thing.
>> > >>
>> > >> Right, so this is the default udev rule:
>> > >>
>> > >> /lib/udev/rules.d/80-drivers.rules:
>> > >>
>> > >> # do not edit this file, it will be overwritten on update
>> > >>
>> > >> ACTION=="remove", GOTO="drivers_end"
>> > >>
>> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
>>
>> So if the new uevents do not have the MODALIAS line in them, then they
>> will not trigger this?  Dmitry, can you see if that would fix this
>> problem without having to fix everyone's old versions of udev/systemd?

Unfortunately MODALIAS= is being added by individual subsystems having
their subsystem specific format. Unless you'd be OK with
kobject_uevent_env() poking into the generated environment and zapping
MODALIAS= environment variables for KOBJ_BIND/KOBJ_UNBIND actions.

Let me know and I can try to come up with a patch.

I'm still going to submit correction for the rule to systemd folks.

>
> Perhaps another option is dropping the unbind event?  From the commit
> message it seems like only bind is really needed ATM.  Do events have
> to be symmetrical?

While you are absolutely right that bind is the most important one,
I'd be hesitant removing unbind even though we do not have concrete
use case for it yet. The bind operation complements unbind, so having
bind uevent but not unbind "feels weird".

Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
> > wrote:  
> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:  
> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:  
> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > > wrote:
> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  
> > >  
> > >> > > >wrote:  
> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> > >   
> > >> > > >> >> Hi!
> > >> > > >> >>
> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod  
> > >succeeds  
> > >> > > >but the  
> > >> > > >> >> module is still loaded and the refcount goes to 1:
> > >> > > >> >>
> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> > >> > > >> >>   lsmod | grep nfp; \
> > >> > > >> >>   rmmod nfp; \
> > >> > > >> >>   lsmod | grep nfp
> > >> > > >> >> nfp   249856  0
> > >> > > >> >> nfp   200704  1
> > >> > > >> >>
> > >> > > >> >> If I rmmod again the module will be actually unloaded.  The  
> > >user  
> > >> > > >space  
> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying  
> > >to  
> > >> > > >bisect  
> > >> > > >> >> now...
> > >> > > >> >
> > >> > > >> > Got 'em!
> > >> > > >> >
> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> > >> > > >refs/bisect/bad)  
> > >> > > >> > Author: Dmitry Torokhov 
> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > >> > > >> >
> > >> > > >> > driver core: emit uevents when device is bound to a  
> > >driver
> > >> > > >> 
> > >> > > >> Does it happen with all modules or only nfp one?
> > >> > > >> 
> > >> > > >> It seems to work here:
> > >> > > >> 
> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > > >> psmouse   135168  0
> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> > >> > > >
> > >> > > >It looks like the driver is actually reloaded.  The driver used  
> > >to  
> > >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding  
> > >the  
> > >> > > >kernel to test that right now).
> > >> > > >
> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> > >> > > >else cause the driver to be loaded again?   
> > >> > > 
> > >> > > It depends on how silly the udev rules are, but yes, this can  
> > >definitely happen.
> > >> > 
> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > >> > 
> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > >> > $
> > >> 
> > >> Not sure why you bring the deferrals here, they have nothing to do  
> > >with  
> > >> module removal. Also, deferrals are rarely issued by the leaf driver,  
> > >and  
> > >> more often by providers of resources (GPIO, regulator, interrupt,  
> > >etc).
> > >
> > >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> > >I brought it up.  Turns out it was irrelevant :)
> > >  
> > >> > I tested without any udev rules in /etc/udev/, just the standard  
> > >distro  
> > >> > ones.  Same thing.
> > >> 
> > >> Right, so this is the default udev rule:
> > >> 
> > >> /lib/udev/rules.d/80-drivers.rules:
> > >> 
> > >> # do not edit this file, it will be overwritten on update
> > >> 
> > >> ACTION=="remove", GOTO="drivers_end"
> > >> 
> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"  
> 
> So if the new uevents do not have the MODALIAS line in them, then they
> will not trigger this?  Dmitry, can you see if that would fix this
> problem without having to fix everyone's old versions of udev/systemd?

Perhaps another option is dropping the unbind event?  From the commit
message it seems like only bind is really needed ATM.  Do events have
to be symmetrical?

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sun, 10 Sep 2017 21:09:08 +0200, Greg Kroah-Hartman wrote:
> On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> > On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  
> > wrote:  
> > >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:  
> > >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:  
> > >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> > >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > > wrote:
> > >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> > >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  
> > >  
> > >> > > >wrote:  
> > >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> > >   
> > >> > > >> >> Hi!
> > >> > > >> >>
> > >> > > >> >> I'm having trouble with modules on linux/master.  rmmod  
> > >succeeds  
> > >> > > >but the  
> > >> > > >> >> module is still loaded and the refcount goes to 1:
> > >> > > >> >>
> > >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> > >> > > >> >>   lsmod | grep nfp; \
> > >> > > >> >>   rmmod nfp; \
> > >> > > >> >>   lsmod | grep nfp
> > >> > > >> >> nfp   249856  0
> > >> > > >> >> nfp   200704  1
> > >> > > >> >>
> > >> > > >> >> If I rmmod again the module will be actually unloaded.  The  
> > >user  
> > >> > > >space  
> > >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying  
> > >to  
> > >> > > >bisect  
> > >> > > >> >> now...
> > >> > > >> >
> > >> > > >> > Got 'em!
> > >> > > >> >
> > >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> > >> > > >refs/bisect/bad)  
> > >> > > >> > Author: Dmitry Torokhov 
> > >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > >> > > >> >
> > >> > > >> > driver core: emit uevents when device is bound to a  
> > >driver
> > >> > > >> 
> > >> > > >> Does it happen with all modules or only nfp one?
> > >> > > >> 
> > >> > > >> It seems to work here:
> > >> > > >> 
> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > > >> psmouse   135168  0
> > >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> > >> > > >
> > >> > > >It looks like the driver is actually reloaded.  The driver used  
> > >to  
> > >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding  
> > >the  
> > >> > > >kernel to test that right now).
> > >> > > >
> > >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> > >> > > >else cause the driver to be loaded again?   
> > >> > > 
> > >> > > It depends on how silly the udev rules are, but yes, this can  
> > >definitely happen.
> > >> > 
> > >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > >> > 
> > >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > >> > $
> > >> 
> > >> Not sure why you bring the deferrals here, they have nothing to do  
> > >with  
> > >> module removal. Also, deferrals are rarely issued by the leaf driver,  
> > >and  
> > >> more often by providers of resources (GPIO, regulator, interrupt,  
> > >etc).
> > >
> > >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> > >I brought it up.  Turns out it was irrelevant :)
> > >  
> > >> > I tested without any udev rules in /etc/udev/, just the standard  
> > >distro  
> > >> > ones.  Same thing.
> > >> 
> > >> Right, so this is the default udev rule:
> > >> 
> > >> /lib/udev/rules.d/80-drivers.rules:
> > >> 
> > >> # do not edit this file, it will be overwritten on update
> > >> 
> > >> ACTION=="remove", GOTO="drivers_end"
> > >> 
> > >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"  
> 
> So if the new uevents do not have the MODALIAS line in them, then they
> will not trigger this?  Dmitry, can you see if that would fix this
> problem without having to fix everyone's old versions of udev/systemd?

Perhaps another option is dropping the unbind event?  From the commit
message it seems like only bind is really needed ATM.  Do events have
to be symmetrical?

Re: [bisected] Re: Module removal-related regression?

[Greg Kroah-Hartman]

On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
> >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
> >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> > wrote:  
> >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
> >
> >> > > >wrote:
> >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> > 
> >> > > >> >> Hi!
> >> > > >> >>
> >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
> >succeeds
> >> > > >but the
> >> > > >> >> module is still loaded and the refcount goes to 1:
> >> > > >> >>
> >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> > > >> >>   lsmod | grep nfp; \
> >> > > >> >>   rmmod nfp; \
> >> > > >> >>   lsmod | grep nfp
> >> > > >> >> nfp   249856  0
> >> > > >> >> nfp   200704  1
> >> > > >> >>
> >> > > >> >> If I rmmod again the module will be actually unloaded.  The
> >user
> >> > > >space
> >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
> >to
> >> > > >bisect
> >> > > >> >> now...  
> >> > > >> >
> >> > > >> > Got 'em!
> >> > > >> >
> >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> >> > > >refs/bisect/bad)
> >> > > >> > Author: Dmitry Torokhov 
> >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> > > >> >
> >> > > >> > driver core: emit uevents when device is bound to a
> >driver  
> >> > > >> 
> >> > > >> Does it happen with all modules or only nfp one?
> >> > > >> 
> >> > > >> It seems to work here:
> >> > > >> 
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> psmouse   135168  0
> >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> >> > > >
> >> > > >It looks like the driver is actually reloaded.  The driver used
> >to
> >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
> >the
> >> > > >kernel to test that right now).
> >> > > >
> >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >> > > >else cause the driver to be loaded again? 
> >> > > 
> >> > > It depends on how silly the udev rules are, but yes, this can
> >definitely happen.  
> >> > 
> >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> >> > 
> >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> >> > $  
> >> 
> >> Not sure why you bring the deferrals here, they have nothing to do
> >with
> >> module removal. Also, deferrals are rarely issued by the leaf driver,
> >and
> >> more often by providers of resources (GPIO, regulator, interrupt,
> >etc).
> >
> >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> >I brought it up.  Turns out it was irrelevant :)
> >
> >> > I tested without any udev rules in /etc/udev/, just the standard
> >distro
> >> > ones.  Same thing.  
> >> 
> >> Right, so this is the default udev rule:
> >> 
> >> /lib/udev/rules.d/80-drivers.rules:
> >> 
> >> # do not edit this file, it will be overwritten on update
> >> 
> >> ACTION=="remove", GOTO="drivers_end"
> >> 
> >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

So if the new uevents do not have the MODALIAS line in them, then they
will not trigger this?  Dmitry, can you see if that would fix this
problem without having to fix everyone's old versions of udev/systemd?

thanks,

greg k-h

Re: [bisected] Re: Module removal-related regression?

[Greg Kroah-Hartman]

On Sun, Sep 10, 2017 at 11:12:17AM -0700, Dmitry Torokhov wrote:
> On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
> >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
> >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> > wrote:  
> >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
> >
> >> > > >wrote:
> >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> > 
> >> > > >> >> Hi!
> >> > > >> >>
> >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
> >succeeds
> >> > > >but the
> >> > > >> >> module is still loaded and the refcount goes to 1:
> >> > > >> >>
> >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> > > >> >>   lsmod | grep nfp; \
> >> > > >> >>   rmmod nfp; \
> >> > > >> >>   lsmod | grep nfp
> >> > > >> >> nfp   249856  0
> >> > > >> >> nfp   200704  1
> >> > > >> >>
> >> > > >> >> If I rmmod again the module will be actually unloaded.  The
> >user
> >> > > >space
> >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
> >to
> >> > > >bisect
> >> > > >> >> now...  
> >> > > >> >
> >> > > >> > Got 'em!
> >> > > >> >
> >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> >> > > >refs/bisect/bad)
> >> > > >> > Author: Dmitry Torokhov 
> >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> > > >> >
> >> > > >> > driver core: emit uevents when device is bound to a
> >driver  
> >> > > >> 
> >> > > >> Does it happen with all modules or only nfp one?
> >> > > >> 
> >> > > >> It seems to work here:
> >> > > >> 
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> psmouse   135168  0
> >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> >> > > >
> >> > > >It looks like the driver is actually reloaded.  The driver used
> >to
> >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
> >the
> >> > > >kernel to test that right now).
> >> > > >
> >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >> > > >else cause the driver to be loaded again? 
> >> > > 
> >> > > It depends on how silly the udev rules are, but yes, this can
> >definitely happen.  
> >> > 
> >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> >> > 
> >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> >> > $  
> >> 
> >> Not sure why you bring the deferrals here, they have nothing to do
> >with
> >> module removal. Also, deferrals are rarely issued by the leaf driver,
> >and
> >> more often by providers of resources (GPIO, regulator, interrupt,
> >etc).
> >
> >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> >I brought it up.  Turns out it was irrelevant :)
> >
> >> > I tested without any udev rules in /etc/udev/, just the standard
> >distro
> >> > ones.  Same thing.  
> >> 
> >> Right, so this is the default udev rule:
> >> 
> >> /lib/udev/rules.d/80-drivers.rules:
> >> 
> >> # do not edit this file, it will be overwritten on update
> >> 
> >> ACTION=="remove", GOTO="drivers_end"
> >> 
> >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"

So if the new uevents do not have the MODALIAS line in them, then they
will not trigger this?  Dmitry, can you see if that would fix this
problem without having to fix everyone's old versions of udev/systemd?

thanks,

greg k-h

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sun, Sep 10, 2017 at 11:39 AM, Jakub Kicinski  wrote:
> On Sun, 10 Sep 2017 11:12:17 -0700, Dmitry Torokhov wrote:
>> On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
>> >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
>> >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
>> >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
>> >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
>> > wrote:
>> >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
>> >
>> >> > > >wrote:
>> >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
>> >
>> >> > > >> >> Hi!
>> >> > > >> >>
>> >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
>> >succeeds
>> >> > > >but the
>> >> > > >> >> module is still loaded and the refcount goes to 1:
>> >> > > >> >>
>> >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> >> > > >> >>   lsmod | grep nfp; \
>> >> > > >> >>   rmmod nfp; \
>> >> > > >> >>   lsmod | grep nfp
>> >> > > >> >> nfp   249856  0
>> >> > > >> >> nfp   200704  1
>> >> > > >> >>
>> >> > > >> >> If I rmmod again the module will be actually unloaded.  The
>> >user
>> >> > > >space
>> >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
>> >to
>> >> > > >bisect
>> >> > > >> >> now...
>> >> > > >> >
>> >> > > >> > Got 'em!
>> >> > > >> >
>> >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>> >> > > >refs/bisect/bad)
>> >> > > >> > Author: Dmitry Torokhov 
>> >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> >> > > >> >
>> >> > > >> > driver core: emit uevents when device is bound to a
>> >driver
>> >> > > >>
>> >> > > >> Does it happen with all modules or only nfp one?
>> >> > > >>
>> >> > > >> It seems to work here:
>> >> > > >>
>> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> >> > > >> psmouse   135168  0
>> >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>> >> > > >
>> >> > > >It looks like the driver is actually reloaded.  The driver used
>> >to
>> >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
>> >the
>> >> > > >kernel to test that right now).
>> >> > > >
>> >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>> >> > > >else cause the driver to be loaded again?
>> >> > >
>> >> > > It depends on how silly the udev rules are, but yes, this can
>> >definitely happen.
>> >> >
>> >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
>> >> >
>> >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
>> >> > $
>> >>
>> >> Not sure why you bring the deferrals here, they have nothing to do
>> >with
>> >> module removal. Also, deferrals are rarely issued by the leaf driver,
>> >and
>> >> more often by providers of resources (GPIO, regulator, interrupt,
>> >etc).
>> >
>> >Yes, it's unusual, but this driver used to do it.  Which is exactly why
>> >I brought it up.  Turns out it was irrelevant :)
>> >
>> >> > I tested without any udev rules in /etc/udev/, just the standard
>> >distro
>> >> > ones.  Same thing.
>> >>
>> >> Right, so this is the default udev rule:
>> >>
>> >> /lib/udev/rules.d/80-drivers.rules:
>> >>
>> >> # do not edit this file, it will be overwritten on update
>> >>
>> >> ACTION=="remove", GOTO="drivers_end"
>> >>
>> >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
>> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load
>> >tifm_sd"
>> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load
>> >tifm_ms"
>> >> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
>> >> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
>> >> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load
>> >ppdev"
>> >> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load
>> >$env{MODALIAS}"
>> >> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
>> >> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load
>> >sm_ftl"
>> >>
>> >> LABEL="drivers_end"
>> >>
>> >> So udev (and systemd) want to load kernel module on any action
>> >besides
>> >> device removal. Shortsighted decision I'd say. I'll send a patch to
>> >> systemd, in the mean time you can simply adjust your local rule to
>> >read
>> >>
>> >> ACTION!="add", GOTO="drivers_end"
>> >
>> >Mm.  That is a silly thing.  You will break a lot of setups, though.
>>
>> I think the priority it to have module loading working properly, and
>> for most users once module is loaded it stays loaded. Unloading is
>> mostly for developers.
>>
>> Luckily newer 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sun, Sep 10, 2017 at 11:39 AM, Jakub Kicinski  wrote:
> On Sun, 10 Sep 2017 11:12:17 -0700, Dmitry Torokhov wrote:
>> On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
>> >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
>> >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
>> >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
>> >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
>> > wrote:
>> >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
>> >
>> >> > > >wrote:
>> >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
>> >
>> >> > > >> >> Hi!
>> >> > > >> >>
>> >> > > >> >> I'm having trouble with modules on linux/master.  rmmod
>> >succeeds
>> >> > > >but the
>> >> > > >> >> module is still loaded and the refcount goes to 1:
>> >> > > >> >>
>> >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> >> > > >> >>   lsmod | grep nfp; \
>> >> > > >> >>   rmmod nfp; \
>> >> > > >> >>   lsmod | grep nfp
>> >> > > >> >> nfp   249856  0
>> >> > > >> >> nfp   200704  1
>> >> > > >> >>
>> >> > > >> >> If I rmmod again the module will be actually unloaded.  The
>> >user
>> >> > > >space
>> >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
>> >to
>> >> > > >bisect
>> >> > > >> >> now...
>> >> > > >> >
>> >> > > >> > Got 'em!
>> >> > > >> >
>> >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>> >> > > >refs/bisect/bad)
>> >> > > >> > Author: Dmitry Torokhov 
>> >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> >> > > >> >
>> >> > > >> > driver core: emit uevents when device is bound to a
>> >driver
>> >> > > >>
>> >> > > >> Does it happen with all modules or only nfp one?
>> >> > > >>
>> >> > > >> It seems to work here:
>> >> > > >>
>> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> >> > > >> psmouse   135168  0
>> >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>> >> > > >
>> >> > > >It looks like the driver is actually reloaded.  The driver used
>> >to
>> >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
>> >the
>> >> > > >kernel to test that right now).
>> >> > > >
>> >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>> >> > > >else cause the driver to be loaded again?
>> >> > >
>> >> > > It depends on how silly the udev rules are, but yes, this can
>> >definitely happen.
>> >> >
>> >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
>> >> >
>> >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
>> >> > $
>> >>
>> >> Not sure why you bring the deferrals here, they have nothing to do
>> >with
>> >> module removal. Also, deferrals are rarely issued by the leaf driver,
>> >and
>> >> more often by providers of resources (GPIO, regulator, interrupt,
>> >etc).
>> >
>> >Yes, it's unusual, but this driver used to do it.  Which is exactly why
>> >I brought it up.  Turns out it was irrelevant :)
>> >
>> >> > I tested without any udev rules in /etc/udev/, just the standard
>> >distro
>> >> > ones.  Same thing.
>> >>
>> >> Right, so this is the default udev rule:
>> >>
>> >> /lib/udev/rules.d/80-drivers.rules:
>> >>
>> >> # do not edit this file, it will be overwritten on update
>> >>
>> >> ACTION=="remove", GOTO="drivers_end"
>> >>
>> >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
>> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load
>> >tifm_sd"
>> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load
>> >tifm_ms"
>> >> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
>> >> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
>> >> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load
>> >ppdev"
>> >> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load
>> >$env{MODALIAS}"
>> >> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
>> >> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load
>> >sm_ftl"
>> >>
>> >> LABEL="drivers_end"
>> >>
>> >> So udev (and systemd) want to load kernel module on any action
>> >besides
>> >> device removal. Shortsighted decision I'd say. I'll send a patch to
>> >> systemd, in the mean time you can simply adjust your local rule to
>> >read
>> >>
>> >> ACTION!="add", GOTO="drivers_end"
>> >
>> >Mm.  That is a silly thing.  You will break a lot of setups, though.
>>
>> I think the priority it to have module loading working properly, and
>> for most users once module is loaded it stays loaded. Unloading is
>> mostly for developers.
>>
>> Luckily newer systemd versions drop events they do not recognize, so
>> exposure is even smaller.
>
> Could 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sun, 10 Sep 2017 11:12:17 -0700, Dmitry Torokhov wrote:
> On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
> >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:  
> >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:  
> >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > wrote:
> >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  
> >  
> >> > > >wrote:  
> >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> >   
> >> > > >> >> Hi!
> >> > > >> >>
> >> > > >> >> I'm having trouble with modules on linux/master.  rmmod  
> >succeeds  
> >> > > >but the  
> >> > > >> >> module is still loaded and the refcount goes to 1:
> >> > > >> >>
> >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> > > >> >>   lsmod | grep nfp; \
> >> > > >> >>   rmmod nfp; \
> >> > > >> >>   lsmod | grep nfp
> >> > > >> >> nfp   249856  0
> >> > > >> >> nfp   200704  1
> >> > > >> >>
> >> > > >> >> If I rmmod again the module will be actually unloaded.  The  
> >user  
> >> > > >space  
> >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying  
> >to  
> >> > > >bisect  
> >> > > >> >> now...
> >> > > >> >
> >> > > >> > Got 'em!
> >> > > >> >
> >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> >> > > >refs/bisect/bad)  
> >> > > >> > Author: Dmitry Torokhov 
> >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> > > >> >
> >> > > >> > driver core: emit uevents when device is bound to a  
> >driver
> >> > > >> 
> >> > > >> Does it happen with all modules or only nfp one?
> >> > > >> 
> >> > > >> It seems to work here:
> >> > > >> 
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> psmouse   135168  0
> >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> >> > > >
> >> > > >It looks like the driver is actually reloaded.  The driver used  
> >to  
> >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding  
> >the  
> >> > > >kernel to test that right now).
> >> > > >
> >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >> > > >else cause the driver to be loaded again?   
> >> > > 
> >> > > It depends on how silly the udev rules are, but yes, this can  
> >definitely happen.
> >> > 
> >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> >> > 
> >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> >> > $
> >> 
> >> Not sure why you bring the deferrals here, they have nothing to do  
> >with  
> >> module removal. Also, deferrals are rarely issued by the leaf driver,  
> >and  
> >> more often by providers of resources (GPIO, regulator, interrupt,  
> >etc).
> >
> >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> >I brought it up.  Turns out it was irrelevant :)
> >  
> >> > I tested without any udev rules in /etc/udev/, just the standard  
> >distro  
> >> > ones.  Same thing.
> >> 
> >> Right, so this is the default udev rule:
> >> 
> >> /lib/udev/rules.d/80-drivers.rules:
> >> 
> >> # do not edit this file, it will be overwritten on update
> >> 
> >> ACTION=="remove", GOTO="drivers_end"
> >> 
> >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load  
> >tifm_sd"  
> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load  
> >tifm_ms"  
> >> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
> >> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
> >> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load  
> >ppdev"  
> >> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load  
> >$env{MODALIAS}"  
> >> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
> >> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load  
> >sm_ftl"  
> >> 
> >> LABEL="drivers_end"
> >> 
> >> So udev (and systemd) want to load kernel module on any action  
> >besides  
> >> device removal. Shortsighted decision I'd say. I'll send a patch to
> >> systemd, in the mean time you can simply adjust your local rule to  
> >read  
> >> 
> >> ACTION!="add", GOTO="drivers_end"  
> >
> >Mm.  That is a silly thing.  You will break a lot of setups, though.  
> 
> I think the priority it to have module loading working properly, and
> for most users once module is loaded it stays loaded. Unloading is
> mostly for developers.
>
> 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sun, 10 Sep 2017 11:12:17 -0700, Dmitry Torokhov wrote:
> On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
> >On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:  
> >> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:  
> >> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> >> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > wrote:
> >> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> >> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  
> >  
> >> > > >wrote:  
> >> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> >   
> >> > > >> >> Hi!
> >> > > >> >>
> >> > > >> >> I'm having trouble with modules on linux/master.  rmmod  
> >succeeds  
> >> > > >but the  
> >> > > >> >> module is still loaded and the refcount goes to 1:
> >> > > >> >>
> >> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> > > >> >>   lsmod | grep nfp; \
> >> > > >> >>   rmmod nfp; \
> >> > > >> >>   lsmod | grep nfp
> >> > > >> >> nfp   249856  0
> >> > > >> >> nfp   200704  1
> >> > > >> >>
> >> > > >> >> If I rmmod again the module will be actually unloaded.  The  
> >user  
> >> > > >space  
> >> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying  
> >to  
> >> > > >bisect  
> >> > > >> >> now...
> >> > > >> >
> >> > > >> > Got 'em!
> >> > > >> >
> >> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> >> > > >refs/bisect/bad)  
> >> > > >> > Author: Dmitry Torokhov 
> >> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> > > >> >
> >> > > >> > driver core: emit uevents when device is bound to a  
> >driver
> >> > > >> 
> >> > > >> Does it happen with all modules or only nfp one?
> >> > > >> 
> >> > > >> It seems to work here:
> >> > > >> 
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> psmouse   135168  0
> >> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> >> > > >
> >> > > >It looks like the driver is actually reloaded.  The driver used  
> >to  
> >> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding  
> >the  
> >> > > >kernel to test that right now).
> >> > > >
> >> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >> > > >else cause the driver to be loaded again?   
> >> > > 
> >> > > It depends on how silly the udev rules are, but yes, this can  
> >definitely happen.
> >> > 
> >> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> >> > 
> >> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> >> > $
> >> 
> >> Not sure why you bring the deferrals here, they have nothing to do  
> >with  
> >> module removal. Also, deferrals are rarely issued by the leaf driver,  
> >and  
> >> more often by providers of resources (GPIO, regulator, interrupt,  
> >etc).
> >
> >Yes, it's unusual, but this driver used to do it.  Which is exactly why
> >I brought it up.  Turns out it was irrelevant :)
> >  
> >> > I tested without any udev rules in /etc/udev/, just the standard  
> >distro  
> >> > ones.  Same thing.
> >> 
> >> Right, so this is the default udev rule:
> >> 
> >> /lib/udev/rules.d/80-drivers.rules:
> >> 
> >> # do not edit this file, it will be overwritten on update
> >> 
> >> ACTION=="remove", GOTO="drivers_end"
> >> 
> >> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load  
> >tifm_sd"  
> >> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load  
> >tifm_ms"  
> >> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
> >> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
> >> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load  
> >ppdev"  
> >> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load  
> >$env{MODALIAS}"  
> >> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
> >> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load  
> >sm_ftl"  
> >> 
> >> LABEL="drivers_end"
> >> 
> >> So udev (and systemd) want to load kernel module on any action  
> >besides  
> >> device removal. Shortsighted decision I'd say. I'll send a patch to
> >> systemd, in the mean time you can simply adjust your local rule to  
> >read  
> >> 
> >> ACTION!="add", GOTO="drivers_end"  
> >
> >Mm.  That is a silly thing.  You will break a lot of setups, though.  
> 
> I think the priority it to have module loading working properly, and
> for most users once module is loaded it stays loaded. Unloading is
> mostly for developers.
>
> Luckily newer systemd versions drop events they do not recognize, so
> 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
>On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
>> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
>> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
>> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> wrote:  
>> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
>
>> > > >wrote:
>> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> 
>> > > >> >> Hi!
>> > > >> >>
>> > > >> >> I'm having trouble with modules on linux/master.  rmmod
>succeeds
>> > > >but the
>> > > >> >> module is still loaded and the refcount goes to 1:
>> > > >> >>
>> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> > > >> >>   lsmod | grep nfp; \
>> > > >> >>   rmmod nfp; \
>> > > >> >>   lsmod | grep nfp
>> > > >> >> nfp   249856  0
>> > > >> >> nfp   200704  1
>> > > >> >>
>> > > >> >> If I rmmod again the module will be actually unloaded.  The
>user
>> > > >space
>> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
>to
>> > > >bisect
>> > > >> >> now...  
>> > > >> >
>> > > >> > Got 'em!
>> > > >> >
>> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>> > > >refs/bisect/bad)
>> > > >> > Author: Dmitry Torokhov 
>> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> > > >> >
>> > > >> > driver core: emit uevents when device is bound to a
>driver  
>> > > >> 
>> > > >> Does it happen with all modules or only nfp one?
>> > > >> 
>> > > >> It seems to work here:
>> > > >> 
>> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > > >> psmouse   135168  0
>> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>> > > >
>> > > >It looks like the driver is actually reloaded.  The driver used
>to
>> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
>the
>> > > >kernel to test that right now).
>> > > >
>> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>> > > >else cause the driver to be loaded again? 
>> > > 
>> > > It depends on how silly the udev rules are, but yes, this can
>definitely happen.  
>> > 
>> > I confirmed the driver doesn't use EPROBE_DEFER any more:
>> > 
>> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
>> > $  
>> 
>> Not sure why you bring the deferrals here, they have nothing to do
>with
>> module removal. Also, deferrals are rarely issued by the leaf driver,
>and
>> more often by providers of resources (GPIO, regulator, interrupt,
>etc).
>
>Yes, it's unusual, but this driver used to do it.  Which is exactly why
>I brought it up.  Turns out it was irrelevant :)
>
>> > I tested without any udev rules in /etc/udev/, just the standard
>distro
>> > ones.  Same thing.  
>> 
>> Right, so this is the default udev rule:
>> 
>> /lib/udev/rules.d/80-drivers.rules:
>> 
>> # do not edit this file, it will be overwritten on update
>> 
>> ACTION=="remove", GOTO="drivers_end"
>> 
>> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
>> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load
>tifm_sd"
>> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load
>tifm_ms"
>> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
>> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
>> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load
>ppdev"
>> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load
>$env{MODALIAS}"
>> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
>> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load
>sm_ftl"
>> 
>> LABEL="drivers_end"
>> 
>> So udev (and systemd) want to load kernel module on any action
>besides
>> device removal. Shortsighted decision I'd say. I'll send a patch to
>> systemd, in the mean time you can simply adjust your local rule to
>read
>> 
>> ACTION!="add", GOTO="drivers_end"
>
>Mm.  That is a silly thing.  You will break a lot of setups, though.

I think the priority it to have module loading working properly, and for most 
users once module is loaded it stays loaded. Unloading is mostly for developers.

Luckily newer systemd versions drop events they do not recognize, so exposure 
is even smaller.


Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On September 10, 2017 11:00:10 AM PDT, Jakub Kicinski  wrote:
>On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
>> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
>> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
>> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski
> wrote:  
>> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski
>
>> > > >wrote:
>> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> 
>> > > >> >> Hi!
>> > > >> >>
>> > > >> >> I'm having trouble with modules on linux/master.  rmmod
>succeeds
>> > > >but the
>> > > >> >> module is still loaded and the refcount goes to 1:
>> > > >> >>
>> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> > > >> >>   lsmod | grep nfp; \
>> > > >> >>   rmmod nfp; \
>> > > >> >>   lsmod | grep nfp
>> > > >> >> nfp   249856  0
>> > > >> >> nfp   200704  1
>> > > >> >>
>> > > >> >> If I rmmod again the module will be actually unloaded.  The
>user
>> > > >space
>> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying
>to
>> > > >bisect
>> > > >> >> now...  
>> > > >> >
>> > > >> > Got 'em!
>> > > >> >
>> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>> > > >refs/bisect/bad)
>> > > >> > Author: Dmitry Torokhov 
>> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> > > >> >
>> > > >> > driver core: emit uevents when device is bound to a
>driver  
>> > > >> 
>> > > >> Does it happen with all modules or only nfp one?
>> > > >> 
>> > > >> It seems to work here:
>> > > >> 
>> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > > >> psmouse   135168  0
>> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>> > > >
>> > > >It looks like the driver is actually reloaded.  The driver used
>to
>> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding
>the
>> > > >kernel to test that right now).
>> > > >
>> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>> > > >else cause the driver to be loaded again? 
>> > > 
>> > > It depends on how silly the udev rules are, but yes, this can
>definitely happen.  
>> > 
>> > I confirmed the driver doesn't use EPROBE_DEFER any more:
>> > 
>> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
>> > $  
>> 
>> Not sure why you bring the deferrals here, they have nothing to do
>with
>> module removal. Also, deferrals are rarely issued by the leaf driver,
>and
>> more often by providers of resources (GPIO, regulator, interrupt,
>etc).
>
>Yes, it's unusual, but this driver used to do it.  Which is exactly why
>I brought it up.  Turns out it was irrelevant :)
>
>> > I tested without any udev rules in /etc/udev/, just the standard
>distro
>> > ones.  Same thing.  
>> 
>> Right, so this is the default udev rule:
>> 
>> /lib/udev/rules.d/80-drivers.rules:
>> 
>> # do not edit this file, it will be overwritten on update
>> 
>> ACTION=="remove", GOTO="drivers_end"
>> 
>> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
>> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load
>tifm_sd"
>> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load
>tifm_ms"
>> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
>> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
>> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load
>ppdev"
>> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load
>$env{MODALIAS}"
>> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
>> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load
>sm_ftl"
>> 
>> LABEL="drivers_end"
>> 
>> So udev (and systemd) want to load kernel module on any action
>besides
>> device removal. Shortsighted decision I'd say. I'll send a patch to
>> systemd, in the mean time you can simply adjust your local rule to
>read
>> 
>> ACTION!="add", GOTO="drivers_end"
>
>Mm.  That is a silly thing.  You will break a lot of setups, though.

I think the priority it to have module loading working properly, and for most 
users once module is loaded it stays loaded. Unloading is mostly for developers.

Luckily newer systemd versions drop events they do not recognize, so exposure 
is even smaller.


Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > > wrote:  
> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski 
> > > >wrote:
> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> > > >> >> Hi!
> > > >> >>
> > > >> >> I'm having trouble with modules on linux/master.  rmmod succeeds
> > > >but the
> > > >> >> module is still loaded and the refcount goes to 1:
> > > >> >>
> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> > > >> >>   lsmod | grep nfp; \
> > > >> >>   rmmod nfp; \
> > > >> >>   lsmod | grep nfp
> > > >> >> nfp   249856  0
> > > >> >> nfp   200704  1
> > > >> >>
> > > >> >> If I rmmod again the module will be actually unloaded.  The user
> > > >space
> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to
> > > >bisect
> > > >> >> now...  
> > > >> >
> > > >> > Got 'em!
> > > >> >
> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> > > >refs/bisect/bad)
> > > >> > Author: Dmitry Torokhov 
> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > > >> >
> > > >> > driver core: emit uevents when device is bound to a driver  
> > > >> 
> > > >> Does it happen with all modules or only nfp one?
> > > >> 
> > > >> It seems to work here:
> > > >> 
> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> psmouse   135168  0
> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> > > >
> > > >It looks like the driver is actually reloaded.  The driver used to
> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
> > > >kernel to test that right now).
> > > >
> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> > > >else cause the driver to be loaded again? 
> > > 
> > > It depends on how silly the udev rules are, but yes, this can definitely 
> > > happen.  
> > 
> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > 
> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > $  
> 
> Not sure why you bring the deferrals here, they have nothing to do with
> module removal. Also, deferrals are rarely issued by the leaf driver, and
> more often by providers of resources (GPIO, regulator, interrupt, etc).

Yes, it's unusual, but this driver used to do it.  Which is exactly why
I brought it up.  Turns out it was irrelevant :)

> > I tested without any udev rules in /etc/udev/, just the standard distro
> > ones.  Same thing.  
> 
> Right, so this is the default udev rule:
> 
> /lib/udev/rules.d/80-drivers.rules:
> 
> # do not edit this file, it will be overwritten on update
> 
> ACTION=="remove", GOTO="drivers_end"
> 
> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load tifm_sd"
> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load tifm_ms"
> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load ppdev"
> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load 
> $env{MODALIAS}"
> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load sm_ftl"
> 
> LABEL="drivers_end"
> 
> So udev (and systemd) want to load kernel module on any action besides
> device removal. Shortsighted decision I'd say. I'll send a patch to
> systemd, in the mean time you can simply adjust your local rule to read
> 
> ACTION!="add", GOTO="drivers_end"

Mm.  That is a silly thing.  You will break a lot of setups, though.

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sun, 10 Sep 2017 09:21:11 -0700, Dmitry Torokhov wrote:
> On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> > On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:  
> > > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  
> > > wrote:  
> > > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> > > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski 
> > > >wrote:
> > > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> > > >> >> Hi!
> > > >> >>
> > > >> >> I'm having trouble with modules on linux/master.  rmmod succeeds
> > > >but the
> > > >> >> module is still loaded and the refcount goes to 1:
> > > >> >>
> > > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> > > >> >>   lsmod | grep nfp; \
> > > >> >>   rmmod nfp; \
> > > >> >>   lsmod | grep nfp
> > > >> >> nfp   249856  0
> > > >> >> nfp   200704  1
> > > >> >>
> > > >> >> If I rmmod again the module will be actually unloaded.  The user
> > > >space
> > > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to
> > > >bisect
> > > >> >> now...  
> > > >> >
> > > >> > Got 'em!
> > > >> >
> > > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
> > > >refs/bisect/bad)
> > > >> > Author: Dmitry Torokhov 
> > > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > > >> >
> > > >> > driver core: emit uevents when device is bound to a driver  
> > > >> 
> > > >> Does it happen with all modules or only nfp one?
> > > >> 
> > > >> It seems to work here:
> > > >> 
> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> psmouse   135168  0
> > > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
> > > >
> > > >It looks like the driver is actually reloaded.  The driver used to
> > > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
> > > >kernel to test that right now).
> > > >
> > > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> > > >else cause the driver to be loaded again? 
> > > 
> > > It depends on how silly the udev rules are, but yes, this can definitely 
> > > happen.  
> > 
> > I confirmed the driver doesn't use EPROBE_DEFER any more:
> > 
> > $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> > $  
> 
> Not sure why you bring the deferrals here, they have nothing to do with
> module removal. Also, deferrals are rarely issued by the leaf driver, and
> more often by providers of resources (GPIO, regulator, interrupt, etc).

Yes, it's unusual, but this driver used to do it.  Which is exactly why
I brought it up.  Turns out it was irrelevant :)

> > I tested without any udev rules in /etc/udev/, just the standard distro
> > ones.  Same thing.  
> 
> Right, so this is the default udev rule:
> 
> /lib/udev/rules.d/80-drivers.rules:
> 
> # do not edit this file, it will be overwritten on update
> 
> ACTION=="remove", GOTO="drivers_end"
> 
> ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load tifm_sd"
> SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load tifm_ms"
> SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
> SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
> SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load ppdev"
> SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load 
> $env{MODALIAS}"
> SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
> KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load sm_ftl"
> 
> LABEL="drivers_end"
> 
> So udev (and systemd) want to load kernel module on any action besides
> device removal. Shortsighted decision I'd say. I'll send a patch to
> systemd, in the mean time you can simply adjust your local rule to read
> 
> ACTION!="add", GOTO="drivers_end"

Mm.  That is a silly thing.  You will break a lot of setups, though.

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  wrote:
> > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski   
> > >wrote:  
> > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> > >> >> Hi!
> > >> >>
> > >> >> I'm having trouble with modules on linux/master.  rmmod succeeds  
> > >but the  
> > >> >> module is still loaded and the refcount goes to 1:
> > >> >>
> > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> > >> >>   lsmod | grep nfp; \
> > >> >>   rmmod nfp; \
> > >> >>   lsmod | grep nfp
> > >> >> nfp   249856  0
> > >> >> nfp   200704  1
> > >> >>
> > >> >> If I rmmod again the module will be actually unloaded.  The user  
> > >space  
> > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to  
> > >bisect  
> > >> >> now...
> > >> >
> > >> > Got 'em!
> > >> >
> > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> > >refs/bisect/bad)  
> > >> > Author: Dmitry Torokhov 
> > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > >> >
> > >> > driver core: emit uevents when device is bound to a driver
> > >> 
> > >> Does it happen with all modules or only nfp one?
> > >> 
> > >> It seems to work here:
> > >> 
> > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> psmouse   135168  0
> > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> > >
> > >It looks like the driver is actually reloaded.  The driver used to
> > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
> > >kernel to test that right now).
> > >
> > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> > >else cause the driver to be loaded again?   
> > 
> > It depends on how silly the udev rules are, but yes, this can definitely 
> > happen.
> 
> I confirmed the driver doesn't use EPROBE_DEFER any more:
> 
> $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> $

Not sure why you bring the deferrals here, they have nothing to do with
module removal. Also, deferrals are rarely issued by the leaf driver, and
more often by providers of resources (GPIO, regulator, interrupt, etc).

> 
> I tested without any udev rules in /etc/udev/, just the standard distro
> ones.  Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load tifm_sd"
SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load tifm_ms"
SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load ppdev"
SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load sm_ftl"

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action besides
device removal. Shortsighted decision I'd say. I'll send a patch to
systemd, in the mean time you can simply adjust your local rule to read

ACTION!="add", GOTO="drivers_end"

Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sun, Sep 10, 2017 at 12:03:38AM +0200, Jakub Kicinski wrote:
> On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> > On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  wrote:
> > >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> > >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski   
> > >wrote:  
> > >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> > >> >> Hi!
> > >> >>
> > >> >> I'm having trouble with modules on linux/master.  rmmod succeeds  
> > >but the  
> > >> >> module is still loaded and the refcount goes to 1:
> > >> >>
> > >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> > >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> > >> >>   lsmod | grep nfp; \
> > >> >>   rmmod nfp; \
> > >> >>   lsmod | grep nfp
> > >> >> nfp   249856  0
> > >> >> nfp   200704  1
> > >> >>
> > >> >> If I rmmod again the module will be actually unloaded.  The user  
> > >space  
> > >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to  
> > >bisect  
> > >> >> now...
> > >> >
> > >> > Got 'em!
> > >> >
> > >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> > >refs/bisect/bad)  
> > >> > Author: Dmitry Torokhov 
> > >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> > >> >
> > >> > driver core: emit uevents when device is bound to a driver
> > >> 
> > >> Does it happen with all modules or only nfp one?
> > >> 
> > >> It seems to work here:
> > >> 
> > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> psmouse   135168  0
> > >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> > >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> > >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> > >
> > >It looks like the driver is actually reloaded.  The driver used to
> > >return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
> > >kernel to test that right now).
> > >
> > >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> > >else cause the driver to be loaded again?   
> > 
> > It depends on how silly the udev rules are, but yes, this can definitely 
> > happen.
> 
> I confirmed the driver doesn't use EPROBE_DEFER any more:
> 
> $ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
> $

Not sure why you bring the deferrals here, they have nothing to do with
module removal. Also, deferrals are rarely issued by the leaf driver, and
more often by providers of resources (GPIO, regulator, interrupt, etc).

> 
> I tested without any udev rules in /etc/udev/, just the standard distro
> ones.  Same thing.

Right, so this is the default udev rule:

/lib/udev/rules.d/80-drivers.rules:

# do not edit this file, it will be overwritten on update

ACTION=="remove", GOTO="drivers_end"

ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="SD", RUN{builtin}="kmod load tifm_sd"
SUBSYSTEM=="tifm", ENV{TIFM_CARD_TYPE}=="MS", RUN{builtin}="kmod load tifm_ms"
SUBSYSTEM=="memstick", RUN{builtin}="kmod load ms_block mspro_block"
SUBSYSTEM=="i2o", RUN{builtin}="kmod load i2o_block"
SUBSYSTEM=="module", KERNEL=="parport_pc", RUN{builtin}="kmod load ppdev"
SUBSYSTEM=="serio", ENV{MODALIAS}=="?*", RUN{builtin}="kmod load $env{MODALIAS}"
SUBSYSTEM=="graphics", RUN{builtin}="kmod load fbcon"
KERNEL=="mtd*ro", ENV{MTD_FTL}=="smartmedia", RUN{builtin}="kmod load sm_ftl"

LABEL="drivers_end"

So udev (and systemd) want to load kernel module on any action besides
device removal. Shortsighted decision I'd say. I'll send a patch to
systemd, in the mean time you can simply adjust your local rule to read

ACTION!="add", GOTO="drivers_end"

Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  wrote:
> >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski   
> >wrote:  
> >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> >> >> Hi!
> >> >>
> >> >> I'm having trouble with modules on linux/master.  rmmod succeeds  
> >but the  
> >> >> module is still loaded and the refcount goes to 1:
> >> >>
> >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> >>   lsmod | grep nfp; \
> >> >>   rmmod nfp; \
> >> >>   lsmod | grep nfp
> >> >> nfp   249856  0
> >> >> nfp   200704  1
> >> >>
> >> >> If I rmmod again the module will be actually unloaded.  The user  
> >space  
> >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to  
> >bisect  
> >> >> now...
> >> >
> >> > Got 'em!
> >> >
> >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> >refs/bisect/bad)  
> >> > Author: Dmitry Torokhov 
> >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> >
> >> > driver core: emit uevents when device is bound to a driver
> >> 
> >> Does it happen with all modules or only nfp one?
> >> 
> >> It seems to work here:
> >> 
> >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> psmouse   135168  0
> >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> >
> >It looks like the driver is actually reloaded.  The driver used to
> >return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
> >kernel to test that right now).
> >
> >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >else cause the driver to be loaded again?   
> 
> It depends on how silly the udev rules are, but yes, this can definitely 
> happen.

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

I tested without any udev rules in /etc/udev/, just the standard distro
ones.  Same thing.

Please find attached the logs from rmmod with kobject debug enabled.
[  155.670272] bus: 'pci': remove driver nfp_netvf
[  155.670398] kobject: 'nfp_netvf' (8807384510a8): kobject_cleanup, parent 88045331bbe8
[  155.670405] kobject: 'nfp_netvf' (8807384510a8): auto cleanup 'remove' event
[  155.670412] kobject: 'nfp_netvf' (8807384510a8): kobject_uevent_env
[  155.670480] kobject: 'nfp_netvf' (8807384510a8): fill_kobj_path: path = '/bus/pci/drivers/nfp_netvf'
[  155.670587] kobject: 'nfp_netvf' (8807384510a8): auto cleanup kobject_del
[  155.670620] kobject: 'nfp_netvf' (8807384510a8): calling ktype release
[  155.670627] driver: 'nfp_netvf': driver_release
[  155.670642] kobject: 'nfp_netvf': free name
[  155.670727] bus: 'pci': remove driver nfp
[  155.670795] device: 'hwmon0': device_unregister
[  155.671127] PM: Removing info for No Bus:hwmon0
[  155.671144] kobject: 'hwmon0' (88075ce44b20): kobject_uevent_env
[  155.671219] kobject: 'hwmon0' (88075ce44b20): fill_kobj_path: path = '/devices/pci:80/:80:02.0/:82:00.0/hwmon/hwmon0'
[  155.671329] kobject: 'hwmon' (88074e448fc8): kobject_cleanup, parent 880751b480b8
[  155.671336] kobject: 'hwmon' (88074e448fc8): auto cleanup kobject_del
[  155.671369] kobject: 'hwmon' (88074e448fc8): calling ktype release
[  155.671395] kobject: 'hwmon': free name
[  155.671403] kobject: 'hwmon0' (88075ce44b20): kobject_cleanup, parent   (null)
[  155.671409] kobject: 'hwmon0' (88075ce44b20): calling ktype release
[  155.671438] kobject: 'hwmon0': free name
[  156.489986] kobject: 'rx-31' (88074867f7d8): kobject_cleanup, parent 88075401a048
[  156.490002] kobject: 'rx-31' (88074867f7d8): auto cleanup 'remove' event
[  156.490009] kobject: 'rx-31' (88074867f7d8): kobject_uevent_env
[  156.490102] kobject: 'rx-31' (88074867f7d8): fill_kobj_path: path = '/devices/pci:80/:80:02.0/:82:00.0/net/p4p1/queues/rx-31'
[  156.490557] kobject: 'rx-31' (88074867f7d8): auto cleanup kobject_del
[  156.490790] kobject: 'rx-31' (88074867f7d8): calling ktype release
[  156.490813] kobject: 'rx-31': free name
[  156.490870] kobject: 'rx-30' (88074867f758): kobject_cleanup, parent 88075401a048
[  156.490877] kobject: 'rx-30' (88074867f758): auto cleanup 'remove' event
[  156.490884] kobject: 'rx-30' (88074867f758): kobject_uevent_env
[  156.490992] kobject: 'rx-30' (88074867f758): fill_kobj_path: path = '/devices/pci:80/:80:02.0/:82:00.0/net/p4p1/queues/rx-30'
[  156.491124] kobject: 'rx-30' (88074867f758): auto cleanup kobject_del
[  156.491208] kobject: 'rx-30' (88074867f758): calling ktype release
[  156.491214] 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sat, 09 Sep 2017 13:59:25 -0700, Dmitry Torokhov wrote:
> On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  wrote:
> >On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:  
> >> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski   
> >wrote:  
> >> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
> >> >> Hi!
> >> >>
> >> >> I'm having trouble with modules on linux/master.  rmmod succeeds  
> >but the  
> >> >> module is still loaded and the refcount goes to 1:
> >> >>
> >> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >> >>   lsmod | grep nfp; \
> >> >>   rmmod nfp; \
> >> >>   lsmod | grep nfp
> >> >> nfp   249856  0
> >> >> nfp   200704  1
> >> >>
> >> >> If I rmmod again the module will be actually unloaded.  The user  
> >space  
> >> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to  
> >bisect  
> >> >> now...
> >> >
> >> > Got 'em!
> >> >
> >> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,  
> >refs/bisect/bad)  
> >> > Author: Dmitry Torokhov 
> >> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >> >
> >> > driver core: emit uevents when device is bound to a driver
> >> 
> >> Does it happen with all modules or only nfp one?
> >> 
> >> It seems to work here:
> >> 
> >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> psmouse   135168  0
> >> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> >> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> >> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse  
> >
> >It looks like the driver is actually reloaded.  The driver used to
> >return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
> >kernel to test that right now).
> >
> >Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
> >else cause the driver to be loaded again?   
> 
> It depends on how silly the udev rules are, but yes, this can definitely 
> happen.

I confirmed the driver doesn't use EPROBE_DEFER any more:

$ grep -nrI EPROBE_DEFER drivers/net/ethernet/netronome/
$

I tested without any udev rules in /etc/udev/, just the standard distro
ones.  Same thing.

Please find attached the logs from rmmod with kobject debug enabled.
[  155.670272] bus: 'pci': remove driver nfp_netvf
[  155.670398] kobject: 'nfp_netvf' (8807384510a8): kobject_cleanup, parent 88045331bbe8
[  155.670405] kobject: 'nfp_netvf' (8807384510a8): auto cleanup 'remove' event
[  155.670412] kobject: 'nfp_netvf' (8807384510a8): kobject_uevent_env
[  155.670480] kobject: 'nfp_netvf' (8807384510a8): fill_kobj_path: path = '/bus/pci/drivers/nfp_netvf'
[  155.670587] kobject: 'nfp_netvf' (8807384510a8): auto cleanup kobject_del
[  155.670620] kobject: 'nfp_netvf' (8807384510a8): calling ktype release
[  155.670627] driver: 'nfp_netvf': driver_release
[  155.670642] kobject: 'nfp_netvf': free name
[  155.670727] bus: 'pci': remove driver nfp
[  155.670795] device: 'hwmon0': device_unregister
[  155.671127] PM: Removing info for No Bus:hwmon0
[  155.671144] kobject: 'hwmon0' (88075ce44b20): kobject_uevent_env
[  155.671219] kobject: 'hwmon0' (88075ce44b20): fill_kobj_path: path = '/devices/pci:80/:80:02.0/:82:00.0/hwmon/hwmon0'
[  155.671329] kobject: 'hwmon' (88074e448fc8): kobject_cleanup, parent 880751b480b8
[  155.671336] kobject: 'hwmon' (88074e448fc8): auto cleanup kobject_del
[  155.671369] kobject: 'hwmon' (88074e448fc8): calling ktype release
[  155.671395] kobject: 'hwmon': free name
[  155.671403] kobject: 'hwmon0' (88075ce44b20): kobject_cleanup, parent   (null)
[  155.671409] kobject: 'hwmon0' (88075ce44b20): calling ktype release
[  155.671438] kobject: 'hwmon0': free name
[  156.489986] kobject: 'rx-31' (88074867f7d8): kobject_cleanup, parent 88075401a048
[  156.490002] kobject: 'rx-31' (88074867f7d8): auto cleanup 'remove' event
[  156.490009] kobject: 'rx-31' (88074867f7d8): kobject_uevent_env
[  156.490102] kobject: 'rx-31' (88074867f7d8): fill_kobj_path: path = '/devices/pci:80/:80:02.0/:82:00.0/net/p4p1/queues/rx-31'
[  156.490557] kobject: 'rx-31' (88074867f7d8): auto cleanup kobject_del
[  156.490790] kobject: 'rx-31' (88074867f7d8): calling ktype release
[  156.490813] kobject: 'rx-31': free name
[  156.490870] kobject: 'rx-30' (88074867f758): kobject_cleanup, parent 88075401a048
[  156.490877] kobject: 'rx-30' (88074867f758): auto cleanup 'remove' event
[  156.490884] kobject: 'rx-30' (88074867f758): kobject_uevent_env
[  156.490992] kobject: 'rx-30' (88074867f758): fill_kobj_path: path = '/devices/pci:80/:80:02.0/:82:00.0/net/p4p1/queues/rx-30'
[  156.491124] kobject: 'rx-30' (88074867f758): auto cleanup kobject_del
[  156.491208] kobject: 'rx-30' (88074867f758): calling ktype release
[  156.491214] kobject: 'rx-30': free name
[  156.491235] kobject: 'rx-29' 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  wrote:
>On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski 
>wrote:
>> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
>> >> Hi!
>> >>
>> >> I'm having trouble with modules on linux/master.  rmmod succeeds
>but the
>> >> module is still loaded and the refcount goes to 1:
>> >>
>> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> >>   lsmod | grep nfp; \
>> >>   rmmod nfp; \
>> >>   lsmod | grep nfp
>> >> nfp   249856  0
>> >> nfp   200704  1
>> >>
>> >> If I rmmod again the module will be actually unloaded.  The user
>space
>> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to
>bisect
>> >> now...  
>> >
>> > Got 'em!
>> >
>> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>refs/bisect/bad)
>> > Author: Dmitry Torokhov 
>> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> >
>> > driver core: emit uevents when device is bound to a driver  
>> 
>> Does it happen with all modules or only nfp one?
>> 
>> It seems to work here:
>> 
>> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> psmouse   135168  0
>> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>
>It looks like the driver is actually reloaded.  The driver used to
>return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
>kernel to test that right now).
>
>Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>else cause the driver to be loaded again? 

It depends on how silly the udev rules are, but yes, this can definitely happen.


Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On September 9, 2017 1:17:26 PM PDT, Jakub Kicinski  wrote:
>On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
>> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski 
>wrote:
>> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
>> >> Hi!
>> >>
>> >> I'm having trouble with modules on linux/master.  rmmod succeeds
>but the
>> >> module is still loaded and the refcount goes to 1:
>> >>
>> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>> >>   lsmod | grep nfp; \
>> >>   rmmod nfp; \
>> >>   lsmod | grep nfp
>> >> nfp   249856  0
>> >> nfp   200704  1
>> >>
>> >> If I rmmod again the module will be actually unloaded.  The user
>space
>> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to
>bisect
>> >> now...  
>> >
>> > Got 'em!
>> >
>> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD,
>refs/bisect/bad)
>> > Author: Dmitry Torokhov 
>> > Date:   Wed Jul 19 17:24:30 2017 -0700
>> >
>> > driver core: emit uevents when device is bound to a driver  
>> 
>> Does it happen with all modules or only nfp one?
>> 
>> It seems to work here:
>> 
>> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> psmouse   135168  0
>> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
>> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
>> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse
>
>It looks like the driver is actually reloaded.  The driver used to
>return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
>kernel to test that right now).
>
>Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
>else cause the driver to be loaded again? 

It depends on how silly the udev rules are, but yes, this can definitely happen.


Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  wrote:
> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> >> Hi!
> >>
> >> I'm having trouble with modules on linux/master.  rmmod succeeds but the
> >> module is still loaded and the refcount goes to 1:
> >>
> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >>   lsmod | grep nfp; \
> >>   rmmod nfp; \
> >>   lsmod | grep nfp
> >> nfp   249856  0
> >> nfp   200704  1
> >>
> >> If I rmmod again the module will be actually unloaded.  The user space
> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to bisect
> >> now...  
> >
> > Got 'em!
> >
> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)
> > Author: Dmitry Torokhov 
> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >
> > driver core: emit uevents when device is bound to a driver  
> 
> Does it happen with all modules or only nfp one?
> 
> It seems to work here:
> 
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> psmouse   135168  0
> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded.  The driver used to
return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
else cause the driver to be loaded again? 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  wrote:
> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> >> Hi!
> >>
> >> I'm having trouble with modules on linux/master.  rmmod succeeds but the
> >> module is still loaded and the refcount goes to 1:
> >>
> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >>   lsmod | grep nfp; \
> >>   rmmod nfp; \
> >>   lsmod | grep nfp
> >> nfp   249856  0
> >> nfp   200704  1
> >>
> >> If I rmmod again the module will be actually unloaded.  The user space
> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to bisect
> >> now...  
> >
> > Got 'em!
> >
> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)
> > Author: Dmitry Torokhov 
> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >
> > driver core: emit uevents when device is bound to a driver  
> 
> Does it happen with all modules or only nfp one?
> 
> It seems to work here:
> 
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> psmouse   135168  0
> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

It looks like the driver is actually reloaded.  The driver used to
return EPROBE_DEFER, but I think it doesn't any more (rebuilding the
kernel to test that right now).

Could the uevent on unbind tickle Ubuntu 14.04's udev or somehow
else cause the driver to be loaded again? 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  wrote:
> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> >> Hi!
> >>
> >> I'm having trouble with modules on linux/master.  rmmod succeeds but the
> >> module is still loaded and the refcount goes to 1:
> >>
> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >>   lsmod | grep nfp; \
> >>   rmmod nfp; \
> >>   lsmod | grep nfp
> >> nfp   249856  0
> >> nfp   200704  1
> >>
> >> If I rmmod again the module will be actually unloaded.  The user space
> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to bisect
> >> now...  
> >
> > Got 'em!
> >
> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)
> > Author: Dmitry Torokhov 
> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >
> > driver core: emit uevents when device is bound to a driver  
> 
> Does it happen with all modules or only nfp one?
> 
> It seems to work here:
> 
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> psmouse   135168  0
> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Yes, and only if FW is loaded/requested successfully.  Hmm... 

I'm building a kernel with more debug, any particular suggestions? 

Re: [bisected] Re: Module removal-related regression?

[Jakub Kicinski]

On Sat, 9 Sep 2017 12:55:51 -0700, Dmitry Torokhov wrote:
> On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  wrote:
> > On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:  
> >> Hi!
> >>
> >> I'm having trouble with modules on linux/master.  rmmod succeeds but the
> >> module is still loaded and the refcount goes to 1:
> >>
> >> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
> >>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
> >>   lsmod | grep nfp; \
> >>   rmmod nfp; \
> >>   lsmod | grep nfp
> >> nfp   249856  0
> >> nfp   200704  1
> >>
> >> If I rmmod again the module will be actually unloaded.  The user space
> >> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to bisect
> >> now...  
> >
> > Got 'em!
> >
> > commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)
> > Author: Dmitry Torokhov 
> > Date:   Wed Jul 19 17:24:30 2017 -0700
> >
> > driver core: emit uevents when device is bound to a driver  
> 
> Does it happen with all modules or only nfp one?
> 
> It seems to work here:
> 
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> psmouse   135168  0
> dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
> dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
> dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Yes, and only if FW is loaded/requested successfully.  Hmm... 

I'm building a kernel with more debug, any particular suggestions? 

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  wrote:
> On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
>> Hi!
>>
>> I'm having trouble with modules on linux/master.  rmmod succeeds but the
>> module is still loaded and the refcount goes to 1:
>>
>> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>>   lsmod | grep nfp; \
>>   rmmod nfp; \
>>   lsmod | grep nfp
>> nfp   249856  0
>> nfp   200704  1
>>
>> If I rmmod again the module will be actually unloaded.  The user space
>> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to bisect
>> now...
>
> Got 'em!
>
> commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)
> Author: Dmitry Torokhov 
> Date:   Wed Jul 19 17:24:30 2017 -0700
>
> driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse   135168  0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Thanks.

-- 
Dmitry

Re: [bisected] Re: Module removal-related regression?

[Dmitry Torokhov]

On Sat, Sep 9, 2017 at 12:27 PM, Jakub Kicinski  wrote:
> On Sat, 9 Sep 2017 19:41:21 +0200, Jakub Kicinski wrote:
>> Hi!
>>
>> I'm having trouble with modules on linux/master.  rmmod succeeds but the
>> module is still loaded and the refcount goes to 1:
>>
>> #rmmod nfp; insmod ./src/nfp.ko nfp_pf_netdev=0 ; \
>>   /opt/netronome/bin/nfp-hwinfo -n 2  assembly.partno \
>>   lsmod | grep nfp; \
>>   rmmod nfp; \
>>   lsmod | grep nfp
>> nfp   249856  0
>> nfp   200704  1
>>
>> If I rmmod again the module will be actually unloaded.  The user space
>> is mostly Ubuntu 14.04.  Has anyone seen this?  I'm trying to bisect
>> now...
>
> Got 'em!
>
> commit 1455cf8dbfd06aa7651dcfccbadb7a093944ca65 (HEAD, refs/bisect/bad)
> Author: Dmitry Torokhov 
> Date:   Wed Jul 19 17:24:30 2017 -0700
>
> driver core: emit uevents when device is bound to a driver

Does it happen with all modules or only nfp one?

It seems to work here:

dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
psmouse   135168  0
dtor@dtor-glaptop3:~ $ sudo rmmod psmouse
dtor@dtor-glaptop3:~ $ lsmod | grep psmouse
dtor@dtor-glaptop3:~ $ sudo modprobe psmouse

Thanks.

-- 
Dmitry