Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
On 12/12/2018 08:57 PM, Michael Ellerman wrote: > Frank Rowand writes: >> On 12/11/18 8:07 AM, Rob Herring wrote: >>> On Tue, Dec 11, 2018 at 7:29 AM Michael Ellerman >>> wrote: > ... diff --git a/drivers/of/base.c b/drivers/of/base.c index 09692c9b32a7..d8e4534c0686 100644 --- a/drivers/of/base.c +++ b/drivers/of/base.c @@ -1190,6 +1190,10 @@ struct device_node *of_find_node_by_phandle(phandle handle) if (phandle_cache[masked_handle] && handle == phandle_cache[masked_handle]->phandle) np = phandle_cache[masked_handle]; + + /* If we find a detached node, remove it */ + if (of_node_check_flag(np, OF_DETACHED)) + np = phandle_cache[masked_handle] = NULL; >> >> The bug you found exposes a couple of different issues, a little bit >> deeper than the proposed fix. I'll work on a fuller fix tonight or >> tomorrow. > > OK thanks. > >>> I'm wondering if we should explicitly remove the node from the cache >>> when we set OF_DETACHED. Otherwise, it could be possible that the node >>> pointer has been freed already. Or maybe we need both? >> >> Yes, it should be explicitly removed. I may also add in a paranoia check in >> of_find_node_by_phandle(). > > That seems best to me. I agree that we should do both. > > cheers Michael -- Michael W. Bringmann Linux I/O, Networking and Security Development IBM Corporation Tie-Line 363-5196 External: (512) 286-5196 Cell: (512) 466-0650 m...@linux.vnet.ibm.com
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
Frank Rowand writes: > On 12/11/18 8:07 AM, Rob Herring wrote: >> On Tue, Dec 11, 2018 at 7:29 AM Michael Ellerman wrote: ... >>> diff --git a/drivers/of/base.c b/drivers/of/base.c >>> index 09692c9b32a7..d8e4534c0686 100644 >>> --- a/drivers/of/base.c >>> +++ b/drivers/of/base.c >>> @@ -1190,6 +1190,10 @@ struct device_node *of_find_node_by_phandle(phandle >>> handle) >>> if (phandle_cache[masked_handle] && >>> handle == phandle_cache[masked_handle]->phandle) >>> np = phandle_cache[masked_handle]; >>> + >>> + /* If we find a detached node, remove it */ >>> + if (of_node_check_flag(np, OF_DETACHED)) >>> + np = phandle_cache[masked_handle] = NULL; > > The bug you found exposes a couple of different issues, a little bit > deeper than the proposed fix. I'll work on a fuller fix tonight or > tomorrow. OK thanks. >> I'm wondering if we should explicitly remove the node from the cache >> when we set OF_DETACHED. Otherwise, it could be possible that the node >> pointer has been freed already. Or maybe we need both? > > Yes, it should be explicitly removed. I may also add in a paranoia check in > of_find_node_by_phandle(). That seems best to me. cheers
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
Rob Herring writes: > On Tue, Dec 11, 2018 at 7:29 AM Michael Ellerman wrote: ... >> diff --git a/drivers/of/base.c b/drivers/of/base.c >> index 09692c9b32a7..d8e4534c0686 100644 >> --- a/drivers/of/base.c >> +++ b/drivers/of/base.c >> @@ -1190,6 +1190,10 @@ struct device_node *of_find_node_by_phandle(phandle >> handle) >> if (phandle_cache[masked_handle] && >> handle == phandle_cache[masked_handle]->phandle) >> np = phandle_cache[masked_handle]; >> + >> + /* If we find a detached node, remove it */ >> + if (of_node_check_flag(np, OF_DETACHED)) >> + np = phandle_cache[masked_handle] = NULL; > > I'm wondering if we should explicitly remove the node from the cache > when we set OF_DETACHED. Otherwise, it could be possible that the node > pointer has been freed already. Yeah good point. > Or maybe we need both? That's probably best, it could even be a WARN_ON() if we find one in of_find_node_by_phandle(). cheers
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
Hi Michael Bringmann, On 12/11/18 8:07 AM, Rob Herring wrote: > On Tue, Dec 11, 2018 at 7:29 AM Michael Ellerman wrote: >> >> Hi Michael, >> >> Please Cc the device tree folks on device tree patches, and also the >> original author of the patch that added the code you're modifying. >> >> So I've added: >> robh...@kernel.org >> frowand.l...@gmail.com >> devicet...@vger.kernel.org >> linux-ker...@vger.kernel.org >> >> Michael Bringmann writes: >>> The PPC mobility code receives RTAS requests to delete nodes with >>> platform-/hardware-specific attributes when restarting the kernel >>> after a migration. My example is for migration between a P8 Alpine >>> and a P8 Brazos. Nodes to be deleted include 'ibm,random-v1', >>> 'ibm,platform-facilities', 'ibm,sym-encryption-v1', and, >>> 'ibm,compression-v1'. >>> >>> The mobility.c code calls 'of_detach_node' for the nodes and their >>> children. This makes calls to detach the properties and to remove >>> the associated sysfs/kernfs files. >>> >>> Then new copies of the same nodes are next provided by the PHYP, >>> local copies are built, and a pointer to the 'struct device_node' >>> is passed to of_attach_node. Before the call to of_attach_node, >>> the phandle is initialized to 0 when the data structure is alloced. >>> During the call to of_attach_node, it calls __of_attach_node which >>> pulls the actual name and phandle from just created sub-properties >>> named something like 'name' and 'ibm,phandle'. >>> >>> This is all fine for the first migration. The problem occurs with >>> the second and subsequent migrations when the PHYP on the new system >>> wants to replace the same set of nodes again, referenced with the >>> same names and phandle values. >>> >>> On the second and subsequent migrations, the PHYP tells the system >>> to again delete the nodes 'ibm,platform-facilities', 'ibm,random-v1', >>> 'ibm,compression-v1', 'ibm,sym-encryption-v1'. It specifies these >>> nodes by its known set of phandle values -- the same handles used >>> by the PHYP on the source system are known on the target system. >>> The mobility.c code calls of_find_node_by_phandle() with these values >>> and ends up locating the first instance of each node that was added >>> during the original boot, instead of the second instance of each node >>> created after the first migration. The detach during the second >>> migration fails with errors like, >>> >>> [ 4565.030704] WARNING: CPU: 3 PID: 4787 at drivers/of/dynamic.c:252 >>> __of_detach_node+0x8/0xa0 >>> [ 4565.030708] Modules linked in: nfsv3 nfs_acl nfs tcp_diag udp_diag >>> inet_diag unix_diag af_packet_diag netlink_diag lockd grace fscache sunrpc >>> xts vmx_crypto sg pseries_rng binfmt_misc ip_tables xfs libcrc32c sd_mod >>> ibmveth ibmvscsi scsi_transport_srp dm_mirror dm_region_hash dm_log dm_mod >>> [ 4565.030733] CPU: 3 PID: 4787 Comm: drmgr Tainted: GW >>> 4.18.0-rc1-wi107836-v05-120+ #201 >>> [ 4565.030737] NIP: c07c1ea8 LR: c07c1fb4 CTR: >>> 00655170 >>> [ 4565.030741] REGS: c003f302b690 TRAP: 0700 Tainted: GW >>> (4.18.0-rc1-wi107836-v05-120+) >>> [ 4565.030745] MSR: 80010282b033 >>> CR: 22288822 XER: 000a >>> [ 4565.030757] CFAR: c07c1fb0 IRQMASK: 1 >>> [ 4565.030757] GPR00: c07c1fa4 c003f302b910 c114bf00 >>> c0038e68 >>> [ 4565.030757] GPR04: 0001 80c008e0b4b8 >>> >>> [ 4565.030757] GPR08: 0001 8003 >>> 2843 >>> [ 4565.030757] GPR12: 8800 c0001ec9ae00 4000 >>> >>> [ 4565.030757] GPR16: 0008 >>> f6ff >>> [ 4565.030757] GPR20: 0007 c003e9f1f034 >>> 0001 >>> [ 4565.030757] GPR24: >>> >>> [ 4565.030757] GPR28: c1549d28 c1134828 c0038e68 >>> c003f302b930 >>> [ 4565.030804] NIP [c07c1ea8] __of_detach_node+0x8/0xa0 >>> [ 4565.030808] LR [c07c1fb4] of_detach_node+0x74/0xd0 >>> [ 4565.030811] Call Trace: >>> [ 4565.030815] [c003f302b910] [c07c1fa4] >>> of_detach_node+0x64/0xd0 (unreliable) >>> [ 4565.030821] [c003f302b980] [c00c33c4] >>> dlpar_detach_node+0xb4/0x150 >>> [ 4565.030826] [c003f302ba10] [c00c3ffc] >>> delete_dt_node+0x3c/0x80 >>> [ 4565.030831] [c003f302ba40] [c00c4380] >>> pseries_devicetree_update+0x150/0x4f0 >>> [ 4565.030836] [c003f302bb70] [c00c479c] >>> post_mobility_fixup+0x7c/0xf0 >>> [ 4565.030841] [c003f302bbe0] [c00c4908] >>> migration_store+0xf8/0x130 >>> [ 4565.030847] [c003f302bc70] [c0998160] >>> kobj_attr_store+0x30/0x60 >>> [ 4565.030852] [c003f302bc90] [c0412f14] >>> sysfs_kf_write+0x64/0xa0 >>> [
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
--- Snip ---
>>
>> The mobility.c code continues on during the second migration, accepts
>> the definitions of the new nodes from the PHYP and ends up renaming
>> the new properties e.g.
>>
>> [ 4565.827296] Duplicate name in base, renamed to "ibm,platform-facilities#1"
>>
>> There is no check like 'of_node_check_flag(np, OF_DETACHED)' within
>> of_find_node_by_phandle to skip nodes that are detached, but still
>> present due to caching or use count considerations. Also, note that
>> of_find_node_by_phandle also uses a 'phandle_cache' which does not
>> appear to be updated when of_detach_node() is invoked.
>
> This seems like the real bug. Since the phandle cache was added we can
> now find detached nodes when we shouldn't be able to.
>
> Does the patch below work?
>
> cheers
>
> diff --git a/drivers/of/base.c b/drivers/of/base.c
> index 09692c9b32a7..d8e4534c0686 100644
> --- a/drivers/of/base.c
> +++ b/drivers/of/base.c
> @@ -1190,6 +1190,10 @@ struct device_node *of_find_node_by_phandle(phandle
> handle)
> if (phandle_cache[masked_handle] &&
> handle == phandle_cache[masked_handle]->phandle)
> np = phandle_cache[masked_handle];
> +
> + /* If we find a detached node, remove it */
> + if (of_node_check_flag(np, OF_DETACHED))
> + np = phandle_cache[masked_handle] = NULL;
> }
>
> if (!np) {
>
>
I think this would be a bit better for cases where masked values overlap:
diff --git a/drivers/of/base.c b/drivers/of/base.c
index 09692c9..ec79129 100644
--- a/drivers/of/base.c
+++ b/drivers/of/base.c
@@ -1188,8 +1188,13 @@ struct device_node *of_find_node_by_phandle(phandle
handle)
if (phandle_cache) {
if (phandle_cache[masked_handle] &&
- handle == phandle_cache[masked_handle]->phandle)
- np = phandle_cache[masked_handle];
+ handle == phandle_cache[masked_handle]->phandle) {
+ np = phandle_cache[masked_handle];
+
+ /* If we find a detached node, remove it */
+ if (of_node_check_flag(np, OF_DETACHED))
+ np = phandle_cache[masked_handle] = NULL;
+ }
}
if (!np) {
Will try it out. Wouldn't it be better to do this when the node is detached
in drivers/of/dynamic.c:__of_detach_node()?
Thanks.
Michael
--
Michael W. Bringmann
Linux Technology Center
IBM Corporation
Tie-Line 363-5196
External: (512) 286-5196
Cell: (512) 466-0650
m...@linux.vnet.ibm.com
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
On Tue, Dec 11, 2018 at 7:29 AM Michael Ellerman wrote: > > Hi Michael, > > Please Cc the device tree folks on device tree patches, and also the > original author of the patch that added the code you're modifying. > > So I've added: > robh...@kernel.org > frowand.l...@gmail.com > devicet...@vger.kernel.org > linux-ker...@vger.kernel.org > > Michael Bringmann writes: > > The PPC mobility code receives RTAS requests to delete nodes with > > platform-/hardware-specific attributes when restarting the kernel > > after a migration. My example is for migration between a P8 Alpine > > and a P8 Brazos. Nodes to be deleted include 'ibm,random-v1', > > 'ibm,platform-facilities', 'ibm,sym-encryption-v1', and, > > 'ibm,compression-v1'. > > > > The mobility.c code calls 'of_detach_node' for the nodes and their > > children. This makes calls to detach the properties and to remove > > the associated sysfs/kernfs files. > > > > Then new copies of the same nodes are next provided by the PHYP, > > local copies are built, and a pointer to the 'struct device_node' > > is passed to of_attach_node. Before the call to of_attach_node, > > the phandle is initialized to 0 when the data structure is alloced. > > During the call to of_attach_node, it calls __of_attach_node which > > pulls the actual name and phandle from just created sub-properties > > named something like 'name' and 'ibm,phandle'. > > > > This is all fine for the first migration. The problem occurs with > > the second and subsequent migrations when the PHYP on the new system > > wants to replace the same set of nodes again, referenced with the > > same names and phandle values. > > > > On the second and subsequent migrations, the PHYP tells the system > > to again delete the nodes 'ibm,platform-facilities', 'ibm,random-v1', > > 'ibm,compression-v1', 'ibm,sym-encryption-v1'. It specifies these > > nodes by its known set of phandle values -- the same handles used > > by the PHYP on the source system are known on the target system. > > The mobility.c code calls of_find_node_by_phandle() with these values > > and ends up locating the first instance of each node that was added > > during the original boot, instead of the second instance of each node > > created after the first migration. The detach during the second > > migration fails with errors like, > > > > [ 4565.030704] WARNING: CPU: 3 PID: 4787 at drivers/of/dynamic.c:252 > > __of_detach_node+0x8/0xa0 > > [ 4565.030708] Modules linked in: nfsv3 nfs_acl nfs tcp_diag udp_diag > > inet_diag unix_diag af_packet_diag netlink_diag lockd grace fscache sunrpc > > xts vmx_crypto sg pseries_rng binfmt_misc ip_tables xfs libcrc32c sd_mod > > ibmveth ibmvscsi scsi_transport_srp dm_mirror dm_region_hash dm_log dm_mod > > [ 4565.030733] CPU: 3 PID: 4787 Comm: drmgr Tainted: GW > > 4.18.0-rc1-wi107836-v05-120+ #201 > > [ 4565.030737] NIP: c07c1ea8 LR: c07c1fb4 CTR: > > 00655170 > > [ 4565.030741] REGS: c003f302b690 TRAP: 0700 Tainted: GW > > (4.18.0-rc1-wi107836-v05-120+) > > [ 4565.030745] MSR: 80010282b033 > > CR: 22288822 XER: 000a > > [ 4565.030757] CFAR: c07c1fb0 IRQMASK: 1 > > [ 4565.030757] GPR00: c07c1fa4 c003f302b910 c114bf00 > > c0038e68 > > [ 4565.030757] GPR04: 0001 80c008e0b4b8 > > > > [ 4565.030757] GPR08: 0001 8003 > > 2843 > > [ 4565.030757] GPR12: 8800 c0001ec9ae00 4000 > > > > [ 4565.030757] GPR16: 0008 > > f6ff > > [ 4565.030757] GPR20: 0007 c003e9f1f034 > > 0001 > > [ 4565.030757] GPR24: > > > > [ 4565.030757] GPR28: c1549d28 c1134828 c0038e68 > > c003f302b930 > > [ 4565.030804] NIP [c07c1ea8] __of_detach_node+0x8/0xa0 > > [ 4565.030808] LR [c07c1fb4] of_detach_node+0x74/0xd0 > > [ 4565.030811] Call Trace: > > [ 4565.030815] [c003f302b910] [c07c1fa4] > > of_detach_node+0x64/0xd0 (unreliable) > > [ 4565.030821] [c003f302b980] [c00c33c4] > > dlpar_detach_node+0xb4/0x150 > > [ 4565.030826] [c003f302ba10] [c00c3ffc] > > delete_dt_node+0x3c/0x80 > > [ 4565.030831] [c003f302ba40] [c00c4380] > > pseries_devicetree_update+0x150/0x4f0 > > [ 4565.030836] [c003f302bb70] [c00c479c] > > post_mobility_fixup+0x7c/0xf0 > > [ 4565.030841] [c003f302bbe0] [c00c4908] > > migration_store+0xf8/0x130 > > [ 4565.030847] [c003f302bc70] [c0998160] > > kobj_attr_store+0x30/0x60 > > [ 4565.030852] [c003f302bc90] [c0412f14] > > sysfs_kf_write+0x64/0xa0 > > [ 4565.030857] [c003f302bcb0] [c0411cac] > >
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
On 12/11/2018 07:29 AM, Michael Ellerman wrote: > Hi Michael, > > Please Cc the device tree folks on device tree patches, and also the > original author of the patch that added the code you're modifying. > > So I've added: > robh...@kernel.org > frowand.l...@gmail.com > devicet...@vger.kernel.org > linux-ker...@vger.kernel.org Thanks. > > Michael Bringmann writes: >> The PPC mobility code receives RTAS requests to delete nodes with >> platform-/hardware-specific attributes when restarting the kernel >> after a migration. My example is for migration between a P8 Alpine >> and a P8 Brazos. Nodes to be deleted include 'ibm,random-v1', >> 'ibm,platform-facilities', 'ibm,sym-encryption-v1', and, >> 'ibm,compression-v1'. >> >> The mobility.c code calls 'of_detach_node' for the nodes and their >> children. This makes calls to detach the properties and to remove >> the associated sysfs/kernfs files. >> >> Then new copies of the same nodes are next provided by the PHYP, >> local copies are built, and a pointer to the 'struct device_node' >> is passed to of_attach_node. Before the call to of_attach_node, >> the phandle is initialized to 0 when the data structure is alloced. >> During the call to of_attach_node, it calls __of_attach_node which >> pulls the actual name and phandle from just created sub-properties >> named something like 'name' and 'ibm,phandle'. >> >> This is all fine for the first migration. The problem occurs with >> the second and subsequent migrations when the PHYP on the new system >> wants to replace the same set of nodes again, referenced with the >> same names and phandle values. >> >> On the second and subsequent migrations, the PHYP tells the system >> to again delete the nodes 'ibm,platform-facilities', 'ibm,random-v1', >> 'ibm,compression-v1', 'ibm,sym-encryption-v1'. It specifies these >> nodes by its known set of phandle values -- the same handles used >> by the PHYP on the source system are known on the target system. >> The mobility.c code calls of_find_node_by_phandle() with these values >> and ends up locating the first instance of each node that was added >> during the original boot, instead of the second instance of each node >> created after the first migration. The detach during the second >> migration fails with errors like, >> >> [ 4565.030704] WARNING: CPU: 3 PID: 4787 at drivers/of/dynamic.c:252 >> __of_detach_node+0x8/0xa0 >> [ 4565.030708] Modules linked in: nfsv3 nfs_acl nfs tcp_diag udp_diag >> inet_diag unix_diag af_packet_diag netlink_diag lockd grace fscache sunrpc >> xts vmx_crypto sg pseries_rng binfmt_misc ip_tables xfs libcrc32c sd_mod >> ibmveth ibmvscsi scsi_transport_srp dm_mirror dm_region_hash dm_log dm_mod >> [ 4565.030733] CPU: 3 PID: 4787 Comm: drmgr Tainted: GW >> 4.18.0-rc1-wi107836-v05-120+ #201 >> [ 4565.030737] NIP: c07c1ea8 LR: c07c1fb4 CTR: >> 00655170 >> [ 4565.030741] REGS: c003f302b690 TRAP: 0700 Tainted: GW >>(4.18.0-rc1-wi107836-v05-120+) >> [ 4565.030745] MSR: 80010282b033 >> CR: 22288822 XER: 000a >> [ 4565.030757] CFAR: c07c1fb0 IRQMASK: 1 >> [ 4565.030757] GPR00: c07c1fa4 c003f302b910 c114bf00 >> c0038e68 >> [ 4565.030757] GPR04: 0001 80c008e0b4b8 >> >> [ 4565.030757] GPR08: 0001 8003 >> 2843 >> [ 4565.030757] GPR12: 8800 c0001ec9ae00 4000 >> >> [ 4565.030757] GPR16: 0008 >> f6ff >> [ 4565.030757] GPR20: 0007 c003e9f1f034 >> 0001 >> [ 4565.030757] GPR24: >> >> [ 4565.030757] GPR28: c1549d28 c1134828 c0038e68 >> c003f302b930 >> [ 4565.030804] NIP [c07c1ea8] __of_detach_node+0x8/0xa0 >> [ 4565.030808] LR [c07c1fb4] of_detach_node+0x74/0xd0 >> [ 4565.030811] Call Trace: >> [ 4565.030815] [c003f302b910] [c07c1fa4] >> of_detach_node+0x64/0xd0 (unreliable) >> [ 4565.030821] [c003f302b980] [c00c33c4] >> dlpar_detach_node+0xb4/0x150 >> [ 4565.030826] [c003f302ba10] [c00c3ffc] delete_dt_node+0x3c/0x80 >> [ 4565.030831] [c003f302ba40] [c00c4380] >> pseries_devicetree_update+0x150/0x4f0 >> [ 4565.030836] [c003f302bb70] [c00c479c] >> post_mobility_fixup+0x7c/0xf0 >> [ 4565.030841] [c003f302bbe0] [c00c4908] >> migration_store+0xf8/0x130 >> [ 4565.030847] [c003f302bc70] [c0998160] >> kobj_attr_store+0x30/0x60 >> [ 4565.030852] [c003f302bc90] [c0412f14] sysfs_kf_write+0x64/0xa0 >> [ 4565.030857] [c003f302bcb0] [c0411cac] >> kernfs_fop_write+0x16c/0x240 >> [ 4565.030862] [c003f302bd00] [c0355f20] __vfs_write+0x40/0x220 >> [
Re: [PATCH v03] powerpc/mobility: Fix node detach/rename problem
Hi Michael, Please Cc the device tree folks on device tree patches, and also the original author of the patch that added the code you're modifying. So I've added: robh...@kernel.org frowand.l...@gmail.com devicet...@vger.kernel.org linux-ker...@vger.kernel.org Michael Bringmann writes: > The PPC mobility code receives RTAS requests to delete nodes with > platform-/hardware-specific attributes when restarting the kernel > after a migration. My example is for migration between a P8 Alpine > and a P8 Brazos. Nodes to be deleted include 'ibm,random-v1', > 'ibm,platform-facilities', 'ibm,sym-encryption-v1', and, > 'ibm,compression-v1'. > > The mobility.c code calls 'of_detach_node' for the nodes and their > children. This makes calls to detach the properties and to remove > the associated sysfs/kernfs files. > > Then new copies of the same nodes are next provided by the PHYP, > local copies are built, and a pointer to the 'struct device_node' > is passed to of_attach_node. Before the call to of_attach_node, > the phandle is initialized to 0 when the data structure is alloced. > During the call to of_attach_node, it calls __of_attach_node which > pulls the actual name and phandle from just created sub-properties > named something like 'name' and 'ibm,phandle'. > > This is all fine for the first migration. The problem occurs with > the second and subsequent migrations when the PHYP on the new system > wants to replace the same set of nodes again, referenced with the > same names and phandle values. > > On the second and subsequent migrations, the PHYP tells the system > to again delete the nodes 'ibm,platform-facilities', 'ibm,random-v1', > 'ibm,compression-v1', 'ibm,sym-encryption-v1'. It specifies these > nodes by its known set of phandle values -- the same handles used > by the PHYP on the source system are known on the target system. > The mobility.c code calls of_find_node_by_phandle() with these values > and ends up locating the first instance of each node that was added > during the original boot, instead of the second instance of each node > created after the first migration. The detach during the second > migration fails with errors like, > > [ 4565.030704] WARNING: CPU: 3 PID: 4787 at drivers/of/dynamic.c:252 > __of_detach_node+0x8/0xa0 > [ 4565.030708] Modules linked in: nfsv3 nfs_acl nfs tcp_diag udp_diag > inet_diag unix_diag af_packet_diag netlink_diag lockd grace fscache sunrpc > xts vmx_crypto sg pseries_rng binfmt_misc ip_tables xfs libcrc32c sd_mod > ibmveth ibmvscsi scsi_transport_srp dm_mirror dm_region_hash dm_log dm_mod > [ 4565.030733] CPU: 3 PID: 4787 Comm: drmgr Tainted: GW > 4.18.0-rc1-wi107836-v05-120+ #201 > [ 4565.030737] NIP: c07c1ea8 LR: c07c1fb4 CTR: > 00655170 > [ 4565.030741] REGS: c003f302b690 TRAP: 0700 Tainted: GW > (4.18.0-rc1-wi107836-v05-120+) > [ 4565.030745] MSR: 80010282b033 > CR: 22288822 XER: 000a > [ 4565.030757] CFAR: c07c1fb0 IRQMASK: 1 > [ 4565.030757] GPR00: c07c1fa4 c003f302b910 c114bf00 > c0038e68 > [ 4565.030757] GPR04: 0001 80c008e0b4b8 > > [ 4565.030757] GPR08: 0001 8003 > 2843 > [ 4565.030757] GPR12: 8800 c0001ec9ae00 4000 > > [ 4565.030757] GPR16: 0008 > f6ff > [ 4565.030757] GPR20: 0007 c003e9f1f034 > 0001 > [ 4565.030757] GPR24: > > [ 4565.030757] GPR28: c1549d28 c1134828 c0038e68 > c003f302b930 > [ 4565.030804] NIP [c07c1ea8] __of_detach_node+0x8/0xa0 > [ 4565.030808] LR [c07c1fb4] of_detach_node+0x74/0xd0 > [ 4565.030811] Call Trace: > [ 4565.030815] [c003f302b910] [c07c1fa4] of_detach_node+0x64/0xd0 > (unreliable) > [ 4565.030821] [c003f302b980] [c00c33c4] > dlpar_detach_node+0xb4/0x150 > [ 4565.030826] [c003f302ba10] [c00c3ffc] delete_dt_node+0x3c/0x80 > [ 4565.030831] [c003f302ba40] [c00c4380] > pseries_devicetree_update+0x150/0x4f0 > [ 4565.030836] [c003f302bb70] [c00c479c] > post_mobility_fixup+0x7c/0xf0 > [ 4565.030841] [c003f302bbe0] [c00c4908] > migration_store+0xf8/0x130 > [ 4565.030847] [c003f302bc70] [c0998160] kobj_attr_store+0x30/0x60 > [ 4565.030852] [c003f302bc90] [c0412f14] sysfs_kf_write+0x64/0xa0 > [ 4565.030857] [c003f302bcb0] [c0411cac] > kernfs_fop_write+0x16c/0x240 > [ 4565.030862] [c003f302bd00] [c0355f20] __vfs_write+0x40/0x220 > [ 4565.030867] [c003f302bd90] [c0356358] vfs_write+0xc8/0x240 > [ 4565.030872] [c003f302bde0] [c03566cc] ksys_write+0x5c/0x100 > [ 4565.030880]
[PATCH v03] powerpc/mobility: Fix node detach/rename problem
The PPC mobility code receives RTAS requests to delete nodes with platform-/hardware-specific attributes when restarting the kernel after a migration. My example is for migration between a P8 Alpine and a P8 Brazos. Nodes to be deleted include 'ibm,random-v1', 'ibm,platform-facilities', 'ibm,sym-encryption-v1', and, 'ibm,compression-v1'. The mobility.c code calls 'of_detach_node' for the nodes and their children. This makes calls to detach the properties and to remove the associated sysfs/kernfs files. Then new copies of the same nodes are next provided by the PHYP, local copies are built, and a pointer to the 'struct device_node' is passed to of_attach_node. Before the call to of_attach_node, the phandle is initialized to 0 when the data structure is alloced. During the call to of_attach_node, it calls __of_attach_node which pulls the actual name and phandle from just created sub-properties named something like 'name' and 'ibm,phandle'. This is all fine for the first migration. The problem occurs with the second and subsequent migrations when the PHYP on the new system wants to replace the same set of nodes again, referenced with the same names and phandle values. On the second and subsequent migrations, the PHYP tells the system to again delete the nodes 'ibm,platform-facilities', 'ibm,random-v1', 'ibm,compression-v1', 'ibm,sym-encryption-v1'. It specifies these nodes by its known set of phandle values -- the same handles used by the PHYP on the source system are known on the target system. The mobility.c code calls of_find_node_by_phandle() with these values and ends up locating the first instance of each node that was added during the original boot, instead of the second instance of each node created after the first migration. The detach during the second migration fails with errors like, [ 4565.030704] WARNING: CPU: 3 PID: 4787 at drivers/of/dynamic.c:252 __of_detach_node+0x8/0xa0 [ 4565.030708] Modules linked in: nfsv3 nfs_acl nfs tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag lockd grace fscache sunrpc xts vmx_crypto sg pseries_rng binfmt_misc ip_tables xfs libcrc32c sd_mod ibmveth ibmvscsi scsi_transport_srp dm_mirror dm_region_hash dm_log dm_mod [ 4565.030733] CPU: 3 PID: 4787 Comm: drmgr Tainted: GW 4.18.0-rc1-wi107836-v05-120+ #201 [ 4565.030737] NIP: c07c1ea8 LR: c07c1fb4 CTR: 00655170 [ 4565.030741] REGS: c003f302b690 TRAP: 0700 Tainted: GW (4.18.0-rc1-wi107836-v05-120+) [ 4565.030745] MSR: 80010282b033 CR: 22288822 XER: 000a [ 4565.030757] CFAR: c07c1fb0 IRQMASK: 1 [ 4565.030757] GPR00: c07c1fa4 c003f302b910 c114bf00 c0038e68 [ 4565.030757] GPR04: 0001 80c008e0b4b8 [ 4565.030757] GPR08: 0001 8003 2843 [ 4565.030757] GPR12: 8800 c0001ec9ae00 4000 [ 4565.030757] GPR16: 0008 f6ff [ 4565.030757] GPR20: 0007 c003e9f1f034 0001 [ 4565.030757] GPR24: [ 4565.030757] GPR28: c1549d28 c1134828 c0038e68 c003f302b930 [ 4565.030804] NIP [c07c1ea8] __of_detach_node+0x8/0xa0 [ 4565.030808] LR [c07c1fb4] of_detach_node+0x74/0xd0 [ 4565.030811] Call Trace: [ 4565.030815] [c003f302b910] [c07c1fa4] of_detach_node+0x64/0xd0 (unreliable) [ 4565.030821] [c003f302b980] [c00c33c4] dlpar_detach_node+0xb4/0x150 [ 4565.030826] [c003f302ba10] [c00c3ffc] delete_dt_node+0x3c/0x80 [ 4565.030831] [c003f302ba40] [c00c4380] pseries_devicetree_update+0x150/0x4f0 [ 4565.030836] [c003f302bb70] [c00c479c] post_mobility_fixup+0x7c/0xf0 [ 4565.030841] [c003f302bbe0] [c00c4908] migration_store+0xf8/0x130 [ 4565.030847] [c003f302bc70] [c0998160] kobj_attr_store+0x30/0x60 [ 4565.030852] [c003f302bc90] [c0412f14] sysfs_kf_write+0x64/0xa0 [ 4565.030857] [c003f302bcb0] [c0411cac] kernfs_fop_write+0x16c/0x240 [ 4565.030862] [c003f302bd00] [c0355f20] __vfs_write+0x40/0x220 [ 4565.030867] [c003f302bd90] [c0356358] vfs_write+0xc8/0x240 [ 4565.030872] [c003f302bde0] [c03566cc] ksys_write+0x5c/0x100 [ 4565.030880] [c003f302be30] [c000b288] system_call+0x5c/0x70 [ 4565.030884] Instruction dump: [ 4565.030887] 38210070 3860 e8010010 eb61ffd8 eb81ffe0 eba1ffe8 ebc1fff0 ebe1fff8 [ 4565.030895] 7c0803a6 4e800020 e9230098 7929f7e2 <0b09> 2f89 4cde0020 e9030040 [ 4565.030903] ---[ end trace 5bd54cb1df9d2976 ]--- The mobility.c code continues on during the second migration, accepts the definitions of the new nodes from the PHYP and ends up renaming the new properties
