Re: [PATCH libpciaccess] Support for 32-bit domains
On 11/08/16 06:16 PM, Mark Kettenis wrote: >> Date: Wed, 10 Aug 2016 22:58:34 + >> From: Keith Busch>> >> If preserving libpciaccess ABI is of high importance, I think the only >> other option is to just ignore domains requiring 32-bits. That should >> be okay for us since X should not need the devices in these domains >> anyway. I'll send a patch for consideration. > > To be honest, bumping the shared library major is perfectly fine with > me. The current "thou shalt never bump the shared library major" > mantra that seems to has taken hold of the Linux community makes no > sense. Why have a shared library major at all if you can never bump > it? Of course it can be bumped. My points are: * The ABI must not be broken without bumping the SONAME. * The benefits and costs of bumping SONAME need to be considered carefully, taking the downstream distribution POV into account (as they are the entities which ultimately have to deal with ABI issues). > In any case the impact of bumping the libpciaccess shared library > should be fairly limited as it's not widely used outside of X. I agree, though at the very least, distributors would need to ensure that Xorg and its drivers don't end up linked against different libpciaccess SONAMEs. -- Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
On Fri, Aug 12, 2016 at 11:43:52AM +0200, Mark Kettenis wrote: > 'struct pci_device' isn't an opaque struct, and the "documentation" > never says that you can't build your own. It isn't unreasonable to > expect to be able to fill in the domain, bus, device and function > yourself and be able to call pci_device_cfg_read(). It also means > that code can copy a 'struct pci_device' instance around. Code using > the old ABI would lose the new 32-bit domain field in that case. Hm, that also has potential memory corruption if a program allocates their own with the older, smaller sized struct. Unless versioning the symbols is ok (yikes!), I think ignoring these types of domains in Linux is the simplest thing we can do. It's certainly okay with us and infinitely better than the current behavior. :) Here's the alternate proposal, assuming this is more readily acceptable: https://lists.x.org/archives/xorg-devel/2016-August/050593.html Inlined here for convenience: --- diff --git a/src/linux_sysfs.c b/src/linux_sysfs.c index 6367b11..f09a832 100644 --- a/src/linux_sysfs.c +++ b/src/linux_sysfs.c @@ -119,18 +119,28 @@ pci_system_linux_sysfs_create( void ) /** - * Filter out the names "." and ".." from the scanned sysfs entries. + * Filter out the names "." and ".." from the scanned sysfs entries, and + * domains requiring 32-bits. * * \param d Directory entry being processed by \c scandir. * * \return - * Zero if the entry name matches either "." or "..", non-zero otherwise. + * Zero if the entry name matches either "." or "..", or the domain requires + * 32 bits, non-zero otherwise. * * \sa scandir, populate_entries */ static int scan_sys_pci_filter( const struct dirent * d ) { +if (d->d_name[0] != '.') { +unsigned dom = 0; + +sscanf(d->d_name, "%x:", ); +if (dom > USHRT_MAX) +return 0; +} + return !((strcmp( d->d_name, "." ) == 0) || (strcmp( d->d_name, ".." ) == 0)); } -- ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
> From: Eric Anholt> Date: Thu, 11 Aug 2016 23:32:04 -0700 > > Keith Busch writes: > > > On Thu, Aug 11, 2016 at 12:03:30PM -0700, Eric Anholt wrote: > >> Given that libpciaccess allocates the struct, and the struct isn't > >> embedded in any other public structs, I think you could just tack a > >> "uint16_t domain_high;" at the end of the struct and fill that with the > >> high bits. > > > > Hmm, then we have to change every usage of domain to combine the two, > > and every usage thereafter must do the same. > > > > How about tacking 'uint32_t domain' to the end for the full domain, > > rename the existing 'uint16_t domain' to 'domain_low', and then just > > copy the lower bits from the full domain into there? That should satisfy > > legacy usage, and everywhere else will automatically use the full value. > > > > Does something like this look more reasonable? > > That looks even better to me! Let's give people a few days to comment, > but I like this solution. 'struct pci_device' isn't an opaque struct, and the "documentation" never says that you can't build your own. It isn't unreasonable to expect to be able to fill in the domain, bus, device and function yourself and be able to call pci_device_cfg_read(). It also means that code can copy a 'struct pci_device' instance around. Code using the old ABI would lose the new 32-bit domain field in that case. ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
Keith Buschwrites: > On Thu, Aug 11, 2016 at 12:03:30PM -0700, Eric Anholt wrote: >> Given that libpciaccess allocates the struct, and the struct isn't >> embedded in any other public structs, I think you could just tack a >> "uint16_t domain_high;" at the end of the struct and fill that with the >> high bits. > > Hmm, then we have to change every usage of domain to combine the two, > and every usage thereafter must do the same. > > How about tacking 'uint32_t domain' to the end for the full domain, > rename the existing 'uint16_t domain' to 'domain_low', and then just > copy the lower bits from the full domain into there? That should satisfy > legacy usage, and everywhere else will automatically use the full value. > > Does something like this look more reasonable? That looks even better to me! Let's give people a few days to comment, but I like this solution. Reviewed-by: Eric Anholt signature.asc Description: PGP signature ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
On Thu, Aug 11, 2016 at 12:03:30PM -0700, Eric Anholt wrote: > Given that libpciaccess allocates the struct, and the struct isn't > embedded in any other public structs, I think you could just tack a > "uint16_t domain_high;" at the end of the struct and fill that with the > high bits. Hmm, then we have to change every usage of domain to combine the two, and every usage thereafter must do the same. How about tacking 'uint32_t domain' to the end for the full domain, rename the existing 'uint16_t domain' to 'domain_low', and then just copy the lower bits from the full domain into there? That should satisfy legacy usage, and everywhere else will automatically use the full value. Does something like this look more reasonable? --- diff --git a/include/pciaccess.h b/include/pciaccess.h index 1d7aa4b..e095dfb 100644 --- a/include/pciaccess.h +++ b/include/pciaccess.h @@ -321,7 +321,7 @@ struct pci_device { * the domain will always be zero. */ /*@{*/ -uint16_tdomain; +uint16_tdomain_low; uint8_t bus; uint8_t dev; uint8_t func; @@ -385,6 +385,12 @@ struct pci_device { * Used by the VGA arbiter. Type of resource decoded by the device * and * the file descriptor (/dev/vga_arbiter). */ int vgaarb_rsrc; + +/** + * The 32-bit pci domain. The lower 16 bits are copied into the + * domain_low field for ABI compatibility. + */ +uint32_tdomain; }; diff --git a/src/linux_sysfs.c b/src/linux_sysfs.c index 6367b11..c640f41 100644 --- a/src/linux_sysfs.c +++ b/src/linux_sysfs.c @@ -159,10 +159,11 @@ populate_entries( struct pci_system * p ) (struct pci_device_private *) >devices[i]; - sscanf(devices[i]->d_name, "%04x:%02x:%02x.%1u", + sscanf(devices[i]->d_name, "%x:%02x:%02x.%1u", & dom, & bus, & dev, & func); device->base.domain = dom; + device->base.domain_low = dom; device->base.bus = bus; device->base.dev = dev; device->base.func = func; -- ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
Michel Dänzerwrites: > On 10/08/16 06:39 AM, Keith Busch wrote: >> A pci "domain" is a purely software construct, and need not be limited >> to the 16-bit ACPI defined segment. The Linux kernel currently supports >> 32-bit domains, so this patch matches up with those capabilities to make >> it usable on systems exporting such domains. >> >> Reported-by: Pawel Baldysiak >> Signed-off-by: Keith Busch >> --- >> include/pciaccess.h | 2 +- >> src/linux_sysfs.c | 2 +- >> 2 files changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/include/pciaccess.h b/include/pciaccess.h >> index 1d7aa4b..93ed76f 100644 >> --- a/include/pciaccess.h >> +++ b/include/pciaccess.h >> @@ -321,7 +321,7 @@ struct pci_device { >> * the domain will always be zero. >> */ >> /*@{*/ >> -uint16_tdomain; >> +uint32_tdomain; > > This change breaks ABI, so as is it would require bumping the library > SONAME (which is painful and thus generally discouraged from a > downstream POV). Given that libpciaccess allocates the struct, and the struct isn't embedded in any other public structs, I think you could just tack a "uint16_t domain_high;" at the end of the struct and fill that with the high bits. signature.asc Description: PGP signature ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
> Date: Wed, 10 Aug 2016 22:58:34 + > From: Keith Busch> > On Thu, Aug 11, 2016 at 12:17:48AM +0200, Mark Kettenis wrote: > > > From: Keith Busch > > > Date: Tue, 9 Aug 2016 15:39:35 -0600 > > > > > > A pci "domain" is a purely software construct, and need not be limited > > > to the 16-bit ACPI defined segment. The Linux kernel currently supports > > > 32-bit domains, so this patch matches up with those capabilities to make > > > it usable on systems exporting such domains. > > > > Well, yes, and no. PCI domains really are a hardware property. There > > are systems out there that have multiple PCI host bridges, each with > > their own separate config/mem/io address spaces and bus numbers > > starting with 0 for the root of the PCI bus hierarchy. Pretty much > > any 64-bit SPARC system falls into this category, and I've seen > > PA-RISC and POWER systems with such a hardware configuration as well. > > And given that HP's Itanium line developed from their PA-RISC hardware > > I expect them to be in the same boat. There is no domain numering > > scheme that is implied by the hardware though, do domain numbers are > > indeed purely a software construct. On OpenBSD we simply number the > > domains sequentially. So 16 bits are more than enough. > > > > The Linux kernel could do the same with ACPI segments (which may or > > may not map onto true PCI domains). That would remove the need to > > change te libpciaccess ABI. Although I can see that having a 1:1 > > mapping of ACPI segments to domains is something that is nice to have. > > I can give a little more background on where this is coming from. The > Intel x86 Skylake E-Series has an option to provide a number of additional > "host bridges". The "vmd" driver in the Linux mainline kernel supports > this hardware. > > For better or worse, Linux does match the segment number to the > domain. The "vmd" hardware is not a segment though, and decoupling _SEG > from domain numbers in the Linux kernel proved difficult and unpopular > with the devs. To avoid the potential clash from letting vmd hardware > occupy the same range that an ACPI _SEG could define, we let VMD start > at 0x1. > > I've already patched pci-utils (provides lspci, setpci) to allow > this, but I missed this library at the time (my dev machines are all > no-graphics). Right now, a system with VMD segfaults startx. I believe > it's down to the error handling that frees the pci devices and sets > pci_system->devices to NULL. It looks like this is dereferenced later, > but I'm very unfamiliar with the code base and not sure which repo to > look into. > > If preserving libpciaccess ABI is of high importance, I think the only > other option is to just ignore domains requiring 32-bits. That should > be okay for us since X should not need the devices in these domains > anyway. I'll send a patch for consideration. To be honest, bumping the shared library major is perfectly fine with me. The current "thou shalt never bump the shared library major" mantra that seems to has taken hold of the Linux community makes no sense. Why have a shared library major at all if you can never bump it? In any case the impact of bumping the libpciaccess shared library should be fairly limited as it's not widely used outside of X. But I fear it does affect the driver API. > > However, I do believe that ACPI segments are actually encoded as > > 64-bit integers. So such a 1:1 mapping may not be achievable. > > True, though only 16 bits are used (ACPI 6, section 6.5.6): > > " > The lower 16 bits of _SEG returned integer is the PCI Segment Group number. > Other bits are reserved. > " At least whoever designed the libpciaccess interface had some reason to pick a 16-bit integer for the domain. ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
On Wed, Aug 10, 2016 at 10:09:14AM +0900, Michel Dänzer wrote: > This change breaks ABI, so as is it would require bumping the library > SONAME (which is painful and thus generally discouraged from a > downstream POV). Well, the current situation is such systems are unbootable without this fix, so the pain from bumping for ABI sounds pretty tame in comparison. :) I suspect, though, the boot halting segfault is probably a bug that can be fixed elsewhere without supporting 32-bit domain devices. I'll see if I can confirm that to be the case. ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
Sorry all, I pointed 'git send-email' to the wrong patch version. Please ignore. ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
[PATCH libpciaccess] Support for 32-bit domains
A pci "domain" is a purely software construct, and need not be limited to the 16-bit ACPI defined segment. The Linux kernel currently supports 32-bit domains, so this patch matches up with those capabilities to make it usable on systems exporting such domains. Reported-by: Pawel BaldysiakSigned-off-by: Keith Busch --- include/pciaccess.h | 2 +- src/linux_sysfs.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/pciaccess.h b/include/pciaccess.h index 1d7aa4b..93ed76f 100644 --- a/include/pciaccess.h +++ b/include/pciaccess.h @@ -321,7 +321,7 @@ struct pci_device { * the domain will always be zero. */ /*@{*/ -uint16_tdomain; +uint32_tdomain; uint8_t bus; uint8_t dev; uint8_t func; diff --git a/src/linux_sysfs.c b/src/linux_sysfs.c index 6367b11..c0c8f2a 100644 --- a/src/linux_sysfs.c +++ b/src/linux_sysfs.c @@ -159,7 +159,7 @@ populate_entries( struct pci_system * p ) (struct pci_device_private *) >devices[i]; - sscanf(devices[i]->d_name, "%04x:%02x:%02x.%1u", + sscanf(devices[i]->d_name, "%x:%02x:%02x.%1u", & dom, & bus, & dev, & func); device->base.domain = dom; -- 2.7.2 ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
On Thu, Aug 11, 2016 at 12:17:48AM +0200, Mark Kettenis wrote: > > From: Keith Busch> > Date: Tue, 9 Aug 2016 15:39:35 -0600 > > > > A pci "domain" is a purely software construct, and need not be limited > > to the 16-bit ACPI defined segment. The Linux kernel currently supports > > 32-bit domains, so this patch matches up with those capabilities to make > > it usable on systems exporting such domains. > > Well, yes, and no. PCI domains really are a hardware property. There > are systems out there that have multiple PCI host bridges, each with > their own separate config/mem/io address spaces and bus numbers > starting with 0 for the root of the PCI bus hierarchy. Pretty much > any 64-bit SPARC system falls into this category, and I've seen > PA-RISC and POWER systems with such a hardware configuration as well. > And given that HP's Itanium line developed from their PA-RISC hardware > I expect them to be in the same boat. There is no domain numering > scheme that is implied by the hardware though, do domain numbers are > indeed purely a software construct. On OpenBSD we simply number the > domains sequentially. So 16 bits are more than enough. > > The Linux kernel could do the same with ACPI segments (which may or > may not map onto true PCI domains). That would remove the need to > change te libpciaccess ABI. Although I can see that having a 1:1 > mapping of ACPI segments to domains is something that is nice to have. I can give a little more background on where this is coming from. The Intel x86 Skylake E-Series has an option to provide a number of additional "host bridges". The "vmd" driver in the Linux mainline kernel supports this hardware. For better or worse, Linux does match the segment number to the domain. The "vmd" hardware is not a segment though, and decoupling _SEG from domain numbers in the Linux kernel proved difficult and unpopular with the devs. To avoid the potential clash from letting vmd hardware occupy the same range that an ACPI _SEG could define, we let VMD start at 0x1. I've already patched pci-utils (provides lspci, setpci) to allow this, but I missed this library at the time (my dev machines are all no-graphics). Right now, a system with VMD segfaults startx. I believe it's down to the error handling that frees the pci devices and sets pci_system->devices to NULL. It looks like this is dereferenced later, but I'm very unfamiliar with the code base and not sure which repo to look into. If preserving libpciaccess ABI is of high importance, I think the only other option is to just ignore domains requiring 32-bits. That should be okay for us since X should not need the devices in these domains anyway. I'll send a patch for consideration. > However, I do believe that ACPI segments are actually encoded as > 64-bit integers. So such a 1:1 mapping may not be achievable. True, though only 16 bits are used (ACPI 6, section 6.5.6): " The lower 16 bits of _SEG returned integer is the PCI Segment Group number. Other bits are reserved. " ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
> From: Keith Busch> Date: Tue, 9 Aug 2016 15:39:35 -0600 > > A pci "domain" is a purely software construct, and need not be limited > to the 16-bit ACPI defined segment. The Linux kernel currently supports > 32-bit domains, so this patch matches up with those capabilities to make > it usable on systems exporting such domains. Well, yes, and no. PCI domains really are a hardware property. There are systems out there that have multiple PCI host bridges, each with their own separate config/mem/io address spaces and bus numbers starting with 0 for the root of the PCI bus hierarchy. Pretty much any 64-bit SPARC system falls into this category, and I've seen PA-RISC and POWER systems with such a hardware configuration as well. And given that HP's Itanium line developed from their PA-RISC hardware I expect them to be in the same boat. There is no domain numering scheme that is implied by the hardware though, do domain numbers are indeed purely a software construct. On OpenBSD we simply number the domains sequentially. So 16 bits are more than enough. The Linux kernel could do the same with ACPI segments (which may or may not map onto true PCI domains). That would remove the need to change te libpciaccess ABI. Although I can see that having a 1:1 mapping of ACPI segments to domains is something that is nice to have. However, I do believe that ACPI segments are actually encoded as 64-bit integers. So such a 1:1 mapping may not be achievable. > Reported-by: Pawel Baldysiak > Signed-off-by: Keith Busch > --- > include/pciaccess.h | 2 +- > src/linux_sysfs.c | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/include/pciaccess.h b/include/pciaccess.h > index 1d7aa4b..93ed76f 100644 > --- a/include/pciaccess.h > +++ b/include/pciaccess.h > @@ -321,7 +321,7 @@ struct pci_device { > * the domain will always be zero. > */ > /*@{*/ > -uint16_tdomain; > +uint32_tdomain; > uint8_t bus; > uint8_t dev; > uint8_t func; > diff --git a/src/linux_sysfs.c b/src/linux_sysfs.c > index 6367b11..c0c8f2a 100644 > --- a/src/linux_sysfs.c > +++ b/src/linux_sysfs.c > @@ -159,7 +159,7 @@ populate_entries( struct pci_system * p ) > (struct pci_device_private *) >devices[i]; > > > - sscanf(devices[i]->d_name, "%04x:%02x:%02x.%1u", > + sscanf(devices[i]->d_name, "%x:%02x:%02x.%1u", > & dom, & bus, & dev, & func); > > device->base.domain = dom; > -- > 2.7.2 > > ___ > xorg-devel@lists.x.org: X.Org development > Archives: http://lists.x.org/archives/xorg-devel > Info: https://lists.x.org/mailman/listinfo/xorg-devel > > ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
Re: [PATCH libpciaccess] Support for 32-bit domains
On 10/08/16 06:39 AM, Keith Busch wrote: > A pci "domain" is a purely software construct, and need not be limited > to the 16-bit ACPI defined segment. The Linux kernel currently supports > 32-bit domains, so this patch matches up with those capabilities to make > it usable on systems exporting such domains. > > Reported-by: Pawel Baldysiak> Signed-off-by: Keith Busch > --- > include/pciaccess.h | 2 +- > src/linux_sysfs.c | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/include/pciaccess.h b/include/pciaccess.h > index 1d7aa4b..93ed76f 100644 > --- a/include/pciaccess.h > +++ b/include/pciaccess.h > @@ -321,7 +321,7 @@ struct pci_device { > * the domain will always be zero. > */ > /*@{*/ > -uint16_tdomain; > +uint32_tdomain; This change breaks ABI, so as is it would require bumping the library SONAME (which is painful and thus generally discouraged from a downstream POV). -- Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel
[PATCH libpciaccess] Support for 32-bit domains
A pci "domain" is a purely software construct, and need not be limited to the 16-bit ACPI defined segment. The Linux kernel currently supports 32-bit domains, so this patch matches up with those capabilities to make it usable on systems exporting such domains. Reported-by: Pawel BaldysiakSigned-off-by: Keith Busch --- include/pciaccess.h | 2 +- src/linux_sysfs.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/pciaccess.h b/include/pciaccess.h index 1d7aa4b..93ed76f 100644 --- a/include/pciaccess.h +++ b/include/pciaccess.h @@ -321,7 +321,7 @@ struct pci_device { * the domain will always be zero. */ /*@{*/ -uint16_tdomain; +uint32_tdomain; uint8_t bus; uint8_t dev; uint8_t func; diff --git a/src/linux_sysfs.c b/src/linux_sysfs.c index 6367b11..c0c8f2a 100644 --- a/src/linux_sysfs.c +++ b/src/linux_sysfs.c @@ -159,7 +159,7 @@ populate_entries( struct pci_system * p ) (struct pci_device_private *) >devices[i]; - sscanf(devices[i]->d_name, "%04x:%02x:%02x.%1u", + sscanf(devices[i]->d_name, "%x:%02x:%02x.%1u", & dom, & bus, & dev, & func); device->base.domain = dom; -- 2.7.2 ___ xorg-devel@lists.x.org: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: https://lists.x.org/mailman/listinfo/xorg-devel