Re: Thunderbolt(/USB4) followup & I'm happy to donate some hardware Re: Feature request: Use the PCIe devices on Thunderbolt (aka PCIe hotplug?)
Kind bump on this thread. As for me I'd like to attach nvme(4) and maybe ethernet and amdgpu(4) to the Thunderbolt-as-PCIe-bridge. Have a good wknd! Joseph ‐‐‐ Original Message ‐‐‐ On Monday, 26 October 2020 13:02, Joseph Mayer wrote: > (If this one belongs on misc@ please say.) > > Hi tech@, > > If anyone is interested in implementing Thunderbolt support for > OpenBSD, I'd like to donate some PCIe expansion Thunderbolt 3 enclosure > and M.2 NVMe SSD Thunderbolt 3 enclosure as appropriate, if so please > let me know. > > BSDCan 2020 presentation by Scott Long of FreeBSD Thunderbolt support > here: https://youtu.be/VbAJf2PBE-M?t=802 > (https://www.bsdcan.org/events/bsdcan_2020/schedule/session/27-thunderbolt-on-freebsd/). > He mentions there that the sources are in > "rc/sys/dev/thunderbolt" but they appear to not have been merged yet. > > Thunderbolt in essence is a hotplugged PCIv3 x4 interface, useful when > a machine especially a laptop lacks other ways to plug in SSD, NIC, > AMDGPU. Not sure how clean the licensing situation is and how bloated > it is. (Note USB4 and Thunderbolt 4 are Thunderbolt 3 but with PCIe > data increased from 22gbps to 32gbps.) > > Apparently Thunderbolt is incorporated in the USB4 spec and this way > will be more ubiquitous and come to more architectures, ref. > https://www.phoronix.com/scan.php?page=news_item=Arm-Thunderbolt-Works , > https://lwn.net/Articles/802961/ . > > Within Linux there's seemingly unending amounts of patches and more: > https://github.com/torvalds/linux/tree/master/drivers/thunderbolt , > Intel devs unhelpful https://lore.kernel.org/patchwork/patch/983864/ , > https://lwn.net/Search/DoSearch?words=thunderbolt , search "thunderbolt > site:lkml.iu.edu/hypermail/linux/kernel/". > > Joseph > > ‐‐‐ Original Message ‐‐‐ > On Tuesday, 24 March 2020 01:45, John-Mark Gurney j...@funkthat.com wrote: > > > Joseph Mayer wrote this message on Sat, Mar 21, 2020 at 02:57 +: > > > > > Thunderbolt support would be awesome. Especially it would allow the use > > > of additional M.2 NVMe SSD:s on a laptop at full performance. > > > Thunderbolt support would also allow the use of an AMDGPU via a PCIe > > > chassi, as well as enable the use of 10gbps Ethernet on laptops [1]. > > > While I like to use Thunderbolt for this pragmatic reason, also Intel > > > apparently promises license etc. generosity to computer makers, which > > > certainly does not hurt. [2] > > > FreeBSD has Thunderbolt support. It appears to me that they call it > > > "PCIe Hot plug". [3] > > > > From my understanding, Thunderbolt is different from PCIe Hot Plug... > > PCIe the spec itself has hot plug capabilities, and this is what is > > used for laptops w/ ExpressCards and some servers... > > Thunderbolt from my understanding is more complicated due to > > display routing and other related features and FreeBSD does NOT > > yet have support for it. > > > > > It was implemented 2015 by John-Mark Gurney j...@freebsd.org. > > > > John Baldwin,j...@freebsd.org ended up implementing it differently > > and not using the code I had written, so he is probably a better > > person to ask on the current state of the code.. > > This was done via: > > https://reviews.freebsd.org/D6136?id=15683 > > I have heard that there may be a proper ThunderBolt support coming > > to FreeBSD in the near future, but not sure exactly when... > > > > > Not sure if a TB device must be attached on boot and cannot be > > > detached, anyhow if that is the case then still totally fine. > > > > The devctl command can detach a device. This allows ejecting > > devices w/o crashing the system for removal, or allowing you to detach > > a device and pass it through to a bhyve vm, etc. Not all drivers are > > written to allow detaching... > > > > > NetBSD appears to have support also but I don't find details. > > > Security-wise Thunderbolt without IOMMU is correlated with physical > > > break-in attack vectors, anyhow that is commonly fine. [4] > > > > From my understanding, all PCIe switches have a built in IOMMU, so > > this shouldn't be a major security issue. I have not done indepth > > analysis to verify this though. and this also depends upon the > > PCIe switch not having bugs... > > There is a relatively inexpensive USB3 to PCIe bridge that lets you > > issue arbitrary PCIe commands that could be used to verify the security > > of implementations... > > > > > One Thunderbolt 3 controller provides 22gbps of PCIe data bandwidth to > > > all the one or two Thunderbolt ports it exports, which is fine. [5] > > > Many Thunderbolt devices allow daisy chaining. An "eGFX" certified [6] > > > Thunderbolt PCIe chassi (such as [7]) has absolutely no performance > > > advantage over a normal Thunderbolt PCIe chassi (such as [8]), > > > including for eGPU (e.g. AMDGPU) use. > > > > Good luck! > > > > > [1] The lowest cost and most common 10gbps Ethernet Thunderbolt chip > > > is Aquantia AQC107S. There are also some
Re: Thunderbolt(/USB4) followup & I'm happy to donate some hardware Re: Feature request: Use the PCIe devices on Thunderbolt (aka PCIe hotplug?)
Hi Tom, I share your understanding that Thunderbolt has a lower security profile due to TB having dynamic memory addresses access while USB does not. I presume the engineering idea is that the IOMMU (when enabled and properly configured) should uphold memory safety. Remember though you have this risk for internal devices already e.g. your PCIe device's firmware might try to mess with you, whether it's in a PCIe slot in your computer or in an external TB3 enclosure. What I wanted to bring attention to with this thread was to request TB3 support and say I'm happy to donate some enclosures. Best regards, Joseph ‐‐‐ Original Message ‐‐‐ On Monday, 26 October 2020 15:36, Tom Smyth wrote: > Hi Joseph,All > > There are some PCI-E attack surfaces that might need to be considered... > perhaps the availability of more devices with thunderbolt connections make > PCI-E / DMA Attacks more viable and hence more prevalent. > > https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-934.pdf > I did come across intel SGX when configuring the bios / firmware > on my lenovo laptop which mentioned Thunderbolt / PCI-E attacks. > > But mitigating this risk could yield security benefits for people who > use PCI-E pass > through / SR-IOV in Virtualized environments. > > I hope this helps, > > Tom Smyth > > On Mon, 26 Oct 2020 at 12:06, Joseph Mayer joseph.ma...@protonmail.com wrote: > > > (If this one belongs on misc@ please say.) > > Hi tech@, > > If anyone is interested in implementing Thunderbolt support for > > OpenBSD, I'd like to donate some PCIe expansion Thunderbolt 3 enclosure > > and M.2 NVMe SSD Thunderbolt 3 enclosure as appropriate, if so please > > let me know. > > BSDCan 2020 presentation by Scott Long of FreeBSD Thunderbolt support > > here: https://youtu.be/VbAJf2PBE-M?t=802 > > (https://www.bsdcan.org/events/bsdcan_2020/schedule/session/27-thunderbolt-on-freebsd/). > > He mentions there that the sources are in > > "rc/sys/dev/thunderbolt" but they appear to not have been merged yet. > > Thunderbolt in essence is a hotplugged PCIv3 x4 interface, useful when > > a machine especially a laptop lacks other ways to plug in SSD, NIC, > > AMDGPU. Not sure how clean the licensing situation is and how bloated > > it is. (Note USB4 and Thunderbolt 4 are Thunderbolt 3 but with PCIe > > data increased from 22gbps to 32gbps.) > > Apparently Thunderbolt is incorporated in the USB4 spec and this way > > will be more ubiquitous and come to more architectures, ref. > > https://www.phoronix.com/scan.php?page=news_item=Arm-Thunderbolt-Works , > > https://lwn.net/Articles/802961/ . > > Within Linux there's seemingly unending amounts of patches and more: > > https://github.com/torvalds/linux/tree/master/drivers/thunderbolt , > > Intel devs unhelpful https://lore.kernel.org/patchwork/patch/983864/ , > > https://lwn.net/Search/DoSearch?words=thunderbolt , search "thunderbolt > > site:lkml.iu.edu/hypermail/linux/kernel/". > > Joseph > > ‐‐‐ Original Message ‐‐‐ > > On Tuesday, 24 March 2020 01:45, John-Mark Gurney j...@funkthat.com wrote: > > > > > Joseph Mayer wrote this message on Sat, Mar 21, 2020 at 02:57 +: > > > > > > > Thunderbolt support would be awesome. Especially it would allow the use > > > > of additional M.2 NVMe SSD:s on a laptop at full performance. > > > > Thunderbolt support would also allow the use of an AMDGPU via a PCIe > > > > chassi, as well as enable the use of 10gbps Ethernet on laptops [1]. > > > > While I like to use Thunderbolt for this pragmatic reason, also Intel > > > > apparently promises license etc. generosity to computer makers, which > > > > certainly does not hurt. [2] > > > > FreeBSD has Thunderbolt support. It appears to me that they call it > > > > "PCIe Hot plug". [3] > > > > > > From my understanding, Thunderbolt is different from PCIe Hot Plug... > > > PCIe the spec itself has hot plug capabilities, and this is what is > > > used for laptops w/ ExpressCards and some servers... > > > Thunderbolt from my understanding is more complicated due to > > > display routing and other related features and FreeBSD does NOT > > > yet have support for it. > > > > > > > It was implemented 2015 by John-Mark Gurney j...@freebsd.org. > > > > > > John Baldwin,j...@freebsd.org ended up implementing it differently > > > and not using the code I had written, so he is probably a better > > > person to ask on the current state of the code.. > > > This was done via: > > > https://reviews.freebsd.org/D6136?id=15683 > > > I have heard that there may be a proper ThunderBolt support coming > > > to FreeBSD in the near future, but not sure exactly when... > > > > > > > Not sure if a TB device must be attached on boot and cannot be > > > > detached, anyhow if that is the case then still totally fine. > > > > > > The devctl command can detach a device. This allows ejecting > > > devices w/o crashing the system for removal, or allowing you to detach > > > a device and pass it through to a bhyve vm, etc.
Thunderbolt(/USB4) followup & I'm happy to donate some hardware Re: Feature request: Use the PCIe devices on Thunderbolt (aka PCIe hotplug?)
(If this one belongs on misc@ please say.) Hi tech@, If anyone is interested in implementing Thunderbolt support for OpenBSD, I'd like to donate some PCIe expansion Thunderbolt 3 enclosure and M.2 NVMe SSD Thunderbolt 3 enclosure as appropriate, if so please let me know. BSDCan 2020 presentation by Scott Long of FreeBSD Thunderbolt support here: https://youtu.be/VbAJf2PBE-M?t=802 (https://www.bsdcan.org/events/bsdcan_2020/schedule/session/27-thunderbolt-on-freebsd/). He mentions there that the sources are in "rc/sys/dev/thunderbolt" but they appear to not have been merged yet. Thunderbolt in essence is a hotplugged PCIv3 x4 interface, useful when a machine especially a laptop lacks other ways to plug in SSD, NIC, AMDGPU. Not sure how clean the licensing situation is and how bloated it is. (Note USB4 and Thunderbolt 4 are Thunderbolt 3 but with PCIe data increased from 22gbps to 32gbps.) Apparently Thunderbolt is incorporated in the USB4 spec and this way will be more ubiquitous and come to more architectures, ref. https://www.phoronix.com/scan.php?page=news_item=Arm-Thunderbolt-Works , https://lwn.net/Articles/802961/ . Within Linux there's seemingly unending amounts of patches and more: https://github.com/torvalds/linux/tree/master/drivers/thunderbolt , Intel devs unhelpful https://lore.kernel.org/patchwork/patch/983864/ , https://lwn.net/Search/DoSearch?words=thunderbolt , search "thunderbolt site:lkml.iu.edu/hypermail/linux/kernel/". Joseph ‐‐‐ Original Message ‐‐‐ On Tuesday, 24 March 2020 01:45, John-Mark Gurney wrote: > Joseph Mayer wrote this message on Sat, Mar 21, 2020 at 02:57 +: > > > Thunderbolt support would be awesome. Especially it would allow the use > > of additional M.2 NVMe SSD:s on a laptop at full performance. > > Thunderbolt support would also allow the use of an AMDGPU via a PCIe > > chassi, as well as enable the use of 10gbps Ethernet on laptops [1]. > > While I like to use Thunderbolt for this pragmatic reason, also Intel > > apparently promises license etc. generosity to computer makers, which > > certainly does not hurt. [2] > > FreeBSD has Thunderbolt support. It appears to me that they call it > > "PCIe Hot plug". [3] > > From my understanding, Thunderbolt is different from PCIe Hot Plug... > > PCIe the spec itself has hot plug capabilities, and this is what is > used for laptops w/ ExpressCards and some servers... > > Thunderbolt from my understanding is more complicated due to > display routing and other related features and FreeBSD does NOT > yet have support for it. > > > It was implemented 2015 by John-Mark Gurney j...@freebsd.org. > > John Baldwin,j...@freebsd.org ended up implementing it differently > and not using the code I had written, so he is probably a better > person to ask on the current state of the code.. > > This was done via: > https://reviews.freebsd.org/D6136?id=15683 > > I have heard that there may be a proper ThunderBolt support coming > to FreeBSD in the near future, but not sure exactly when... > > > Not sure if a TB device must be attached on boot and cannot be > > detached, anyhow if that is the case then still totally fine. > > The devctl command can detach a device. This allows ejecting > devices w/o crashing the system for removal, or allowing you to detach > a device and pass it through to a bhyve vm, etc. Not all drivers are > written to allow detaching... > > > NetBSD appears to have support also but I don't find details. > > Security-wise Thunderbolt without IOMMU is correlated with physical > > break-in attack vectors, anyhow that is commonly fine. [4] > > From my understanding, all PCIe switches have a built in IOMMU, so > this shouldn't be a major security issue. I have not done indepth > analysis to verify this though. and this also depends upon the > PCIe switch not having bugs... > > There is a relatively inexpensive USB3 to PCIe bridge that lets you > issue arbitrary PCIe commands that could be used to verify the security > of implementations... > > > One Thunderbolt 3 controller provides 22gbps of PCIe data bandwidth to > > all the one or two Thunderbolt ports it exports, which is fine. [5] > > Many Thunderbolt devices allow daisy chaining. An "eGFX" certified [6] > > Thunderbolt PCIe chassi (such as [7]) has absolutely no performance > > advantage over a normal Thunderbolt PCIe chassi (such as [8]), > > including for eGPU (e.g. AMDGPU) use. > > Good luck! > > > [1] The lowest cost and most common 10gbps Ethernet Thunderbolt chip > > is Aquantia AQC107S. There are also some adapters based on a normal > > PCIe 10gbps chip and a separate Thunderbolt to PCIe controller. > > [2] https://www.theregister.co.uk/2017/05/24/intel_thunderbolt_3forall/ > > [3] > > https://www.freebsd.org/news/status/report-2015-01-2015-03.html#Adding-PCIe-Hot-plug-Support > > https://www.freebsd.org/news/status/report-2015-07-2015-09.html#Adding-PCIe-Hot-plug-Support > > [4] > >
Re: Thunderbolt(/USB4) followup & I'm happy to donate some hardware Re: Feature request: Use the PCIe devices on Thunderbolt (aka PCIe hotplug?)
Hi Joseph,All There are some PCI-E attack surfaces that might need to be considered... perhaps the availability of more devices with thunderbolt connections make PCI-E / DMA Attacks more viable and hence more prevalent. https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-934.pdf I did come across intel SGX when configuring the bios / firmware on my lenovo laptop which mentioned Thunderbolt / PCI-E attacks. But mitigating this risk could yield security benefits for people who use PCI-E pass through / SR-IOV in Virtualized environments. I hope this helps, Tom Smyth On Mon, 26 Oct 2020 at 12:06, Joseph Mayer wrote: > > (If this one belongs on misc@ please say.) > > Hi tech@, > > If anyone is interested in implementing Thunderbolt support for > OpenBSD, I'd like to donate some PCIe expansion Thunderbolt 3 enclosure > and M.2 NVMe SSD Thunderbolt 3 enclosure as appropriate, if so please > let me know. > > BSDCan 2020 presentation by Scott Long of FreeBSD Thunderbolt support > here: https://youtu.be/VbAJf2PBE-M?t=802 > (https://www.bsdcan.org/events/bsdcan_2020/schedule/session/27-thunderbolt-on-freebsd/). > He mentions there that the sources are in > "rc/sys/dev/thunderbolt" but they appear to not have been merged yet. > > Thunderbolt in essence is a hotplugged PCIv3 x4 interface, useful when > a machine especially a laptop lacks other ways to plug in SSD, NIC, > AMDGPU. Not sure how clean the licensing situation is and how bloated > it is. (Note USB4 and Thunderbolt 4 are Thunderbolt 3 but with PCIe > data increased from 22gbps to 32gbps.) > > Apparently Thunderbolt is incorporated in the USB4 spec and this way > will be more ubiquitous and come to more architectures, ref. > https://www.phoronix.com/scan.php?page=news_item=Arm-Thunderbolt-Works , > https://lwn.net/Articles/802961/ . > > Within Linux there's seemingly unending amounts of patches and more: > https://github.com/torvalds/linux/tree/master/drivers/thunderbolt , > Intel devs unhelpful https://lore.kernel.org/patchwork/patch/983864/ , > https://lwn.net/Search/DoSearch?words=thunderbolt , search "thunderbolt > site:lkml.iu.edu/hypermail/linux/kernel/". > > Joseph > > ‐‐‐ Original Message ‐‐‐ > On Tuesday, 24 March 2020 01:45, John-Mark Gurney wrote: > > > Joseph Mayer wrote this message on Sat, Mar 21, 2020 at 02:57 +: > > > > > Thunderbolt support would be awesome. Especially it would allow the use > > > of additional M.2 NVMe SSD:s on a laptop at full performance. > > > Thunderbolt support would also allow the use of an AMDGPU via a PCIe > > > chassi, as well as enable the use of 10gbps Ethernet on laptops [1]. > > > While I like to use Thunderbolt for this pragmatic reason, also Intel > > > apparently promises license etc. generosity to computer makers, which > > > certainly does not hurt. [2] > > > FreeBSD has Thunderbolt support. It appears to me that they call it > > > "PCIe Hot plug". [3] > > > > From my understanding, Thunderbolt is different from PCIe Hot Plug... > > > > PCIe the spec itself has hot plug capabilities, and this is what is > > used for laptops w/ ExpressCards and some servers... > > > > Thunderbolt from my understanding is more complicated due to > > display routing and other related features and FreeBSD does NOT > > yet have support for it. > > > > > It was implemented 2015 by John-Mark Gurney j...@freebsd.org. > > > > John Baldwin,j...@freebsd.org ended up implementing it differently > > and not using the code I had written, so he is probably a better > > person to ask on the current state of the code.. > > > > This was done via: > > https://reviews.freebsd.org/D6136?id=15683 > > > > I have heard that there may be a proper ThunderBolt support coming > > to FreeBSD in the near future, but not sure exactly when... > > > > > Not sure if a TB device must be attached on boot and cannot be > > > detached, anyhow if that is the case then still totally fine. > > > > The devctl command can detach a device. This allows ejecting > > devices w/o crashing the system for removal, or allowing you to detach > > a device and pass it through to a bhyve vm, etc. Not all drivers are > > written to allow detaching... > > > > > NetBSD appears to have support also but I don't find details. > > > Security-wise Thunderbolt without IOMMU is correlated with physical > > > break-in attack vectors, anyhow that is commonly fine. [4] > > > > From my understanding, all PCIe switches have a built in IOMMU, so > > this shouldn't be a major security issue. I have not done indepth > > analysis to verify this though. and this also depends upon the > > PCIe switch not having bugs... > > > > There is a relatively inexpensive USB3 to PCIe bridge that lets you > > issue arbitrary PCIe commands that could be used to verify the security > > of implementations... > > > > > One Thunderbolt 3 controller provides 22gbps of PCIe data bandwidth to > > > all the one or two Thunderbolt ports it exports, which is fine. [5] > > > Many Thunderbolt
Re: Feature request: Use the PCIe devices on Thunderbolt (aka PCIe hotplug?)
Joseph Mayer wrote this message on Sat, Mar 21, 2020 at 02:57 +: > Thunderbolt support would be awesome. Especially it would allow the use > of additional M.2 NVMe SSD:s on a laptop at full performance. > > Thunderbolt support would also allow the use of an AMDGPU via a PCIe > chassi, as well as enable the use of 10gbps Ethernet on laptops [1]. > > > While I like to use Thunderbolt for this pragmatic reason, also Intel > apparently promises license etc. generosity to computer makers, which > certainly does not hurt. [2] > > > FreeBSD has Thunderbolt support. It appears to me that they call it > "PCIe Hot plug". [3] >From my understanding, Thunderbolt is different from PCIe Hot Plug... PCIe the spec itself has hot plug capabilities, and this is what is used for laptops w/ ExpressCards and some servers... Thunderbolt from my understanding is more complicated due to display routing and other related features and FreeBSD does NOT yet have support for it. > It was implemented 2015 by John-Mark Gurney . John Baldwin, j...@freebsd.org ended up implementing it differently and not using the code I had written, so he is probably a better person to ask on the current state of the code.. This was done via: https://reviews.freebsd.org/D6136?id=15683 I have heard that there may be a proper ThunderBolt support coming to FreeBSD in the near future, but not sure exactly when... > Not sure if a TB device must be attached on boot and cannot be > detached, anyhow if that is the case then still totally fine. The devctl command can detach a device. This allows ejecting devices w/o crashing the system for removal, or allowing you to detach a device and pass it through to a bhyve vm, etc. Not all drivers are written to allow detaching... > NetBSD appears to have support also but I don't find details. > > Security-wise Thunderbolt without IOMMU is correlated with physical > break-in attack vectors, anyhow that is commonly fine. [4] >From my understanding, all PCIe switches have a built in IOMMU, so this shouldn't be a major security issue. I have not done indepth analysis to verify this though. and this also depends upon the PCIe switch not having bugs... There is a relatively inexpensive USB3 to PCIe bridge that lets you issue arbitrary PCIe commands that could be used to verify the security of implementations... > One Thunderbolt 3 controller provides 22gbps of PCIe data bandwidth to > all the one or two Thunderbolt ports it exports, which is fine. [5] > Many Thunderbolt devices allow daisy chaining. An "eGFX" certified [6] > Thunderbolt PCIe chassi (such as [7]) has absolutely no performance > advantage over a normal Thunderbolt PCIe chassi (such as [8]), > including for eGPU (e.g. AMDGPU) use. Good luck! > [1] The lowest cost and most common 10gbps Ethernet Thunderbolt chip > is Aquantia AQC107S. There are also some adapters based on a normal > PCIe 10gbps chip and a separate Thunderbolt to PCIe controller. > > [2] https://www.theregister.co.uk/2017/05/24/intel_thunderbolt_3forall/ > > [3] > https://www.freebsd.org/news/status/report-2015-01-2015-03.html#Adding-PCIe-Hot-plug-Support > https://www.freebsd.org/news/status/report-2015-07-2015-09.html#Adding-PCIe-Hot-plug-Support > > [4] > https://www.osnews.com/story/129501/thunderbolt-enables-severe-security-threats/ > > [5] And not 40gbps as common marketing makes it sound like. > > [6] https://thunderbolttechnology.net/egfx > https://thunderbolttechnology.net/blog/the-difference-between-egfx-and-egpu > = marketing mumbo jumbo. > > [7] https://www.asus.com/Graphics-Cards-Accessories/XG-STATION-PRO/ > > [8] https://www.akitio.com/expansion/node-pro -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
