Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-10-02 Thread Jörg Saßmannshausen
# ib_write_bw > > The test should fail. > > Hope that helps > Ade > > > > -Original Message- > From: Beowulf On Behalf Of Jörg Saßmannshausen > Sent: 02 October 2018 22:20 > To: beowulf@beowulf.org > Subject: Re: [Beowulf] RHEL7 kernel upda

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-10-02 Thread Ade Fewings
On client side run : # ib_write_bw The test should fail. Hope that helps Ade -Original Message- From: Beowulf On Behalf Of Jörg Saßmannshausen Sent: 02 October 2018 22:20 To: beowulf@beowulf.org Subject: Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA Dear all

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-10-02 Thread Jörg Saßmannshausen
Dear all, is there some kind of quick test to demonstrate the patch does have or does not cause a problem with RDMA? I have been asked to look into that but I don't really want to use a large cp2k calculation which, I believe, makes use of RDMA. All the best from London Jörg Am Mittwoch,

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-10-01 Thread Ryan Novosielski
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/10/2018 08:41 PM, Kilian Cavalotti wrote: > On Mon, Sep 10, 2018 at 4:18 PM Ryan Novosielski > wrote: >> So we’ve learned what, here, that RedHat doesn’t test the RDMA >> stack at all? > > Looks like Spectre-like vulns take all precedence,

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-12 Thread John Hearns via Beowulf
Regarding CentOS, Karanbir Singh is the leader of the project and has a job at Redhat https://www.linuxfoundation.org/blog/2014/01/centos-project-leader-karanbir-singh-opens-up-on-red-hat-deal/ On Tue, 11 Sep 2018 at 18:03, Peter St. John wrote: > > I mean the RH QA that tests RH products isn't

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-11 Thread Peter St. John
I mean the RH QA that tests RH products isn't the same team as tests (or not) CentOS, but I only know from the wiki that RH has an expanding agreement with CentOS so may be this is all merging. As I said, my buddy doesn't work in this area, and I sure don't. Probably all you guys are more up to

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-11 Thread Peter St. John
I mean the RH QA that tests RH products isn't the same team as tests (or not) CentOS, but I only know from the wiki that RH has an expanding agreement with CentOS so may be this is all merging. As I said, my buddy doesn't work in this area, and I sure don't. Probably all you guys are more up to

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-11 Thread Peter Kjellström
On Tue, 11 Sep 2018 12:37:18 -0400 "Peter St. John" wrote: > A friend at RH (who works in a different area) tells me RH does not > themselves test the downstream CentOS. > Peter That isn't surprising is it? But in this case we're talking about them not testing their own product.. :-D /Peter K

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-11 Thread Peter St. John
A friend at RH (who works in a different area) tells me RH does not themselves test the downstream CentOS. Peter On Tue, Sep 11, 2018 at 8:32 AM, Peter Kjellström wrote: > On Mon, 10 Sep 2018 23:17:21 + > Ryan Novosielski wrote: > ... > > So we’ve learned what, here, that RedHat doesn’t

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-11 Thread Peter Kjellström
On Mon, 10 Sep 2018 23:17:21 + Ryan Novosielski wrote: ... > So we’ve learned what, here, that RedHat doesn’t test the RDMA stack > at all? This we knew already since last time they completely destroyed it with an update (and took >month to fix). /Peter -- Sent from my Android device with

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Chris Samuel
On Tuesday, 11 September 2018 10:41:24 AM AEST Kilian Cavalotti wrote: > Last I heard, the fix will be in 862.14.1 to be released on the 25th Ah interesting, I wonder if that fix is already in the 3.10.0-933 kernel that's meant to be in the RHEL 7.6 beta? -- Chris Samuel :

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Chris Samuel
On Tuesday, 11 September 2018 9:17:21 AM AEST Ryan Novosielski wrote: > So we’ve learned what, here, that RedHat doesn’t test the RDMA stack at all? It certainly does seem to be the case. Unlike other issues I've hit in the past with bugs introduced in the IB stack in 6.x -> 6.y transitions

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Kilian Cavalotti
On Mon, Sep 10, 2018 at 4:18 PM Ryan Novosielski wrote: > So we’ve learned what, here, that RedHat doesn’t test the RDMA stack at all? Looks like Spectre-like vulns take all precedence, these days, indeed. Last I heard, the fix will be in 862.14.1 to be released on the 25th Cheers, -- Kilian

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Ryan Novosielski
> On Sep 10, 2018, at 18:15, Chris Samuel wrote: > >> On Tuesday, 11 September 2018 1:25:55 AM AEST Peter St. John wrote: >> >> I had wanted to say that such a bug would be caught by compiling with some >> reasonalbe warning level; but I think I was wrong. > > Interesting - looks like it

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Peter St. John
yes the gcc I used is 5.1, I guess that's how long I've had this laptop :-) And I like that "not guarding" that sounds useful. On Mon, Sep 10, 2018 at 6:15 PM, Chris Samuel wrote: > On Tuesday, 11 September 2018 1:25:55 AM AEST Peter St. John wrote: > > > I had wanted to say that such a bug

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Chris Samuel
On Tuesday, 11 September 2018 1:25:55 AM AEST Peter St. John wrote: > I had wanted to say that such a bug would be caught by compiling with some > reasonalbe warning level; but I think I was wrong. Interesting - looks like it depends on your GCC version, 7.3.0 catches it with -Wall here:

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Peter St. John
I had wanted to say that such a bug would be caught by compiling with some reasonalbe warning level; but I think I was wrong. I compiled if(1==1); with some wrapper and got nothing with whatever gcc I have on this laptop, until gcc -Wextra which is more persnickety than -Wall, and just got

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread John Hearns via Beowulf
Linux should have coded the kernel in Python then. Easily caught there. (Yes. I am making a joke) On Mon, 10 Sep 2018 at 09:23, Chris Samuel wrote: > > On Friday, 17 August 2018 2:47:37 PM AEST Chris Samuel wrote: > > > Just a heads up that the 3.10.0-862.11.6.el7.x86_64 kernel from RHEL/CentOS

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-09-10 Thread Chris Samuel
On Friday, 17 August 2018 2:47:37 PM AEST Chris Samuel wrote: > Just a heads up that the 3.10.0-862.11.6.el7.x86_64 kernel from RHEL/CentOS > that was released to address the most recent Intel CPU problem "L1TF" seems > to break RDMA (found by a colleague here at Swinburne). So this CentOS bug

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-23 Thread John Hearns via Beowulf
My bad. The license has been updated now https://www.theregister.co.uk/2018/08/23/intel_microcode_license/ On Thu, 23 Aug 2018 at 20:11, John Hearns wrote: > https://www.theregister.co.uk/2018/08/21/intel_cpu_patch_licence/ > > >

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-23 Thread John Hearns via Beowulf
https://www.theregister.co.uk/2018/08/21/intel_cpu_patch_licence/ https://perens.com/2018/08/22/new-intel-microcode-license-restriction-is-not-acceptable/ On Tue, 21 Aug 2018 at 16:18, Lux, Jim (337K) wrote: > > > On 8/21/18, 1:37 AM, "Beowulf on behalf of Chris Samuel" < >

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-21 Thread Lux, Jim (337K)
On 8/21/18, 1:37 AM, "Beowulf on behalf of Chris Samuel" wrote: On Tuesday, 21 August 2018 3:27:59 AM AEST Lux, Jim (337K) wrote: > I'd find it hard to believe that Intel's CPU designers sat around > implementing deliberate flaws ( the Bosch engine controller for VW model).

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-21 Thread Chris Samuel
On Tuesday, 21 August 2018 3:27:59 AM AEST Lux, Jim (337K) wrote: > I'd find it hard to believe that Intel's CPU designers sat around > implementing deliberate flaws ( the Bosch engine controller for VW model). Not to mention that Spectre variants affected AMD, ARM & IBM (at least). This

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-20 Thread Lux, Jim (337K)
wulf [mailto:beowulf-boun...@beowulf.org] On Behalf Of Jörg Saßmannshausen Sent: Sunday, August 19, 2018 2:00 PM To: beowulf@beowulf.org Subject: Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA Dear all, whereas I am accepting that no system is 100% secure ans bug-free, I am

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-19 Thread Jonathan Engwall
Thank you On August 19, 2018, at 2:10 PM, Chris Samuel wrote: On Monday, 20 August 2018 6:32:26 AM AEST Jonathan Engwall wrote: > I am not shocked that my previous message may have been removed. To clarify: nothing has been removed to my knowledge. Your email is in the list archives.

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-19 Thread Chris Samuel
On Monday, 20 August 2018 6:32:26 AM AEST Jonathan Engwall wrote: > I am not shocked that my previous message may have been removed. To clarify: nothing has been removed to my knowledge. Your email is in the list archives. http://beowulf.org/pipermail/beowulf/2018-August/035219.html All the

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-19 Thread Jörg Saßmannshausen
Dear all, whereas I am accepting that no system is 100% secure ans bug-free, I am beginning to wonder whether the current problems we are having are actually design flaws and whether, and that is the more important bit, Intel and other vendors did know about it. I am thinking of the famous

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-19 Thread Jonathan Engwall
As far as vulnerabilities go, here is a terrible idea: Write a little login patch that grabs your own email address and uses it to attempt to login to Facebook without a password 1000 times per second. Kill the script after two seconds. You want to read the Facebook head first so you can kick

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread John Hearns via Beowulf
Rather more seriously, this is a topic which is well worth discussing, What are best practices on patching HPC systems? Perhaps we need a separate thread here. I will throw in one thought, which I honestly do not want to see happening. I recently took a trip to Bletchley Park in the UK. On

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread John Hearns via Beowulf
*To patch, or not to patch, that is the question:* Whether 'tis nobler in the mind to suffer The loops and branches of speculative execution, Or to take arms against a sea of exploits And by opposing end them. To die—to sleep, No more; and by a sleep to say we end The heart-ache and the thousand

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Chris Samuel
On Sunday, 19 August 2018 5:19:07 AM AEST Jeff Johnson wrote: > With the spate of security flaws over the past year and the impacts their > fixes have on performance and functionality it might be worthwhile to just > run airgapped. For me none of the HPC systems I've been involved with here in

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Chris Samuel
On Saturday, 18 August 2018 11:55:22 PM AEST Jörg Saßmannshausen wrote: > So I don't really understand about "Cannot make this public, as the patch > that caused it was due to embargo'd security fix." issue. I don't think any of us do, unless there's another fix there that is for an undisclosed

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Joe Landman
FWIW: it looks like this is the CVE that keeps on giving. Yesterday some of the mitigation hit, and this morning a new rev of kernel with a single CVE patch came out.   Don't know when it might show up in distro kernels, but its already in mine. We are not done with Spectre/Meltdown vulns by

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Jeff Johnson
With the spate of security flaws over the past year and the impacts their fixes have on performance and functionality it might be worthwhile to just run airgapped. On Thu, Aug 16, 2018 at 22:48 Chris Samuel wrote: > Hi all, > > Just a heads up that the 3.10.0-862.11.6.el7.x86_64 kernel from >

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Jörg Saßmannshausen
Hi Chris, unless there is something I miss but I read about that in 'Der Spiegel Online' on Wednesday http://www.spiegel.de/netzwelt/gadgets/foreshadow-neue-angriffsmethode-trifft-intel-chips-und-cloud-dienste-a-1223289.html and the link was to this page here: https://foreshadowattack.eu/

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Chris Samuel
On 18/8/18 8:47 pm, Jörg Saßmannshausen wrote: Hi Chris, Hiya, these are bad news if InfiniBand will be affected here as well as that is what we need to use for parallel calculations. They make use of RMDA and if that has a problem. well, you get the idea I guess. Oh yes, this is why

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Jörg Saßmannshausen
Hi Chris, these are bad news if InfiniBand will be affected here as well as that is what we need to use for parallel calculations. They make use of RMDA and if that has a problem. well, you get the idea I guess. Has anybody contacted the vendors like Mellanox or Intel regarding this? All

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Christopher Samuel
On 18/08/18 17:22, Jörg Saßmannshausen wrote: if the problem is RMDA, how about InfiniBand? Will that be broken as well? For RDMA it appears yes, though IPoIB still works for us (though ours is OPA rather than IB Kilian reported the same). All the best, Chris -- Chris Samuel :

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-18 Thread Jörg Saßmannshausen
Dear all, if the problem is RMDA, how about InfiniBand? Will that be broken as well? All the best Jörg Am Samstag, 18. August 2018, 13:33:55 BST schrieb Chris Samuel: > On Saturday, 18 August 2018 12:54:03 AM AEST Kilian Cavalotti wrote: > > That's true: RH mentioned an "embargo'd security

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-17 Thread Chris Samuel
On Saturday, 18 August 2018 12:54:03 AM AEST Kilian Cavalotti wrote: > That's true: RH mentioned an "embargo'd security fix" but didn't refer > to L1TF explicitly (which I think is not under embargo anymore). Agreed, though I'm not sure any of the listed fixes are embargoed now. > As the

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-17 Thread Jörg Saßmannshausen
Hi all, I came across the 'foreshadow' problem 2 days ago. This is what I got back from my colleagues: https://access.redhat.com/security/vulnerabilities/L1TF-perf This is more a performance investigation though but I thought I might add a bit more information to the whole problem. All the

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-17 Thread Kilian Cavalotti
Hi Chris, On Thu, Aug 16, 2018 at 10:05 PM, Chris Samuel wrote: > There's 6 CVE's addressed in that update from the look of it, so it might not > be the L1TF fix itself that has triggered it. > > https://access.redhat.com/errata/RHSA-2018:2384 That's true: RH mentioned an "embargo'd security

Re: [Beowulf] RHEL7 kernel update for L1TF vulnerability breaks RDMA

2018-08-16 Thread Chris Samuel
On Friday, 17 August 2018 2:47:37 PM AEST Chris Samuel wrote: > Just a heads up that the 3.10.0-862.11.6.el7.x86_64 kernel from RHEL/CentOS > that was released to address the most recent Intel CPU problem "L1TF" seems > to break RDMA (found by a colleague here at Swinburne). There's 6 CVE's