Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Alexandre Abadie
Hi Kees, Indeed that's a lot of failures! Regarding the failures during flash, the flashing over USB workflow is rather fragile: if something went wrong (a crash) when testing an application, the board might no recover and cannot be flashed automatically anymore after. There's also the issue

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Kees Bakker
Let me explain a little bit more why I am struggling right now. First thing you should know is that I'm creating the support for a new SODAQ board (Sara SFF). Second, I haven't run the automated tests before, so I don't know what to expect. The complete run gives me the result below. No need to

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Kees Bakker
On 31-05-2020 12:33, Martine Sophie Lenders wrote: > Hi, > > Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: >> "... FAILED (due to $reason)", and maybe not change return code to >> something that's an error. > While I like this idea I foresee a usage problem since some tests in > that script fail

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Martine Sophie Lenders
Hi, Am 31.05.20 um 12:25 schrieb Kaspar Schleiser: > "... FAILED (due to $reason)", and maybe not change return code to > something that's an error. While I like this idea I foresee a usage problem since some tests in that script fail due to missing root permissions (or missing scapy_unroot). If

Re: [riot-devel] Which tests are expected to succeed for my board

2020-05-31 Thread Kaspar Schleiser
Hi, On 5/30/20 10:14 PM, Alexandre Abadie wrote: > You can just put all of them in the same issue. It will be easier to track. > That is what is done in [1]. I think we should add this information to compile_and_test.py. Some way to expect failure for specific tests or board/test combinations,