Thanks for the info, Rahul! Is there any way to specify a different compiler (e.g., clang) when building on kokoro?
Thanks, Jason On Fri, May 3, 2019 at 9:12 AM Rahul Thakur <[email protected]> wrote: > Hi Giacomo, Jason, > > Sorry for late reply. > Re: tool chain - scons and python are available in the test environment > AFAIK. > Re: test throughput - How long does kokoro take to finish current > presubmit test? > > If you have much longer tests to run at periodic frequency, say once in a > few days, on HEAD, kokoro also has a periodic continuous build mode. Such > test will send out email report, serving as a pulse check for ToT. > Presubmit jobs will not be influenced. I can look in to periodic builds if > it fits for gem5 long regression use case. > > RT > > On Fri, May 3, 2019 at 6:17 AM Giacomo Travaglini < > [email protected]> wrote: > >> Hi Jason, >> >> I have seen patches being under review for a long time, and IMHO adding >> an extra 10-30 mins is not the real bottleneck. >> I'd rather wait a little bit more but being sure I am not breaking >> anything... >> >> about ruby protocols, let me say that we (in arm) are relatively happy >> with the current setup: >> >> 1) Quick regressions are run on a commit base (your presubmit.sh). This >> means it won't take a lot of time/computation >> 2) Long regressions (ruby protocols are tested here) are run every night >> on a batch of MERGED commits. >> Which means at a certain point we just checkout origin/HEAD and we run >> long regressions. >> >> This is the setup I would actually recommend: a sanity suite (quick) >> being run on a commit base, and more serious tests >> (long) being run periodically. If you are scared about overloading the >> framework, you can always scale down our ambition >> and run long regressions every two nights. >> Running less frequently is better than not running at all 🙂 >> >> Please let me know what you think about this; other devs are welcome to >> comment as well, >> >> Giacomo >> >> >> ------------------------------ >> *From:* Jason Lowe-Power <[email protected]> >> *Sent:* 02 May 2019 22:59 >> *To:* Giacomo Travaglini; Rahul Thakur >> *Cc:* gem5 Developer List >> *Subject:* Re: [gem5-dev] Continuous integration is live! >> >> Hi Giacomo, >> >> In tests/main.py we call scons and use the current environment defaults >> to build gem5. I don't know if the kokoro infrastructure supports other >> compilers. This might be something that Rahul can address. >> >> I'm also not sure if we can find a way to run more compilations in >> parallel on Kokoro. I'm happy to refactor the test scripts to do this. >> However, as it is, we are currently compiling at least 4 binaries mostly >> sequentially, which is making the testing take a significant amount of >> time. If we add more compilers (and more Ruby protocols), this is going to >> begin to get out of hand. It would also be good to compile .fast, .opt, and >> .debug, but I believe we're only compiling .opt right now. >> >> Cheers, >> Jason >> >> On Thu, May 2, 2019 at 5:53 AM Giacomo Travaglini < >> [email protected]> wrote: >> >> Hi Jason, >> >> I understand; Another thing I would like to ask: >> >> Which script is building gem5 in jenkins? Ideally it would be nice to >> build with BOTH gcc and clang (so that we avoid >> periodic "fix clang build" patches. I would also make the version >> configurable/visible from the script so that >> we can track changes in compiler support and people can compare failures >> in case they managed to build >> seamlessly on their local workspace >> >> Giacomo >> ------------------------------ >> *From:* Jason Lowe-Power <[email protected]> >> *Sent:* 26 April 2019 17:49 >> *To:* Giacomo Travaglini >> *Cc:* gem5 Developer List >> *Subject:* Re: [gem5-dev] Continuous integration is live! >> >> Hi Giacomo, >> >> You *do* have permission :). Anyone can modify >> tests/jenkins/presubmit.cfg and presubmit.sh. In fact, if you look at the >> history of the presubmit.sh, it *was* running the old tests. See >> https://gem5-review.googlesource.com/c/testing/jenkins-gem5-prod/+/18028, >> for instance. >> >> The problem is that we can't distribute most of the binaries (e.g., SPEC >> binaries). We could probably upload them to a private location on the >> Google Cloud and have jenkins consume them that way, but I believe that >> will be more work than it's worth. >> >> I personally believe that putting effort into porting tests is more worth >> everyone's time than trying to get the old tests to run, but that's just >> my opinion. I'm happy to merge changes to run the old tests. I personally >> believe we should only merge tests into the verification tester which >> everyone can run locally, but I'm open to proprietary tests, especially in >> the short term if we have a plan to make them not proprietary. >> >> Cheers, >> Jason >> >> On Fri, Apr 26, 2019 at 9:36 AM Giacomo Travaglini < >> [email protected]> wrote: >> >> Hi Jason, >> >> It's really amazing that we have a testing framework in place, thanks for >> your effort! >> At the moment as far as I can tell we are only running tests registered >> within the new >> testing library. >> >> I was wondering if we could temporarily enable the system to run legacy >> quick regressions as well, >> while waiting for porting those to the new library. I guess it is >> something that shouldn't require a lot of work >> >> (just calling .util/regress I guess) >> >> I am saying this since a patch recently merged broke some syscall >> emulation tests and I think it would >> be beneficial for us to run the entire test suite straightaway while >> porting tests manually. >> I could even handle it myself if I had permission to configure the system. >> >> Let me know your thoughts, >> >> Giacomo >> >> ------------------------------ >> *From:* gem5-dev <[email protected]> on behalf of Jason >> Lowe-Power <[email protected]> >> *Sent:* 16 April 2019 16:30 >> *To:* gem5 Developer List; Rahul Thakur >> *Subject:* [gem5-dev] Continuous integration is live! >> >> Hi all, >> >> We now have initial support for continuous integration testing! We should >> all thank Google for donating the CPU time and infrastructure to run these >> tests. Specifically, Rahul Thakur has been incredibly helpful for the past >> two years in getting this off the ground. Thanks, Rahul and the rest of >> the >> team at Google who has been helping us set this up! >> >> Now, if you submit a patch to gerrit and receive a maintainer +1, "kokoro" >> will kick off a build / test of gem5. Once that is complete, you will >> receive a verified +1. If it fails, you will receive a verified -1. The >> logs can be viewed by anyone once the job is completed by following the >> link posted by kokoro (the https://source.cloud.google.com, not the >> sponge >> link). You can see an example on a patch I recently submitted here: >> https://gem5-review.googlesource.com/c/public/gem5/+/18068. Note that the >> tests take a couple of hours to run. However, I believe there is no limit >> to the number of different changes that can be tested at the same time. >> >> Soon, we are going to enable commit gating with the verified +1 tag. I.e., >> you will have to pass the continuous integration tests before you can >> commit your code. >> >> Note that this is using the "new" testing infrastructure. You can run this >> locally by running "./main.py" in the tests directory. More information >> about how to run tests and add tests can be found in the TESTING.md file. >> If there are any questions/issues do not hesitate to contact me or the >> list. The documentation for the new infrastructure can still be improved. >> Right now, we're running about 30 tests. You can find the tests that we >> are >> running in the tests/gem5 directory. >> >> We are looking for volunteers to help us port more of the old tests to the >> new infrastructure and to expand the coverage of our tests. I'm happy to >> help anyone get started on this and point out which tests still need to be >> migrated, where our biggest coverage holes are, etc. >> >> Cheers, >> Jason >> _______________________________________________ >> gem5-dev mailing list >> [email protected] >> http://m5sim.org/mailman/listinfo/gem5-dev >> IMPORTANT NOTICE: The contents of this email and any attachments are >> confidential and may also be privileged. If you are not the intended >> recipient, please notify the sender immediately and do not disclose the >> contents to any other person, use it for any purpose, or store or copy the >> information in any medium. Thank you. >> >> IMPORTANT NOTICE: The contents of this email and any attachments are >> confidential and may also be privileged. If you are not the intended >> recipient, please notify the sender immediately and do not disclose the >> contents to any other person, use it for any purpose, or store or copy the >> information in any medium. Thank you. >> >> IMPORTANT NOTICE: The contents of this email and any attachments are >> confidential and may also be privileged. If you are not the intended >> recipient, please notify the sender immediately and do not disclose the >> contents to any other person, use it for any purpose, or store or copy the >> information in any medium. Thank you. >> > _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
