Hi Jason, I have seen patches being under review for a long time, and IMHO adding an extra 10-30 mins is not the real bottleneck. I'd rather wait a little bit more but being sure I am not breaking anything...
about ruby protocols, let me say that we (in arm) are relatively happy with the current setup: 1) Quick regressions are run on a commit base (your presubmit.sh). This means it won't take a lot of time/computation 2) Long regressions (ruby protocols are tested here) are run every night on a batch of MERGED commits. Which means at a certain point we just checkout origin/HEAD and we run long regressions. This is the setup I would actually recommend: a sanity suite (quick) being run on a commit base, and more serious tests (long) being run periodically. If you are scared about overloading the framework, you can always scale down our ambition and run long regressions every two nights. Running less frequently is better than not running at all 🙂 Please let me know what you think about this; other devs are welcome to comment as well, Giacomo ________________________________ From: Jason Lowe-Power <ja...@lowepower.com> Sent: 02 May 2019 22:59 To: Giacomo Travaglini; Rahul Thakur Cc: gem5 Developer List Subject: Re: [gem5-dev] Continuous integration is live! Hi Giacomo, In tests/main.py we call scons and use the current environment defaults to build gem5. I don't know if the kokoro infrastructure supports other compilers. This might be something that Rahul can address. I'm also not sure if we can find a way to run more compilations in parallel on Kokoro. I'm happy to refactor the test scripts to do this. However, as it is, we are currently compiling at least 4 binaries mostly sequentially, which is making the testing take a significant amount of time. If we add more compilers (and more Ruby protocols), this is going to begin to get out of hand. It would also be good to compile .fast, .opt, and .debug, but I believe we're only compiling .opt right now. Cheers, Jason On Thu, May 2, 2019 at 5:53 AM Giacomo Travaglini <giacomo.travagl...@arm.com<mailto:giacomo.travagl...@arm.com>> wrote: Hi Jason, I understand; Another thing I would like to ask: Which script is building gem5 in jenkins? Ideally it would be nice to build with BOTH gcc and clang (so that we avoid periodic "fix clang build" patches. I would also make the version configurable/visible from the script so that we can track changes in compiler support and people can compare failures in case they managed to build seamlessly on their local workspace Giacomo ________________________________ From: Jason Lowe-Power <ja...@lowepower.com<mailto:ja...@lowepower.com>> Sent: 26 April 2019 17:49 To: Giacomo Travaglini Cc: gem5 Developer List Subject: Re: [gem5-dev] Continuous integration is live! Hi Giacomo, You *do* have permission :). Anyone can modify tests/jenkins/presubmit.cfg and presubmit.sh. In fact, if you look at the history of the presubmit.sh, it *was* running the old tests. See https://gem5-review.googlesource.com/c/testing/jenkins-gem5-prod/+/18028, for instance. The problem is that we can't distribute most of the binaries (e.g., SPEC binaries). We could probably upload them to a private location on the Google Cloud and have jenkins consume them that way, but I believe that will be more work than it's worth. I personally believe that putting effort into porting tests is more worth everyone's time than trying to get the old tests to run, but that's just my opinion. I'm happy to merge changes to run the old tests. I personally believe we should only merge tests into the verification tester which everyone can run locally, but I'm open to proprietary tests, especially in the short term if we have a plan to make them not proprietary. Cheers, Jason On Fri, Apr 26, 2019 at 9:36 AM Giacomo Travaglini <giacomo.travagl...@arm.com<mailto:giacomo.travagl...@arm.com>> wrote: Hi Jason, It's really amazing that we have a testing framework in place, thanks for your effort! At the moment as far as I can tell we are only running tests registered within the new testing library. I was wondering if we could temporarily enable the system to run legacy quick regressions as well, while waiting for porting those to the new library. I guess it is something that shouldn't require a lot of work (just calling .util/regress I guess) I am saying this since a patch recently merged broke some syscall emulation tests and I think it would be beneficial for us to run the entire test suite straightaway while porting tests manually. I could even handle it myself if I had permission to configure the system. Let me know your thoughts, Giacomo ________________________________ From: gem5-dev <gem5-dev-boun...@gem5.org<mailto:gem5-dev-boun...@gem5.org>> on behalf of Jason Lowe-Power <ja...@lowepower.com<mailto:ja...@lowepower.com>> Sent: 16 April 2019 16:30 To: gem5 Developer List; Rahul Thakur Subject: [gem5-dev] Continuous integration is live! Hi all, We now have initial support for continuous integration testing! We should all thank Google for donating the CPU time and infrastructure to run these tests. Specifically, Rahul Thakur has been incredibly helpful for the past two years in getting this off the ground. Thanks, Rahul and the rest of the team at Google who has been helping us set this up! Now, if you submit a patch to gerrit and receive a maintainer +1, "kokoro" will kick off a build / test of gem5. Once that is complete, you will receive a verified +1. If it fails, you will receive a verified -1. The logs can be viewed by anyone once the job is completed by following the link posted by kokoro (the https://source.cloud.google.com, not the sponge link). You can see an example on a patch I recently submitted here: https://gem5-review.googlesource.com/c/public/gem5/+/18068. Note that the tests take a couple of hours to run. However, I believe there is no limit to the number of different changes that can be tested at the same time. Soon, we are going to enable commit gating with the verified +1 tag. I.e., you will have to pass the continuous integration tests before you can commit your code. Note that this is using the "new" testing infrastructure. You can run this locally by running "./main.py" in the tests directory. More information about how to run tests and add tests can be found in the TESTING.md file. If there are any questions/issues do not hesitate to contact me or the list. The documentation for the new infrastructure can still be improved. Right now, we're running about 30 tests. You can find the tests that we are running in the tests/gem5 directory. We are looking for volunteers to help us port more of the old tests to the new infrastructure and to expand the coverage of our tests. I'm happy to help anyone get started on this and point out which tests still need to be migrated, where our biggest coverage holes are, etc. Cheers, Jason _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org<mailto:gem5-dev@gem5.org> http://m5sim.org/mailman/listinfo/gem5-dev IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
