Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
> A typical run takes around 20 mins and the time spent besides running the > tests themselves is a small part. Oh, nice! That's a lot quicker than I expected :) > Well, this is information about the code, so I think it makes sense to store > it alongside it. It really needs to be kept in sync so we can know what code > changes broke the expectations. > > But as Eric mentioned, we should probably store only failures, which will > (hopefully :)) reduce with time. There you go, then :) > Well, a diff will be produced after a failed run. One should be able to just > add that diff to the commit that fixed stuff and submit that for code > review. ...Sounds automated to me! :) > Will checkout a release tag as per Eric's comment. +1 > It's only once every 5k tests, so it may not be that bad :) Sure :P ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
+1 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
On 4/26/19 9:14 PM, Eric Anholt wrote: Alyssa Rosenzweig writes: We start by building a container in Docker that contains a suitable rootfs and kernel for the DUT, deqp and all dependencies for building Mesa itself. Out of curiosity, what's the performance impact of this? If there are no changes to the kernel or to deqp (but mesa had a commit somewhere in Panfrost space), do we have to rebuild the former two? Does ccache maybe pick that up? I'm trying to get a sense for how long it takes between pushing a commit and getting a CI answer, and maybe if that can be shortened. the expectations that are stored in git. Might it be better to track this outside so we don't pollute mesa with changes to that largely autogenerated file? Or I guess that's problematic since then we lose branch information / etc. Hopefully just current expected fails get stored in git. Actually, passes were being stored as well :) Thanks for the idea! Is there an automated way to do this based on the results of LAVA/CI? + git clone --depth 1 https://github.com/KhronosGroup/VK-GL-CTS.git . && \ Is this the right repo? I recall getting deqp source from Google's servers (Chromium git). I suppose it's the same. VK-GL-CTS is the official conformance suite, and it includes dEQP. You need to use a release tag, or you'll have extra garbage tests expecting nonstandardized behavior being run. Same for dEQP master. Done. Thanks, Tomeu ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
On 4/26/19 6:24 PM, Alyssa Rosenzweig wrote: We start by building a container in Docker that contains a suitable rootfs and kernel for the DUT, deqp and all dependencies for building Mesa itself. Out of curiosity, what's the performance impact of this? If there are no changes to the kernel or to deqp (but mesa had a commit somewhere in Panfrost space), do we have to rebuild the former two? Does ccache maybe pick that up? I'm trying to get a sense for how long it takes between pushing a commit and getting a CI answer, and maybe if that can be shortened. A typical run takes around 20 mins and the time spent besides running the tests themselves is a small part. https://gitlab.freedesktop.org/tomeu/mesa/pipelines/33402 We use Docker to cache as much as possible, and then use ccache when building Mesa. Depending on which runner is assigned, we may find a useful ccache dir, or not. the expectations that are stored in git. Might it be better to track this outside so we don't pollute mesa with changes to that largely autogenerated file? Or I guess that's problematic since then we lose branch information / etc. Well, this is information about the code, so I think it makes sense to store it alongside it. It really needs to be kept in sync so we can know what code changes broke the expectations. But as Eric mentioned, we should probably store only failures, which will (hopefully :)) reduce with time. Any code that changes the expectations (hopefully tests are fixed) needs to also update the expectations file. Is there an automated way to do this based on the results of LAVA/CI? Well, a diff will be produced after a failed run. One should be able to just add that diff to the commit that fixed stuff and submit that for code review. + git clone --depth 1 https://github.com/KhronosGroup/VK-GL-CTS.git . && \ Is this the right repo? I recall getting deqp source from Google's servers (Chromium git). I suppose it's the same. Will checkout a release tag as per Eric's comment. + git clone --depth 1 https://gitlab.freedesktop.org/tomeu/mesa.git -b panfrost-ci . && \ U Oops :) +# To prevent memory leaks from slowing throughput, restart everything between batches *blush* It's only once every 5k tests, so it may not be that bad :) Thanks, Tomeu ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
> Hopefully just current expected fails get stored in git. ATM it looks like both passes and fails are there. Also, we're failing thousands of tests in GLES2 alone... > VK-GL-CTS is the official conformance suite, and it includes dEQP. You > need to use a release tag, or you'll have extra garbage tests expecting > nonstandardized behavior being run. Same for dEQP master. Ah-ha, gotcha, thank you! ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
Alyssa Rosenzweig writes: >> We start by building a container in Docker that contains a suitable >> rootfs and kernel for the DUT, deqp and all dependencies for building >> Mesa itself. > > Out of curiosity, what's the performance impact of this? If there are no > changes to the kernel or to deqp (but mesa had a commit somewhere in > Panfrost space), do we have to rebuild the former two? Does ccache maybe > pick that up? I'm trying to get a sense for how long it takes between > pushing a commit and getting a CI answer, and maybe if that can be > shortened. > >> the expectations that are stored >> in git. > > Might it be better to track this outside so we don't pollute mesa with > changes to that largely autogenerated file? Or I guess that's > problematic since then we lose branch information / etc. Hopefully just current expected fails get stored in git. > Is there an automated way to do this based on the results of LAVA/CI? >> + git clone --depth 1 https://github.com/KhronosGroup/VK-GL-CTS.git . >> && \ > > Is this the right repo? I recall getting deqp source from Google's > servers (Chromium git). I suppose it's the same. VK-GL-CTS is the official conformance suite, and it includes dEQP. You need to use a release tag, or you'll have extra garbage tests expecting nonstandardized behavior being run. Same for dEQP master. signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] [PATCH] panfrosti/ci: Initial commit
> We start by building a container in Docker that contains a suitable > rootfs and kernel for the DUT, deqp and all dependencies for building > Mesa itself. Out of curiosity, what's the performance impact of this? If there are no changes to the kernel or to deqp (but mesa had a commit somewhere in Panfrost space), do we have to rebuild the former two? Does ccache maybe pick that up? I'm trying to get a sense for how long it takes between pushing a commit and getting a CI answer, and maybe if that can be shortened. > the expectations that are stored > in git. Might it be better to track this outside so we don't pollute mesa with changes to that largely autogenerated file? Or I guess that's problematic since then we lose branch information / etc. > Any code that changes the expectations (hopefully tests are > fixed) needs to also update the expectations file. Is there an automated way to do this based on the results of LAVA/CI? > + git clone --depth 1 https://github.com/KhronosGroup/VK-GL-CTS.git . >&& \ Is this the right repo? I recall getting deqp source from Google's servers (Chromium git). I suppose it's the same. > + git clone --depth 1 https://gitlab.freedesktop.org/tomeu/mesa.git -b > panfrost-ci . && \ U > +# To prevent memory leaks from slowing throughput, restart everything > between batches *blush* ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
