Re: Re: CI caching improvement

On Mon, Mar 21, 2022 at 04:19:38PM +0500, Илья Шипицин wrote: > > I think we can adjust build-ssl.sh script to download tagged quictls (and > cache it in the way we do cache openssl itself) > Tags · quictls/openssl (github.com) > > Unfortunately that

Re: CI caching improvement

William, On 3/18/22 11:31, William Lallemand wrote: It looks like it is available as well on our repositories, I just test it and it works correctly. Honestly I really don't like the dependency to another repository with a format specific to github. I agree that a cleaner integration with

Re: CI caching improvement

пт, 18 мар. 2022 г. в 15:32, William Lallemand : > On Wed, Mar 16, 2022 at 09:31:56AM +0100, Tim Düsterhus wrote: > > Willy, > > > > On 3/8/22 20:43, Tim Düsterhus wrote: > > >> Yes my point was about VTest. However you made me think about a very > good > > >> reason for caching haproxy builds as

Re: CI caching improvement

On Wed, Mar 16, 2022 at 09:31:56AM +0100, Tim Düsterhus wrote: > Willy, > > On 3/8/22 20:43, Tim Düsterhus wrote: > >> Yes my point was about VTest. However you made me think about a very good > >> reason for caching haproxy builds as well :-) Very commonly, some VTest > >> randomly fails.

Re: CI caching improvement

Willy, On 3/8/22 20:43, Tim Düsterhus wrote: Yes my point was about VTest. However you made me think about a very good reason for caching haproxy builds as well :-) Very commonly, some VTest randomly fails. Timing etc are involved. And at the moment, it's impossible to restart the tests

Re: CI caching improvement

script/build-vtest.sh was/is reused for cirrus,travis On Wed, Mar 9, 2022, 12:05 AM Tim Düsterhus wrote: > William, > > On 3/8/22 16:06, William Lallemand wrote: > > Let me know if we can improve the attached patch, otherwise I'll merge > > it. > > > > Let me make a competing proposal that: > >

Re: [EXTERNAL] Re: CI caching improvement

William, On 3/8/22 21:30, Tim Düsterhus wrote: - The action is also easily reusable by other projects. For testing my Adding to that: It's also easily reusable by the other workflows. We currently have the separate musl.yml workflow that does this:

Re: [EXTERNAL] Re: CI caching improvement

William, On 3/8/22 20:54, William Lallemand wrote: Honestly I'm confused, it is overcomplicated in my opinion :( I don't really see the benefits in creating a whole new repository instead of the few lines in the yaml file. I believe that having a non-trivial amount of logic in a YAML file

Re: [EXTERNAL] Re: CI caching improvement

On Tue, Mar 08, 2022 at 08:05:31PM +0100, Tim Düsterhus wrote: > > Let me make a competing proposal that: > > 1. Keeps the complexity out of the GitHub workflow configuration in > haproxy/haproxy. > 2. Still allows VTest caching. > > For my https://github.com/TimWolla/haproxy-auth-request

Re: CI caching improvement

Willy, On 3/8/22 20:11, Willy Tarreau wrote: Yes my point was about VTest. However you made me think about a very good reason for caching haproxy builds as well :-) Very commonly, some VTest randomly fails. Timing etc are involved. And at the moment, it's impossible to restart the tests

Re: CI caching improvement

On Tue, Mar 08, 2022 at 04:40:40PM +0100, Tim Düsterhus wrote: > > > Please don't. You always want to run a clean build, otherwise you're going > > > to get super-hard-to-debug failures if some object file is accidentally > > > taken from the cache instead of a clean build. > > > > What risk are

Re: CI caching improvement

William, On 3/8/22 16:06, William Lallemand wrote: Let me know if we can improve the attached patch, otherwise I'll merge it. Let me make a competing proposal that: 1. Keeps the complexity out of the GitHub workflow configuration in haproxy/haproxy. 2. Still allows VTest caching. For my

Re: CI caching improvement

On Tue, Mar 08, 2022 at 04:17:00PM +0100, Tim Düsterhus wrote: > William > > On 3/8/22 16:06, William Lallemand wrote: > > Also, I'm wondering if we could also cache the build of HAProxy, you > > could think that weird, but in fact it will help relaunch the tests when > > one is failing, without

Re: CI caching improvement

вт, 8 мар. 2022 г. в 21:13, William Lallemand : > On Tue, Mar 08, 2022 at 08:38:00PM +0500, Илья Шипицин wrote: > > > > I'm fine with swapping "vtest" <--> "haproxy" order. > > > > Ok, I can do that. > > > also, I do not think current patch is ugly, it is acceptable for me (if > we > > agree to

Re: CI caching improvement

On Tue, Mar 08, 2022 at 08:38:00PM +0500, Илья Шипицин wrote: > > I'm fine with swapping "vtest" <--> "haproxy" order. > Ok, I can do that. > also, I do not think current patch is ugly, it is acceptable for me (if we > agree to save 8 sec). Honestly I don't see the value in building the same

Re: CI caching improvement

Willy, On 3/8/22 16:24, Willy Tarreau wrote: Hi Tim, On Tue, Mar 08, 2022 at 04:17:00PM +0100, Tim Düsterhus wrote: William On 3/8/22 16:06, William Lallemand wrote: Also, I'm wondering if we could also cache the build of HAProxy, you could think that weird, but in fact it will help

Re: CI caching improvement

I thought to build "vtest" just once and deliver using artifacts to all jobs. It will save some electricity, also GitHub sometimes throw 429 when we download "vtest" in too many parallel ways. however, it will not speed up, so I postoned that idea (something like that

Re: CI caching improvement

Hi Tim, On Tue, Mar 08, 2022 at 04:17:00PM +0100, Tim Düsterhus wrote: > William > > On 3/8/22 16:06, William Lallemand wrote: > > Also, I'm wondering if we could also cache the build of HAProxy, you > > could think that weird, but in fact it will help relaunch the tests when > > one is failing,

Re: CI caching improvement

Willy, William, On 3/8/22 16:14, Willy Tarreau wrote: Due to this I think we should move the build of vtest after the build of haproxy (and generally, anything that's not needed for the build ought to be moved after). This will at least save whatever can be saved on failed builds. That on the

Re: CI caching improvement

William On 3/8/22 16:06, William Lallemand wrote: Also, I'm wondering if we could also cache the build of HAProxy, you could think that weird, but in fact it will help relaunch the tests when one is failing, without rebuilding the whole thing. Please don't. You always want to run a clean

Re: CI caching improvement

Hi William, On Tue, Mar 08, 2022 at 04:06:45PM +0100, William Lallemand wrote: > Hello, > > The attached patch implements a somehow ugly way to cache the VTest > binary, basically it gets the commit ID by doing a curl of the > master.patch on the github URL. > > It allows to save ~8s per matrix