kgyrtkirk commented on PR #15106: URL: https://github.com/apache/druid/pull/15106#issuecomment-1760824575
> > could you please reference a failed build which have failed because of that caching? > I have experienced this locally where I had artifacts from a different build causing integration tests to fail. This change removes that possibility. See also further below in my comment for an example that happened as part of this change. I don't understand how these caches affect your local builds - if you can't point to a build which have failed/misbehaved then I don't really understand why do we need to change this. those artifacts could be there and cached and reloaded - but they will not affect things; as they will get overwritten by the local build because incremental compile could not kick in for a fresh checkout - so for the install target it will need to compile and create the artifacats from 0. > [...] the goal of caching is to download as many dependencies as possible without affecting the integrity [...] I don't think it should be a best-effort thing...we should try to avoid as external (re)downloads as possible... let's say you miss 1 artifact; and you cache stuff and fan out to a 100 jobs; that will be cause 100 hits on that artifact I think as an alternative we could probably purge the local druid artifacts (remove `~/.m2/repository/org/apache/druid/` ?) at the end of the job so that they are not placed in the caches > The pom hash is not sufficient, different code could have the same pom hash, causing incorrect artifacts in the maven repository from being used at runtime. Some tests may also accidentally pass due to artifacts being present instead of failing if they are not rebuilt. That depends on the usecase: * If it just builds the project from zero it should work - since the actual code should not affect the referenced dependencies. * if its used during testing to *avoid* the rebuild of the project; like [here](https://github.com/apache/druid/blob/d25caaefa4282a6b61663e8184d3c0b4092f6f35/.github/workflows/reusable-revised-its.yml#L84-L105) then yes - all files of the project should be accounted for - as it will try to *evade* the build phase Could you describe the scenario you have in mind? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
