kaxil commented on PR #73477: URL: https://github.com/apache/airflow/pull/73477#issuecomment-5773367697
> I got to the deep layers of memory and recalled the errors that led to the weird (but I think needed) up to 4 rebuilds scenarios needed sometimes @kaxil -> I am afraid the way it is simplified here will make some edge (and some not-that-edge) cases of inter-package dependent PRs to fail. > > The issue is that: > > a) spellcheck-only builds do not create inventories when they fail (!). They only use existing ones. This is why they are faster in general. > > b) when you run PR you cannot update inventories in "S3" - what you have there is result of last successful main build > > c) when you have PRs that modifies several providars and they add new methods that are used in other providers (almost always when you modify some common provideds) -> you **need** new inventories from the providers you depend on even if you are just doing spellchecking. > > d) we have no topological sort and no way to hold building new providers until provider dependencies that add new methods build their inventories. > > e) when you have unrelated errors and they prevent from creating inventories, all you really see in the output are "missing things" which are not really missing. > > See the detailed scenario in the inline comment. > > I am sure we **could** optimize it more but it's not as easy as just repeating spellcheck buit Replied inline -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
