potiuk edited a comment on pull request #14531: URL: https://github.com/apache/airflow/pull/14531#issuecomment-789392406
> Have you double checked the build time with this change on one of the proposed AWS instances? I'm nervous of blowing throw our AWS credits in 1 month if this change _doesn't_ help. Yes. I already explained it when I created the PR (just scroll the PR few pages up).: https://github.com/apache/airflow/pull/14531#issuecomment-787506971 but I can explain it again and again, and again no problem. I am super patient person, though sometimes I am quite surprised that I have to answer questions that already have been answered (and ones that you even commented on). This is not the first time over the last few weeks when I shared the data with you. The aim of this change (as I also explained already several time is to actually drive the cost down). This was my goal from the very beginning and I even did some initial estimations which I shared with you in this document when I actually run those tests manually https://docs.google.com/document/d/1ZZeZ4BYMNX7ycGRUKAXv0s6etz1g-90Onn5nRQQHOfE/edit#heading=h.mkysbyj2zg1t - you even commented on that document. So answering your short question in long form - yes I did, and you had all chances to see all the results before. With the current 32 GB instances the whole suite of tests runs 26 minutes instead of > 1 hour with this solution. It fails and runs slower than it can because it swaps out memory (it needs around 35 GB of memory when I tested it locally) so it will be much faster when we have bigger memory. My rough estimations is that it will run for ~ 15 minutes. But for sure it will not run for > 30 minutes which means that we save at least 6 build hours with every PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
