potiuk commented on issue #19891:
URL: https://github.com/apache/airflow/issues/19891#issuecomment-982946395
Just checked.
Parallel pre-commit is useless for `mypy` - seems that it will not speedup
the tests anyway as mypy pretty much loads as many python code as it reaches
out from the files, and with our code structure, any import to airflow will
pretty much load everything always. So when we run MyPy in parallel, My 16
cores are fully busy and produce the same result in 2m6s as single busy core
in 2m13s. Results are the same.
I will turn on serial mode :).
Serial:
```
[jarek:~/code/airflow] enable-mypy-as-non-failing-step(+3/-4)+ 8s ±
pre-commit run mypy --all-files 2>&1 | sed -r
"s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" | grep "error:" | sort | awk
'FS=" " { print $1 }' | uniq > /tmp/res-serial.out
[jarek:~/code/airflow] enable-mypy-as-non-failing-step(+3/-4)+ 2m13s ± wc
/tmp/res-serial.out
963 963 48752 /tmp/res-serial.out
```
Paralel:
```
[jarek:~/code/airflow] enable-mypy-as-non-failing-step(+3/-4)+ ± pre-commit
run mypy --all-files 2>&1 | sed -r
"s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" | grep "error:" | sort | awk
'FS=" " { print $1 }' | uniq > /tmp/res-paralle.out
[jarek:~/code/airflow] enable-mypy-as-non-failing-step(+1/-2)+ 2m6s ± wc
/tmp/res-paralle.out
963 963 48752 /tmp/res-paralle.out
[jarek:~/code/airflow] enable-mypy-as-non-failing-step(+1/-2)+ ±
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
