potiuk commented on pull request #17911:
URL: https://github.com/apache/airflow/pull/17911#issuecomment-909741345
I believe if those are correct observations (about relation between
different timeouts/thresholds), then we should check those values while
starting components of Airflow and raise Warnings instead (or on top of )
documentation update.
People will not look in the documentation usually, even if they have an
issue they will not look in the docs, but they **might** monitor logs for
warnings and they **willl** find such warnings if we ask them to provide
"suspicious logs" when the see a problem.
Also - we have PLENTY of parameters. Maybe there are other relations between
different values that we should also check and warn the users about @ashb
@kaxil @ephraimbuddy @jedcunningham ?
I think you've been recently chasing quite a number of similar reports/jobs
and I believe there are some of those inter-parameter relations that come to
your mind immediately as invalid and once that we should flag? I saw at least
few advises in the issues in the form ("Hey, your configuration is really wrong
- you should not use X parallelism when you have just Y cores". Maybe we
should figure out and codify some of those in the forms of warning messages?
This eventually means less work for those who answer to the common issues of
various people and far less frustration at the side of the users.
I think that might bring down a number of issues in the future significantly
if we have such checks in-place and raise them as warnings with an explanation
why the warnings are raised.
WDYT? Any other candidates for such "sanity checks with the parameter
configurations" ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
