potiuk commented on a change in pull request #18356:
URL: https://github.com/apache/airflow/pull/18356#discussion_r712916297
##########
File path: docs/apache-airflow/best-practices.rst
##########
@@ -255,9 +297,34 @@ No additional code needs to be written by the user to run
this test.
.. code-block:: bash
- python your-dag-file.py
+ python your-dag-file.py
+
+Running the above command without any error ensures your DAG does not contain
any uninstalled dependency,
+syntax errors, etc. Make sure that you load your DAG in an environment that
corresponds to your
+scheduler environment - with the same dependencies, environment variables,
common code referred from the
+DAG.
+
+This is also a great way to check if your DAG loads faster after an
optimization, if you want to attempt
+to optimize DAG loading time. Simply run the DAG and measure the time it
takes, but again you have to
+make sure your DAG runs with the same dependencies, environment variables,
common code.
+Make sure to run it several time in succession to account for caching effects.
Compare the results
+before and after the optimization in order to assess the impact of the
optimization.
+
+There are many ways to measure the time of processing, one of them in Linux
environment is to
+use built-in ``time`` command
+
+.. code-block:: bash
+
+ time python your-dag-file.py
+
+Result:
+
+.. code-block:: text
+
+ python your-dag-file.py 0.05s user 0.02s system 1% cpu 1.033 total
-Running the above command without any error ensures your DAG does not contain
any uninstalled dependency, syntax errors, etc.
+The important metrics is the "total time" - which tells you how long elapsed
time it took
+to process the DAG.
Review comment:
Thought about it, yeah. Might be worth mentioning indeed (though I tried
to be quite clear here that it's about the "improvements" indeed it's good to
make this statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
