Hey All, +1 binding - signatures, sums, licences all work. I run it with some example dags and it looks generally OK.
I have not run detailed tests for all cases but I checked #18504. The cached (not real problem but will save headeache to some people who run airflow) memory used by scheduler will not grow because of logs any more: Page Cache Before (rc1): RES PAGES SIZE FILE 8K 2 6.8K ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log 12K 3 9.1K ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log 12K 3 9.6K ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log 12K 3 10.2K ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log 12K 3 10K ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log 12K 3 9.7K ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log 12K 3 9K ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log 12K 3 10.7K ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log 12K 3 8.8K ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log 12K 3 8.8K ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log 12K 3 10K ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log 12K 3 9.3K ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log 20K 5 16.4K ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log 12K 3 9.4K ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log 12K 3 9.9K ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log 12K 3 10.1K ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log 12K 3 9.3K ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log 12K 3 10K ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log 12K 3 9.8K ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log 12K 3 10.5K ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log 12K 3 9.5K ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log 12K 3 9.7K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log 12K 3 10.3K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log 12K 3 9K ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log 12K 3 9.4K ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log 12K 3 11.7K ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log 12K 3 9.7K ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log 12K 3 9.5K ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log 12K 3 10.2K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log 12K 3 10.2K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log 12K 3 9.5K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log 8K 2 6.8K ./scheduler/2021-09-17/test_dag.py.log 8K 2 7.1K ./scheduler/2021-09-17/test_dag_failure.py.log 8K 2 7.2K ./scheduler/2021-09-17/example_druid.py.log 12K 3 10.3K ./scheduler/2021-09-17/example_docker.py.log Page Cache After (rc2): I read fully the subdag.py.log and it was fully cached but all other logs generated used just one or two pages of the cache even if they were much bigger (if they were not actually read). RES PAGES SIZE FILE 24K 6 20.3K ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log 4K 1 30.6K ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log 4K 1 26.5K ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log 4K 1 27.3K ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log 8K 2 28.8K ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log 8K 2 28.9K ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log 4K 1 27.1K ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log 8K 2 32.1K ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log 4K 1 26.3K ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log 8K 2 28.1K ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log 8K 2 29.8K ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log 8K 2 30.1K ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log 8K 2 28.4K ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log 8K 2 30.1K ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log 4K 1 27.8K ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log 8K 2 49.2K ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log 4K 1 27.9K ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log 4K 1 30K ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log 4K 1 29.3K ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log 4K 1 31.6K ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log 4K 1 34.9K ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log 4K 1 30.2K ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log 8K 2 29.1K ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log 8K 2 28.9K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log 4K 1 30.8K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log 4K 1 27.1K ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log 8K 2 28.1K ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log 8K 2 28.6K ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log 8K 2 30.5K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log 4K 1 30.7K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log 8K 2 28.6K ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log 8K 2 20.4K ./scheduler/2021-09-17/test_dag.py.log 8K 2 21.2K ./scheduler/2021-09-17/test_dag_failure.py.log 8K 2 21.6K ./scheduler/2021-09-17/example_druid.py.log 4K 1 30.8K ./scheduler/2021-09-17/example_docker.py.log J. On Fri, Sep 17, 2021 at 1:50 PM Kaxil Naik <[email protected]> wrote: > Gentle ping to test rc2 > > On Wed, Sep 15, 2021 at 11:18 PM Kaxil Naik <[email protected]> wrote: > >> Just 3 code changes from 2.1.4rc1: >> >> - Advises the kernel to not cache log files generated by Airflow (#18054 >> <https://github.com/apache/airflow/pull/18054>) >> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method (#18208 >> <https://github.com/apache/airflow/pull/18208>) >> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886 >> <https://github.com/apache/airflow/pull/17886>) >> >> >> On Wed, Sep 15, 2021 at 10:50 PM Kaxil Naik <[email protected]> wrote: >> >>> Hey fellow Airflowers, >>> >>> I have cut Airflow 2.1.4rc2. This email is calling a vote on the >>> release, which will last for 72 hours, from September 15, 2021 at 9:50pm >>> UTC until September 18, 2021 at 9:50pm UTC, or until 3 binding +1 votes >>> have been received. >>> >>> Airflow 2.1.4 RC2 is available at: >>> https://dist.apache.org/repos/dist/dev/airflow/2.1.4rc2/ >>> >>> *apache-airflow-2.1.4-source.tar.gz* is a source release that comes >>> with INSTALL instructions. >>> *apache-airflow-2.1.4.tar.gz* is the binary Python "sdist" release. >>> *apache_airflow-2.1.4-py3-none-any.whl* is the binary Python wheel >>> "binary" release. >>> >>> Public keys are available at: >>> https://dist.apache.org/repos/dist/release/airflow/KEYS >>> >>> Consider this my +1 binding vote >>> >>> Please vote accordingly: >>> >>> [ ] +1 approve >>> [ ] +0 no opinion >>> [ ] -1 disapprove with the reason >>> >>> Only votes from PMC members are binding, but all members of the >>> community are encouraged to test the release and vote with "(non-binding)". >>> >>> The test procedure for PMCs and Contributors who would like to test this >>> RC are described in >>> >>> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmcs >>> >>> Please note that the version number excludes the `rcX` string, so it's >>> now >>> simply 2.1.4. This will allow us to rename the artifact without modifying >>> the artifact checksums when we actually release. >>> >>> Changes since 2.1.4rc1 below ( >>> https://github.com/apache/airflow/blob/2.1.4rc2/CHANGELOG.txt). >>> >>> *Bug Fixes* >>> >>> - Advises the kernel to not cache log files generated by Airflow (#18054) >>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method (#18208) >>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886) >>> >>> *Doc-only Changes* >>> >>> - Doc: Use ``closer.lua`` script for downloading sources (#18179) >>> - Doc: Improve installing from sources (#18194) >>> - Improves installing from sources pages for all components (#18251) >>> >>> >>> Thanks, >>> Kaxil >>> >>
