+1 (non-binding) On Fri, Sep 17, 2021 at 6:58 AM Jarek Potiuk <[email protected]> wrote:
> Hey All, > > +1 binding - signatures, sums, licences all work. I run it with some > example dags and it looks generally OK. > > I have not run detailed tests for all cases but I checked #18504. The > cached (not real problem but will save headeache to some people who run > airflow) memory used by scheduler will not grow because of logs any more: > > Page Cache Before (rc1): > > RES PAGES SIZE FILE > 8K 2 6.8K > ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log > 12K 3 9.1K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log > 12K 3 9.6K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log > 12K 3 10.2K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log > 12K 3 10K > ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log > 12K 3 9.7K > ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log > 12K 3 9K > ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log > 12K 3 10.7K > ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log > 12K 3 8.8K > ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log > 12K 3 8.8K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log > 12K 3 10K > ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log > 12K 3 9.3K > ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log > 20K 5 16.4K > ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log > 12K 3 9.4K > ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log > 12K 3 9.9K > ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log > 12K 3 10.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log > 12K 3 9.3K > ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log > 12K 3 10K > ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log > 12K 3 9.8K > ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log > 12K 3 10.5K > ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log > 12K 3 9.5K > ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log > 12K 3 9.7K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log > 12K 3 10.3K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log > 12K 3 9K > ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log > 12K 3 9.4K > ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log > 12K 3 11.7K > ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log > 12K 3 9.7K > ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log > 12K 3 9.5K > ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log > 12K 3 10.2K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log > 12K 3 10.2K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log > 12K 3 9.5K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log > 8K 2 6.8K ./scheduler/2021-09-17/test_dag.py.log > 8K 2 7.1K ./scheduler/2021-09-17/test_dag_failure.py.log > 8K 2 7.2K ./scheduler/2021-09-17/example_druid.py.log > 12K 3 10.3K ./scheduler/2021-09-17/example_docker.py.log > > Page Cache After (rc2): > > I read fully the subdag.py.log and it was fully cached but all other logs > generated used just one or two pages of the cache even if they were much > bigger (if they were not actually read). > > RES PAGES SIZE FILE > 24K 6 20.3K > ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log > 4K 1 30.6K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log > 4K 1 26.5K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log > 4K 1 27.3K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log > 8K 2 28.8K > ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log > 8K 2 28.9K > ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log > 4K 1 27.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log > 8K 2 32.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log > 4K 1 26.3K > ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log > 8K 2 28.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log > 8K 2 29.8K > ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log > 8K 2 30.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log > 8K 2 28.4K > ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log > 8K 2 30.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log > 4K 1 27.8K > ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log > 8K 2 49.2K > ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log > 4K 1 27.9K > ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log > 4K 1 30K > ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log > 4K 1 29.3K > ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log > 4K 1 31.6K > ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log > 4K 1 34.9K > ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log > 4K 1 30.2K > ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log > 8K 2 29.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log > 8K 2 28.9K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log > 4K 1 30.8K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log > 4K 1 27.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log > 8K 2 28.1K > ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log > 8K 2 28.6K > ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log > 8K 2 30.5K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log > 4K 1 30.7K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log > 8K 2 28.6K > ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log > 8K 2 20.4K ./scheduler/2021-09-17/test_dag.py.log > 8K 2 21.2K ./scheduler/2021-09-17/test_dag_failure.py.log > 8K 2 21.6K ./scheduler/2021-09-17/example_druid.py.log > 4K 1 30.8K ./scheduler/2021-09-17/example_docker.py.log > > J. > > On Fri, Sep 17, 2021 at 1:50 PM Kaxil Naik <[email protected]> wrote: > >> Gentle ping to test rc2 >> >> On Wed, Sep 15, 2021 at 11:18 PM Kaxil Naik <[email protected]> wrote: >> >>> Just 3 code changes from 2.1.4rc1: >>> >>> - Advises the kernel to not cache log files generated by Airflow (#18054 >>> <https://github.com/apache/airflow/pull/18054>) >>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method (#18208 >>> <https://github.com/apache/airflow/pull/18208>) >>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886 >>> <https://github.com/apache/airflow/pull/17886>) >>> >>> >>> On Wed, Sep 15, 2021 at 10:50 PM Kaxil Naik <[email protected]> >>> wrote: >>> >>>> Hey fellow Airflowers, >>>> >>>> I have cut Airflow 2.1.4rc2. This email is calling a vote on the >>>> release, which will last for 72 hours, from September 15, 2021 at 9:50pm >>>> UTC until September 18, 2021 at 9:50pm UTC, or until 3 binding +1 >>>> votes have been received. >>>> >>>> Airflow 2.1.4 RC2 is available at: >>>> https://dist.apache.org/repos/dist/dev/airflow/2.1.4rc2/ >>>> >>>> *apache-airflow-2.1.4-source.tar.gz* is a source release that comes >>>> with INSTALL instructions. >>>> *apache-airflow-2.1.4.tar.gz* is the binary Python "sdist" release. >>>> *apache_airflow-2.1.4-py3-none-any.whl* is the binary Python wheel >>>> "binary" release. >>>> >>>> Public keys are available at: >>>> https://dist.apache.org/repos/dist/release/airflow/KEYS >>>> >>>> Consider this my +1 binding vote >>>> >>>> Please vote accordingly: >>>> >>>> [ ] +1 approve >>>> [ ] +0 no opinion >>>> [ ] -1 disapprove with the reason >>>> >>>> Only votes from PMC members are binding, but all members of the >>>> community are encouraged to test the release and vote with "(non-binding)". >>>> >>>> The test procedure for PMCs and Contributors who would like to test >>>> this RC are described in >>>> >>>> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmcs >>>> >>>> Please note that the version number excludes the `rcX` string, so it's >>>> now >>>> simply 2.1.4. This will allow us to rename the artifact without >>>> modifying >>>> the artifact checksums when we actually release. >>>> >>>> Changes since 2.1.4rc1 below ( >>>> https://github.com/apache/airflow/blob/2.1.4rc2/CHANGELOG.txt). >>>> >>>> *Bug Fixes* >>>> >>>> - Advises the kernel to not cache log files generated by Airflow (#18054) >>>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method >>>> (#18208) >>>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886) >>>> >>>> *Doc-only Changes* >>>> >>>> - Doc: Use ``closer.lua`` script for downloading sources (#18179) >>>> - Doc: Improve installing from sources (#18194) >>>> - Improves installing from sources pages for all components (#18251) >>>> >>>> >>>> Thanks, >>>> Kaxil >>>> >>>
