+1 (non-binding) - Verified the same issues and tested with the same DAGs as done with the previous RC
On Fri, Sep 17, 2021 at 8:27 PM Jed Cunningham <[email protected]> wrote: > +1 (non-binding) > > On Fri, Sep 17, 2021 at 6:58 AM Jarek Potiuk <[email protected]> wrote: > >> Hey All, >> >> +1 binding - signatures, sums, licences all work. I run it with some >> example dags and it looks generally OK. >> >> I have not run detailed tests for all cases but I checked #18504. The >> cached (not real problem but will save headeache to some people who run >> airflow) memory used by scheduler will not grow because of logs any more: >> >> Page Cache Before (rc1): >> >> RES PAGES SIZE FILE >> 8K 2 6.8K >> ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log >> 12K 3 9.1K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log >> 12K 3 9.6K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log >> 12K 3 10.2K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log >> 12K 3 10K >> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log >> 12K 3 9.7K >> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log >> 12K 3 9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log >> 12K 3 10.7K >> ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log >> 12K 3 8.8K >> ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log >> 12K 3 8.8K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log >> 12K 3 10K >> ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log >> 12K 3 9.3K >> ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log >> 20K 5 16.4K >> ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log >> 12K 3 9.4K >> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log >> 12K 3 9.9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log >> 12K 3 10.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log >> 12K 3 9.3K >> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log >> 12K 3 10K >> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log >> 12K 3 9.8K >> ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log >> 12K 3 10.5K >> ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log >> 12K 3 9.5K >> ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log >> 12K 3 9.7K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log >> 12K 3 10.3K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log >> 12K 3 9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log >> 12K 3 9.4K >> ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log >> 12K 3 11.7K >> ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log >> 12K 3 9.7K >> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log >> 12K 3 9.5K >> ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log >> 12K 3 10.2K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log >> 12K 3 10.2K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log >> 12K 3 9.5K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log >> 8K 2 6.8K ./scheduler/2021-09-17/test_dag.py.log >> 8K 2 7.1K ./scheduler/2021-09-17/test_dag_failure.py.log >> 8K 2 7.2K ./scheduler/2021-09-17/example_druid.py.log >> 12K 3 10.3K ./scheduler/2021-09-17/example_docker.py.log >> >> Page Cache After (rc2): >> >> I read fully the subdag.py.log and it was fully cached but all other logs >> generated used just one or two pages of the cache even if they were much >> bigger (if they were not actually read). >> >> RES PAGES SIZE FILE >> 24K 6 20.3K >> ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log >> 4K 1 30.6K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log >> 4K 1 26.5K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log >> 4K 1 27.3K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log >> 8K 2 28.8K >> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log >> 8K 2 28.9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log >> 4K 1 27.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log >> 8K 2 32.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log >> 4K 1 26.3K >> ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log >> 8K 2 28.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log >> 8K 2 29.8K >> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log >> 8K 2 30.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log >> 8K 2 28.4K >> ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log >> 8K 2 30.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log >> 4K 1 27.8K >> ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log >> 8K 2 49.2K >> ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log >> 4K 1 27.9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log >> 4K 1 30K >> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log >> 4K 1 29.3K >> ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log >> 4K 1 31.6K >> ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log >> 4K 1 34.9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log >> 4K 1 30.2K >> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log >> 8K 2 29.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log >> 8K 2 28.9K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log >> 4K 1 30.8K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log >> 4K 1 27.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log >> 8K 2 28.1K >> ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log >> 8K 2 28.6K >> ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log >> 8K 2 30.5K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log >> 4K 1 30.7K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log >> 8K 2 28.6K >> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log >> 8K 2 20.4K ./scheduler/2021-09-17/test_dag.py.log >> 8K 2 21.2K ./scheduler/2021-09-17/test_dag_failure.py.log >> 8K 2 21.6K ./scheduler/2021-09-17/example_druid.py.log >> 4K 1 30.8K ./scheduler/2021-09-17/example_docker.py.log >> >> J. >> >> On Fri, Sep 17, 2021 at 1:50 PM Kaxil Naik <[email protected]> wrote: >> >>> Gentle ping to test rc2 >>> >>> On Wed, Sep 15, 2021 at 11:18 PM Kaxil Naik <[email protected]> >>> wrote: >>> >>>> Just 3 code changes from 2.1.4rc1: >>>> >>>> - Advises the kernel to not cache log files generated by Airflow (#18054 >>>> <https://github.com/apache/airflow/pull/18054>) >>>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method >>>> (#18208 <https://github.com/apache/airflow/pull/18208>) >>>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886 >>>> <https://github.com/apache/airflow/pull/17886>) >>>> >>>> >>>> On Wed, Sep 15, 2021 at 10:50 PM Kaxil Naik <[email protected]> >>>> wrote: >>>> >>>>> Hey fellow Airflowers, >>>>> >>>>> I have cut Airflow 2.1.4rc2. This email is calling a vote on the >>>>> release, which will last for 72 hours, from September 15, 2021 at 9:50pm >>>>> UTC until September 18, 2021 at 9:50pm UTC, or until 3 binding +1 >>>>> votes have been received. >>>>> >>>>> Airflow 2.1.4 RC2 is available at: >>>>> https://dist.apache.org/repos/dist/dev/airflow/2.1.4rc2/ >>>>> >>>>> *apache-airflow-2.1.4-source.tar.gz* is a source release that comes >>>>> with INSTALL instructions. >>>>> *apache-airflow-2.1.4.tar.gz* is the binary Python "sdist" release. >>>>> *apache_airflow-2.1.4-py3-none-any.whl* is the binary Python wheel >>>>> "binary" release. >>>>> >>>>> Public keys are available at: >>>>> https://dist.apache.org/repos/dist/release/airflow/KEYS >>>>> >>>>> Consider this my +1 binding vote >>>>> >>>>> Please vote accordingly: >>>>> >>>>> [ ] +1 approve >>>>> [ ] +0 no opinion >>>>> [ ] -1 disapprove with the reason >>>>> >>>>> Only votes from PMC members are binding, but all members of the >>>>> community are encouraged to test the release and vote with >>>>> "(non-binding)". >>>>> >>>>> The test procedure for PMCs and Contributors who would like to test >>>>> this RC are described in >>>>> >>>>> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmcs >>>>> >>>>> Please note that the version number excludes the `rcX` string, so it's >>>>> now >>>>> simply 2.1.4. This will allow us to rename the artifact without >>>>> modifying >>>>> the artifact checksums when we actually release. >>>>> >>>>> Changes since 2.1.4rc1 below ( >>>>> https://github.com/apache/airflow/blob/2.1.4rc2/CHANGELOG.txt). >>>>> >>>>> *Bug Fixes* >>>>> >>>>> - Advises the kernel to not cache log files generated by Airflow (#18054) >>>>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method >>>>> (#18208) >>>>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886) >>>>> >>>>> *Doc-only Changes* >>>>> >>>>> - Doc: Use ``closer.lua`` script for downloading sources (#18179) >>>>> - Doc: Improve installing from sources (#18194) >>>>> - Improves installing from sources pages for all components (#18251) >>>>> >>>>> >>>>> Thanks, >>>>> Kaxil >>>>> >>>>
