+1 (binding) Verified some issues and ran a couple of DAGs
> On 18 Sep 2021, at 02:49, Josh Fell <[email protected]> wrote: > > > +1 (non-binding) - Verified the same issues and tested with the same DAGs as > done with the previous RC > >> On Fri, Sep 17, 2021 at 8:27 PM Jed Cunningham <[email protected]> >> wrote: >> +1 (non-binding) >> >>> On Fri, Sep 17, 2021 at 6:58 AM Jarek Potiuk <[email protected]> wrote: >>> Hey All, >>> >>> +1 binding - signatures, sums, licences all work. I run it with some >>> example dags and it looks generally OK. >>> >>> I have not run detailed tests for all cases but I checked #18504. The >>> cached (not real problem but will save headeache to some people who run >>> airflow) memory used by scheduler will not grow because of logs any more: >>> >>> Page Cache Before (rc1): >>> >>> RES PAGES SIZE FILE >>> 8K 2 6.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log >>> 12K 3 9.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log >>> 12K 3 9.6K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log >>> 12K 3 10.2K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log >>> 12K 3 10K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log >>> 12K 3 9.7K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log >>> 12K 3 9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log >>> 12K 3 10.7K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log >>> 12K 3 8.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log >>> 12K 3 8.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log >>> 12K 3 10K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log >>> 12K 3 9.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log >>> 20K 5 16.4K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log >>> 12K 3 9.4K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log >>> 12K 3 9.9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log >>> 12K 3 10.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log >>> 12K 3 9.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log >>> 12K 3 10K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log >>> 12K 3 9.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log >>> 12K 3 10.5K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log >>> 12K 3 9.5K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log >>> 12K 3 9.7K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log >>> 12K 3 10.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log >>> 12K 3 9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log >>> 12K 3 9.4K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log >>> 12K 3 11.7K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log >>> 12K 3 9.7K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log >>> 12K 3 9.5K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log >>> 12K 3 10.2K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log >>> 12K 3 10.2K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log >>> 12K 3 9.5K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log >>> 8K 2 6.8K ./scheduler/2021-09-17/test_dag.py.log >>> 8K 2 7.1K ./scheduler/2021-09-17/test_dag_failure.py.log >>> 8K 2 7.2K ./scheduler/2021-09-17/example_druid.py.log >>> 12K 3 10.3K ./scheduler/2021-09-17/example_docker.py.log >>> >>> Page Cache After (rc2): >>> >>> I read fully the subdag.py.log and it was fully cached but all other logs >>> generated used just one or two pages of the cache even if they were much >>> bigger (if they were not actually read). >>> >>> RES PAGES SIZE FILE >>> 24K 6 20.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/subdags/subdag.py.log >>> 4K 1 30.6K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl_virtualenv.py.log >>> 4K 1 26.5K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial.py.log >>> 4K 1 27.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_etl_dag.py.log >>> 8K 2 28.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/tutorial_taskflow_api_etl.py.log >>> 8K 2 28.9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_target_dag.py.log >>> 4K 1 27.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_xcom.py.log >>> 8K 2 32.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_xcomargs.py.log >>> 4K 1 26.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/test_utils.py.log >>> 8K 2 28.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group.py.log >>> 8K 2 29.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_task_group_decorator.py.log >>> 8K 2 30.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_trigger_controller_dag.py.log >>> 8K 2 28.4K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_python_operator.py.log >>> 8K 2 30.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_short_circuit_operator.py.log >>> 4K 1 27.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_skip_dag.py.log >>> 8K 2 49.2K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_subdag_operator.py.log >>> 4K 1 27.9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only.py.log >>> 4K 1 30K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_latest_only_with_trigger.py.log >>> 4K 1 29.3K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_nested_branch_dag.py.log >>> 4K 1 31.6K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_passing_params_via_test_command.py.log >>> 4K 1 34.9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_external_task_marker_dag.py.log >>> 4K 1 30.2K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor_config.py.log >>> 8K 2 29.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_kubernetes_executor.py.log >>> 8K 2 28.9K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_operator.py.log >>> 4K 1 30.8K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_python_dop_operator_3.py.log >>> 4K 1 27.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_complex.py.log >>> 8K 2 28.1K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_dag_decorator.py.log >>> 8K 2 28.6K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_bash_operator.py.log >>> 8K 2 30.5K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_datetime_operator.py.log >>> 4K 1 30.7K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_day_of_week_operator.py.log >>> 8K 2 28.6K >>> ./scheduler/2021-09-17/native_dags/example_dags/example_branch_labels.py.log >>> 8K 2 20.4K ./scheduler/2021-09-17/test_dag.py.log >>> 8K 2 21.2K ./scheduler/2021-09-17/test_dag_failure.py.log >>> 8K 2 21.6K ./scheduler/2021-09-17/example_druid.py.log >>> 4K 1 30.8K ./scheduler/2021-09-17/example_docker.py.log >>> >>> J. >>> >>>> On Fri, Sep 17, 2021 at 1:50 PM Kaxil Naik <[email protected]> wrote: >>>> Gentle ping to test rc2 >>>> >>>>> On Wed, Sep 15, 2021 at 11:18 PM Kaxil Naik <[email protected]> wrote: >>>>> Just 3 code changes from 2.1.4rc1: >>>>> >>>>> - Advises the kernel to not cache log files generated by Airflow (#18054) >>>>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method >>>>> (#18208) >>>>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886) >>>>> >>>>>> On Wed, Sep 15, 2021 at 10:50 PM Kaxil Naik <[email protected]> wrote: >>>>>> Hey fellow Airflowers, >>>>>> >>>>>> I have cut Airflow 2.1.4rc2. This email is calling a vote on the >>>>>> release, which will last for 72 hours, from September 15, 2021 at 9:50pm >>>>>> UTC until September 18, 2021 at 9:50pm UTC, or until 3 binding +1 votes >>>>>> have been received. >>>>>> >>>>>> Airflow 2.1.4 RC2 is available at: >>>>>> https://dist.apache.org/repos/dist/dev/airflow/2.1.4rc2/ >>>>>> >>>>>> apache-airflow-2.1.4-source.tar.gz is a source release that comes with >>>>>> INSTALL instructions. >>>>>> apache-airflow-2.1.4.tar.gz is the binary Python "sdist" release. >>>>>> apache_airflow-2.1.4-py3-none-any.whl is the binary Python wheel >>>>>> "binary" release. >>>>>> >>>>>> Public keys are available at: >>>>>> https://dist.apache.org/repos/dist/release/airflow/KEYS >>>>>> >>>>>> Consider this my +1 binding vote >>>>>> >>>>>> Please vote accordingly: >>>>>> >>>>>> [ ] +1 approve >>>>>> [ ] +0 no opinion >>>>>> [ ] -1 disapprove with the reason >>>>>> >>>>>> Only votes from PMC members are binding, but all members of the >>>>>> community are encouraged to test the release and vote with >>>>>> "(non-binding)". >>>>>> >>>>>> The test procedure for PMCs and Contributors who would like to test this >>>>>> RC are described in >>>>>> https://github.com/apache/airflow/blob/main/dev/README_RELEASE_AIRFLOW.md#verify-the-release-candidate-by-pmcs >>>>>> >>>>>> Please note that the version number excludes the `rcX` string, so it's >>>>>> now >>>>>> simply 2.1.4. This will allow us to rename the artifact without modifying >>>>>> the artifact checksums when we actually release. >>>>>> >>>>>> Changes since 2.1.4rc1 below >>>>>> (https://github.com/apache/airflow/blob/2.1.4rc2/CHANGELOG.txt). >>>>>> >>>>>> Bug Fixes >>>>>> - Advises the kernel to not cache log files generated by Airflow (#18054) >>>>>> - Sort adopted tasks in ``_check_for_stalled_adopted_tasks`` method >>>>>> (#18208) >>>>>> - Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886) >>>>>> Doc-only Changes >>>>>> - Doc: Use ``closer.lua`` script for downloading sources (#18179) >>>>>> - Doc: Improve installing from sources (#18194) >>>>>> - Improves installing from sources pages for all components (#18251) >>>>>> >>>>>> Thanks, >>>>>> Kaxil
