This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-2-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit e994d36ff4a22bff602a4ab1df81ae2759174775 Author: Jed Cunningham <[email protected]> AuthorDate: Fri Sep 10 17:04:16 2021 -0600 Add 2.2.0b1 changelog fixup! Add 2.2.0b1 changelog --- CHANGELOG.txt | 276 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 276 insertions(+) diff --git a/CHANGELOG.txt b/CHANGELOG.txt index 442942d..0a51759 100644 --- a/CHANGELOG.txt +++ b/CHANGELOG.txt @@ -1,3 +1,279 @@ +Airflow 2.2.0b1, TBD +------------------------- + +New Features +"""""""""""" + +- AIP-39: Handle DAG scheduling with timetables (#15397) +- AIP-39: ``DagRun.data_interval_start|end`` (#16352) +- AIP-40: Add Deferrable "Async" Tasks (#15389) +- Add Airflow Standalone command (#15826) +- Ability to test connections from UI or API (#15795) +- Add Next Run to UI (#17732) +- Allow custom timetable as a DAG argument (#17414) +- Add a calendar field to choose the execution date of the DAG when triggering it (#16141) +- Allow setting specific ``cwd`` for BashOperator (#17751) +- Show import errors in DAG views (#17818) +- Add pre/post execution hooks [Experimental] (#17576) +- Add triggerer to ``docker-compose.yaml`` file (#17745) +- Added table to view providers in Airflow ui under admin tab (#15385) +- Adds secrets backend/logging/auth information to provider yaml (#17625) +- Add date format filters to Jinja environment (#17451) +- Introduce ``RESTARTING`` state (#16681) +- Webserver: Unpause DAG on manual trigger (#16569) +- API endpoint to create new user (#16609) +- Add ``insert_args`` for support transfer replace (#15825) +- Add recursive flag to glob in filesystem sensor (#16894) +- Add conn to jinja template context (#16686) +- Add ``default_args`` for ``TaskGroup`` (#16557) +- Allow adding duplicate connections from UI (#15574) +- Allow specifying multiple URLs via the CORS config option (#17941) +- Implement API endpoint for DAG deletion (#17980) +- Add DAG run endpoint for marking a dagrun success or failed(#17839) +- Add Triggerer warning banner in UI (#17565) +- Add support for ``kinit`` options ``[-f|-F]`` and ``[-a|-A]`` (#17816) +- Queue support for DaskExecutor using Dask Worker Resources (#16829) +- Make auto refresh interval configurable (#18107) + +Improvements +"""""""""""" +- Adding ``TaskGroup`` support in ``BaseOperator.chain()`` (#17456) +- Fix deprecation error message rather than silencing it (#18126) +- Make next_dagrun_info take a data interval (#18088) +- Change XCom class methods to accept run_id argument (#18084) +- Advises the kernel to not cache log files generated by Airflow (#18054) +- Allow filtering DAGS by tags in the REST API (#18090) +- Optimize imports of Providers Manager (#18052) +- Adds capability of Warnings for incompatible community providers (#18020) +- Serialize the ``template_ext`` attribute to show it in UI (#17985) +- Add ``robots.txt`` and ``X-Robots-Tag`` header (#17946) +- Refactor BranchDayOfWeekOperator, DayOfWeekSensor (#17940) +- Update error message to guide the user into self-help mostly (#17929) +- Update to Celery 5 (#17397) +- Avoid redirect loop for users with no permissions (#17838) +- Improve graph view load time for dags with open groups (#17821) +- Add links to provider's documentation (#17736) +- Separate infer_data_interval for data interval timetables (#17755) +- Remove Marshmallow schema warnings (#17753) +- Add ``logical_date`` to OpenAPI DAGRun schema (#17122) +- Rename ``none_failed_or_skipped`` by ``none_failed_min_one_success`` trigger rule (#17683) +- Remove ``[core] store_dag_code`` & use DB to get Dag Code (#16342) +- Rename ``task_concurrency`` to ``max_active_tis_per_dag`` (#17708) +- Import Hooks lazily individually in providers manager (#17682) +- Adding support for multiple task-ids in the external task sensor (#17339) +- Replace execution_date with run_id in airflow tasks run command (#16666) +- Avoid endless redirect loop when user has no roles (#17613) +- Make output from users cli command more consistent (#17642) +- Open relative extra links in place (#17477) +- Move worker_log_server_port option to the logging section (#17621) +- Use gunicorn to serve logs generated by worker (#17591) +- Improve validation of Group id (#17578) +- Simplify 404 page (#17501) +- Add XCom.clear so it's hookable in custom XCom backend (#17405) +- add deprecation notice for SubDagOperator (#17488) +- Make schema in DBApiHook private (#17423) +- Support DAGS folder being in different location on scheduler and runners (#16860) +- Remove /dagrun/create and disable edit form generated by F.A.B (#17376) +- Enable specifying dictionary paths in `template_fields_renderers` (#17321) +- error early if virtualenv is missing (#15788) +- Handle connection parameters added to Extra and custom fields (#17269) +- Fix `airflow celery stop` to accept the pid file. (#17278) +- Remove DAG refresh buttons (#17263) +- deprecate dummy trigger rule infavor of always (#17144) +- Be verbose about failure to import airflow_local_settings (#17195) +- Include exit code in AirflowException str when BashOperator fails. (#17151) +- Adding EdgeModifier support for chain() (#17099) +- Only allows supported field types to be used in custom connections (#17194) +- Secrets backend failover (#16404) +- Warn on Webserver when using ``SQLite`` or ``SequentialExecutor`` (#17133) +- Extend ``init_containers`` defined in ``pod_override`` (#17537) +- Client-side filter dag dependencies (#16253) +- Improve executor validation in CLI (#17071) +- Prevent running `airflow db init\upgrade` migrations and setup in parallel. (#17078) +- Update chain() and cross_downstream() to support XComArgs (#16732) +- Improve graph view refresh (#16696) +- When a task instance fails with exception, log it (#16805) +- Set process title for ``serve-logs`` and ``LocalExecutor`` (#16644) +- Rename test_cycle to check_cycle (#16617) +- Set Process title for Worker when using ``LocalExecutor`` (#16623) +- Add schema as DbApiHook instance attribute (#16521) +- Improve compatibility with MSSQL (#9973) +- Add transparency for unsupported connection type (#16220) +- Call resource based fab methods (#16190) +- Format more dates with timezone (#16129) +- Replace deprecated ``dag.sub_dag`` with ``dag.partial_subset`` (#16179) +- Treat `AirflowSensorTimeout` as immediate failure without retrying (#12058) +- Marking success/failed automatically clears failed downstream tasks (#13037) +- UI: Updated table component (#15805) +- Add close/open indicator for import dag errors (#16073) +- Add collapsible import errors (#16072) +- Always return a response in TI's ``action_clear`` view (#15980) +- Add cli command to delete user by email (#15873) +- Update `airflow tasks *` commands to lookup TaskInstances from DagRun Table (#16030) +- Use resource and action names for FAB permissions (#16410) +- Rename DAG concurrency (``[core] dag_concurrency``) settings for easier understanding (#16267) +- Calendar UI improvements (#16226) +- Refactor: ``SKIPPED`` should not be logged again as ``SUCCESS`` (#14822) +- Remove limits for dnspython (#18162) + +Bug Fixes +""""""""" +- Change id collation for MySQL to case-sensitive (#18072) +- Fix failed tests from run_as_user+error_file changes (#18013) +- Logs task launch exception in StandardTaskRunner (#17967) +- Require timetable class be registered via plugin (#17989) +- Applied permissions to self._error_file (#15947) +- Hide variable import form if user lacks permission (#18000) +- Fix blank dag dependencies view (#17990) +- Add missing menu access for dag dependencies and configurations pages (#17450) +- Fix passing Jinja templates in ``DateTimeSensor`` (#17959) +- Fixing bug which restricted the visibility of ImportErrors (#17924) +- Fix grammar in `traceback.html` (#17942) +- Fix ``DagRunState`` enum query for ``MySQLdb`` driver (#17886) +- BugFix: ``TimeSensorAsync`` returns a naive datetime (#17875) +- Fixed button size in "Actions" group. (#17902) +- Fix log links on graph TI modal (#17862) +- Only show import errors for DAGs a user can access (#17835) +- Show all import_errors from zip files (#17759) +- fix EXTRA_LOGGER_NAMES param and related docs (#17808) +- Use one interpreter for Airflow and gunicorn (#17805) +- Fix: Mysql 5.7 id utf8mb3 (#14535) +- Fix dag_processing.last_duration metric random holes (#17769) +- Automatically use utf8mb3_general_ci collation for mysql (#17729) +- Fix ``TimeSensorAsync`` (#17748) +- Ensure ``DateTimeTrigger`` receives a datetime object (#17747) +- Fix wrong query on running tis (#17631) +- Add root to tree refresh url (#17633) +- Do not delete running DAG from the UI (#17630) +- Fix ``triggerer`` query where limit is not supported in some MySQL version (#17601) +- Rescue if a DagRun's DAG was removed from db (#17544) +- fix: filter condition of TaskInstance does not work #17535 (#17548) +- Fixed long delays in the Temporal trigger (#17564) +- Regression on pid reset to allow task start after heartbeat (#17333) +- Dont use TaskInstance in CeleryExecutor.trigger_tasks (#16248) +- Remove locks for upgrades in mssql (#17213) +- Create virtualenv via python call (#17156) +- Ensure a DAG is acyclic when running DAG.cli() (#17105) +- Translate non-ascii characters (#17057) +- Change the logic of None comparison in model_list template (#16893) +- Have UI and POST /task_instances_state API endpoint have same behaviour (#16539) +- ensure task is skipped if missing sla (#16719) +- Fix direct use of cached_property module. (#16710) +- Fix TI success confirm page (#16650) +- Cattrs 1.7.0 released by the end of May 2021 break lineage usage (#16173) +- Fix Celery executor getting stuck randomly because of reset_signals in multiprocessing (#15989) +- Modify return value check in python virtualenv jinja template (#16049) +- Fix dag dependency search (#15924) +- Make custom JSON encoder support ``Decimal`` (#16383) +- Bugfix: Allow clearing tasks with just ``dag_id`` and empty ``subdir`` (#16513) +- Convert port value to a number before calling test connection (#16497) +- Handle missing/null serialized DAG dependencies (#16393) +- Correctly set ``dag.fileloc`` when using the ``@dag`` decorator (#16384) +- Fix TI success/failure links (#16233) + +Doc only changes +"""""""""""""""" +- Update non-working example in documentation (#18067) +- Improve the description of how to handle dynamic task generation (#17963) +- Remove default_args pattern + added get_current_context() use for Core Airflow example DAGs (#16866) +- Update max_tis_per_query to better render on the webpage (#17971) +- Adds Github Oauth example with team based authorization (#17896) +- Update docker.rst (#17882) +- Example xcom update (#17749) +- Add doc warning about connections added via envvars (#17915) +- fix wrong documents around upgrade-check.rst (#17903) +- Add Brent to Committers list (#17873) +- Improves documentation about modules management (#17757) +- Remove deprecated metrics from metrics.rst (#17772) +- Make sure "podcution-readiness` of docker-compose is well explained (#17731) +- Doc: Update Upgrade to 2 docs with Airflow 1.10.x EOL dates (#17710) +- Doc: Replace deprecated param from docstrings (#17709) +- docs(dagowner): describe dag owner more carefully (#17699) +- docs(impersonation): update note so avoid misintrepretation (#17701) +- Docs: Make ``DAG.is_active`` read-only in API (#17667) +- Improve cross-links to operators and hooks references (#17622) +- Update documentation regarding Python 3.9 support (#17611) +- Fix MySQL database character set instruction (#17603) +- Document overriding ``XCom.clear`` for data lifecycle management (#17589) +- Path correction in docs for airflow core (#17567) +- docs(celery): reworded, add actual multiple queues example (#17541) +- Doc: Add FAQ to speed up parsing with tons of dag files (#17519) +- Improve image building documentation for new users (#17409) +- Doc: Strip unnecessary arguments from MariaDB JIRA URL (#17296) +- Update warning about MariaDB and multiple schedulers (#17287) +- Doc: Recommend using same configs on all Airflow components (#17146) +- Move docs about masking to a new page (#17007) +- Docs: Suggest use of Env vars instead of Airflow Vars in best practises doc (#16926) +- Docs: Better description for `pod_template_file` (#16861) +- Prevent Redis access directly from host (#16885) +- Add Aneesh Joseph as Airflow Committer (#16835) +- Docs: Added new pipeline example for the tutorial docs (#16548) +- Remove upstart from docs (#16672) +- Add new committers: ``Jed`` and ``TP`` (#16671) +- Docs: Fix ``flask-ouathlib`` to ``flask-oauthlib`` in Upgrading docs (#16320) +- Docs: Fix creating a connection docs (#16312) +- Docker-compose: Don't run migration for adhoc command (#16255) +- Docs: Fix url for ``Elasticsearch`` (#16275) +- Small improvements for README.md files (#16244) +- Fix docs for ``dag_concurrency`` (#16177) +- Check synctatic correctness for code-snippets (#16005) +- Improvements for Docker Image docs (#14843) +- Add proper link for wheel packages in docs. (#15999) +- Add Docs for ``default_pool`` slots (#15997) +- Add memory usage warning in quick-start documentation (#15967) +- Update example `KubernetesExecutor` `git-sync` pod template file (#15904) +- Docs: Fix Taskflow API docs (#16574) +- Added new pipeline example for the tutorial docs (#16084) +- Updating the DAG docstring to include ``render_template_as_native_obj`` (#16534) +- Update docs on setting up SMTP (#16523) +- Docs: Fix API verb from ``POST`` to ``PATCH`` (#16511) + +Misc/Internal +""""""""""""" +- Test coverage on the autocomplete view (#15943) +- Update version added fields in airflow/config_templates/config.yml (#18128) +- Add "packaging" to core requirements (#18122) +- Check for missing DagRun rows for "downstream" tables before migrating DB (#17030) +- Rename FileToWasbOperator to LocalFilesystemToWasbOperator (#18109) +- Adds LoggingMixins to BaseTrigger (#18106) +- Limit colorlog version (6.x is incompatible) (#18099) +- Change TaskInstance and TaskReschedule PK from execution_date to run_id (#17719) +- Fix building documentation broken by upgrade of dnspython (#18046) +- Fix building docs in `main` builds (#18035) +- Adding missing init file in example_dags directory (#18019) +- Bump `pip` version to `21.2.4` (#17746) +- Do not let create_dagrun overwrite explicit run_id (#17728) +- Remove upper-limit on ``tenacity`` (#17593) +- Remove redundant ``numpy`` dependency (#17594) +- Bump `mysql-connector-python` to latest version (#17596) +- Make `pandas` an optional core dependency (#17575) +- Add more typing to airflow.utils.helpers (#15582) +- Chore: Some code cleanup in `airflow/utils/db.py` (#17090) +- Refactor: Remove processor_factory from DAG processing (#16659) +- Remove AbstractDagFileProcessorProcess from dag processing (#16816) +- Update TaskGroup typing (#16811) +- Update ``click`` to 8.x (#16779) +- Remove remaining Pylint disables (#16760) +- Remove duplicated try, there is already a try in create_session (#16701) +- Removes pylint from our toolchain (#16682) +- Refactor usage of unneeded function call (#16653) +- Add type annotations to setup.py (#16658) +- Remove SQLAlchemy <1.4 constraint (#16630) +- Refactor `dag.clear` method (#16086) +- Use ``DAG_ACTIONS`` constant (#16232) +- Use updated ``_get_all_non_dag_permissions`` method (#16317) +- Add updated-name wrappers for built-in FAB methods (#16077) +- Pins ``docutils`` to <0.17 until breaking behaviour is fixed (#16133) +- Remove ``TaskInstance.log_filepat``h attribute (#15217) +- Bump ``pyupgrade`` `` v2.13.0`` to ``v2.18.1`` (#15991) +- Removes unnecessary function call in ``airflow/www/app.py`` (#15956) +- Correctly implement autocomplete early return in ``airflow/www/views.py`` (#15940) +- Move ``plyvel`` to google provider extra (#15812) +- Update ``Watchtower`` version to ``1.0.6`` (#16469) +- Update permission migrations to use new naming scheme (#16400) +- Use resource and action names for FAB (#16380) +- Swap out calls to find_permission_view_menu for get_permission wrapper (#16377) + Airflow 2.1.3, 2021-08-23 -------------------------
