This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch v1-10-stable in repository https://gitbox.apache.org/repos/asf/airflow.git.
from 8b05289 Cache 1 10 ci images (#8955) add 69eeeda Add Local Filesystem Secret Backend (v1-10) (#8596) add ac257fe Reduce response payload size of /dag_stats and /task_stats (#8655) add 313d09e Backport Airflow config command (1.10.*) (#8694) add 8eb4565 Add airflow info command (v1-10-test) (#8704) add c79e7df Latest debian-buster release broke image build (#8758) add a8d8903 Show Deprecation warning on duplicate Task ids (#8728) add 3b70308 [8650] Add Yandex.Cloud custom connection to 1.10 (#8791) add 908962a [AIRFLOW-4052] Allow filtering using "event" and "owner" in "Log" view (#4881) add cd32afa Azure storage 0.37.0 is not installable any more (#8833) add 0c17935 Avoid failure on transient requirements in CI image add f5d89ed Use Debian's provided JRE from Buster (#8919) add a2d3acd Hive/Hadoop minicluster needs JDK8 and JAVA_HOME to work (#8938) add d0b0207 Fix new flake8 warnings on v1-10-test branch (#8953) add b2a4032 [AIRFLOW-3367] Run celery integration test with redis broker. (#4207) add 64db6e6 Fix race in Celery tests by pre-creating result tables (#8909) add a3aa995 Pin Version of Azure Cosmos to <4 (#8956) add 5664e36 Fix timing-based flakey test in TestLocalTaskJob (#8405) add 23d5ea0 Use production image for k8s tests (#9038) add 3437663 Move k8sexecutor out of contrib to closer match master (#8904) add 0925741 [AIRFLOW-4851] Refactor K8S codebase with k8s API models (#5481) add a5e7b99 [AIRFLOW-5443] Use alpine image in Kubernetes's sidecar (#6059) add 9444b4c [AIRFLOW-5445] Reduce the required resources for the Kubernetes's sidecar (#6062) add 4c484ef [AIRFLOW-5873] KubernetesPodOperator fixes and test (#6524) add 4918b85 [AIRFLOW-6959] Use NULL as dag.description default value (#7593) add 2fa5157 Add note about using dag_run.conf in BashOperator (#9143) add 570c9fa Fix --forward-credentials flag in Breeze (#8554) add 79d34ea Fixed optimistions of non-py-code builds (#8601) add c3d4396 Fix the process of requirements generations (#8648) add 264a94b Fixed test-target command (#8795) add c057430 Add comments to breeze scripts (#8797) add 6ba874b Useful help information in test-target and docker-compose commands (#8796) add 1831e79 The librabbitmq library stopped installing for python3.7 (#8853) add 41808a7 Use Debian's provided JRE from Buster (#8919) add cf25e53 Re-run all tests when Dockerfile or Github worflow change (#8924) add 0efaa00 Hive/Hadoop minicluster needs JDK8 and JAVA_HOME to work (#8938) add 7b4e1a4 Python base images are stored in cache (#8943) add a41801c Add ADDITIONAL_PYTHON_DEPS (#9031) add ffe496a Add ADDITIONAL_AIRFLOW_EXTRAS (#9032) add 5683783 Additional python extras and deps can be set in breeze (#9035) add dbb4284 detect incompatible docker server version in breeze (#9042) add 214b508 Adds hive as extra in pyhive (#9075) add 7d3dab1 Prevents failure on fixing permissions for files with space in it (#9076) add 4f1a319 Enable configurable git sync depth (#9094) add 5c45091 Don't reuse MY_DIR in breeze to mean different folder from ci/_utils.sh (#9098) add e5df858 You can push with Breeze as separate command and to cache (#8976) add d83331b Produce less verbose output when building docker mount options (#9103) add f099416 Display docs errors summary (#8392) add 66ab8c3 Remove Hive/Hadoop/Java dependency from unit tests (#9029) add 32ed3c6 Kubernetes Cluster is started on host not in the container (#8265) add d505e8d Fixes a bug where `build-image` command did not calculate md5 (#9130) add c7c3561 Fix INTEGRATIONS[*]: unbound variable error in breeze (#9135) add 77998f5 Cope with multiple processes get_remote_image_info in parallel (#9105) add 6d07eac Remove remnant kubernetes stuff from breeze scripts (#9138) add 19f6065 Restrict google-cloud-texttospeach to <v2 (#9137) add aa52507 Use static binary linked docker client in CI image (#9126) add d55de34 Remove vendored nvd3 and slugify libraries (#9136) add ebe7317 Replaces cloud-provider CLIs in CI image with scripts running containers (#9129) add 100f407 Split utils sh (#9132) add 339184f Add PR/issue note in Contribution Workflow Example (#9177) add 9a3a26e Fixes failure of the build scripts when remote repo does not exist (#9188) add 88533b0 Improved cloud tool available in the trimmed down CI container (#9167) add c136b82 Chown should work now when building the documentation (#8600) add 0a955b6 Validate only task commands are run by executors (#9178) add 0df2df6 Don't use the `|safe` filter in code, it's risky (#9180) add 83ca3bb Fix up coredns pods in KIND CI (#9224) add ecc15fe Get all pod logs on k8s launching failure (#9317) add 5757042 CI: Propogate Exit Code Correctly (#9247) add 21fac5b fixup! KubernetesPodOperator tests were not running add 29176d0 fixup! change mock import for 2.7 add 469b80b Disable KIND git-sync tests for now (#9229) add 709d0d7 Upload kind logs to Github Actions artifact (#9230) add 435b072 Further validation that only task commands are run by executors (#9240) add 7f20bed Fix Celery default to no longer allow pickle (#7205) add de8f1a3 Fix Migration for MSSQL (#8385) add 252203b Make Kubernetes tests pass (#9332) add ec346ae Add dev script to compare GH issues against merges (#9270) add 51d2841 Fix list formatting of plugins doc. (#8873) add c9f0c2e Fix KubernetesPodOperator pod name length validation (#8829) add 39df72a Fix awkward log info in dbapi_hook (#8482) add b1315b3 Fix performance degradation when updating dagrun state (#8435) add 450f400 Correctly deserialize dagrun_timeout field on DAGs (#8735) add 1ebb058 Fix migration message (#8988) add 65659e5 set name via validator for KubernetesPodOperator add f73003c Remove redundant count query in BaseOperator.clear() (#9362) add 60856b1 Make it possible to silence warnings from Airflow (#9208) add 678b261 Update AWS connection example to show how to set from env var (#9191) add f38a9a1 Add SQL Branch Operator add 4b808ed [AIRFLOW-4472] Use json.dumps/loads for templating lineage data (#5253) add d1a9830 Parameterized bash/python in the prod image (#9157) add c349a22 Add 'main' param to template_fields in DataprocSubmitPySparkJobOperator (#9154) add 4ddc13b Fix retries causing constraint violation on MySQL with DAG Serialization (#9336) add aca7559 Merging multiple sql operators (#9124) add 994221d Don't use the term "whitelist" - language matters (#9174) add 13461c5 Add PR/issue note in Contribution Workflow Example (#9177) add 7d89c9f Remove httplib2 from Google requirements (#9194) add b623ba8 Fix typo in BREEZE.rst (#9199) add d4328d5 Updated missing parameters for docker image building (#9039) add 30f459e Support additional apt dependencies (#9189) add 07720c9 n Improved compatibility with Python 3.5+ - Convert signal.SIGTERM to int (#9207) add 89fc739 Correctly restore colour in logs after format arg (#9222) add f962392 Add generic CLI tool wrapper (#9223) add eb5abe9 Additional apt dependencies options in breeze (#9231) add 1a25906 Add missing variable in run_cli_tool.sh (#9239) add d09d328 Remove generating temp remote manifest file in project dir (#9267) add 07246a4 Update pre-commit-hooks repo version (#9195) add 6b701f2 Fix broken CI image optimisation (#9313) add 18cdeef Improve production image iteration speed (#9162) add 5642b47 Fixes unbound variable on MacOS (#9335) add 9ffb939 clarify breeze initialize virtualenv instructions (#9319) add a9e4339 Fix Airflow Stable version in README.md (#9360) add 6ce8242 Fixed crashing webserver after /tmp is mounted from the host (#9378) add 679e787 Fixes Breeze 'tests' command (#9384) add 7db94ce Fix in-breeze CLI tools to work also on Linux (#9376) add d248dcd add guidance re yarn build for local virtualenv development (#9411) add f2f9ca2 Fixed rendering of IMAGES.rst (#9433) add 76971dd In case of worktree .git might be a file - rat-check fails (#9435) add 88797e9 Fixes pushing prod image directly from breeze (#9449) add d9dd684 Added more precise Python requirements to README.md (#8455) add d59b506 Update README to remove Py 3.8 limitation for Master (#9451) add 674d849 Show Dag's Markdown docs on Tree View (#9448) add 265d536 Include some missing RBAC roles on User and Viewer roles (#9133) add 47af74b [AIRFLOW-5902] avoid unnecessary sleep to maintain local task job heart rate (#6553) add b5c29bc [AIRFLOW-6231] Display DAG run conf in the list view (#6794) add 62fb0e3 Update the version of cattrs from 0.9 to 1.0 to support Python 3.8 (#7100) add 7b7c576 [AIRFLOW-6535] Add AirflowFailException to fail without any retry (#7133) add c174b3d JSON escape text in test_list_dagrun_includes_conf (#9264) add 37c769c Flush pending Sentry exceptions before exiting (#7232) add c7e5e72 [AIRFLOW-7066] Use sphinx syntax in concepts.rst (#7729) add 9af65e2 [AIRFLOW-4357] Fix SVG tooltip positioning with custom scripting (#8269) add 6c7fd5b Use existing DagBag for 'dag_details' & `trigger` Endpoints (#8501) add 553bf87 Make hive macros py3 compatible (#8598) add 65673f5 Enhanced documentation around Cluster Policy (#8661) add ca96f79 Improve tutorial - Include all imports statements (#8670) add 8b03773 Fix docs on creating CustomOperator (#8678) add 540d92f Fix displaying Executor Class Name in "Base Job" table (#8679) add 1247267 Prevent clickable sorting on non sortable columns in TI view (#8681) add 14b5f3b Fix connection add/edit for spark (#8685) add e2adde3 Update example webserver_config.py to show correct CSRF config (#8944) add 5d73344 Pinning max pandas version to 2.0 (lesser than) to allow pandas 1.0. (#7954) add d6eae98 [AIRFLOW-3900] Error on undefined template variables in unit tests. (#4719) add 1663760 UX Fix: Prevent undesired text selection with DAG title selection in Chrome (#8912) add 6f23740 Use Markup for htmlcontent for landing_times (#9242) add 2ca64b5 Decrypt secrets from SystemsManagerParameterStoreBackend (#9214) add b69f180 Add a tip to trigger DAG screen (#9049) add c497fcf Add TaskInstance state to TI Tooltip to be colour-blind friendlier (#8910) add 55a0841 [AIRFLOW-8902] Fix Dag Run UI execution date with timezone cannot be saved issue (#8902) add e289e6c Fix tree view if config contains " (#9250) add b056e65 Fix json string escape in tree view (#8551) add 882f0f0 Fix failing tests from #9250 (#9307) add b70cd1c Monkey patch greenlet celery pools (#8559) add f148b63 Optimize count query on /home (#8729) add b9e6498 [AIRFLOW-6940] Improve test isolation in test_views.py (#7564) add 3c42b97 Add extra options to fix SSL issue and be more flexible (#9409) add 67752ec Monitor pods by labels instead of names (#6377) add cdfab10 Support k8s auth method in Vault Secrets provider (#8640) add 5fec88e Filter dags by clicking on tag (#8897) add c7af88c Allow passing backend_kwargs to AWS SSM client (#8802) add 358eb56 Add default `conf` parameter to Spark JDBC Hook (#8787) add a093b91 [AIRFLOW-5413] Refactor worker config (#7114) add 7ff7352 [AIRFLOW-5641] Support running git sync container as root (#6312) add 45ce4f9 make k8s tests pass add f73f09d [AIRFLOW-5349] Add schedulername option for KubernetesPodOperator (#6088) add 501cc0d [AIRFLOW-5659] Add support for ephemeral storage on KubernetesPodOperator (#6337) add 613677c [AIRFLOW-6778] Add a configurable DAGs volume mount path for Kubernetes (#8147) add f70150c Correctly store non-default Nones in serialized tasks/dags (#8772) add 36ce7a2 Correctly restore upstream_task_ids when deserializing Operators (#8775) add 2f1a544 Make loading plugins from entrypoint fault-tolerant (#8732) add bb53283 [AIRFLOW-5898] fix alembic crash due to typing import (#6547) add bde12c8 Avoid color info in response of /dag_stats & /task_stats (#8742) add 7aa0f47 [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file (#6230) add 2445d4a Document default timeout value for SSHOperator (#8744) add e0e53ad [AIRFLOW-5500] Fix the trigger_dag api in the case of nested subdags add e36f462 Add context to execution_date_fn in ExternalTaskSensor (#8702) add e39e107 Ensure "started"/"ended" in tooltips are not shown if job not started (#8667) add 9182527 Persist start/end date and duration for DummyOperator Task Instance (#8663) add 3f55dc5 Fix logging issue when running tasks (#9363) add ef74fad [AIRFLOW-6856] Bulk fetch paused_dag_ids add ed54dff [AIRFLOW-6957] Make retrieving Paused Dag ids a separate method add 16b3113 Remove redundant code from breeze initialization (#9375) add 55354ff Add missing precommit-hook ids to breeze-complete (#9524) add c94554f Gunicorn works better if temporary folder uses tmpfs (#9534) add 0a8b9c4 Make Production Dockerfile OpenShift-compatible (#9545) add 8222388 More sensible docker caching strategy for Prod images (#9547) add 15b5d06 Add support for fetching logs from running pods (#8626) add ee23fe9 Fix the default value for store_dag_code (#9554) add a0e2753 Fix failing test in DagCode (#9565) add 80089ba Allow changing Task States Colors (#9520) add f2c9c90 Add more info on dry-run CLI option (#9582) add 9db0d43 Update commands in docs for v1.10+ (#9585) add 66e0b61 Add copy button to Code Blocks in Airflow Docs (#9450) add c107146 Reload gunicorn when plugins has beeen changed (#8997) add 8763759 Fix failing tests from #8997 (#9576) add ffa0295 Add docs about reload_on_plugin_change opiton (#9575) add 332651b [AIRFLOW-6897] Simplify DagFileProcessorManager (#7521) add b93f746 Use pfromat instead of str to render arguments in WebUI (#9587) add 2e652b2 Change worker_refresh_interval fallback to default of 30 (#9588) add 7e0fa65 Fix failing test in test_webserver_command (#9589) add 967aa90 Show "Task Reschedule" table in Airflow Webserver (#9521) add e47fbe0 [AIRFLOW-1156] BugFix: Unpausing a DAG with catchup=False creates an extra DAG run (#8776) add ef82cc1 [AIRFLOW-5906] Add authenticator parameter to snowflake_hook (#8642) add aa0848b [AIRFLOW-4363] Fix JSON encoding error (#8287) add f8f89a4 Add Webserver tests to Quarantine add 1b632ea Fix pickling failure when spawning processes (#8671) add 0a4ffee Use fork when test relies on mock.patch in parent process. (#8794) add 2c9bb89 Access function to be pickled as attribute, not method, to avoid error. (#8823) add d25524c Add 3.8 to the test matrices (#8836) add 97b3944 Fix Failing test for JSON Formatter on Python 3.8 (#9278) add 6031846 [AIRFLOW-6708] Set unique logger names (#7330) add bb17dc2 [AIRFLOW-XXXX] Fix outdated doc on settings.policy (#7532) add b03665e Add task instance mutation hook (#8852) add 5825b16 [AIRFLOW-XXX] GSoD: Adding Task re-run documentation (#6295) add b80a396 [AIRFLOW-XXX] Add a structural dag validation example (#6727) add bd1e2bc [AIRFLOW-XXXX] Add scheduler in production section (#7351) add e1317d5 fix typo in DAG Serialization documentation (#8317) add c9cd3ef Add documentation for CLI command airflow dags test (#8251) add 862a527 Add Local and Sequential Executors to Doc (#8084) add 88d38f4 [AIRFLOW-XXXX] fix pools doc for LocalExecutor (#7643) add ad7d51c Make doc clearer about Airflow Variables using Environment Variables (#8427) add 1bb82a4 Fix formatting of Pool docs in concepts.rst (#8443) add 935515d Improve language in Pod Mutation Hook docs (#8445) add 4859077 Make KubernetesPodOperator clear in docs (#8444) add 4f08d4d Carefully parse warning messages when building documentation (#8693) add 45a583f Enforce code-block directives in doc (#9443) add 641e110 Add docs on using DAGRun.conf (#9578) add ec7ad93 Bump version to 1.10.11 add 99d8852 fixup! Add Local and Sequential Executors to Doc (#8084) add 836f717 Fix task and dag stats on home page (#8865) add 17070df Restrict editing DagRun State in the old UI (#9612) add 0e55421 Fix quarantined tests - TestCliWebServer (#9596) add bdca88c fixup! Switches to Helm Chart for Kubernetes tests (#9468) add 180bca4 Change default auth for experimental backend to deny_all (#9611) add 1303b85 Add docs to change Colors on the Webserver (#9607) add 61dd9b7 Change 'initiate' to 'initialize' in installation.rst (#9619) add a831d39 Replace old Variables View Screenshot with new (#9620) add b8f2cd3 Restrict changing XCom values from the Webserver (#9614) add 02497a2 Replace old SubDag zoom screenshot with new (#9621) add 233cd2b Update docs about the change to default auth for experimental API (#9617) add c685ce3 Update version_added of configs added in 1.10.11 add c08a35c Add __repr__ for DagTag so tags display properly in /dagmodel/show (#8719) add 34aabac Update the tree view of dag on Concepts Last Run Only (#8268) add 3e080c2 Add Changelog for 1.10.11 add 317b041 Update README.md for 1.10.11 No new revisions were added by this update. Summary of changes: .coveragerc | 1 - .dockerignore | 4 +- .flake8 | 2 +- .github/workflows/ci.yml | 130 +- .gitignore | 4 +- .pre-commit-config.yaml | 130 +- .rat-excludes | 8 +- .travis.yml | 81 -- BREEZE.rst | 536 ++++--- CHANGELOG.txt | 194 +++ CI.rst | 33 +- CONTRIBUTING.rst | 13 +- Dockerfile | 83 +- Dockerfile.ci | 159 +- IMAGES.rst | 447 +++++- LICENSE | 2 +- LOCAL_VIRTUALENV.rst | 9 +- MANIFEST.in | 1 - README.md | 23 +- STATIC_CODE_CHECKS.rst | 2 + TESTING.rst | 251 +++- UPDATING.md | 47 +- airflow/__init__.py | 3 +- airflow/_vendor/README | 13 - airflow/_vendor/__init__.py | 18 - airflow/_vendor/nvd3/LICENSE | 24 - airflow/_vendor/nvd3/NVD3Chart.py | 506 ------- airflow/_vendor/nvd3/__init__.py | 29 - airflow/_vendor/nvd3/cumulativeLineChart.py | 104 -- airflow/_vendor/nvd3/discreteBarChart.py | 91 -- airflow/_vendor/nvd3/ipynb.py | 91 -- airflow/_vendor/nvd3/lineChart.py | 120 -- airflow/_vendor/nvd3/linePlusBarChart.py | 131 -- airflow/_vendor/nvd3/lineWithFocusChart.py | 105 -- airflow/_vendor/nvd3/multiBarChart.py | 95 -- airflow/_vendor/nvd3/multiBarHorizontalChart.py | 100 -- airflow/_vendor/nvd3/pieChart.py | 101 -- airflow/_vendor/nvd3/scatterChart.py | 121 -- airflow/_vendor/nvd3/stackedAreaChart.py | 99 -- airflow/_vendor/nvd3/templates/base.html | 35 - airflow/_vendor/nvd3/templates/content.html | 123 -- .../nvd3/templates/cumulativelinechart.html | 10 - .../_vendor/nvd3/templates/discretebarchart.html | 31 - .../_vendor/nvd3/templates/linebarwfocuschart.html | 60 - airflow/_vendor/nvd3/templates/linechart.html | 46 - .../_vendor/nvd3/templates/lineplusbarchart.html | 43 - .../_vendor/nvd3/templates/linewfocuschart.html | 10 - airflow/_vendor/nvd3/templates/multibarchart.html | 10 - .../nvd3/templates/multibarcharthorizontal.html | 10 - airflow/_vendor/nvd3/templates/page.html | 12 - airflow/_vendor/nvd3/templates/piechart.html | 80 - airflow/_vendor/nvd3/templates/scatterchart.html | 52 - .../_vendor/nvd3/templates/stackedareachart.html | 7 - airflow/_vendor/nvd3/translator.py | 71 - airflow/_vendor/slugify/LICENSE | 21 - airflow/_vendor/slugify/__init__.py | 6 - airflow/_vendor/slugify/slugify.py | 185 --- airflow/api/__init__.py | 9 +- airflow/api/auth/backend/kerberos_auth.py | 14 +- airflow/api/common/experimental/delete_dag.py | 7 +- airflow/api/common/experimental/mark_tasks.py | 8 +- airflow/api/common/experimental/trigger_dag.py | 12 +- airflow/bin/cli.py | 871 +++++++++-- airflow/config_templates/config.yml | 44 +- airflow/config_templates/default_airflow.cfg | 25 +- airflow/config_templates/default_celery.py | 10 +- .../config_templates/default_webserver_config.py | 3 +- airflow/configuration.py | 87 +- .../auth/backends/github_enterprise_auth.py | 4 +- airflow/contrib/auth/backends/google_auth.py | 4 +- airflow/contrib/auth/backends/ldap_auth.py | 18 +- airflow/contrib/auth/backends/password_auth.py | 8 +- .../example_dags/example_kubernetes_operator.py | 12 +- airflow/contrib/executors/__init__.py | 19 - airflow/contrib/hooks/bigquery_hook.py | 7 +- airflow/contrib/hooks/cloudant_hook.py | 5 +- airflow/contrib/hooks/gcp_api_base_hook.py | 8 +- airflow/contrib/hooks/gcp_mlengine_hook.py | 6 +- airflow/contrib/hooks/qubole_check_hook.py | 8 +- airflow/contrib/hooks/qubole_hook.py | 6 +- airflow/contrib/hooks/salesforce_hook.py | 11 +- airflow/contrib/hooks/slack_webhook_hook.py | 10 +- airflow/contrib/hooks/snowflake_hook.py | 8 +- airflow/contrib/hooks/spark_jdbc_hook.py | 2 +- airflow/contrib/hooks/spark_submit_hook.py | 2 +- airflow/contrib/kubernetes/__init__.py | 5 + .../kubernetes_request_factory.py | 258 ---- .../pod_request_factory.py | 135 -- airflow/contrib/kubernetes/pod.py | 150 -- airflow/contrib/kubernetes/pod_generator.py | 181 --- airflow/contrib/kubernetes/volume_mount.py | 37 - airflow/contrib/kubernetes/worker_configuration.py | 432 ------ airflow/contrib/operators/dataproc_operator.py | 4 +- airflow/contrib/operators/file_to_wasb.py | 5 +- .../contrib/operators/kubernetes_pod_operator.py | 296 +++- airflow/contrib/operators/mlengine_operator.py | 5 +- airflow/contrib/operators/qubole_check_operator.py | 2 +- .../contrib/operators/slack_webhook_operator.py | 9 +- airflow/contrib/operators/snowflake_operator.py | 13 +- airflow/contrib/operators/ssh_operator.py | 2 +- airflow/contrib/plugins/metastore_browser/main.py | 16 +- .../templates/metastore_browser/dbs.html | 2 +- airflow/contrib/secrets/aws_systems_manager.py | 15 +- airflow/contrib/secrets/hashicorp_vault.py | 17 +- airflow/contrib/utils/gcp_field_validator.py | 2 +- airflow/contrib/utils/sendgrid.py | 9 +- airflow/dag/base_dag.py | 8 - .../example_kubernetes_executor_config.py | 6 +- .../example_latest_only_with_trigger.py | 3 + airflow/example_dags/example_subdag_operator.py | 2 + airflow/example_dags/example_trigger_target_dag.py | 4 +- airflow/example_dags/subdags/subdag.py | 2 + airflow/example_dags/tutorial.py | 2 +- airflow/exceptions.py | 40 + airflow/executors/__init__.py | 7 +- airflow/executors/celery_executor.py | 17 +- airflow/executors/dask_executor.py | 10 + airflow/executors/debug_executor.py | 7 +- .../{contrib => }/executors/kubernetes_executor.py | 195 +-- airflow/executors/local_executor.py | 32 +- airflow/executors/sequential_executor.py | 9 + airflow/hooks/base_hook.py | 4 +- airflow/hooks/dbapi_hook.py | 2 +- airflow/hooks/hive_hooks.py | 7 +- airflow/hooks/webhdfs_hook.py | 6 +- airflow/jobs/backfill_job.py | 5 +- airflow/jobs/base_job.py | 23 +- airflow/jobs/local_task_job.py | 8 - airflow/jobs/scheduler_job.py | 64 +- .../__init__.py | 0 airflow/kubernetes/k8s_model.py | 62 + airflow/{contrib => }/kubernetes/kube_client.py | 2 +- airflow/kubernetes/pod.py | 122 ++ airflow/kubernetes/pod_generator.py | 606 ++++++++ airflow/{contrib => }/kubernetes/pod_launcher.py | 102 +- .../kubernetes/pod_runtime_info_env.py | 28 +- airflow/{contrib => }/kubernetes/refresh_config.py | 1 - airflow/{contrib => }/kubernetes/secret.py | 63 +- airflow/{contrib => }/kubernetes/volume.py | 20 +- airflow/kubernetes/volume_mount.py | 71 + airflow/kubernetes/worker_configuration.py | 455 ++++++ airflow/lineage/__init__.py | 8 +- airflow/lineage/datasets.py | 11 +- ...15af_add_rendered_task_instance_fields_table.py | 2 +- ...ea_add_precision_to_execution_date_in_mysql.py} | 43 +- .../b3b105409875_add_root_dag_id_to_dag.py | 2 + airflow/models/baseoperator.py | 18 +- airflow/models/connection.py | 2 + airflow/models/crypto.py | 6 +- airflow/models/dag.py | 36 +- airflow/models/dagcode.py | 4 +- airflow/models/dagrun.py | 10 +- airflow/models/serialized_dag.py | 5 +- airflow/models/taskinstance.py | 112 +- airflow/models/variable.py | 9 +- airflow/models/xcom.py | 4 +- airflow/operators/bash_operator.py | 33 +- airflow/operators/check_operator.py | 425 +----- airflow/operators/docker_operator.py | 10 +- airflow/operators/hive_operator.py | 15 + airflow/operators/hive_stats_operator.py | 24 +- airflow/operators/hive_to_samba_operator.py | 14 +- airflow/operators/sql.py | 636 ++++++++ .../{dummy_operator.py => sql_branch_operator.py} | 28 +- airflow/plugins_manager.py | 21 +- airflow/secrets/local_filesystem.py | 268 ++++ airflow/security/kerberos.py | 5 +- airflow/sensors/external_task_sensor.py | 22 +- airflow/sensors/hdfs_sensor.py | 8 +- airflow/sentry.py | 16 +- airflow/serialization/json_schema.py | 2 +- airflow/serialization/serialized_objects.py | 38 +- airflow/settings.py | 66 +- airflow/task/task_runner/standard_task_runner.py | 15 +- tests/compat.py => airflow/typing_compat.py | 20 +- airflow/utils/cli.py | 21 + airflow/utils/code_utils.py | 44 + airflow/utils/dag_processing.py | 184 +-- airflow/utils/db.py | 4 +- airflow/utils/email.py | 5 +- airflow/utils/helpers.py | 2 +- airflow/utils/log/colored_log.py | 13 +- airflow/utils/log/es_task_handler.py | 4 +- airflow/utils/log/file_task_handler.py | 24 + airflow/utils/log/logging_mixin.py | 2 +- airflow/{hooks/sqlite_hook.py => utils/mixins.py} | 27 +- airflow/utils/{configuration.py => platform.py} | 46 +- airflow/utils/sqlalchemy.py | 4 +- airflow/utils/state.py | 3 + airflow/version.py | 2 +- airflow/www/api/experimental/endpoints.py | 35 +- airflow/www/app.py | 10 +- airflow/www/static/connection_form.js | 8 + airflow/www/static/main.css | 4 + airflow/www/static/underscore.js | 4 +- airflow/www/templates/admin/master.html | 7 + airflow/www/templates/airflow/chart.html | 2 +- airflow/www/templates/airflow/code.html | 2 +- airflow/www/templates/airflow/config.html | 2 +- airflow/www/templates/airflow/dag.html | 9 +- airflow/www/templates/airflow/dag_code.html | 2 +- airflow/www/templates/airflow/dags.html | 284 ++-- airflow/www/templates/airflow/duration_chart.html | 4 +- airflow/www/templates/airflow/gantt.html | 11 +- airflow/www/templates/airflow/graph.html | 32 +- airflow/www/templates/airflow/list_dags.html | 4 +- airflow/www/templates/airflow/query.html | 2 +- airflow/www/templates/airflow/task.html | 8 +- airflow/www/templates/airflow/task_instance.html | 2 +- airflow/www/templates/airflow/ti_code.html | 2 +- airflow/www/templates/airflow/tree.html | 32 +- airflow/www/templates/airflow/trigger.html | 3 + airflow/www/templates/airflow/xcom.html | 1 - airflow/www/views.py | 107 +- airflow/www_rbac/api/experimental/endpoints.py | 32 +- airflow/www_rbac/app.py | 7 +- airflow/www_rbac/forms.py | 61 +- airflow/www_rbac/security.py | 2 + airflow/www_rbac/static/css/main.css | 4 + airflow/www_rbac/static/js/connection_form.js | 4 + airflow/www_rbac/static/js/task-instances.js | 12 +- airflow/www_rbac/templates/airflow/chart.html | 2 +- airflow/www_rbac/templates/airflow/code.html | 2 +- airflow/www_rbac/templates/airflow/config.html | 2 +- airflow/www_rbac/templates/airflow/dag.html | 23 +- airflow/www_rbac/templates/airflow/dag_code.html | 2 +- airflow/www_rbac/templates/airflow/dags.html | 330 +++-- .../www_rbac/templates/airflow/duration_chart.html | 4 +- airflow/www_rbac/templates/airflow/gantt.html | 13 +- airflow/www_rbac/templates/airflow/graph.html | 32 +- airflow/www_rbac/templates/airflow/master.html | 8 +- airflow/www_rbac/templates/airflow/model_list.html | 2 +- airflow/www_rbac/templates/airflow/task.html | 8 +- .../www_rbac/templates/airflow/task_instance.html | 2 +- airflow/www_rbac/templates/airflow/ti_code.html | 2 +- airflow/www_rbac/templates/airflow/tree.html | 32 +- airflow/www_rbac/templates/airflow/trigger.html | 3 + airflow/www_rbac/templates/airflow/xcom.html | 1 - airflow/www_rbac/utils.py | 29 +- airflow/www_rbac/views.py | 128 +- breeze | 822 ++++++----- breeze-complete | 76 +- chart/charts/postgresql-6.3.12.tgz | Bin 0 -> 22754 bytes dev/airflow-github | 230 +++ dev/airflow-jira | 6 +- dev/requirements.txt | 1 + docs/Makefile | 194 --- docs/autoapi_templates/index.rst | 16 +- docs/best-practices.rst | 61 +- docs/build | 298 ++++ docs/build.sh | 108 -- docs/concepts.rst | 409 +++--- docs/conf.py | 7 +- docs/dag-run.rst | 233 +++ docs/dag-serialization.rst | 6 +- docs/executor/celery.rst | 2 + docs/executor/dask.rst | 2 + docs/executor/debug.rst | 3 + docs/executor/index.rst | 9 +- docs/executor/kubernetes.rst | 2 +- docs/executor/local.rst | 49 + .../gcp/index.rst => executor/sequential.rst} | 14 +- docs/faq.rst | 2 +- docs/howto/connection/aws.rst | 53 +- docs/howto/custom-operator.rst | 20 +- docs/howto/customize-state-colors-ui.rst | 70 + docs/howto/email-config.rst | 4 +- docs/howto/index.rst | 1 + docs/howto/operator/bash.rst | 31 + docs/howto/secure-connections.rst | 2 +- docs/howto/use-alternative-secrets-backend.rst | 99 +- docs/img/change-ui-colors/dags-page-new.png | Bin 0 -> 483599 bytes docs/img/change-ui-colors/dags-page-old.png | Bin 0 -> 493009 bytes docs/img/change-ui-colors/graph-view-new.png | Bin 0 -> 56973 bytes docs/img/change-ui-colors/graph-view-old.png | Bin 0 -> 54884 bytes docs/img/change-ui-colors/tree-view-new.png | Bin 0 -> 36934 bytes docs/img/change-ui-colors/tree-view-old.png | Bin 0 -> 21601 bytes docs/img/example_passing_conf.png | Bin 0 -> 97482 bytes docs/img/latest_only_with_trigger.png | Bin 49510 -> 42887 bytes docs/img/subdag_zoom.png | Bin 150185 -> 255915 bytes docs/img/variable_hidden.png | Bin 154299 -> 121301 bytes docs/index.rst | 1 + docs/installation.rst | 6 +- docs/kubernetes.rst | 23 +- docs/lineage.rst | 4 +- docs/plugins.rst | 22 +- docs/scheduler.rst | 186 +-- docs/security.rst | 32 +- docs/timezone.rst | 10 +- docs/tutorial.rst | 7 +- docs/ui.rst | 2 +- .../ci/docker-compose/empty => empty}/.gitignore | 0 .../kubernetes => kubernetes_tests}/__init__.py | 0 .../test_kubernetes_executor.py | 105 +- kubernetes_tests/test_kubernetes_pod_operator.py | 926 ++++++++++++ requirements/requirements-python2.7.txt | 177 +-- requirements/requirements-python3.5.txt | 210 +-- requirements/requirements-python3.6.txt | 229 +-- requirements/requirements-python3.7.txt | 227 +-- ...ts-python3.7.txt => requirements-python3.8.txt} | 228 +-- requirements/setup-2.7.md5 | 2 +- requirements/setup-3.5.md5 | 2 +- requirements/setup-3.6.md5 | 2 +- requirements/setup-3.7.md5 | 2 +- requirements/setup-3.8.md5 | 1 + scripts/ci/_all_libs.sh | 49 + scripts/ci/_script_init.sh | 9 +- scripts/ci/_utils.sh | 1525 -------------------- scripts/ci/ci_build_dockerhub.sh | 33 +- scripts/ci/ci_count_changed_files.sh | 6 +- .../{ci_docs.sh => ci_deploy_app_to_kubernetes.sh} | 38 +- scripts/ci/ci_docs.sh | 7 +- scripts/ci/ci_fix_ownership.sh | 12 + scripts/ci/ci_flake8.sh | 6 + ...oduction_images.sh => ci_load_image_to_kind.sh} | 17 +- scripts/ci/ci_mypy.sh | 3 + ...nts.sh => ci_perform_kind_cluster_operation.sh} | 13 +- ...image_on_ci.sh => ci_prepare_ci_image_on_ci.sh} | 3 - ...ace_on_ci.sh => ci_prepare_prod_image_on_ci.sh} | 6 +- scripts/ci/ci_run_airflow_testing.sh | 54 +- scripts/ci/ci_run_kubernetes_tests.sh | 110 ++ scripts/ci/docker-compose/backend-sqlite.yml | 2 +- scripts/ci/docker-compose/base.yml | 8 - scripts/ci/docker-compose/forward-credentials.yml | 9 +- scripts/ci/docker-compose/local-prod.yml | 7 +- scripts/ci/docker-compose/local.yml | 16 +- scripts/ci/docker-compose/runtime-kubernetes.yml | 25 - scripts/ci/in_container/_in_container_utils.sh | 112 +- scripts/ci/in_container/check_environment.sh | 61 +- scripts/ci/in_container/entrypoint_ci.sh | 138 +- scripts/ci/in_container/entrypoint_exec.sh | 2 +- .../ci/in_container/kubernetes/app/deploy_app.sh | 186 --- .../app/templates/init_git_sync.template.yaml | 36 - .../kubernetes/docker/airflow-test-env-init.sh | 28 - .../ci/in_container/kubernetes/docker/bootstrap.sh | 27 - .../kubernetes/docker/rebuild_airflow_image.sh | 93 -- .../in_container/kubernetes/setup_kind_cluster.sh | 187 --- scripts/ci/in_container/run_ci_tests.sh | 16 +- scripts/ci/in_container/run_docs_build.sh | 2 +- .../ci/in_container/run_generate_requirements.sh | 39 +- scripts/ci/in_container/run_system_tests.sh | 1 - .../kubernetes/app/postgres.yaml | 0 .../{in_container => }/kubernetes/app/secrets.yaml | 4 +- .../kubernetes/app/templates/airflow.template.yaml | 133 +- .../app/templates/configmaps.template.yaml | 10 +- .../app/templates/init_git_sync.template.yaml | 40 +- .../{in_container => }/kubernetes/app/volumes.yaml | 8 +- .../docker/airflow-test-env-init-dags.sh} | 19 +- .../kubernetes/docker/airflow-test-env-init-db.sh | 26 +- scripts/ci/kubernetes/docker/bootstrap.sh | 74 + .../kubernetes/kind-cluster-conf.yaml | 0 scripts/ci/libraries/_build_images.sh | 752 ++++++++++ scripts/ci/libraries/_initialization.sh | 318 ++++ scripts/ci/libraries/_kind.sh | 459 ++++++ scripts/ci/libraries/_local_mounts.sh | 66 + scripts/ci/libraries/_md5sum.sh | 137 ++ scripts/ci/libraries/_parameters.sh | 58 + scripts/ci/libraries/_permissions.sh | 59 + scripts/ci/libraries/_push_pull_remove_images.sh | 182 +++ scripts/ci/{ci_docs.sh => libraries/_runs.sh} | 39 +- scripts/ci/libraries/_sanity_checks.sh | 166 +++ scripts/ci/libraries/_spinner.sh | 53 + scripts/ci/libraries/_start_end.sh | 83 ++ scripts/ci/libraries/_verbosity.sh | 54 + scripts/ci/pre_commit_local_yml_mounts.sh | 32 +- scripts/ci/run_cli_tool.sh | 154 ++ scripts/{ci/_script_init.sh => prod/clean-logs.sh} | 21 +- entrypoint.sh => scripts/prod/entrypoint_prod.sh | 17 +- setup.cfg | 28 +- setup.py | 37 +- tests/api/common/experimental/test_pool.py | 64 +- tests/api/common/experimental/test_trigger_dag.py | 25 + tests/bats/bats_utils.bash | 4 +- tests/bats/test_local_mounts.bats | 79 +- tests/cli/test_cli.py | 484 ++++++- tests/conftest.py | 37 - .../contrib/executors/test_kubernetes_executor.py | 1102 -------------- tests/contrib/hooks/test_aws_glue_catalog_hook.py | 1 + tests/contrib/hooks/test_cassandra_hook.py | 2 +- tests/contrib/hooks/test_discord_webhook_hook.py | 6 +- tests/contrib/hooks/test_slack_webhook_hook.py | 3 +- tests/contrib/hooks/test_snowflake_hook.py | 6 +- tests/contrib/hooks/test_spark_submit_hook.py | 2 +- .../test_kubernetes_request_factory.py | 397 ----- .../test_pod_request_factory.py | 176 --- .../contrib/operators/test_databricks_operator.py | 6 +- tests/contrib/operators/test_dataproc_operator.py | 2 +- .../test_gcp_sql_operator_system_helper.py | 11 +- .../contrib/operators/test_gcs_to_gcs_operator.py | 6 +- tests/contrib/operators/test_hive_stats.py | 340 +++++ tests/contrib/operators/test_hive_to_mysql.py | 154 ++ tests/contrib/operators/test_hive_to_samba.py | 88 ++ .../operators/test_mssql_to_gcs_operator.py | 7 +- .../operators/test_qubole_check_operator.py | 7 +- tests/contrib/operators/test_sftp_operator.py | 6 +- .../operators/test_slack_webhook_operator.py | 6 +- tests/contrib/operators/test_ssh_operator.py | 6 +- tests/contrib/operators/test_winrm_operator.py | 6 +- tests/contrib/secrets/test_aws_systems_manager.py | 30 + tests/contrib/secrets/test_hashicorp_vault.py | 27 + .../sensors/test_hive_partition.py} | 39 +- tests/contrib/sensors/test_metastore_partition.py | 43 + tests/contrib/sensors/test_webhdfs.py | 42 + tests/contrib/sensors/test_weekday_sensor.py | 18 +- tests/contrib/utils/base_gcp_system_test_case.py | 2 +- tests/contrib/utils/gcp_authenticator.py | 16 +- tests/contrib/utils/logging_command_executor.py | 20 +- .../contrib/utils/test_mlengine_operator_utils.py | 16 +- ...test_mark_success.py => test_logging_in_dag.py} | 34 +- tests/executors/test_celery_executor.py | 202 ++- tests/executors/test_dask_executor.py | 6 +- tests/executors/test_kubernetes_executor.py | 307 ++++ tests/executors/test_local_executor.py | 20 +- .../test_snowflake_operator.py => hive.py} | 50 +- tests/hooks/test_hive_hook.py | 626 +++++--- tests/insert_extras.py | 2 +- tests/jobs/test_backfill_job.py | 9 +- tests/jobs/test_base_job.py | 24 +- tests/jobs/test_local_task_job.py | 122 +- tests/jobs/test_scheduler_job.py | 127 +- tests/{runtime => kubernetes}/__init__.py | 0 .../models}/__init__.py | 0 tests/kubernetes/models/test_pod.py | 76 + tests/kubernetes/models/test_secret.py | 117 ++ .../app/secrets.yaml => tests/kubernetes/pod.yaml | 21 +- tests/kubernetes/test_client.py | 8 +- tests/kubernetes/test_pod_generator.py | 771 ++++++++++ tests/kubernetes/test_pod_launcher.py | 76 +- tests/kubernetes/test_worker_configuration.py | 709 +++++++++ tests/{contrib/kubernetes => minikube}/__init__.py | 0 tests/models/test_baseoperator.py | 5 +- tests/models/test_dag.py | 67 +- tests/models/test_dagcode.py | 6 + tests/models/test_dagrun.py | 29 + tests/models/test_taskinstance.py | 164 ++- tests/operators/test_check_operator.py | 327 ----- tests/operators/test_docker_operator.py | 6 +- tests/operators/test_hive_operator.py | 411 +++--- tests/operators/test_operators.py | 249 +++- tests/operators/test_s3_to_hive_operator.py | 12 +- tests/operators/test_sql.py | 746 ++++++++++ tests/plugins/test_plugins_manager_rbac.py | 27 + .../kubernetes/operators/test_kubernetes_pod.py | 145 ++ .../kubernetes/test_kubernetes_pod_operator.py | 555 ------- tests/secrets/test_local_filesystem.py | 229 +++ tests/secrets/test_secrets.py | 14 + tests/security/test_kerberos.py | 4 +- tests/sensors/test_external_task_sensor.py | 22 + tests/sensors/test_http_sensor.py | 3 +- tests/sensors/test_named_hive_partition_sensor.py | 107 +- tests/serialization/test_dag_serialization.py | 179 ++- tests/test_configuration.py | 44 + tests/test_core.py | 210 ++- tests/test_local_settings.py | 4 +- tests/test_utils/mock_executor.py | 9 +- tests/test_utils/mock_hooks.py | 96 ++ tests/test_utils/mock_operators.py | 9 + tests/test_utils/mock_process.py | 87 ++ tests/utils/log/test_es_task_handler.py | 5 +- tests/utils/test_compression.py | 16 +- tests/utils/test_dag_processing.py | 133 +- tests/utils/test_decorators.py | 10 +- tests/utils/test_json.py | 11 +- tests/utils/test_module_loading.py | 4 +- tests/www/test_validators.py | 11 +- tests/www/test_views.py | 9 + tests/www_rbac/test_validators.py | 9 +- tests/www_rbac/test_views.py | 214 ++- 468 files changed, 20607 insertions(+), 14511 deletions(-) delete mode 100644 .travis.yml delete mode 100644 airflow/_vendor/README delete mode 100644 airflow/_vendor/__init__.py delete mode 100644 airflow/_vendor/nvd3/LICENSE delete mode 100644 airflow/_vendor/nvd3/NVD3Chart.py delete mode 100755 airflow/_vendor/nvd3/__init__.py delete mode 100644 airflow/_vendor/nvd3/cumulativeLineChart.py delete mode 100644 airflow/_vendor/nvd3/discreteBarChart.py delete mode 100644 airflow/_vendor/nvd3/ipynb.py delete mode 100644 airflow/_vendor/nvd3/lineChart.py delete mode 100644 airflow/_vendor/nvd3/linePlusBarChart.py delete mode 100644 airflow/_vendor/nvd3/lineWithFocusChart.py delete mode 100644 airflow/_vendor/nvd3/multiBarChart.py delete mode 100644 airflow/_vendor/nvd3/multiBarHorizontalChart.py delete mode 100644 airflow/_vendor/nvd3/pieChart.py delete mode 100644 airflow/_vendor/nvd3/scatterChart.py delete mode 100644 airflow/_vendor/nvd3/stackedAreaChart.py delete mode 100644 airflow/_vendor/nvd3/templates/base.html delete mode 100644 airflow/_vendor/nvd3/templates/content.html delete mode 100644 airflow/_vendor/nvd3/templates/cumulativelinechart.html delete mode 100644 airflow/_vendor/nvd3/templates/discretebarchart.html delete mode 100644 airflow/_vendor/nvd3/templates/linebarwfocuschart.html delete mode 100644 airflow/_vendor/nvd3/templates/linechart.html delete mode 100644 airflow/_vendor/nvd3/templates/lineplusbarchart.html delete mode 100644 airflow/_vendor/nvd3/templates/linewfocuschart.html delete mode 100644 airflow/_vendor/nvd3/templates/multibarchart.html delete mode 100644 airflow/_vendor/nvd3/templates/multibarcharthorizontal.html delete mode 100644 airflow/_vendor/nvd3/templates/page.html delete mode 100644 airflow/_vendor/nvd3/templates/piechart.html delete mode 100644 airflow/_vendor/nvd3/templates/scatterchart.html delete mode 100644 airflow/_vendor/nvd3/templates/stackedareachart.html delete mode 100644 airflow/_vendor/nvd3/translator.py delete mode 100644 airflow/_vendor/slugify/LICENSE delete mode 100644 airflow/_vendor/slugify/__init__.py delete mode 100644 airflow/_vendor/slugify/slugify.py delete mode 100644 airflow/contrib/executors/__init__.py delete mode 100644 airflow/contrib/kubernetes/kubernetes_request_factory/kubernetes_request_factory.py delete mode 100644 airflow/contrib/kubernetes/kubernetes_request_factory/pod_request_factory.py delete mode 100644 airflow/contrib/kubernetes/pod.py delete mode 100644 airflow/contrib/kubernetes/pod_generator.py delete mode 100644 airflow/contrib/kubernetes/volume_mount.py delete mode 100644 airflow/contrib/kubernetes/worker_configuration.py rename airflow/{contrib => }/example_dags/example_kubernetes_executor_config.py (96%) rename airflow/{contrib => }/executors/kubernetes_executor.py (85%) rename airflow/{contrib/kubernetes/kubernetes_request_factory => kubernetes}/__init__.py (100%) create mode 100644 airflow/kubernetes/k8s_model.py rename airflow/{contrib => }/kubernetes/kube_client.py (97%) create mode 100644 airflow/kubernetes/pod.py create mode 100644 airflow/kubernetes/pod_generator.py rename airflow/{contrib => }/kubernetes/pod_launcher.py (73%) rename airflow/{contrib => }/kubernetes/pod_runtime_info_env.py (63%) rename airflow/{contrib => }/kubernetes/refresh_config.py (99%) rename airflow/{contrib => }/kubernetes/secret.py (55%) rename airflow/{contrib => }/kubernetes/volume.py (72%) create mode 100644 airflow/kubernetes/volume_mount.py create mode 100644 airflow/kubernetes/worker_configuration.py copy airflow/migrations/versions/{f23433877c24_fix_mysql_not_null_constraint.py => a66efa278eea_add_precision_to_execution_date_in_mysql.py} (51%) create mode 100644 airflow/operators/sql.py copy airflow/operators/{dummy_operator.py => sql_branch_operator.py} (61%) create mode 100644 airflow/secrets/local_filesystem.py copy tests/compat.py => airflow/typing_compat.py (67%) create mode 100644 airflow/utils/code_utils.py copy airflow/{hooks/sqlite_hook.py => utils/mixins.py} (61%) copy airflow/utils/{configuration.py => platform.py} (51%) create mode 100644 chart/charts/postgresql-6.3.12.tgz create mode 100755 dev/airflow-github delete mode 100644 docs/Makefile create mode 100755 docs/build delete mode 100755 docs/build.sh create mode 100644 docs/dag-run.rst create mode 100644 docs/executor/local.rst copy docs/{howto/operator/gcp/index.rst => executor/sequential.rst} (65%) create mode 100644 docs/howto/customize-state-colors-ui.rst create mode 100644 docs/img/change-ui-colors/dags-page-new.png create mode 100644 docs/img/change-ui-colors/dags-page-old.png create mode 100644 docs/img/change-ui-colors/graph-view-new.png create mode 100644 docs/img/change-ui-colors/graph-view-old.png create mode 100644 docs/img/change-ui-colors/tree-view-new.png create mode 100644 docs/img/change-ui-colors/tree-view-old.png create mode 100644 docs/img/example_passing_conf.png copy {scripts/ci/docker-compose/empty => empty}/.gitignore (100%) rename {tests/runtime/kubernetes => kubernetes_tests}/__init__.py (100%) rename {tests/runtime/kubernetes => kubernetes_tests}/test_kubernetes_executor.py (71%) create mode 100644 kubernetes_tests/test_kubernetes_pod_operator.py copy requirements/{requirements-python3.7.txt => requirements-python3.8.txt} (65%) create mode 100644 requirements/setup-3.8.md5 create mode 100755 scripts/ci/_all_libs.sh delete mode 100644 scripts/ci/_utils.sh copy scripts/ci/{ci_docs.sh => ci_deploy_app_to_kubernetes.sh} (56%) copy scripts/ci/{ci_push_production_images.sh => ci_load_image_to_kind.sh} (61%) copy scripts/ci/{ci_generate_requirements.sh => ci_perform_kind_cluster_operation.sh} (78%) rename scripts/ci/{ci_prepare_image_on_ci.sh => ci_prepare_ci_image_on_ci.sh} (91%) copy scripts/ci/{ci_free_space_on_ci.sh => ci_prepare_prod_image_on_ci.sh} (91%) create mode 100755 scripts/ci/ci_run_kubernetes_tests.sh delete mode 100644 scripts/ci/docker-compose/runtime-kubernetes.yml delete mode 100755 scripts/ci/in_container/kubernetes/app/deploy_app.sh delete mode 100644 scripts/ci/in_container/kubernetes/app/templates/init_git_sync.template.yaml delete mode 100755 scripts/ci/in_container/kubernetes/docker/airflow-test-env-init.sh delete mode 100644 scripts/ci/in_container/kubernetes/docker/bootstrap.sh delete mode 100755 scripts/ci/in_container/kubernetes/docker/rebuild_airflow_image.sh delete mode 100755 scripts/ci/in_container/kubernetes/setup_kind_cluster.sh rename scripts/ci/{in_container => }/kubernetes/app/postgres.yaml (100%) copy scripts/ci/{in_container => }/kubernetes/app/secrets.yaml (84%) rename scripts/ci/{in_container => }/kubernetes/app/templates/airflow.template.yaml (56%) rename scripts/ci/{in_container => }/kubernetes/app/templates/configmaps.template.yaml (98%) copy airflow/api/auth/backend/deny_all.py => scripts/ci/kubernetes/app/templates/init_git_sync.template.yaml (55%) rename scripts/ci/{in_container => }/kubernetes/app/volumes.yaml (94%) copy scripts/ci/{pre_commit_update_extras.sh => kubernetes/docker/airflow-test-env-init-dags.sh} (72%) copy hooks/push => scripts/ci/kubernetes/docker/airflow-test-env-init-db.sh (64%) create mode 100755 scripts/ci/kubernetes/docker/bootstrap.sh rename scripts/ci/{in_container => }/kubernetes/kind-cluster-conf.yaml (100%) create mode 100644 scripts/ci/libraries/_build_images.sh create mode 100644 scripts/ci/libraries/_initialization.sh create mode 100644 scripts/ci/libraries/_kind.sh create mode 100644 scripts/ci/libraries/_local_mounts.sh create mode 100644 scripts/ci/libraries/_md5sum.sh create mode 100644 scripts/ci/libraries/_parameters.sh create mode 100644 scripts/ci/libraries/_permissions.sh create mode 100644 scripts/ci/libraries/_push_pull_remove_images.sh copy scripts/ci/{ci_docs.sh => libraries/_runs.sh} (51%) mode change 100755 => 100644 create mode 100644 scripts/ci/libraries/_sanity_checks.sh create mode 100644 scripts/ci/libraries/_spinner.sh create mode 100644 scripts/ci/libraries/_start_end.sh create mode 100644 scripts/ci/libraries/_verbosity.sh create mode 100755 scripts/ci/run_cli_tool.sh copy scripts/{ci/_script_init.sh => prod/clean-logs.sh} (66%) rename entrypoint.sh => scripts/prod/entrypoint_prod.sh (89%) delete mode 100644 tests/contrib/executors/test_kubernetes_executor.py delete mode 100644 tests/contrib/kubernetes/kubernetes_request_factory/test_kubernetes_request_factory.py delete mode 100644 tests/contrib/kubernetes/kubernetes_request_factory/test_pod_request_factory.py create mode 100644 tests/contrib/operators/test_hive_stats.py create mode 100644 tests/contrib/operators/test_hive_to_mysql.py create mode 100644 tests/contrib/operators/test_hive_to_samba.py copy tests/{sensors/test_timedelta_sensor.py => contrib/sensors/test_hive_partition.py} (50%) create mode 100644 tests/contrib/sensors/test_metastore_partition.py create mode 100644 tests/contrib/sensors/test_webhdfs.py copy tests/dags/{test_mark_success.py => test_logging_in_dag.py} (64%) create mode 100644 tests/executors/test_kubernetes_executor.py copy tests/{contrib/operators/test_snowflake_operator.py => hive.py} (50%) rename tests/{runtime => kubernetes}/__init__.py (100%) rename tests/{contrib/kubernetes/kubernetes_request_factory => kubernetes/models}/__init__.py (100%) create mode 100644 tests/kubernetes/models/test_pod.py create mode 100644 tests/kubernetes/models/test_secret.py rename scripts/ci/in_container/kubernetes/app/secrets.yaml => tests/kubernetes/pod.yaml (71%) create mode 100644 tests/kubernetes/test_pod_generator.py create mode 100644 tests/kubernetes/test_worker_configuration.py rename tests/{contrib/kubernetes => minikube}/__init__.py (100%) delete mode 100644 tests/operators/test_check_operator.py create mode 100644 tests/operators/test_sql.py create mode 100644 tests/providers/cncf/kubernetes/operators/test_kubernetes_pod.py delete mode 100644 tests/runtime/kubernetes/test_kubernetes_pod_operator.py create mode 100644 tests/secrets/test_local_filesystem.py create mode 100644 tests/test_utils/mock_hooks.py create mode 100644 tests/test_utils/mock_process.py