potiuk opened a new issue #10014: URL: https://github.com/apache/airflow/issues/10014
**Description** I think we are really close to release the second wave of Backport Packages. We accumulated a number of changes already and few more are about to be completed so that we are, I think close to release the second wave. The changes since the last backport release are listed at the end but there are a few changes that we might want to complete first: - [ ] Enabling `cncf.kubernetes` provider after 1.10.11 backported changes and spark-on-kubernetes operator (@turbaszek works on it) - [ ] Implementing Impersonation for all applicable Google operators (@olchas works on it with Hooks already merged) - [ ] DataProc operator improvements (@turbaszek works on it) **Use case / motivation** We want the users of airflow 1.10 to benefit from fixes and improvements in the latest packages. **More info** Current set of changes in the provider's package (not grouped by provider(: * 0aff69fbd Add typing to ImapHook (#9887) * 81b87d48e Add unit tests for GcpBodyFieldSanitizer in Google providers (#9996) * 7d24b088c Stop using start_date in default_args in example_dags (2) (#9985) * 7cc1c8bc0 Updates the slack WebClient call to use the instance variable - token (#9995) * 8b10a4b35 Stop using start_date in default_args in example_dags (#9982) * 0bf330ba8 Add get_blobs_list method to WasbHook (#9950) * ef98edf4d Add more information about using GoogleAdsHook (#9951) * 867bc44ac Add type annotations to providers/vertica (#9936) * 33f0cd265 apply_default keeps the function signature for mypy (#9784) * 39a0288a4 Add Google Authentication for experimental API (#9848) * ac93419d1 Add response_filter parameter to SimpleHttpOperator (#9885) * c2db0dfeb More strict rules in mypy (#9705) (#9906) * 1427e4acb Update Spark submit operator for Spark 3 support (#8730) * c4244e18b Fix calling `get_client` in BigQueryHook.table_exists (#9916) * 719ae2bf6 Dump Pod as YAML in logs for KubernetesPodOperator (#9895) * 5eacc1642 Add support for impersonation in GCP hooks (#9915) * 4eddce22a Add typing for grpc provider (#9884) * d5d119bab Increase typing coverage for Elasticsearch (#9911) * 1cfdebf5f Fix insert_job method of BigQueryHook (#9899) * c8c52e69c Remove type hint causing DeprecationWarning in Firestore operators (#9819) * fc8d38d5f improve typing for openfaas provider (#9883) * e7c87fe45 Refactor AwsBaseHook._get_credentials (#9878) * 840799d55 Improve KubernetesPodOperator guide (#9079) * 5013fda8f Add drop_partition functionality for HiveMetastoreHook (#9472) * 89fe5feb7 improve typing for datadog provider (#9775) * 4d74ac211 Increase typing for Apache and http provider package (#9729) * 750555f26 Add guide for Cassandra Operators (#9877) * 46cdb0e08 Add type hinting for mongo provider (#9875) * 027cc1682 Improve type annotations for Ftp provider (#9868) * eb6f1d1cf Fix typo in datafusion operator (#9859) * 2577f9334 Fix S3FileTransformOperator to support S3 Select transformation only (#8936) * b01d95ec2 Change DAG.clear to take dag_run_state (#9824) * 6d65c15d1 Add guide for AI Platform (previously Machine Learning Engine) Operators (#9798) * 770de53eb BigQueryTableExistenceSensor needs to specify keyword arguments (#9832) * 2d8dbacdf Add CloudVisionDeleteReferenceImageOperator (#9698) * 52b6efe1e Add option to delete by prefix to S3DeleteObjectsOperator (#9350) * 35fe97225 Improve type hinting to provider cloudant (#9825) * 9f017951b Add Google Deployment Manager Hook (#9159) * a2c5389a6 Add kylin operator (#9149) * ed5004cca Allow `replace` flag in gcs_to_gcs operator. (#9667) * 0a2acf0b6 Add type annotations for redis provider (#9815) * 553bb7af7 Keep functions signatures in decorators (#9786) * 68925904e Add multiple file upload functionality to GCS hook (#8849) * d3c76da95 Improve type hinting to provider microsoft (#9774) * 1de78e8f9 Add Google Stackdriver link (#9765) * a518801f8 Add type hinting for discord provider (#9773) * 5bb228d84 improve type hinting for celery provider (#9762) * 092d33f29 Fix StackdriverTaskHandler + add system tests (#9761) * 0873070e0 Mask other forms of password arguments in SparkSubmitOperator (#9615) * b2305660f Update example DAG for AI Platform operators (#9727) * 13a827d80 Ensure Kerberos token is valid in SparkSubmitOperator before running `yarn kill` (#9044) * 2f31b3060 Get Airflow configs with sensitive data from Secret Backends (#9645) * 07b81029e Allow AWSAthenaHook to get more than 1000/first page of results (#6075) * 23f80f34a Move gcs & wasb task handlers to their respective provider packages (#9714) * 564192c16 Add AWS StepFunctions integrations to the aws provider (#8749) * ecce1ace7 [AIRFLOW-XXXX] Remove unnecessary docstring in AWSAthenaOperator * 44d4ae809 Upgrade to latest pre-commit checks (#9686) * a79e2d4c4 Move provider's log task handlers to the provider package (#9604) * a99aaeb49 Allow setting Hashicorp Vault token from File (#9644) * ee20086b8 Move S3TaskHandler to the AWS provider package (#9602) * cd3d9d934 Fix using .json template extension in GMP operators (#9566) * 8bd15ef63 Switches to Helm Chart for Kubernetes tests (#9468) * 4799af30e Extend BigQuery example with include clause (#9572) * e33f1a12d Add template_ext to BigQueryInsertJobOperator (#9568) * 067806d59 Add tests for spark_jdbc_script (#9491) * 40add26d4 Remove almost all references to airflow.contrib (#9559) * 2248a5da1 Expose option: look_for_keys in ssh_hook via extras (#8793) * a97400d0d Move out sendgrid emailer from airflow.contrib (#9355) * c420dbd6e Bump Pylint to 2.5.3 (#9294) * c858babdd Remove kwargs from Super calls in AWS Secrets Backends (#9523) * 0051c89cb nitpick fix (#9527) * 87fdbd070 Use literal syntax instead of function calls to create data structure (#9516) * 3190db524 [AIRFLOW-9347] Fix QuboleHook unable to add list to tags (#9349) * 7256f4caa Pylint fixes and deprecation of rare used methods in Connection (#9419) * c7a454aa3 Add AWS ECS system test (#8888) * 5d61580c5 Enable 'Public function Missing Docstrings' PyDocStyle Check (#9463) * df8efd04f Enable & Fix "Docstring Content Issues" PyDocStyle Check (#9460) * e13a14c87 Enable & Fix Whitespace related PyDocStyle Checks (#9458) * 5b680e27e Don't use connection to store task handler credentials (#9381) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
