potiuk commented on a change in pull request #19237:
URL: https://github.com/apache/airflow/pull/19237#discussion_r737873020
##########
File path:
airflow/providers/amazon/aws/example_dags/example_google_api_to_s3_transfer_advanced.py
##########
@@ -33,13 +33,13 @@
on a YouTube channel you want to retrieve.
"""
+from datetime import datetime
from os import getenv
from airflow import DAG
from airflow.operators.dummy import DummyOperator
-from airflow.operators.python import BranchPythonOperator, get_current_context
+from airflow.operators.python import BranchPythonOperator
from airflow.providers.amazon.aws.transfers.google_api_to_s3 import
GoogleApiToS3Operator
-from airflow.utils.dates import days_ago
# [START howto_operator_google_api_to_s3_transfer_advanced_env_variables]
YOUTUBE_CONN_ID = getenv("YOUTUBE_CONN_ID", "google_cloud_default")
Review comment:
> There is documentation that references these variables and
specifically states \"This example relies on the following variables, which can
be passed via OS environment variables\". I didn't want to remove any context
related to main operator documentation.
The example days in many cases (mostly GCP) serve also as executable
examples. This is why the variables are there because by setting those
variables and using accompanying SystemTests we can actuallub RUN those example
DAGS and they will prove that operators are working. Also in many cases where
Operators are modified or refactored we can (and often do) we just run the
example DAGs with real services and when they work, we have the ultimate proof
that the operators work. This is implemented not only by GCP by also many other
providers. We have outstanding AIP-4 (one of the oldest apps) to make those
tests run automatically regularly - has not happened yet but maybe soon :). For
now periodically various people tests and use those on ad-hoc level and the
examples require usually very little fixes to work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
