potiuk commented on a change in pull request #7506: [AIRFLOW-6877] [WIP] check
cross providers dependencies depends on [AIRFLOW-6663]
URL: https://github.com/apache/airflow/pull/7506#discussion_r383025733
##########
File path:
airflow/providers/google/marketing_platform/operators/campaign_manager.py
##########
@@ -20,16 +20,23 @@
"""
import tempfile
import uuid
+import warnings
from typing import Any, Dict, List, Optional
from googleapiclient import http
from airflow.exceptions import AirflowException
from airflow.models import BaseOperator
-from airflow.providers.google.cloud.hooks.gcs import GCSHook
from airflow.providers.google.marketing_platform.hooks.campaign_manager import
GoogleCampaignManagerHook
from airflow.utils.decorators import apply_defaults
+try:
+ from airflow.providers.google.cloud.hooks.gcs import GCSHook
+except ImportError:
+ warnings.warn("The package google-cloud must be installed to use the
operator.",
Review comment:
I agree - but either we decide to move it all to one folder/provider or we
leave it as three dependent packages. I do not want to make an exception to the
structure of provider packages. It makes it much easier to have one approach
where provider package (e.g. google.cloud) is under "providers/google/cloud/"
directory and "operators/hooks/sensors" are right directly under it.
This way we do not have to keep any mapping between folders and provider
packages - it is straightforward and can be easily derived from where the
operators/hooks/sensors folders are (in fact I am already doing it when I check
for import-cross-dependencies).
I am ok with both - separate packages and dependencies between them and
moving everything back to "google" provider. Alternatively we could have
google/common but I think it's the worst idea of the three. I have slight
preference for option 1) because with limited risk and effort it will let us
explore consequences of having multiple packages with dependencies but 2) is
also fine for me.
Whatever we do - I think we should keep consistency between folder structure
and "provider packages" structure (provider name is formed from the directories
between "providers" and "operators" basically).
What do you think @mik-laj @nuclearpinguin @michalslowikowski00 @ANiteckiP
@kaxil @ashb ?
Option 1):
1 single "google" provider package:
```
providers/google/
operators
hooks
sensors
```
Option 2)
3 separate providers: "google.cloud", "google.marketing", "google.suite".
Both "gogle.marketing" and "google.suite" is pretty much useless without
"google.cloud" (but it will be printed as warning when there is no google.cloud
already installed..
```
providers/google/cloud
operators
...
providers/google/marketing
operators
...
providers/google/suite
operators
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
