[
https://issues.apache.org/jira/browse/AIRFLOW-22?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278287#comment-15278287
]
Chris Riccomini commented on AIRFLOW-22:
----------------------------------------
What's GDELT?
In the case of GCS, yes I believe we can use public buckets. I'm uncomfortable
baking in someone's own buckets, though. It'd have to be public buckets that
Google, themselves, provide. For BQ, unfortunately not. The problem with BQ is
that it requires a project to run its queries on. Even if the datasets are
public, you still need to have access to a whole project, and be a viewer for
it, in order to execute any queries against the data.
> Google cloud code needs tests
> -----------------------------
>
> Key: AIRFLOW-22
> URL: https://issues.apache.org/jira/browse/AIRFLOW-22
> Project: Apache Airflow
> Issue Type: Improvement
> Components: gcp
> Reporter: Chris Riccomini
> Labels: gcp
>
> The Google cloud contrib hooks and operators need tests to verify that they
> work. This is somewhat tricky since they're hard to mock, and most of the
> value comes from integration tests, not unit tests.
> I spoke with [~alexvanboxel], and he suggested just writing a script that can
> be run manually, and exercises the hooks/operators to verify that they work
> as expected. I think this is about the best we can do without spending a ton
> of time on it, since most of the integration tests require a real Google
> cloud project, with real buckets/BQ datasets/DBs, etc. Running it locally
> means we can run the tests against whatever projects we have access to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
