[
https://issues.apache.org/jira/browse/AIRFLOW-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883306#comment-16883306
]
Elad commented on AIRFLOW-4899:
-------------------------------
Seems like a bug in the hook.
https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets/list
list returns nextPageToken - A token that can be used to request the next
results page. This property is omitted on the final results page.
So the {code} return datasets_list{code}
doesn't actually return the full list. It returns only the first page.
> get_dataset_list from bigquery hook return only 50 first datasets
> -----------------------------------------------------------------
>
> Key: AIRFLOW-4899
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4899
> Project: Apache Airflow
> Issue Type: Bug
> Components: hooks
> Affects Versions: 1.10.2, 1.10.3
> Environment: composer-1.7.1-airflow-1.10.2
> Python 3
> Reporter: Benjamin
> Priority: Major
>
> Linked with AIRFLOW-3055
> The next method must return "full list of BigQuery datasets in the current
> project" such as is documented, but return only the 50 first datasets.
> {code:java}
> def get_datasets_list(self, project_id=None){code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
