vanchaxy opened a new issue, #29396:
URL: https://github.com/apache/airflow/issues/29396
### Apache Airflow Provider(s)
google
### Versions of Apache Airflow Providers
apache-airflow-providers-google==7.0.0
But the problem exists in all newer versions.
### Apache Airflow version
apache-airflow==2.3.2
### Operating System
Ubuntu 20.04.4 LTS
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### What happened
The `list_rows` method in the BigQuery Hook does not return the page_token
value, which is necessary for paginating query results. Same problem with
`get_datasets_list` method.
The documentation for the `get_datasets_list` method even states that the
page_token parameter can be accessed:
```
:param page_token: Token representing a cursor into the
datasets. If not passed,
the API will return the first page of datasets. The token marks
the beginning of the
iterator to be returned and the value of the ``page_token`` can
be accessed at
``next_page_token`` of the
:class:`~google.api_core.page_iterator.HTTPIterator`.
```
but it doesn't return HTTPIterator. Instead, it converts the HTTPIterator to
a list[DatasetListItem] using list(datasets), making it impossible to retrieve
the original HTTPIterator and thus impossible to obtain the next_page_token.
### What you think should happen instead
`list_rows` \ `get_datasets_list` methods should return `Iterator` OR both
the list of rows\datasets and the page_token value to allow users to retrieve
multiple results pages. For backward compatibility, we can have a parameter
like return_iterator=True or smth like that.
### How to reproduce
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
