Sabbir02 opened a new issue, #45872: URL: https://github.com/apache/airflow/issues/45872
### Description In the event that I must know the page count beforehand. Two inefficient calls to the API are required for this, and neither one guarantees that every page will be viewed on the second call. HttpOperator by enabling it to handle custom pagination logic, providing users with more flexibility in managing API pagination. The goal is to replace the deprecated SimpleHttpOperator, offering a more efficient and adaptable way to process paginated API data, which improves how workflows interact with APIs within Airflow. ### Use case/motivation For APIs where the pagination logic is not fixed (e.g., using a cursor or other complex structures), this update allows users to implement their custom logic for paging through results. Support for dynamic pagination in APIs is critical for many use cases. By allowing users to pass their own pagination functions, the operator can handle different API structures and pagination methods without modifications. ### Related issues Included in Airflow version 2.8.0 Reference PR: [#34606] ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
