marcosmamorim opened a new pull request, #35622: URL: https://github.com/apache/superset/pull/35622
### SUMMARY Fixes rate limiting issues when fetching Slack channels in large workspaces (10k+ channels). **Note:** This PR takes a more comprehensive approach than #35588 by: - Making retry count configurable instead of hardcoded - **Preventing** rate limits through throttling rather than just detecting them - Providing configuration options for different workspace sizes - Including complete documentation for large workspaces **Problem:** Workspaces with 20k+ channels require ~21 paginated API calls. Without throttling, rapid consecutive requests trigger Slack's rate limits ([Tier 2/3: 20-50 req/min](https://api.slack.com/apis/rate-limits)). The previous hardcoded retry count of 2 was insufficient for large workspaces. **Solution:** - Add `SLACK_API_REQUEST_DELAY` config (default: 0.5s) to throttle paginated API requests - Add `SLACK_API_RATE_LIMIT_RETRY_COUNT` config (default: 2) to make retry count configurable - Implement `time.sleep()` between paginated requests in `get_channels()` - Add detailed progress logging (page count, channels fetched) - Improve error messages with configuration suggestions - Update documentation for large workspaces (10k+ channels) **Impact:** - Backwards compatible: Default values maintain current behavior for small workspaces - Large workspaces: Users can now configure higher retry counts (5-10) and delays (1.0-2.0s) - Better observability with detailed logging ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A - Backend-only changes ### TESTING INSTRUCTIONS **Prerequisites:** - Set `SLACK_API_TOKEN` in config - Enable `ALERT_REPORTS` feature flag **Manual Testing:** 1. Create an Alert/Report with Slack notification 2. Select a Slack channel from the dropdown 3. Verify channels load successfully For large workspaces (10k+ channels), configure: ```python from datetime import timedelta SLACK_CACHE_TIMEOUT = int(timedelta(days=2).total_seconds()) SLACK_API_REQUEST_DELAY = 1.0 SLACK_API_RATE_LIMIT_RETRY_COUNT = 10 ``` Check logs for progress messages: "Fetched page X: Y channels (total: Z)" ### ADDITIONAL INFORMATION - [x] Has associated issue: Fixes #32480 - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
