[
https://issues.apache.org/jira/browse/NIFI-13146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-13146.
-------------------------------
Fix Version/s: 2.0.0-M3
Resolution: Fixed
> ConsumeSlack processor rate limited
> -----------------------------------
>
> Key: NIFI-13146
> URL: https://issues.apache.org/jira/browse/NIFI-13146
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Zsihovszki Krisztina
> Assignee: Zsihovszki Krisztina
> Priority: Major
> Fix For: 2.0.0-M3
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> ConsumeSlack processor was not able to start when it was running in a Slack
> workplace with thousands of channels. It got stuck in initialization phase
> and reported rate limit error continuously.
> The processor fetches all available channels (conversation list) during its
> setup and creates a channel id/name mapping.
> Fetching the conversation list items is executed in batches, 1000
> channels/batch. Since the fetch is done continously, after a while Slack API
> returns rate limit error. (Rate limit settings were at default in Slack.)
> Even if some delay was added after each API call, 30 seconds was not enough
> to fetch all the channels (since it is in onScheduled, after 30 seconds the
> initialization is re-attempted)
> As a mitigation for the problem I'd like to add a logic which checks if
> "Channels" property contains only IDs. In case no channel name is specified,
> another Slack API call (
> [conversations.info|https://api.slack.com/methods/conversations.info]) can be
> used to fetch the channel names for the channel IDs and it is not necessary
> to fetch the whole conversation list.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
