Hello,
We implemented the custom process that are similar to the InvokeHTTP that the
part of URL can be replaced with the Context Data List, then write the weather
to the flowfile. For example, URL to get the weather feed have to include the
zip code in URL, and the ZIP code is {0} in the URL and replaced the zip code
from the Context Data List property.
URL
http://example{0}/weather<http://example%7b0%7d/weather>
Context Data List:
00000
11111
22222
Processor with make the following requests:
http://example{0}/weather<http://example%7b0%7d/weather>
http://example00000/weather
http://example11111/weather
http://example22222/weather
This processor is processed in one request at a time and have a perf issue.
I'd like to modify to process in batches. What are the best way to process in
batches? And also, would the Nifi keep track how many requests the processor
is processed? If so, how the Nifi keep track this and how long the Nifi keep
track of data? I'd like to add the quota priorities in this processor to keep
track of quota. For example, if the weather feeds can be requested only 100
requests a day, I don't want to processor to executed once the quota is reached.
Thanks
Kumiko
