Hi! I've added a diagram[1] detailing the at-scale properties of PYFI for those who are interested. In a nutshell, PYFI scales at the processor level (vs. the flow), allowing you to have say, 50 processors of the same type running on 50 CPUs, whereas another (less intensive) processor in your flow, might only require 10 CPUs.
At-Scale means there is a 1-1 correspondence between logical and physical compute units. [1] https://github.com/radiantone/pyfi#at-scale-design If a data ingress or queue is particularly demanding, then scale only that processor. Furthermore, there are auto-scaling elastic qualities as well where PYFI will scale a processor based on temporary demand. This design is inherently load-balanced at the processor level and has the following additional qualities: * hardware redundancy * high-availability * fault-tolerance * fail-over * performance * ease of maintenance [cid:0375d7ee-31cf-439f-9273-2312d2325801] Cheers! Darren
