Given supervisors are also processes, they may also become bottlenecks. While this is unlikely to happen to a Supervisor, since it is mostly static, it can happen to a DynamicSupervisor.
We can address this by partitioning the dynamic supervisor. Imagine the following dynamic supervisor: defmodule MyApp.DynamicSupervisor do use DynamicSupervisor def start_link(opts) do DynamicSupervisor.start_link(__MODULE__, arg, opts) end def init(_arg) do DynamicSupervisor.init(strategy: :one_for_one) end end In order to partition it, we can start 8 instances of said supervisor inside a regular Supervisor, and then pick one partition at random when starting a child. For example: defmodule MyApp.Supervisor do use Supervisor @partitions 8 @name __MODULE__ def start_child(module, arg) do i = :erlang.phash2(self(), @partitions) + 1 DynamicSupervisor.start_child(:"#{__MODULE__}#{i}", {module, arg}) end def start_link do Supervisor.start_link(__MODULE__, arg, name: @name) end def init(arg) do children = for i <- 1..@partitions do name = :"#{__MODULE__}#{i}" Supervisor.child_spec({MyApp.DynamicSupervisor, name: name}, id: name) end Supervisor.init(children, strategy: :one_for_one) end end I would like to make the above more convenient by introducing a :partitions option to DynamicSupervisor.start_link. When given, the new option will automatically start N dynamic supervisors under a supervisor, like above: DynamicSupervisor.start_link(__MODULE__, :ok, partitions: 8, name: @name) For now, the :name option will be required. Now, when spawning child processes, you will use via tuples: DynamicSupervisor.start_child({:via, DynamicSupervisor, {@name, self()}}, {module, arg}) The via tuple has the format {:via, DynamicSupervisor, {supervisor_name, value_to_partition_on}}. Once invoked, it will take care of partitioning based on the current process and dispatching it. Overall, encapsulating the partitioning of DynamicSupervisor (and consequently of Task.Supervisor) into an easy to use API can help to vertically scale up applications that use those constructs. ## Open questions One of the confusing aspects of the above is that the :name option no longer reflects the name of the DynamicSupervisor but of the parent Supervisor. One alternative is to *not* accept the :name option when :partitions is given, instead we could have a :root_name option instead (or something more appropriately named). Implementation wise, we will store the available processes in ETS or using a Registry (to be started alongside the Elixir application). Feedback? -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JsSj1_qN9DMfHjn9ovta_hYoziRE%3D-E%2B%2BFjQcAm%3DqhZQ%40mail.gmail.com.