Thank you Ben. Those are very good points and questions which I am not sure
I can get satisfactory answers for. I will investigate more and follow up.

On Thu, Sep 2, 2021 at 5:48 PM Ben Wilson <benwilson...@gmail.com> wrote:

> Implemented this way, options like max_restarts max_children and so on
> would occur per partition. I take it the plan would be to simply note that
> in the docs? I don't see any easy way to enforce those values across all
> partitions, which I think is just fine.
>
> which_children/1 and so on won't work as expected either, so maybe we want
> a helper function inside of DynamicSupervisor that flat_maps that over the
> child supervisors?
>
> Overall though, +1 from me.
>
> On Thursday, September 2, 2021 at 11:28:19 AM UTC-4 José Valim wrote:
>
>> Given supervisors are also processes, they may also become bottlenecks.
>> While this is unlikely to happen to a Supervisor, since it is mostly
>> static, it can happen to a DynamicSupervisor.
>>
>> We can address this by partitioning the dynamic supervisor. Imagine the
>> following dynamic supervisor:
>>
>> defmodule MyApp.DynamicSupervisor do
>>   use DynamicSupervisor
>>
>>   def start_link(opts) do
>>     DynamicSupervisor.start_link(__MODULE__, arg, opts)
>>   end
>>
>>   def init(_arg) do
>>     DynamicSupervisor.init(strategy: :one_for_one)
>>   end
>> end
>>
>> In order to partition it, we can start 8 instances of said supervisor
>> inside a regular Supervisor, and then pick one partition at random when
>> starting a child. For example:
>>
>> defmodule MyApp.Supervisor do
>>   use Supervisor
>>
>>   @partitions 8
>>   @name __MODULE__
>>
>>   def start_child(module, arg) do
>>     i = :erlang.phash2(self(), @partitions) + 1
>>     DynamicSupervisor.start_child(:"#{__MODULE__}#{i}", {module, arg})
>>   end
>>
>>   def start_link do
>>     Supervisor.start_link(__MODULE__, arg, name: @name)
>>   end
>>
>>   def init(arg) do
>>     children =
>>       for i <- 1..@partitions do
>>         name = :"#{__MODULE__}#{i}"
>>         Supervisor.child_spec({MyApp.DynamicSupervisor, name: name}, id:
>> name)
>>       end
>>
>>     Supervisor.init(children, strategy: :one_for_one)
>>   end
>> end
>>
>> I would like to make the above more convenient by introducing a
>> :partitions option to DynamicSupervisor.start_link. When given, the new
>> option will automatically start N dynamic supervisors under a supervisor,
>> like above:
>>
>> DynamicSupervisor.start_link(__MODULE__, :ok, partitions: 8, name: @name)
>>
>> For now, the :name option will be required.
>>
>> Now, when spawning child processes, you will use via tuples:
>>
>> DynamicSupervisor.start_child({:via, DynamicSupervisor, {@name, self()}},
>> {module, arg})
>>
>> The via tuple has the format {:via, DynamicSupervisor, {supervisor_name,
>> value_to_partition_on}}. Once invoked, it will take care of partitioning
>> based on the current process and dispatching it.
>>
>> Overall, encapsulating the partitioning of DynamicSupervisor (and
>> consequently of Task.Supervisor) into an easy to use API can help to
>> vertically scale up applications that use those constructs.
>>
>> ## Open questions
>>
>> One of the confusing aspects of the above is that the :name option no
>> longer reflects the name of the DynamicSupervisor but of the parent
>> Supervisor. One alternative is to *not* accept the :name option when
>> :partitions is given, instead we could have a :root_name option instead (or
>> something more appropriately named).
>>
>> Implementation wise, we will store the available processes in ETS or
>> using a Registry (to be started alongside the Elixir application).
>>
>> Feedback?
>>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/bae8be24-a2e5-4a07-89db-7758ccfaf0d4n%40googlegroups.com
> <https://groups.google.com/d/msgid/elixir-lang-core/bae8be24-a2e5-4a07-89db-7758ccfaf0d4n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KehKW1x9jRfF98_M4FjpxZay8Rq_efNy5UWUUj-tmvSg%40mail.gmail.com.

Reply via email to