Hello!
Very interesting. It does seem like this would be easier to learn :)
Little nitpick: I find the name child_spec/1 a little confusing. I would
expect it to be the spec of the children of a supervisor defined by the
module, due to the word "child".
MySup.child_spec([])
#=> spec of child of MySup
MySup.spec([])
#=> spec of MySup
Perhaps I just feel this way because of familiarity with the current
system.
Cheers,
Louis
On Tue, 21 Feb 2017 at 18:16 José Valim <[email protected]>
wrote:
> Note: this proposal is also available in a gist -
> https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6
>
> Streamlining child specs
>
> Hello everyone,
>
> This is a proposal for improving how supervisors are defined and used in
> Elixir. Before we go to the improvements, let's discuss the pain points of
> the current API.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#the-current-state-of-the-art>The
> current state of the art
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#module-based-supervisors>Module-based
> supervisors
>
> Supervisors in Elixir can be defined using modules:
>
> defmodule MySupervisor do
> use Supervisor
>
> def start_link do
> Supervisor.start_link(__MODULE__, [])
> end
>
> def init([]) do
> children = [
> worker(Child, [])
> ]
>
> supervise(children, strategy: :one_for_one)
> endend
>
> Notice we introduced the functions worker and supervise, which are part
> of the Supervisor.Spec module, to take care of defining the tuples used
> behind the scenes.
>
> However, the above is still verbose for the cases we want to simply start
> a supervisor. That's when we introduced callback-less supervisors.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#callback-less-supervisors>Callback-less
> supervisors
>
> The idea behind callback-less supervisors is that you can start them
> directly without needing callbacks:
>
> import Supervisor.Spec
>
> children = [
> worker(Child, [])
> ]
> Supervisor.start_link(children, strategy: :one_for_one)
>
> While the example above is an improvement compared to previous versions,
> especially the early ones that required tuples to be passed around instead
> of using the worker/supervisor functions, it has the following issues:
>
> -
>
> Importing Supervisor.Spec which exists in a separate module from
> Supervisor is confusing in terms of learning and exploring the API and
> documentation
> -
>
> Children inside callback-less supervisors cannot be hot code upgraded,
> since the definition is provided before the supervisor even starts
>
> * The worker(Child, []) API is confusing in regards to which function
> it will invoke and with which arguments. The fact worker(Child, [1, 2,
> 3]) invokes Child.start_link(1, 2, 3) which is then packed into an
> argument given to GenServer.start_link/3 start is not helpful.
>
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#application-integration>Application
> integration
>
> Perhaps the most troubling aspect of supervisors is the amount of code
> necessary to integrate any supervised process into a new application. For
> example, imagine you have created a new application with mix new my_app and
> after a while you decide it needs an agent. In order to correct supervise
> your agent implenentation, you will need to:
>
> 1.
>
> Define a new module that use Application and implements the start/2
> callback
> which starts a supervisor with the agent as a worker
>
> defmodule MyApp do
> use Application
>
> def start(_type, _args) do
> import Supervisor.Spec
>
> children = [
> worker(MyAgent, [], restart: :transient)
> ]
>
> Supervisor.start_link(children, strategy: :one_for_one)
> endend
>
> 2. Make sure to define the `:mod key in the mix.exs:
>
> elixir def application do [mod: {MyApp, []}] end
>
> The amount of code to go from unsupervised to supervised often lead people
> down the wrong road of not adding supervision at all. Another sign the code
> above is a common boilerplate is that it has been automatized under the mix
> new my_app --sup flag. The --sup flag has been helpful but it is useless
> in projects you have already created.
>
> The other problem with the code above is that the agent configuration
> ends-up spread through multiple files. Properties such as :shutdown and
> its type (worker or supervisor) is most times better to be co-located
> with its implementation rather than specified separately in the supervision
> tree. Specifying it at the supervision tree is useful only in cases the
> same implementation is started multiple times under the same or different
> trees, which is not the most common scenario.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#a-streamlined-solution>A
> streamlined solution
>
> In the previous section we have explored the current state of application
> and supervisors in Elixir. In this section, we propose a streamlined
> solution that is going to touch all of the problems outlined above,
> starting with the application one.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#default-application-callbacks>Default
> application callbacks
>
> Ideally, we want to make it as straight-forward as possible to go from
> non-supervised to supervised code. If most applications callbacks end-up
> having the exact same structure, such boilerplate can likely be addressed
> by Elixir.
>
> Therefore, instead of forcing users to define an application callback with
> a boilerplate supervision tree, Elixir could ship with the application
> definition where we only need to tell Elixir which processes to supervise:
>
> def application do
> [extra_applications: [:logger],
> supervise: [MyAgent, {Registry, name: MyRegistry}]]end
>
> The :supervise option allows us to specify a list of module names or
> tuples with two elements, where the first element is the module name and
> second element is any argument. When the application starts, Elixir will
> traverse the list invoking the child_spec/1 function on each module,
> passing the arguments.
>
> In other words, we will push developers to co-locate their supervision
> specification with the module definition.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#the-child_spec1-function>The
> child_spec/1 function
>
> The heart of the proposal is in allowing modules to specify the
> supervision specification for the processes they implement. For example,
> our agent definition may look like this now:
>
> defmodule MyAgent do
> def child_spec(_) do
> %{id: MyAgent, start: {__MODULE__, :start_link, []}, restart: :transient}
> end
> # ...end
>
> With the definition above, we are now able to use MyAgent as a supervised
> process in our application, as shown in the previous section.
>
> If your application requires multiple supervisors, Supervisors can also be
> added to the application's :superviseoption, as long as they also
> implement the child_spec/1 function. Furthermore, notice the Supervisor API,
> such as start_link/2 will also be updated to allow a list of modules, in
> the same format as the application's :supervise:
>
> defmodule MySup do
> def child_spec(_) do
> %{id: MySup, start: {__MODULE__, :start_link, []}, type: :supervisor}
> end
>
> def start_link do
> Supervisor.start_link([OtherAgent], strategy: :simple_one_for_one, name:
> MySup)
> endend
>
> Besides keeping the agent configuration in the same module it is defined,
> the module based child_spec/1 has a big advantage of also being hot code
> swap friendly, because by changing the child_spec/1 function in the
> MyAgentmodule, we can patch how the agent runs in production.
>
> Not only that, by colocating child_spec/1 with the code, we make it
> simpler to start abstractions such as Phoenix.Endpoint or Ecto.Repo under
> a supervision tree, as those abstractions can automatically define the
> child_spec/1 function, no longer forcing developers to know if those
> entries have the type worker or supervisor.
>
> Today developers likely have the following code in their Phoenix
> applications:
>
> children = [
> supervisor(MyApp.Endpoint, []),
> supervisor(MyApp.Repo, [])
> ]
>
> which constraints both Ecto and Phoenix because now they are forced to
> always run a supervisor at the top level in order to maintain backwards
> compatibility. Even worse, if a developer add those entries by hand with
> the wrong type, their applications will be misconfigured.
>
> By storing the child_spec directly in the repository and endpoint,
> developers only need to write:
>
> supervise: [MyApp.Endpoint, MyApp.Repo]
>
> which is cleaner and less error prone.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#catching-up-with-otp>Catching
> up with OTP
>
> In the previous section we have mentioned that Supervisor.start_link/2 will
> also accept a list of modules or a list of tuples with two elements on
> Supervisor.start_link/2. In such cases, the supervisor will invoke
> child_spec/1appropriately on the list elements.
>
> You may have also noticed we have used maps to build the child_spec/1
> functions:
>
> def child_spec(_) do
> %{id: MySup, start: {__MODULE__, :start_link, []}, type: :supervisor}end
>
> The reason we chose a map with the keys above is to mirror the API
> introduced in Erlang/OTP 18. Overall, Supervisor.start_link/2 expect a
> list of children where each child must be:
>
> - an atom - such as MyApp. In such cases, MyApp.child_spec([]) will be
> invoked when the supervisor starts
> - a tuple with two elements - such as {MyApp, arg}. In such cases,
> MyApp.child_spec(arg) will be invoked when the supervisor starts
> - a map - containing the keys :id (required), :start (required),
> :shutdown, :restart, :type or :modules
>
> Introducing maps will allow a more natural and data-centric approach for
> configuring supervisors, no longer needing the helper functions defined in
> Supervised.Spec today.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#a-practical-example>A
> practical example
>
> In order to show how the changes above will affect existing applications,
> we checked how applications generated by Phoenix would leverage them. The
> answer is quite positive. The whole lib/demo.ex file will no longer exist:
>
> defmodule Demo do
> use Application
> # See http://elixir-lang.org/docs/stable/elixir/Application.html # for
> more information on OTP Applications
> def start(_type, _args) do
> import Supervisor.Spec
> # Define workers and child supervisors to be supervised
> children = [ # Start the Ecto repository
> supervisor(Demo.Repo, []), # Start the endpoint when the
> application starts
> supervisor(Demo.Endpoint, []), # Start your own worker by calling:
> Demo.Worker.start_link(arg1, arg2, arg3) # worker(Demo.Worker, [arg1,
> arg2, arg3]),
> ]
> # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html # for
> other strategies and supported options
> opts = [strategy: :one_for_one, name: Demo.Supervisor]
> Supervisor.start_link(children, opts)
> endend
>
> instead it will be replaced by:
>
> def application do
> [extra_applications: [:logger],
> supervise: [Demo.Repo, Demo.Endpoint]]end
>
> which Mix will translate behind the scenes to:
>
> def application do
> [extra_applications: [:logger],
> mod: {Application.Default, %{supervise: [Demo.Repo, Demo.Endpoint]}}]end
>
> Overall we propose the following additions to def application:
>
> - :supervise - expects a list of module names to supervise when the
> application starts (maps as childspecs are not supported as that would make
> them compile time)
> - :max_restarts - the amount of restarts supported by application
> supervision tree in :max_seconds (defaults to 1)
> - :max_seconds - the amount of time for :max_restarts (defaults to 5)
>
> Those options will be handled by the default application module
> implemented in Application.Default. Mix will error building the
> application file if any of the options above are given alongside :mod.
>
> The application tree supervisor will have strategy of :one_for_one and
> such is not configurable. If you need other supervision strategies, then
> all you need to do is define your own supervisor with a custom strategy and
> list it under :supervise in def application.
>
> At the end of the day, the changes above will render both mix new --sup
> and Supervisor.Spec obsolete, as we now co-locate the child_spec/1 with
> the module definition, leading to better code and hot code upgrades, also
> making it easier to move from non-supervised to supervised applications.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#automating-child_spec1>
> Automating child_spec/1
>
> We have explicitly defined the child_spec/1 function in all of the
> examples above. This poses a problem: what if someone defines a Supervisor
> module
> and forget to mark its type as :supervisor in the specification?
>
> Given the Supervisor module already knows sensible defaults, we should
> rely on them for building specs. Therefore, we also propose use Agent, use
> Task, use Supervisor and use GenServer automatically define a
> child_spec/1function
> that is overridable:
>
> defmodule MyAgent do
> use Agent, restart: :transient
> # Generated on: use Agent # def child_spec(opts) do # %{id: MyAgent,
> start: {__MODULE__, :start_link, [opts]}, restart: :transient} # end
>
> def start_link(opts) do
> Agent.start_link(fn -> %{} end, opts)
> end
>
> ...end
>
> The default child_spec/1 will pass the argument given to child_spec/1 directly
> to start_link/1. This means MyAgentcould be started with no name in a
> supervision tree as follows:
>
> supervise: [MyAgent]
>
> and with a given name as:
>
> supervise: [{MyAgent, name: FooBar}]
>
> In other words, the changes in this proposal will push developers towards
> a single start_link/1 function that will receive options as a single
> argument. This mirrors the API exposed in GenServer, Supervisor and
> friends, removing the common confusion with passing more than one argument
> to start_link.
>
> Of course, any of the defaults above, as well as the rare cases developers
> wants to dynamically customize child_spec/1 can be achieved by overriding
> the default implementation of child_spec/1.
>
> <https://gist.github.com/josevalim/82fb1d20c9738bb3cc96449e67407df6#summing-up>Summing
> up
>
> This proposal aims to streamline the move from non-supervised applications
> to supervised applications. To do so, we introduced the ability to colocate
> the child_spec/1 definition with the module implementation, which leads
> to more self-contained abstractions, such as Ecto.Repo and
> Phoenix.Endpoint. Those changes also allowed us to overhaul the
> supervision system, getting rid of Supervisor.Spec and making it more
> data-centric.
>
> While those changes remove some of the importance given to application
> callbacks today, we believe this is a positive change. There is no need to
> talk about application callbacks if the majority of developers are using
> application callbacks simply to start supervision trees. The application
> callback will still remain import for scenarios where custom start,
> config_change or stop are necessary, such as those relying on included
> applications.
>
> Since Elixir is a stable language, all of the code available today will
> continue to work, although this proposal opens up the possibility to
> deprecate Supervisor.Spec and mix new --sup in a far ahead future,
> preparing for an eventual removal in Elixir 2.0.
>
> *Thanks to Dave Thomas for initiating the proposal and the Elixir team for
> further discussions and reviews. Also thanks to Chris McCord and Saša Jurić
> for feedback.*
>
>
> *José Valim*
> www.plataformatec.com.br
> Skype: jv.ptec
> Founder and Director of R&D
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J1mmJQ_QUML5VeSnT81kprkTWBtjekD3hpHMUhVDJBZg%40mail.gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J1mmJQ_QUML5VeSnT81kprkTWBtjekD3hpHMUhVDJBZg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFBTW6GcaCFSJ_z6KYzRN%3Dxt9tdRpxq6woXBcapz4st%2B5w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
