One of the major differences between running your application as a release
and as a Mix project is the differences in configuration. Mix evaluates the
configuration right before the application starts, releases evaluates the
configuration when your application is compiled.
This implies in a large mismatch of how those two environments are used.
For releases, environment variables (read by `System.get_env/1`) need to be
set when the application is compiled and such information may not be
available at this point.
Ideally, we would want a release to evaluate the configurations files in
`config` when the release starts. One approach would be to copy the
configuration files as is to the release but that's hard to achieve in
practice for two reasons:
1. A config file may import other config files and often importing those
files happen dynamically. For example: `import_config "#{Mix.env()}.exs"`.
The dynamic import makes it hard for release tools to know which
configuration files must be copied to a release, especially in cases like
umbrella projects, where a developer may load configuration across projects
2. Even we copy today's configuration files to a release, those
configuration files rely on `Mix`, which is a build tool and therefore it
is not available during releases
To solve those issues, we need to make sure we can discover all imports of
a configuration file without evaluating its contents. We also need to
introduce a new module for configuration that does not depend on Mix.
This is the goal of this proposal.
*## Application.Config*
This proposal is about introducing a module named `Application.Config`. It
will work similarly to the existing `Mix.Config`, except it belongs to the
`:elixir` application instead of `:mix`. This allows releases to leverage
configuration without depending on Mix.
The user API of `Application.Config` is quite similar to `Mix.Config`.
There is `config/2` and `config/3` to define configurations. There still is
`import_config/1` to import new configuration files with one important
difference: the argument to `import_config/1` must be a literal string. So
interpolation, variables or any other dynamic pattern is no longer allowed.
In order to help with configuration management, we will introduce a project
option in your `mix.exs`, named `:config_paths` to help manage multiple
required and optional configuration files.
In the next section we will provide an example of how configuration files
used by projects like Nerves and Phoenix will have to be rewritten and then
we will discuss how integration with release tools such as distillery will
work.
*### A common example*
Projects like Nerves and Phoenix generate files with built-in
multi-environment configuration. Today, this configuration has an entry
point `config/config.exs` file that imports an environment specific
configuration at the bottom:
# config/config.exs
use Mix.Config
config :my_app, :some_shared_configuration, ...
import_config "#{Mix.env()}.exs"
And then each `config/{dev,test,prod}.exs` provides environment specific
configuration. For instance:
# config/dev.exs
use Mix.Config
config :my_app, :some_dev_configuration, ...
The issue in the example above is the use of dynamic imports, such as
`import_config "#{Mix.env()}.exs"`. We will address this by defining both
`config/config.exs` and `config/#{Mix.env()}.exs` as configuration entry
points in your `mix.exs`:
# mix.exs
def project do
[
...,
config_paths: ~w(config/config.exs config/#{Mix.env()}.exs),
...
]
end
And now we can define those configuration files without dynamic imports:
# config/config.exs
use Application.Config
config :my_app, :some_shared_configuration, ...
# config/dev.exs
use Application.Config
config :my_app, :some_dev_configuration, ...
In Phoenix, the `config/prod.exs` case may link to a separate
`prod.secret.exs` file. While we could also refer to this file in the
`:config_paths` configuration in the `mix.exs` file, because it is only
specific to production, it is more straight-forward to continue importing
it at the bottom. So a `config/prod.exs` would look like this:
# config/prod.exs
use Application.Config
config :my_app, :some_prod_configuration, ...
import_config "prod.secret.exs"
By adding `:config_paths`, we are able to move the dynamic configuration to
the `mix.exs` file and make the order that configuration files are loaded
clearer.
*### A FarmBot example*
Nerves projects tend to rely extensively on configuration files. So let's
look into existing open source Nerves projects and see how this proposal
will fare. Let's take a look at [FarmBot v6.4.1](
https://github.com/FarmBot/farmbot_os/tree/v6.4.1).
The questions we want to answer are: if we move the FarmBot project to the
proposed `Application.Config`, will they be able to express of all the
existing idioms they do today? And, even further, will their configuration
files become simpler or more complex?
>From looking at its [config/config.exs](
https://github.com/FarmBot/farmbot_os/blob/v6.4.1/config/config.exs), we
can already see a pattern that won't work in releases: [the use of
`Mix.env` and `Mix.Project.config`](
https://github.com/FarmBot/farmbot_os/blob/v6.4.1/config/config.exs#L3-L5).
We can see [those variables are used to dynamically import configuration](
https://github.com/FarmBot/farmbot_os/blob/v6.4.1/config/config.exs#L69-L77),
which `Application.Config` won't allow.
Those idioms are perfectly fine with how configurations work in Mix today.
But they will no longer with a release built on top of `Application.Config`.
The solution is to move all of those imports to the `:config_paths` option
in `mix.exs`. However, note that some of those dynamic imports are
optional, so we will also need the ability to explicitly tag them as such:
# farmbot/mix.exs
def project do
[
...,
config_path: ~w(config/config.exs config/#{Mix.env()}.exs) ++
optional_config_paths(@target, Mix.env())
...
]
end
defp optional_config_paths("host", env),
do: [{:optional, "config/host/#{env}.exs"}]
defp optional_config_paths(target, env),
do: [{:optional, "config/target/#{env}.exs"}, {:optional,
"config/target/#{target}.exs"}]
We believe this approach is an improvement to the previous one because it
allows all environment and target specific handling to remain in the
`mix.exs` file and not scattered around multiple configuration files.
*## Using it in releases*
In the previous sections, we have outlined `Application.Config` which no
longer depends on Mix and has a restricted `import_config`.
Now that we are able to see all of the configuration files that affect our
system, a release tool, such as discovery, should be able to traverse all
of those configuration files and merge them into a final
`config/release.exs` that will be part of your release. In fact, Elixir
will provide a convenient API that performs such operation, streamlining
the release assembling process.
*## Unresolved topics*
There are two important topics that we have not included in this proposal
and they will be discussed in a further step.
1. What about umbrella projects? Umbrella projects also rely on
configuration and we need to make sure the listed mechanisms also work well
with umbrellas.
2. How to avoid common pitfalls? Even though we will migrate to
`Application.Config`, there is nothing stopping a developer from accessing
Mix (and the module defined in the `mix.exs` file) from their new config
files. As we have seen, this may lead to errors when running releases, as
releases do not have Mix available. To address this, we may introduce
checks when assembling releases that make sure `Mix` is not invoked in
configuration files, raising appropriate error messages in case they do.
*## Summing up*
We propose a new `Application.Config` module and a new `:config_paths`
project option that allows release tools to discover all of the relevant
configurations in a system. Release tools can then merge and copy those
configuration into releases and execute them as part of the release
process, allowing dynamic calls such as `System.get_env/1` to work in
development and in production transparently, with or without releases.
*José Valimwww.plataformatec.com.br
<http://www.plataformatec.com.br/>Founder and Director of R&D*
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4L%2BAsNewcj%2BPF0fogpgDQEAc6WDMDy7X-W5evx19WPkrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
