Hi Stefan, That's a possibility, yes. If we make config/runtime.exs the main entry point, then we will want to move away from config/config.exs, and renaming it to config/compile.exs would be a good idea. But right now it is too early to commit to such a change. We need to validate config/runtime.exs first.
On Thu, Mar 5, 2020 at 3:11 PM Stefan Chrobot <[email protected]> wrote: > What do you think about renaming "config.exs" and friends to > something like "compiletime" (or "compile", "static" "build", "mix")? I see > the proposal to add "runtime.exs" as a way to embrace the fact that there > are two separate concepts: build configuration and runtime configuration, > so why not make it even more explicit? It would make things like reading > ENV during build time more obvious (i.e. System.get_env in build.exs). > > > Best, > > Stefan > > śr., 4 mar 2020 o 20:17 José Valim <[email protected]> napisał(a): > >> Hi Paul! >> >> I believe we are mostly in tune then! >> >> > My take is that the config should be evaluated after compilation, after >> applications are loaded, but before any applications are started, as this >> can be handled properly both when run under Mix and when run in releases by >> having the boot script invoke the configuration entry point. >> >> This is a good point. But if we move in this direction, then we will >> definitely need to introduce a "mix app.load" task, which means existing >> projects won't leverage runtime config unless they invoke "mix app.load" >> instead. Although I like the dichotomy here: users have to call either >> "app.load" or "app.start" in their Mix tasks. This also helps avoid subtle >> differences in behaviour, so I really like it. >> >> >> On Wed, Mar 4, 2020 at 7:45 PM Paul Schoenfelder < >> [email protected]> wrote: >> >>> This was always the hardest/most annoying part about the release >>> tooling, and I think this is probably an improvement, but it does seem to >>> me to trade one set of annoyances for another. That said, I think I'd >>> prefer the new set. >>> >>> My take is that the config should be evaluated after compilation, after >>> applications are loaded, but before any applications are started, as this >>> can be handled properly both when run under Mix and when run in releases by >>> having the boot script invoke the configuration entry point. >>> >>> For those cases where one needs an application started in order to >>> configure the system (which I would assert indicates a different problem), >>> then one can use the `Application` APIs to temporarily start the >>> application, perform that config, and then stop the Application so that the >>> rest of startup can proceed normally (this is more important for releases >>> than under Mix, but I think its important to treat them the same). >>> >>> It's also my experience that most of the time you only need the >>> applications _loaded_, not started, since the APIs you want to call during >>> the configuration stage typically don't require the system to be running >>> (and wouldn't ever, if I had my way). That isn't universally true today of >>> course, but a proper solution to that particular problem is going to >>> require some concrete examples that aren't themselves resolvable by other, >>> more appropriate, means. >>> >>> Anyway, that's my two cents :) >>> >>> Paul >>> >>> On Wed, Mar 4, 2020, at 5:20 PM, José Valim wrote: >>> >>> Over the last releases we have been making improvements to the >>> configuration system. However, there is still one large gap to be >>> addressed. Today config files are treated as compile-time configuration by >>> default. This behaviour, alongside overuse from libraries, made using the >>> application environment via config files more rigid and error prone than >>> they have to be. >>> >>> Today, Mix will load "config/config.exs" before any of the dependencies >>> are compiled and on every command. This makes basic scenarios like the ones >>> below impossible: >>> >>> - >>> >>> If your project requires some environment variables in development, >>> you cannot enforce them. Otherwise, basic commands such as mix help >>> or editor integration will stop working >>> >>> >>> - >>> >>> If you need to configure one application based on another >>> application, that is impossible to do as well, as no applications are >>> available at this point >>> >>> Also, because configuration files run during compilation by default, >>> library authors often accidentally rely on compile-time configuration. >>> Luckily, Elixir v1.10 has added Application.compile_env/2. In the >>> future we will deprecate Application.get_env/3 in the module body, >>> which will help steer people to the correct direction in their own >>> applications. >>> >>> On the opposite side, we have releases, where the configuration works at >>> runtime only. Given there is no way to execute configuration at runtime in >>> Mix, we end-up with two completely disjoint approaches. This introduces a >>> couple issues of their own: >>> >>> - >>> >>> Since "config/releases.exs" run only during releases, if you have a >>> syntax error (or similar), you will find about it just way too late >>> >>> >>> - >>> >>> There is no way for frameworks like Phoenix to provide a >>> configuration file that works for both Mix and release deployments >>> >>> Our goal is to address these issues. However, it is important to >>> consider that a complex project today may already have many configuration >>> files: >>> >>> - config/config.exs (compile time) - shared configuration settings >>> - config/{dev,test,prod}.exs (compile time) - per env configuration >>> settings >>> - config/releases.exs (runtime) - release specific configuration >>> settings >>> >>> Therefore, we would like to propose a new configuration file that can >>> address the problem above while replacing the need to use >>> "config/releases.exs" in 99% of the cases. >>> Proposal: introduce config/runtime.exs >>> >>> Our proposal is simple: we will introduce "config/runtime.exs". >>> "config/runtime.exs" will be loaded both by Mix and Releases, closing the >>> gap between them. >>> >>> For Mix, "config/runtime.exs" will load after the code is compiled and >>> before the application starts, this allows "config/runtime.exs" to rely on >>> code from dependencies, as long as you keep in mind that any application >>> that is started during "config/runtime.exs" cannot be configured by >>> "config/runtime.exs" itself. Furthermore, given "config/runtime.exs" works >>> at runtime, changing it won't require the whole application to be >>> recompiled. >>> >>> For Releases, it will work precisely the same as "config/releases.exs". >>> If both are available, "config/runtime.exs" is executed first, followed by >>> "config/releases.exs". >>> >>> There are a couple pitfalls to be aware though: >>> >>> - >>> >>> Since "config/runtime.exs" is used by both Mix and releases, it >>> cannot configure :kernel, :stdlib, :elixir, and :mix themselves. >>> Attempting to configure those will emit an error. For those rare >>> scenarios, >>> you will need to use "config/releases.exs" - but "config/releases.exs" >>> will >>> remain simple, which will reduce the odds of syntax errors. >>> >>> >>> - >>> >>> Since "config/runtime.exs" is used by both Mix and releases, it >>> cannot invoke "Mix" directly. Therefore, for conditional environment >>> compilation, we will add a env/2 macro to Config that will be >>> available for all config files. For example, instead of a >>> "config/runtime.prod.exs", one will have to: >>> >>> import Config >>> >>> env :prod do >>> config :my_app, :secret_key, System.get_env!("SECRET_KEY")end >>> >>> >>> One may argue that "config/runtime.exs" should eventually replace >>> "config/config.exs" as the default file for application configuration. We >>> will certainly evaluate this option in the future but it is important to >>> take baby steps. And the first step is to support "config/runtime.exs". :) >>> Implementation considerations >>> >>> This section covers implementation details. It is not part of the >>> proposal per se. Although the feature is relatively small, it requires many >>> improvements to Mix and the underlying config engine. The tasks are: >>> >>> - Load config/runtime.exs inside Mix >>> - Copy config/runtime.exs inside escripts and load them when the >>> escript runs >>> - Copy config/runtime.exs inside releases (similar to >>> config/releases.exs) >>> - Add a feature to Config.Reader that allows a warning to be emitted >>> if an undesired module is used (for example, Mix) >>> - Add the env/2 macro to Config >>> - Raise if "import_config" is used in "config/runtime.exs" and >>> "config/releases.exs" - providing proper guidance to users >>> >>> One aspect to consider is exactly when runtime config should be loaded >>> inside Mix. We need to choose between doing it after the "compile" task or >>> before "app.start". The issue is that many projects have tasks that only >>> need the application to be compiled but not started. For example, Ecto Repo >>> management tasks or Phoenix routes tasks. Those tasks today simply run >>> Mix.Task.run("compile"). However, if we were to introduce >>> "config/runtime.exs" and load it before "app.start", those tasks will now >>> run without "config/runtime.exs" and behave incorrectly. >>> >>> Therefore there is an argument to be made to load the runtime >>> configuration right after the code is compiled - even though this is a bit >>> unintuitive. The other option is to ask users to always run "app.start" as >>> the entry point and pass the "--no-start" if they actually don't want to >>> start their apps, which is also a bit counter intuitive. Unfortunately, the >>> second option means projects will behave incorrectly until they are updated. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BS7nefgLCXCb9OWQGj9d9eE1TrDstq%3Dkm%3D%2B2a7iT9DAw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BS7nefgLCXCb9OWQGj9d9eE1TrDstq%3Dkm%3D%2B2a7iT9DAw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/d92115bd-a9fc-4236-b041-a80aa5fc5610%40www.fastmail.com >>> <https://groups.google.com/d/msgid/elixir-lang-core/d92115bd-a9fc-4236-b041-a80aa5fc5610%40www.fastmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JR65fkD_uyOKup1NqcNcJGywt8xe6ZDC-tdXtnvV_M4w%40mail.gmail.com >> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JR65fkD_uyOKup1NqcNcJGywt8xe6ZDC-tdXtnvV_M4w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CACzMe7aMLEKTMXaGcENBjSFuei69V9g-VOqNNuwAqmWyo_cq2w%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CACzMe7aMLEKTMXaGcENBjSFuei69V9g-VOqNNuwAqmWyo_cq2w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2Brhusgd%3D%3Dt6DM8bC9fZ%2B7ZKfzY-PtWR-k5%3DysZoX%3D5TA%40mail.gmail.com.
