What I am saying is that you need to rethink the approach altogether. :) As you said, the issue is with compile-time dependencies. Rewriting compile-time dependencies to export dependencies and changing signatures for recompilation is a brittle work-around the compiler.
With @after_verify, you should be able to treat all of them as runtime dependencies, and let the @after_verify callback be invoked whenever you are supposed to verify them again. :) This means you no longer need to track compile, exports, etc. The compiler is doing all the job for you! On Mon, Sep 19, 2022 at 11:18 PM Marlus Saraiva <marlus.sara...@gmail.com> wrote: > Ok, but as far as I remember, @after_verify can be used for validations > but not for triggering recompilation of caller modules. Does the compiler > validate the result of that callback? I tried to return different values > than :ok but it does not seem to make any difference. > > On Friday, September 16, 2022 at 10:43:35 AM UTC-3 José Valim wrote: > >> Yes, the functionality you ask is the @after_verify callback in Elixir >> v1.14. It is used by LiveView to provide functionality similar to Surface >> and also the new Phoenix.VerifiedRoutes. :) >> >> On Fri, Sep 16, 2022 at 14:43 Marlus Saraiva <marlus....@gmail.com> >> wrote: >> >>> Hi Zach! >>> >>> The only way you can minimize the transitive dependency problem is by >>> removing the compile-time deps. I struggled with this for years with >>> Surface, and as you did, I tried the approach of removing the runtime deps >>> to avoid them from turning into transitive compile-time deps. Eventually, I >>> realized that avoiding compile-time deps would be the only way to have a >>> permanent solution and started removing any implementation that required >>> information at compile-time. As soon as LV v0.8 is out, we'll wrap up that >>> work, so we don't expect further issues after that. In the meantime, to >>> solve the problem, I added a fix that drastically cut down those >>> dependencies, which was to convert the `compile` deps into `export` deps, >>> by using `import` instead of `require`. For this to work properly, I have >>> to generate and automatically rename a signature function to force the >>> recompilation of direct callers whenever the component's metadata (props, >>> slots, etc) changes. Otherwise, it would only recompile them if you added >>> or removed another function. >>> >>> It seems to me that Elixir could provide a way to hook into the compiler >>> allowing DSL developers to inform a criteria that could be used to trigger >>> callers' recompilation. Similar to `__mix_recompile__?`. Maybe something >>> like `__export_changed__?` or anything else. With this approach, DSL >>> authors could replace the `compile` deps with `export` deps as long as the >>> changes in that dependency don't need to be propagated to other modules >>> other than the callers. This is the case for Surface and many other libs, >>> like Commanded, for instance. I'm not sure the proposed solution would >>> solve your issue with Ash but I thought it was worth bringing it here since >>> it may solve issues on other libs. >>> >>> José, is something like that feasible? >>> >>> >>> >>> On Tuesday, September 13, 2022 at 8:37:24 PM UTC-3 zachary....@gmail.com >>> wrote: >>> >>>> Well, its runtime dependencies that create transitive compile time >>>> dependencies, isn't it? So if I have a function like >>>> >>>> defmodule Foo do >>>> # callers are aware that they can't use the module at compile time >>>> def returns_a_module() do >>>> SomeSpecificModule >>>> end >>>> >>>> def something_used_at_compile_time() do >>>> 10 >>>> end >>>> end >>>> >>>> # and then some other module >>>> >>>> defmodule Bar do >>>> @something Foo.something_used_at_compile_time() >>>> end >>>> >>>> This induces a transitive compile time dependency from Bar to >>>> SomeSpecificModule. >>>> >>>> In the case of Ash DSLs, for example, this happens because we store the >>>> resulting configuration in a module attribute. Users are aware that you >>>> can't call these pluggable modules at compile time. Only the framework code >>>> calls those modules. The above example is contrived, of course, I'm not >>>> suggesting that we need a feature to make *that* work, just trying to draw >>>> some kind of parallel. >>>> >>>> I don't see a reasonable way to handle this without essentially >>>> removing the ability to use modules in the way that we are. It would be a >>>> pretty unreasonable amount of work/change for users to change the way that >>>> we plug behavior in Ash. >>>> >>>> To me, another kind of module dependency, like `runtime-only`, would >>>> solve for this, and it would have to be explicitly requested, i.e >>>> `expand_literal(.., runtime_only: true)`. Then when determining transitive >>>> compile time dependencies, the compiler would not use those modules. >>>> >>>> >>>> On Tue, Sep 13, 2022 at 6:51 PM, José Valim <jose....@dashbit.co> >>>> wrote: >>>> >>>>> As I mentioned, the issue on transitive compile-time dependencies are >>>>> the compile-time deps, not the runtime ones. So I would focus on how to >>>>> eliminate those. Otherwise I am not sure we will agree on the problem >>>>> statement. :) >>>>> >>>>> On Wed, Sep 14, 2022 at 12:49 AM Zach Daniel <zachary....@gmail.com> >>>>> wrote: >>>>> >>>> Although the solution I originally proposed may not be correct (totally >>>>>> fine with me [image: 😃]), I think the problem statement is still >>>>>> valid. Can we agree that there are some cases where you may want to >>>>>> reference a module without creating transitive compile time dependencies, >>>>>> i.e in the case of a DSL like what Ash provides? >>>>>> >>>>>> >>>>>> On Sun, Sep 11, 2022 at 3:08 PM, Zach Daniel <zachary....@gmail.com> >>>>>> wrote: >>>>>> >>>>> Hm...yeah, that makes sense. Are there other things a runtime >>>>>>> dependency is meant to do aside from ensure that transitive compile time >>>>>>> dependencies are properly tracked? If so, is there a way to solve for >>>>>>> the >>>>>>> transitive dependencies without removing the runtime dependency? Because >>>>>>> ultimately the idea here is that this module is essentially just a big >>>>>>> validated/extensible configuration. The resource itself doesn't do >>>>>>> anything. So requiring things that refer to a resource to recompile when >>>>>>> any of the modules it refers to in a `change` like that is unnecessary. >>>>>>> However we can express that reality is totally fine with me. >>>>>>> >>>>>>> >>>>>>> On Sunday, September 11, 2022 at 3:05:18 PM UTC-4 José Valim wrote: >>>>>>> >>>>>>>> The issue is in the transitive compile time dependencies, not the >>>>>>>> runtime dependencies. >>>>>>>> >>>>>>>> I don't think we should be encouraging removing the runtime >>>>>>>> dependencies when they are explicitly listed in the code as above. When >>>>>>>> done via configuration, at least you are not literally listing it. >>>>>>>> >>>>>>>> On Sun, Sep 11, 2022 at 8:59 PM Zach Daniel <zachary....@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> So all we we do is hold onto the module, and then at runtime we go >>>>>>>>> through the list of modules that we should call and call a specific >>>>>>>>> function on them. Requiring a runtime dependency for that is causing >>>>>>>>> really >>>>>>>>> slow compile times because of transitive dependencies. Maybe there is >>>>>>>>> some >>>>>>>>> consequence I don't see, but I removed the runtime dependencies by >>>>>>>>> disabling the lexical tracker when expanding the alias, and its been >>>>>>>>> that >>>>>>>>> way for months w/o anyone reporting any issues with that >>>>>>>>> implementation. >>>>>>>>> Aside from having to use `warn: false` if they use aliases. >>>>>>>>> >>>>>>>>> To me, its the same as if they gave us, instead of a module, an >>>>>>>>> `atom` that referred to application configuration, i.e the adapter >>>>>>>>> pattern. >>>>>>>>> That would work without a runtime dependency, so why couldn't this? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Sep 11, 2022 at 2:56 PM, José Valim <jose....@dashbit.co> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>> Sorry, correction: If, at any moment, you call any function from >>>>>>>>>> that module at runtime, you must not remove the *runtime* time >>>>>>>>>> dependency. >>>>>>>>>> >>>>>>>>>> On Sun, Sep 11, 2022 at 8:55 PM José Valim <jose....@dashbit.co> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>> Why do you want to remove the runtime dependency when, per your >>>>>>>>>>> description: >>>>>>>>>>> >>>>>>>>>>> > In Ash Framework, we have declarative ways to construct >>>>>>>>>>> *runtime* behavior using behaviors. >>>>>>>>>>> >>>>>>>>>>> Emphasis mine. If, at any moment, you call any function from >>>>>>>>>>> that module at runtime, you must not remove the compile time >>>>>>>>>>> dependency. >>>>>>>>>>> >>>>>>>>>>> On Sun, Sep 11, 2022 at 8:52 PM Zach Daniel < >>>>>>>>>>> zachary....@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>> `expand_literal` removes the compile time dependency, but leaves >>>>>>>>>>>> a runtime dependency when used inside of a module. >>>>>>>>>>>> >>>>>>>>>>>> What I'm trying to do is remove both the compile time >>>>>>>>>>>> dependency *and* the runtime dependency, without requiring the use >>>>>>>>>>>> of >>>>>>>>>>>> `warn: false` on aliases. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sunday, September 11, 2022 at 2:31:42 PM UTC-4 José Valim >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Sorry, I don't understand the proposal. You mentioned >>>>>>>>>>>>> expand_literal, which already removes the compile-time dependency >>>>>>>>>>>>> but keeps >>>>>>>>>>>>> the remaining functionality such as warnings. Can you please >>>>>>>>>>>>> expand? >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Sep 11, 2022 at 7:57 PM Zach Daniel < >>>>>>>>>>>>> zachary....@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> For clarity, the dependency I'm talking about there is the >>>>>>>>>>>>>> dependency from `MyApp.User` to >>>>>>>>>>>>>> `MyApp.User.Changes.HashPassword`. >>>>>>>>>>>>>> On Sunday, September 11, 2022 at 1:55:51 PM UTC-4 Zach Daniel >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> In Ash Framework, we have declarative ways to construct >>>>>>>>>>>>>>> runtime behavior using behaviors. So an Ash resource might look >>>>>>>>>>>>>>> like this: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ```elixir >>>>>>>>>>>>>>> defmodule MyApp.User do >>>>>>>>>>>>>>> use Ash.Resource >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> alias MyApp.User.Changes.HashPassword >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> attributes do >>>>>>>>>>>>>>> uuid_primary_key :id >>>>>>>>>>>>>>> .... >>>>>>>>>>>>>>> end >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> actions do >>>>>>>>>>>>>>> create :register do >>>>>>>>>>>>>>> change HashPassword >>>>>>>>>>>>>>> end >>>>>>>>>>>>>>> end >>>>>>>>>>>>>>> end >>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> However, by default, this would incur a compile time >>>>>>>>>>>>>>> dependency. This compile time dependency is unnecessary, as we >>>>>>>>>>>>>>> won't call >>>>>>>>>>>>>>> any functions on this module or use it in any way until runtime. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> That optimization is well and good, but due to transitive >>>>>>>>>>>>>>> compile time dependencies, we see some interesting behavior. >>>>>>>>>>>>>>> Something >>>>>>>>>>>>>>> you'd often see in a change module is things like pattern >>>>>>>>>>>>>>> matching on other >>>>>>>>>>>>>>> resources, or the resource in question in function heads. >>>>>>>>>>>>>>> Resources are >>>>>>>>>>>>>>> meant to be introspectable at compile time, and so this runtime >>>>>>>>>>>>>>> dependency >>>>>>>>>>>>>>> on a change, with a compile time dependency on a resource, >>>>>>>>>>>>>>> incurs a >>>>>>>>>>>>>>> transitive compile time dependency. This problem multiplies >>>>>>>>>>>>>>> over time, and >>>>>>>>>>>>>>> causes users to have to do things solely to optimize compile >>>>>>>>>>>>>>> times, like >>>>>>>>>>>>>>> only use map pattern matches instead of struct pattern matches. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So what we do is we actually disable the lexical tracker >>>>>>>>>>>>>>> when accepting certain parts of the DSL. This prevents *any* >>>>>>>>>>>>>>> dependency. >>>>>>>>>>>>>>> Naturally, at compile time you are no longer safe to call a >>>>>>>>>>>>>>> resource's >>>>>>>>>>>>>>> change module as changes in that module won't cause recompiles, >>>>>>>>>>>>>>> but that >>>>>>>>>>>>>>> was never a thing you should have done in the first place so >>>>>>>>>>>>>>> I'm not >>>>>>>>>>>>>>> worried about that. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This leads us to the primary issue: disabling the lexical >>>>>>>>>>>>>>> tracker when expanding aliases also causes warnings about >>>>>>>>>>>>>>> unused aliases, >>>>>>>>>>>>>>> even though they *are* used. I believe I've brought this issue >>>>>>>>>>>>>>> up before >>>>>>>>>>>>>>> and we were hoping that the feature introduced in 1.14 for >>>>>>>>>>>>>>> `defimpl` would >>>>>>>>>>>>>>> help, but that only helps prevent compile time issues, which is >>>>>>>>>>>>>>> something I >>>>>>>>>>>>>>> had already solved for in the same way it was solved for 1.14. >>>>>>>>>>>>>>> I've laid it >>>>>>>>>>>>>>> all out to help clarify exactly *why* I need it so perhaps >>>>>>>>>>>>>>> someone can >>>>>>>>>>>>>>> point me in a better direction. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The simplest thing that could help: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> A way to tell the lexical tracker that an alias has just >>>>>>>>>>>>>>> been referenced without inducing any kind of compile or runtime >>>>>>>>>>>>>>> dependency. >>>>>>>>>>>>>>> The idea is to just prevent it from warning about the alias. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm open to other solutions as well. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>> Google Groups "elixir-lang-core" group. >>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>>> it, send an email to elixir-lang-co...@googlegroups.com. >>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/54627973-7b74-47d7-9e35-4270621e6c91n%40googlegroups.com >>>>>>>>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/54627973-7b74-47d7-9e35-4270621e6c91n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>> . >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "elixir-lang-core" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>> it, send an email to elixir-lang-co...@googlegroups.com. >>>>>>>>>>>> >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/22887a7e-a1b6-4ccc-98bf-69a5ad3551a5n%40googlegroups.com >>>>>>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/22887a7e-a1b6-4ccc-98bf-69a5ad3551a5n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "elixir-lang-core" group. >>>>>>>>>> >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to elixir-lang-co...@googlegroups.com. >>>>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BVkL%2B1V7LTDVyzhCqcNWrmHFPoWx2Fp916Ur%2ByL%2BiVBA%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BVkL%2B1V7LTDVyzhCqcNWrmHFPoWx2Fp916Ur%2ByL%2BiVBA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "elixir-lang-core" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to elixir-lang-co...@googlegroups.com. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/l7xp6eut.444ff913-f528-4b88-805a-cac120cdb4d6%40we.are.superhuman.com >>>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/l7xp6eut.444ff913-f528-4b88-805a-cac120cdb4d6%40we.are.superhuman.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "elixir-lang-core" group. >>>>>>> >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to elixir-lang-co...@googlegroups.com. >>>>>>> >>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/072bb99d-09a0-4567-a934-f8893015dd91n%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/072bb99d-09a0-4567-a934-f8893015dd91n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elixir-lang-core" group. >>>>>> >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to elixir-lang-co...@googlegroups.com. >>>>> >>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/elixir-lang-core/l80s9s37.13dfe243-83b3-4133-8aad-7a86910f56fa%40we.are.superhuman.com >>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/l80s9s37.13dfe243-83b3-4133-8aad-7a86910f56fa%40we.are.superhuman.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>> >>>>> >>>>>> -- >>>>> >>>> You received this message because you are subscribed to the Google >>>>> Groups "elixir-lang-core" group. >>>>> >>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to elixir-lang-co...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J8qxM%3DKxMUUW6X%2Bt%2BrD9rKn0yDHbDY1fDgopxRvtNn5A%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J8qxM%3DKxMUUW6X%2Bt%2BrD9rKn0yDHbDY1fDgopxRvtNn5A%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elixir-lang-co...@googlegroups.com. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/325ea272-674a-4a5a-9228-6af11579bd05n%40googlegroups.com >>> <https://groups.google.com/d/msgid/elixir-lang-core/325ea272-674a-4a5a-9228-6af11579bd05n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/73a308e4-59e9-4f23-bb92-1417855cc996n%40googlegroups.com > <https://groups.google.com/d/msgid/elixir-lang-core/73a308e4-59e9-4f23-bb92-1417855cc996n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KrE3xy3PnQecUP0HM%2BMNx6dboS0Ar50p8cBW9qAAwP-w%40mail.gmail.com.
