Sorry, for the short replies, I was on my phone. :) What I mean is, are the measurements across examples guaranteed to have the same amount of garbage collector calls (or no calls at all)? I am worried that, for quick snippets, the memory measurements are being influenced by other factors. But according to my understanding the anonymous function should not be allocated on Erlang/OTP 24 (and I think some further improvements are coming on 25).
Plus comparing against OTP 23 and 24 will be tough due to the JIT. On Mon, Jan 3, 2022 at 8:38 PM Wiebe-Marten Wijnja <w...@resilia.nl> wrote: > Yes, across benchmark runs the memory measurements are the same. > On 03-01-2022 20:17, José Valim wrote: > > Ah, df has no effect on a JIT system, I forgot about that. Is the memory > measurements guaranteed to have consistent effect of the GC across > benchmarks? > > On Mon, Jan 3, 2022 at 20:06 Wiebe-Marten Wijnja <w...@resilia.nl> wrote: > >> I have run some benchmarks (comparing OTP23 with JIT-enabled OTP24). >> Full results here: https://github.com/Qqwy/elixir-test-benchmrking_then/ >> >> It compares, in a situation where no tail recursion optimization is >> possible, `Kernel.then/2` vs. writing the same code manually vs. using >> `Kernel.then/2` with `@compile :inline`. >> >> >> A brief summary of the results: >> >> - OTP24 is able to get roughly twice as many iterations per second as >> OTP23. However: >> - On OTP24: >> - using `Kernel.then/2` requires (when tail recursion is not possible) >> 2.5x the memory of the other two variants. >> - using `Kernel.then/2`is roughly 30% slower than the other two >> variants. >> - On OTP23: >> - all three techniques use the same amount of memory. >> - using `Kernel.then/2`is roughly 8% slower than the other two variants. >> >> Strange... >> >> >> I also took a look at the disassembled code using :erts_debug.df as you >> suggested. >> Details here: >> https://github.com/Qqwy/elixir-test-benchmrking_then/#looking-at-the-disassembled-code >> *(Note that under OTP24 the *.dis-files only contained 1-5 empty lines, >> so the output is from OTP23. Should I file a bug with the OTP team for >> this?)* >> >> It seems that also during loading, no optimization of immediately-called >> anonymous functions is taking place. >> Above benchmarks seem to support this fact, although the results w.r.t. >> memory usage and the difference in slowdown vs OTP23/24 seems very odd to >> me. >> >> >> How to continue? >> >> >> ~Marten/Qqwy >> On 03-01-2022 17:30, José Valim wrote: >> >> The optimization may happen on the loader. Use erts_debug:df(Mod, Fun, >> Arity) and see that. >> >> On Mon, Jan 3, 2022 at 5:03 PM Wiebe-Marten Wijnja <w...@resilia.nl> >> wrote: >> >>> I've been running my tests on Elixir v1.13.1 built for OTP24 with OTP >>> 24.1.2. >>> When decompiling the resulting BEAM bytecode, the anonymous functions >>> are still visible. >>> >>> I will do some benchmarks to see how the resulting performance is. Maybe >>> the JIT will do something which is not visible in the BEAM bytecode. >>> On 03-01-2022 16:57, José Valim wrote: >>> >>> then/2 is a macro and the emitted code should be optimized from >>> Erlang/OTP 24+. >>> >>> On Mon, Jan 3, 2022 at 4:28 PM w...@resilia.nl <w...@resilia.nl> wrote: >>> >>>> Since v1.12 we have the macro `Kernel.then(value, function)` which >>>> expects an arity-1 function and will call it with the given value. >>>> >>>> This makes code which used to be written as follows: >>>> >>>> ``` >>>> def update(params, socket) do >>>> socket = >>>> socket >>>> |> assign(:myvar, params["myvar"]) >>>> |> assign_new(:some_default, fn -> 42 end) >>>> >>>> {:noreply, socket} >>>> end >>>> ``` >>>> >>>> more readable, by allowing it to be written as: >>>> >>>> ``` >>>> def update(params, socket) do >>>> socket >>>> |> assign(:myvar, params["myvar"]) >>>> |> assign_new(:some_default, fn -> 42 end) >>>> |> then(&{:noreply, &1}) >>>> end >>>> ``` >>>> >>>> This pattern seems to be common in codebases using Elixir 1.12 and up >>>> (At least according to anecdotal evidence). >>>> >>>> All is well. Except there is a little snag: The new code does not have >>>> the same runtime characteristics (both in performance and in memory usage) >>>> as `then`desugars to `(function).(value)`: An anonymous function is created >>>> and immediately run (and then garbage collected soon after). >>>> >>>> The Erlang compiler is clever enough to optimize these >>>> immediately-called anonymous functions away, but it will only do so when >>>> `@compile :inline` is set in the given module, to not mess with the call >>>> stack that might be returned when an exception is thrown. >>>> >>>> Now `@compile :inline` is quite the sledgehammer, as it will inline >>>> *all* functions in the current module (as long as they are not 'too >>>> big', which can also be configured, and only in the places where they are >>>> called statically). >>>> But since we're dealing with anonymous functions here which do not have >>>> clear names, there is no way to predict the name one should pass to the >>>> `@compile` option. >>>> >>>> >>>> It seems like this situation could be improved, although I am not sure >>>> how. >>>> >>>> Is there a way to mark these anonymous functions in some kind of way, >>>> to allow only them to be inlined? >>>> Or is there maybe a way to have the Elixir-compiler already inline >>>> common patterns like a capture with a datatype, rather than relying on the >>>> Erlang compiler for this? >>>> Your input is greatly appreciated. >>>> >>>> ~Marten/Qqwy >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elixir-lang-core" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to elixir-lang-core+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/elixir-lang-core/f0da2df2-432e-423c-a02b-27d8b916a0ecn%40googlegroups.com >>>> <https://groups.google.com/d/msgid/elixir-lang-core/f0da2df2-432e-423c-a02b-27d8b916a0ecn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elixir-lang-core+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2Bu8RTb8sMAJyGiuw6%2BgGgyuVZVxjpFad9M%2BbEgYrwkbg%40mail.gmail.com >>> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2Bu8RTb8sMAJyGiuw6%2BgGgyuVZVxjpFad9M%2BbEgYrwkbg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elixir-lang-core+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elixir-lang-core/b02049e0-8d86-a7b4-e8e0-396bb9ecd4f0%40resilia.nl >>> <https://groups.google.com/d/msgid/elixir-lang-core/b02049e0-8d86-a7b4-e8e0-396bb9ecd4f0%40resilia.nl?utm_medium=email&utm_source=footer> >>> . >> >> >>> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elixir-lang-core+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KqHRqTEisWYLNi7n2UQzP5XtVMUYkLbkHyiVyjcvKFOg%40mail.gmail.com >> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KqHRqTEisWYLNi7n2UQzP5XtVMUYkLbkHyiVyjcvKFOg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elixir-lang-core+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/84dbf8a8-4f9f-5aa7-efc3-1658e097a8c5%40resilia.nl >> <https://groups.google.com/d/msgid/elixir-lang-core/84dbf8a8-4f9f-5aa7-efc3-1658e097a8c5%40resilia.nl?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Jod05LOG61Wf08gkNR0FTSDx8W4gWSZdr96k7BZ94UrQ%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Jod05LOG61Wf08gkNR0FTSDx8W4gWSZdr96k7BZ94UrQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/178e943c-84c3-31c4-9e41-903ad2f8da32%40resilia.nl > <https://groups.google.com/d/msgid/elixir-lang-core/178e943c-84c3-31c4-9e41-903ad2f8da32%40resilia.nl?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LqMq6oLpncmWethkon3Xpbp%3DTQAw8kOm96sU%2Bf3qvj0Q%40mail.gmail.com.
