My gut reaction is the same as Louis'. Looking at the problem statement, the solution, to me, is ultimately a map over the array. The only "problem" is maintaining some state between elements, which is managed well with the map_reduce. All the proposed variations just seem "overworked" to me -- the original Elixir solution seemed more concise than either of the comprehension proposals. Also like Louis, I rarely use the "for" construction -- I will more likely use Enum.each or Enum.with_index or Enum.map to iterate over a list, rather than "for". Maybe I should use "for" more often, but it's not my go-to iterator. :)
...Paul On Thu, Dec 16, 2021 at 7:46 AM Louis Pilfold <lo...@lpil.uk> wrote: > Heya > > My initial impression is that this is quite a large departure from the > semantics of all other constructs in the language, and the usual rules for > reading Elixir no longer apply. The "return value" of a loop block is no > longer the final line of the block, it could be spread throughout it. > > To what extent can one use these mutable values with other language > features? > > Is this permitted? > > for lesson <- lessons, let: lesson_counter do > if lesson.active? do lesson_counter = lesson_counter + 1 > endend > > Or this? > > for lesson <- lessons, let: lesson_counter do > Enum.each(lesson.sessions, fn(_) -> lesson_counter = lesson_counter + 1 > end)end > > If the above snippet with an anonymous function does not work, I think > that may be quite surprising to newcomers as both `Enum.*` and `for` would > work in languages which have conventional mutable variables (which they > will likely be mentally using as a reference). > > Overall I don't feel this added complexity and potential for confusion is > worth the benefits here. This brings no more expressive power to the > language, and personally I don't have any issues with the "before" examples > here using existing language features. > Having said that, I'm not a big user of `for`, so the opinions of others > may be more useful here! > > Cheers, > Louis > > > On Thu, 16 Dec 2021 at 15:02, José Valim <jose.va...@dashbit.co> wrote: > >> Note: This proposal contains images and rich text that may not display >> correctly in your email. If so, you can read this proposal in a gist >> <https://gist.github.com/josevalim/5c6735a4b90acc1bafdafec09acabe4f>. >> >> There is prior art in languages like Common Lisp, Haskell, and even in C# >> with LINQ on having very powerful comprehensions as part of the language. >> While Elixir comprehensions are already very expressive, allowing you to >> map, filter, reduce, and collect over multiple enumerables at the same >> time, it is still not capable of expressing other constructs, such as >> map_reduce. >> >> The challenge here is how to continue adding more expressive power to >> comprehensions without making the API feel massive. That's why, 7 years >> after v1.0, only two new options have been added to comprehensions, :uniq >> and :reduce, to a total of 3 (:into, :uniq, and :reduce). >> Imperative loops >> >> I have been on the record a couple times saying that, while many problems >> are more cleanly solved with recursion, there is a category of problems >> that are much more elegant with imperative loops. One of those problems >> have been described in the "nested-data-structures-traversal" >> <https://github.com/josevalim/nested-data-structure-traversal> >> repository, with solutions available in many different languages. Please >> read the problem statement in said repository, as I will assume from now on >> that you are familiar with it. >> >> Personally speaking, the most concise and clear solution is the Python >> one, which I reproduce here: >> >> section_counter = 1lesson_counter = 1 >> for section in sections: >> if section["reset_lesson_position"]: >> lesson_counter = 1 >> >> section["position"] = section_counter >> section_counter += 1 >> >> for lesson in section["lessons"]: >> lesson["position"] = lesson_counter >> lesson_counter += 1 >> >> There are many things that make this solution clear: >> >> - Reassignment >> - Mutability >> - Sensitive whitespace >> >> Let's compare it with the Elixir solution I wrote and personally prefer >> <https://github.com/josevalim/nested-data-structure-traversal/blob/master/elixir/map_reduce.exs>. >> I am pasting an image below which highlights certain aspects: >> >> [image: Screenshot 2021-12-13 at 10 02 48] >> <https://user-images.githubusercontent.com/9582/145821890-6557ea21-e61f-4813-8c54-53c4ea1a9438.png> >> >> - >> >> Lack of reassignment: in Elixir, we can't reassign variables, we can >> only rebind them. The difference is, when you do var = some_value >> inside a if, for, etc, the value won't "leak" to the outer scope. >> This implies two things in the snippet above: >> 1. We need to use Enum.map_reduce/3 and pass the state in and out >> (highlighted in red) >> 2. When resetting the lesson counter, we need both sides of the >> conditional (hihhlighted in yellow) >> - >> >> Lack of mutability: even though we set the lesson counter inside the >> inner map_reduce, we still need to update the lesson inside the >> session (highlighted in green) >> - >> >> Lack of sensitive whitespace: we have two additional lines with end >> in them (highlighted in blue) >> >> As you can see, do-end blocks add very litte noise to the final solution >> compared to sensitive whitespace. In fact, the only reason I brought it up >> is so we can confidently discard it from the discussion from now on. And >> also because there is zero chance of the language suddenly becoming >> whitespace sensitive. >> >> There is also zero chance of us introducing reassignment and making >> mutability first class in Elixir too. The reason for this is because we all >> agree that, the majority of the time, lack of reassignment and lack of >> mutability are features that make our code more readable and understandable >> in the long term. The snippet above is one of the few examples where we are >> on the wrong end of the trade-offs. >> >> Therefore, how can we move forward? >> Comprehensions >> >> Comprehensions in Elixir have always been a syntax sugar to more complex >> data-structure traversals. Do you want to have the cartesian product >> between all points in x and y? You could write this: >> >> Enum.flat_map(x, fn i -> >> Enum.map(y, fn j -> {i, j} end)end) >> >> Or with a comprehension: >> >> for i <- x, j <- y, do: {i, j} >> >> Or maybe you want to brute force your way into finding Pythagorean >> Triples? >> >> Enum.flat_map(1..20, fn a -> >> Enum.flat_map(1..20, fn b -> >> 1..20 >> |> Enum.filter(fn c -> a*a + b*b == c*c end) >> |> Enum.map(fn c -> {a, b, c} end) >> end)end) >> >> Or with a comprehension: >> >> for a <- 1..20, >> b <- 1..20, >> c <- 1..20, >> a*a + b*b == c*c, >> do: {a, b, c} >> >> There is no question the comprehensions are more concise and clearer, >> once you understand their basic syntax elements (which are, at this point, >> common to many languages). >> >> As mentioned in the introduction, we can express map, filter, reduce, >> and collect inside comprehensions. But how can we represent map_reduce >> in a clear and concise way? >> The :map_reduce option >> >> Since we have :reduce in comprehensions, we could introduce :map_reduce. >> The solution above would look like this: >> >> {sections, _acc} = >> for section <- sections, map_reduce: {1, 1} do >> {section_counter, lesson_counter} -> >> lesson_counter = if section["reset_lesson_position"], do: 1, else: >> lesson_counter >> >> {lessons, lesson_counter} = >> for lesson <- section["lessons"], map_reduce: lesson_counter do >> lesson_counter -> >> {Map.put(lesson, "position", lesson_counter), lesson_counter + 1} >> end >> >> section = >> section >> |> Map.put("lessons", lessons) >> |> Map.put("position", section_counter) >> >> {section, {section_counter + 1, lesson_counter}} >> end >> >> While there is a bit less noise compared to the original solution, the >> reduction of noise mostly happened by the removal of modules names and a >> few tokens, such as fn, (, and ). In terms of implementation, there is >> still a lot of book keeping required to manage the variables. Can we do >> better? >> Introducing :let >> >> Our goal is to declare variables that are automatically looped within the >> comprehension. So let's introduce a new option that does exactly that: >> :let. :let expects one or a tuple of variables that will be reused >> across the comprehension. At the end, :let returns a tuple with the >> comprehension elements and the let variables. >> >> Here is how the solution would look like: >> >> section_counter = 1lesson_counter = 1 >> {sections, _} = >> for section <- sections, >> let: {section_counter, lesson_counter} do >> lesson_counter = if section["reset_lesson_position"], do: 1, else: >> lesson_counter >> >> {lessons, lesson_counter} = >> for lesson <- section["lessons"], let: lesson_counter do >> lesson = Map.put(lesson, "position", lesson_counter) >> lesson_counter = lesson_counter + 1 >> lesson >> end >> >> section = >> section >> |> Map.put("lessons", lessons) >> |> Map.put("position", section_counter) >> >> section_counter = section_counter + 1 >> section >> end >> >> The :let option automatically takes care of passing the variables across >> the comprehension, considerably cutting down the noise, without introducing >> any mutability into the language. At the end, for+:let returns the >> result of the comprehension plus the :let variables wrapped in a tuple. >> Extensions >> >> Here are some extensions to the proposal above. Not all of them might be >> available on the initial implementation. >> Let initialization >> >> You can also initialize the variables within let for convenience: >> >> {sections, _} = >> for section <- sections, >> let: {section_counter = 1, lesson_counter = 1} do >> >> This should be available in the initial implementation. >> :reduce vs :let >> >> With :let, :reduce becomes somewhat redundant. For example, >> Enum.group_by/2 could be written as: >> >> for {k, v} <- Enum.reverse(list), reduce: %{} do >> acc -> Map.update(acc, k, [v], &[v | &1])end >> >> with :let: >> >> {_, acc} = >> for {k, v} <- Enum.reverse(list), let: acc = %{} do >> acc = Map.update(acc, k, [v], &[v | &1]) >> end >> >> The difference, however, is that :let returns the collection, while >> :reduce does not. While the Elixir compiler could be smart enough to >> optimize away building the collection in the :let case if we don't use >> it, we may want to keep both :let and :reduce options for clarity. If >> this is the case, I propose to align the syntaxes such that :reduce uses >> the same semantics as :let. The only difference is the return type: >> >> for {k, v} <- Enum.reverse(list), reduce: acc = %{} do >> acc = Map.update(acc, k, [v], &[v | &1])end >> >> This can be done in a backwards compatible fashion. >> after >> >> When you look at our solution to the problem using let, we had to >> introduce temporary variables in order to update our let variables: >> >> {lessons, lesson_counter} = >> for lesson <- section["lessons"], let: lesson_counter do >> lesson = Map.put(lesson, "position", lesson_counter) >> lesson_counter = lesson_counter + 1 >> lesson >> end >> >> One extension is to add after to the comprehensions, which are computed >> after the result is returned: >> >> {lessons, lesson_counter} = >> for lesson <- section["lessons"], let: lesson_counter do >> Map.put(lesson, "position", lesson_counter) >> after >> lesson_counter = lesson_counter + 1 >> end >> >> This does not need to be part of the initial implementation. >> Summary >> >> Feedback on the proposal and extensions is welcome! >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elixir-lang-core+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD_%3DW%2BM9mSizFHJXFfqnQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD_%3DW%2BM9mSizFHJXFfqnQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFAMx3hTfFbu%2B%2BN9XXnG5sEJ4zTSXdugSa96-u9uEGq%2B5g%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFAMx3hTfFbu%2B%2BN9XXnG5sEJ4zTSXdugSa96-u9uEGq%2B5g%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAD3kWz99qSby_7PjBT%3D68ztzmX-2tq%2B-xV7RXL_ze4ZXsaFp7w%40mail.gmail.com.
