Heya My initial impression is that this is quite a large departure from the semantics of all other constructs in the language, and the usual rules for reading Elixir no longer apply. The "return value" of a loop block is no longer the final line of the block, it could be spread throughout it.
To what extent can one use these mutable values with other language features? Is this permitted? for lesson <- lessons, let: lesson_counter do if lesson.active? do lesson_counter = lesson_counter + 1 endend Or this? for lesson <- lessons, let: lesson_counter do Enum.each(lesson.sessions, fn(_) -> lesson_counter = lesson_counter + 1 end)end If the above snippet with an anonymous function does not work, I think that may be quite surprising to newcomers as both `Enum.*` and `for` would work in languages which have conventional mutable variables (which they will likely be mentally using as a reference). Overall I don't feel this added complexity and potential for confusion is worth the benefits here. This brings no more expressive power to the language, and personally I don't have any issues with the "before" examples here using existing language features. Having said that, I'm not a big user of `for`, so the opinions of others may be more useful here! Cheers, Louis On Thu, 16 Dec 2021 at 15:02, José Valim <jose.va...@dashbit.co> wrote: > Note: This proposal contains images and rich text that may not display > correctly in your email. If so, you can read this proposal in a gist > <https://gist.github.com/josevalim/5c6735a4b90acc1bafdafec09acabe4f>. > > There is prior art in languages like Common Lisp, Haskell, and even in C# > with LINQ on having very powerful comprehensions as part of the language. > While Elixir comprehensions are already very expressive, allowing you to > map, filter, reduce, and collect over multiple enumerables at the same > time, it is still not capable of expressing other constructs, such as > map_reduce. > > The challenge here is how to continue adding more expressive power to > comprehensions without making the API feel massive. That's why, 7 years > after v1.0, only two new options have been added to comprehensions, :uniq > and :reduce, to a total of 3 (:into, :uniq, and :reduce). > Imperative loops > > I have been on the record a couple times saying that, while many problems > are more cleanly solved with recursion, there is a category of problems > that are much more elegant with imperative loops. One of those problems > have been described in the "nested-data-structures-traversal" > <https://github.com/josevalim/nested-data-structure-traversal> > repository, with solutions available in many different languages. Please > read the problem statement in said repository, as I will assume from now on > that you are familiar with it. > > Personally speaking, the most concise and clear solution is the Python > one, which I reproduce here: > > section_counter = 1lesson_counter = 1 > for section in sections: > if section["reset_lesson_position"]: > lesson_counter = 1 > > section["position"] = section_counter > section_counter += 1 > > for lesson in section["lessons"]: > lesson["position"] = lesson_counter > lesson_counter += 1 > > There are many things that make this solution clear: > > - Reassignment > - Mutability > - Sensitive whitespace > > Let's compare it with the Elixir solution I wrote and personally prefer > <https://github.com/josevalim/nested-data-structure-traversal/blob/master/elixir/map_reduce.exs>. > I am pasting an image below which highlights certain aspects: > > [image: Screenshot 2021-12-13 at 10 02 48] > <https://user-images.githubusercontent.com/9582/145821890-6557ea21-e61f-4813-8c54-53c4ea1a9438.png> > > - > > Lack of reassignment: in Elixir, we can't reassign variables, we can > only rebind them. The difference is, when you do var = some_value > inside a if, for, etc, the value won't "leak" to the outer scope. This > implies two things in the snippet above: > 1. We need to use Enum.map_reduce/3 and pass the state in and out > (highlighted in red) > 2. When resetting the lesson counter, we need both sides of the > conditional (hihhlighted in yellow) > - > > Lack of mutability: even though we set the lesson counter inside the > inner map_reduce, we still need to update the lesson inside the > session (highlighted in green) > - > > Lack of sensitive whitespace: we have two additional lines with end in > them (highlighted in blue) > > As you can see, do-end blocks add very litte noise to the final solution > compared to sensitive whitespace. In fact, the only reason I brought it up > is so we can confidently discard it from the discussion from now on. And > also because there is zero chance of the language suddenly becoming > whitespace sensitive. > > There is also zero chance of us introducing reassignment and making > mutability first class in Elixir too. The reason for this is because we all > agree that, the majority of the time, lack of reassignment and lack of > mutability are features that make our code more readable and understandable > in the long term. The snippet above is one of the few examples where we are > on the wrong end of the trade-offs. > > Therefore, how can we move forward? > Comprehensions > > Comprehensions in Elixir have always been a syntax sugar to more complex > data-structure traversals. Do you want to have the cartesian product > between all points in x and y? You could write this: > > Enum.flat_map(x, fn i -> > Enum.map(y, fn j -> {i, j} end)end) > > Or with a comprehension: > > for i <- x, j <- y, do: {i, j} > > Or maybe you want to brute force your way into finding Pythagorean Triples? > > Enum.flat_map(1..20, fn a -> > Enum.flat_map(1..20, fn b -> > 1..20 > |> Enum.filter(fn c -> a*a + b*b == c*c end) > |> Enum.map(fn c -> {a, b, c} end) > end)end) > > Or with a comprehension: > > for a <- 1..20, > b <- 1..20, > c <- 1..20, > a*a + b*b == c*c, > do: {a, b, c} > > There is no question the comprehensions are more concise and clearer, once > you understand their basic syntax elements (which are, at this point, > common to many languages). > > As mentioned in the introduction, we can express map, filter, reduce, and > collect inside comprehensions. But how can we represent map_reduce in a > clear and concise way? > The :map_reduce option > > Since we have :reduce in comprehensions, we could introduce :map_reduce. > The solution above would look like this: > > {sections, _acc} = > for section <- sections, map_reduce: {1, 1} do > {section_counter, lesson_counter} -> > lesson_counter = if section["reset_lesson_position"], do: 1, else: > lesson_counter > > {lessons, lesson_counter} = > for lesson <- section["lessons"], map_reduce: lesson_counter do > lesson_counter -> > {Map.put(lesson, "position", lesson_counter), lesson_counter + 1} > end > > section = > section > |> Map.put("lessons", lessons) > |> Map.put("position", section_counter) > > {section, {section_counter + 1, lesson_counter}} > end > > While there is a bit less noise compared to the original solution, the > reduction of noise mostly happened by the removal of modules names and a > few tokens, such as fn, (, and ). In terms of implementation, there is > still a lot of book keeping required to manage the variables. Can we do > better? > Introducing :let > > Our goal is to declare variables that are automatically looped within the > comprehension. So let's introduce a new option that does exactly that: > :let. :let expects one or a tuple of variables that will be reused across > the comprehension. At the end, :let returns a tuple with the > comprehension elements and the let variables. > > Here is how the solution would look like: > > section_counter = 1lesson_counter = 1 > {sections, _} = > for section <- sections, > let: {section_counter, lesson_counter} do > lesson_counter = if section["reset_lesson_position"], do: 1, else: > lesson_counter > > {lessons, lesson_counter} = > for lesson <- section["lessons"], let: lesson_counter do > lesson = Map.put(lesson, "position", lesson_counter) > lesson_counter = lesson_counter + 1 > lesson > end > > section = > section > |> Map.put("lessons", lessons) > |> Map.put("position", section_counter) > > section_counter = section_counter + 1 > section > end > > The :let option automatically takes care of passing the variables across > the comprehension, considerably cutting down the noise, without introducing > any mutability into the language. At the end, for+:let returns the result > of the comprehension plus the :let variables wrapped in a tuple. > Extensions > > Here are some extensions to the proposal above. Not all of them might be > available on the initial implementation. > Let initialization > > You can also initialize the variables within let for convenience: > > {sections, _} = > for section <- sections, > let: {section_counter = 1, lesson_counter = 1} do > > This should be available in the initial implementation. > :reduce vs :let > > With :let, :reduce becomes somewhat redundant. For example, > Enum.group_by/2 could be written as: > > for {k, v} <- Enum.reverse(list), reduce: %{} do > acc -> Map.update(acc, k, [v], &[v | &1])end > > with :let: > > {_, acc} = > for {k, v} <- Enum.reverse(list), let: acc = %{} do > acc = Map.update(acc, k, [v], &[v | &1]) > end > > The difference, however, is that :let returns the collection, while > :reduce does not. While the Elixir compiler could be smart enough to > optimize away building the collection in the :let case if we don't use > it, we may want to keep both :let and :reduce options for clarity. If > this is the case, I propose to align the syntaxes such that :reduce uses > the same semantics as :let. The only difference is the return type: > > for {k, v} <- Enum.reverse(list), reduce: acc = %{} do > acc = Map.update(acc, k, [v], &[v | &1])end > > This can be done in a backwards compatible fashion. > after > > When you look at our solution to the problem using let, we had to > introduce temporary variables in order to update our let variables: > > {lessons, lesson_counter} = > for lesson <- section["lessons"], let: lesson_counter do > lesson = Map.put(lesson, "position", lesson_counter) > lesson_counter = lesson_counter + 1 > lesson > end > > One extension is to add after to the comprehensions, which are computed > after the result is returned: > > {lessons, lesson_counter} = > for lesson <- section["lessons"], let: lesson_counter do > Map.put(lesson, "position", lesson_counter) > after > lesson_counter = lesson_counter + 1 > end > > This does not need to be part of the initial implementation. > Summary > > Feedback on the proposal and extensions is welcome! > > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD_%3DW%2BM9mSizFHJXFfqnQ%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD_%3DW%2BM9mSizFHJXFfqnQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFAMx3hTfFbu%2B%2BN9XXnG5sEJ4zTSXdugSa96-u9uEGq%2B5g%40mail.gmail.com.
