I was skeptical at first as well until I started thinking more about how
convoluted some scenarios are today, and I think we on this mailing list
especially are all likely a bit stuck in our ways. If you think about it from
the beginner perspective, there is a ton of ceremony for certain operations,
and some even today I have to go lookup to do basic things after almost 9 years
of elixir. Consider the "make every other item highlighted" scenario. You
either have to use `Enum.with_index` and `rem(index, 2) == 0`, which already is
introducing two new APIs to the user, or you need to use Enum.zip with
Stream.cycle:
<%= for {item, highlighted?} <- Enum.zip(items, Stream.cycle([true,
false])) %>
This looks reasonable, but is extremely non intuitive for newcomers, and even I
have to think hard or lookup with I've done previously to remember it zip +
cycle. Consider the alternative:
<%= for item <- items, let: {highlighted? = false} do
...
<% highlighted? = not highlighted? %>
<% end %>
This will also help cut down on the number of unnecessary traversals folks need
to make in general, but performance is not close the main benefits here. While
this at first felt like mutation to me and unlike anything we do in elixir, in
my mind it actually aligns really well with the way variable rebinding already
works – it is simply being extended to for bodies.
+1 from me.
> On Dec 16, 2021, at 11:54 AM, Ben Wilson <benwilson...@gmail.com> wrote:
>
> I am with Louis and Paul so far I think. I won't repeat their comments but I
> think I can extend the issue by pointing out that this breaks refactoring for
> the inner contents of `for`. Previously, if you have:
>
> ```
> for lesson <- section["lessons"], reduce: 0 do
> counter ->
> # complex multi-line-thing using the lesson and counter
> end
> ```
>
> I can refactor this into:
>
> ```
> for lesson <- section["lessons"], reduce: 0 do
> counter ->
> complex_operation(lesson, counter)
> end
>
> def complex_thing(lesson, counter) do
> # complex multi-line-thing using the lesson and counter
> end
> ```
>
> And everything just works, as is normal in Elixir code. The proposed changes
> would (as far as I can see) break this and that feels very unexpected and
> foreign.
>
> I sympathize with the problem space, but so far it's a -1 for me on this
> particular proposed improvement.
>
> - Ben
> On Thursday, December 16, 2021 at 10:02:49 AM UTC-5 José Valim wrote:
> Note: This proposal contains images and rich text that may not display
> correctly in your email. If so, you can read this proposal in a gist
> <https://gist.github.com/josevalim/5c6735a4b90acc1bafdafec09acabe4f>.
>
> There is prior art in languages like Common Lisp, Haskell, and even in C#
> with LINQ on having very powerful comprehensions as part of the language.
> While Elixir comprehensions are already very expressive, allowing you to map,
> filter, reduce, and collect over multiple enumerables at the same time, it is
> still not capable of expressing other constructs, such as map_reduce.
>
> The challenge here is how to continue adding more expressive power to
> comprehensions without making the API feel massive. That's why, 7 years after
> v1.0, only two new options have been added to comprehensions, :uniq and
> :reduce, to a total of 3 (:into, :uniq, and :reduce).
>
> Imperative loops
>
> I have been on the record a couple times saying that, while many problems are
> more cleanly solved with recursion, there is a category of problems that are
> much more elegant with imperative loops. One of those problems have been
> described in the "nested-data-structures-traversal"
> <https://github.com/josevalim/nested-data-structure-traversal> repository,
> with solutions available in many different languages. Please read the problem
> statement in said repository, as I will assume from now on that you are
> familiar with it.
>
> Personally speaking, the most concise and clear solution is the Python one,
> which I reproduce here:
>
> section_counter = 1
> lesson_counter = 1
>
> for section in sections:
> if section["reset_lesson_position"]:
> lesson_counter = 1
>
> section["position"] = section_counter
> section_counter += 1
>
> for lesson in section["lessons"]:
> lesson["position"] = lesson_counter
> lesson_counter += 1
> There are many things that make this solution clear:
>
> Reassignment
> Mutability
> Sensitive whitespace
> Let's compare it with the Elixir solution I wrote and personally prefer
> <https://github.com/josevalim/nested-data-structure-traversal/blob/master/elixir/map_reduce.exs>.
> I am pasting an image below which highlights certain aspects:
>
>
> <https://user-images.githubusercontent.com/9582/145821890-6557ea21-e61f-4813-8c54-53c4ea1a9438.png>
> Lack of reassignment: in Elixir, we can't reassign variables, we can only
> rebind them. The difference is, when you do var = some_value inside a if,
> for, etc, the value won't "leak" to the outer scope. This implies two things
> in the snippet above:
>
> We need to use Enum.map_reduce/3 and pass the state in and out (highlighted
> in red)
> When resetting the lesson counter, we need both sides of the conditional
> (hihhlighted in yellow)
> Lack of mutability: even though we set the lesson counter inside the inner
> map_reduce, we still need to update the lesson inside the session
> (highlighted in green)
>
> Lack of sensitive whitespace: we have two additional lines with end in them
> (highlighted in blue)
>
> As you can see, do-end blocks add very litte noise to the final solution
> compared to sensitive whitespace. In fact, the only reason I brought it up is
> so we can confidently discard it from the discussion from now on. And also
> because there is zero chance of the language suddenly becoming whitespace
> sensitive.
>
> There is also zero chance of us introducing reassignment and making
> mutability first class in Elixir too. The reason for this is because we all
> agree that, the majority of the time, lack of reassignment and lack of
> mutability are features that make our code more readable and understandable
> in the long term. The snippet above is one of the few examples where we are
> on the wrong end of the trade-offs.
>
> Therefore, how can we move forward?
>
> Comprehensions
>
> Comprehensions in Elixir have always been a syntax sugar to more complex
> data-structure traversals. Do you want to have the cartesian product between
> all points in x and y? You could write this:
>
> Enum.flat_map(x, fn i ->
> Enum.map(y, fn j -> {i, j} end)
> end)
> Or with a comprehension:
>
> for i <- x, j <- y, do: {i, j}
> Or maybe you want to brute force your way into finding Pythagorean Triples?
>
> Enum.flat_map(1..20, fn a ->
> Enum.flat_map(1..20, fn b ->
> 1..20
> |> Enum.filter(fn c -> a*a + b*b == c*c end)
> |> Enum.map(fn c -> {a, b, c} end)
> end)
> end)
> Or with a comprehension:
>
> for a <- 1..20,
> b <- 1..20,
> c <- 1..20,
> a*a + b*b == c*c,
> do: {a, b, c}
> There is no question the comprehensions are more concise and clearer, once
> you understand their basic syntax elements (which are, at this point, common
> to many languages).
>
> As mentioned in the introduction, we can express map, filter, reduce, and
> collect inside comprehensions. But how can we represent map_reduce in a clear
> and concise way?
>
> The :map_reduce option
>
> Since we have :reduce in comprehensions, we could introduce :map_reduce. The
> solution above would look like this:
>
> {sections, _acc} =
> for section <- sections, map_reduce: {1, 1} do
> {section_counter, lesson_counter} ->
> lesson_counter = if section["reset_lesson_position"], do: 1, else:
> lesson_counter
>
> {lessons, lesson_counter} =
> for lesson <- section["lessons"], map_reduce: lesson_counter do
> lesson_counter ->
> {Map.put(lesson, "position", lesson_counter), lesson_counter + 1}
> end
>
> section =
> section
> |> Map.put("lessons", lessons)
> |> Map.put("position", section_counter)
>
> {section, {section_counter + 1, lesson_counter}}
> end
> While there is a bit less noise compared to the original solution, the
> reduction of noise mostly happened by the removal of modules names and a few
> tokens, such as fn, (, and ). In terms of implementation, there is still a
> lot of book keeping required to manage the variables. Can we do better?
>
> Introducing :let
>
> Our goal is to declare variables that are automatically looped within the
> comprehension. So let's introduce a new option that does exactly that: :let.
> :let expects one or a tuple of variables that will be reused across the
> comprehension. At the end, :let returns a tuple with the comprehension
> elements and the let variables.
>
> Here is how the solution would look like:
>
> section_counter = 1
> lesson_counter = 1
>
> {sections, _} =
> for section <- sections,
> let: {section_counter, lesson_counter} do
> lesson_counter = if section["reset_lesson_position"], do: 1, else:
> lesson_counter
>
> {lessons, lesson_counter} =
> for lesson <- section["lessons"], let: lesson_counter do
> lesson = Map.put(lesson, "position", lesson_counter)
> lesson_counter = lesson_counter + 1
> lesson
> end
>
> section =
> section
> |> Map.put("lessons", lessons)
> |> Map.put("position", section_counter)
>
> section_counter = section_counter + 1
> section
> end
> The :let option automatically takes care of passing the variables across the
> comprehension, considerably cutting down the noise, without introducing any
> mutability into the language. At the end, for+:let returns the result of the
> comprehension plus the :letvariables wrapped in a tuple.
>
> Extensions
>
> Here are some extensions to the proposal above. Not all of them might be
> available on the initial implementation.
>
> Let initialization
>
> You can also initialize the variables within let for convenience:
>
> {sections, _} =
> for section <- sections,
> let: {section_counter = 1, lesson_counter = 1} do
> This should be available in the initial implementation.
>
> :reduce vs :let
>
> With :let, :reduce becomes somewhat redundant. For example, Enum.group_by/2
> could be written as:
>
> for {k, v} <- Enum.reverse(list), reduce: %{} do
> acc -> Map.update(acc, k, [v], &[v | &1])
> end
> with :let:
>
> {_, acc} =
> for {k, v} <- Enum.reverse(list), let: acc = %{} do
> acc = Map.update(acc, k, [v], &[v | &1])
> end
> The difference, however, is that :let returns the collection, while :reduce
> does not. While the Elixir compiler could be smart enough to optimize away
> building the collection in the :let case if we don't use it, we may want to
> keep both :let and :reduceoptions for clarity. If this is the case, I propose
> to align the syntaxes such that :reduce uses the same semantics as :let. The
> only difference is the return type:
>
> for {k, v} <- Enum.reverse(list), reduce: acc = %{} do
> acc = Map.update(acc, k, [v], &[v | &1])
> end
> This can be done in a backwards compatible fashion.
>
> after
>
> When you look at our solution to the problem using let, we had to introduce
> temporary variables in order to update our let variables:
>
> {lessons, lesson_counter} =
> for lesson <- section["lessons"], let: lesson_counter do
> lesson = Map.put(lesson, "position", lesson_counter)
> lesson_counter = lesson_counter + 1
> lesson
> end
> One extension is to add after to the comprehensions, which are computed after
> the result is returned:
>
> {lessons, lesson_counter} =
> for lesson <- section["lessons"], let: lesson_counter do
> Map.put(lesson, "position", lesson_counter)
> after
> lesson_counter = lesson_counter + 1
> end
> This does not need to be part of the initial implementation.
>
> Summary
>
> Feedback on the proposal and extensions is welcome!
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elixir-lang-core+unsubscr...@googlegroups.com
> <mailto:elixir-lang-core+unsubscr...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/22703d4b-60cb-4b0e-83d2-4a122f9147afn%40googlegroups.com
>
> <https://groups.google.com/d/msgid/elixir-lang-core/22703d4b-60cb-4b0e-83d2-4a122f9147afn%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/657C7140-D8C2-4FF6-8616-3CD92400F1EB%40chrismccord.com.
