I do not have any issues with the existing syntax as well. I don’t think `let:` is somewhat I am going to use on daily basis, but I also don’t see any harm while the old syntax for `reduce` stays.
While we are on topic, I’d share another thing I really miss in `for` comprehensions: early returns aka `reduce_while` aka `take`. When one wants to use the power of comprehension to find the first element meeting some requirements, `for` cannot be used because it will greedily iterate through the tail of the iterations. Consider the following example: for i <- 0..1_000, j <- 0..1_000, value = some_get_fun(some_matrix, i, j), not is_nil(value), do: value If I want to find a first non-nil value, I have to either go through all the 1M iterations and then pipe the result to hd() or use `throw value` and wrap the call into `try/catch`. It’d be great to have somewhat like `take: 1` option to stop evaluation immediately after it has been emitted into `do:` clause. On Thu, Dec 16, 2021 at 5:12 PM Paul Clegg <dotdotdotp...@gmail.com> wrote: > My gut reaction is the same as Louis'. Looking at the problem statement, > the solution, to me, is ultimately a map over the array. The only > "problem" is maintaining some state between elements, which is managed well > with the map_reduce. All the proposed variations just seem "overworked" to > me -- the original Elixir solution seemed more concise than either of the > comprehension proposals. Also like Louis, I rarely use the "for" > construction -- I will more likely use Enum.each or Enum.with_index or > Enum.map to iterate over a list, rather than "for". Maybe I should use > "for" more often, but it's not my go-to iterator. :) > > ...Paul > > > > > On Thu, Dec 16, 2021 at 7:46 AM Louis Pilfold <lo...@lpil.uk> wrote: > >> Heya >> >> My initial impression is that this is quite a large departure from the >> semantics of all other constructs in the language, and the usual rules for >> reading Elixir no longer apply. The "return value" of a loop block is no >> longer the final line of the block, it could be spread throughout it. >> >> To what extent can one use these mutable values with other language >> features? >> >> Is this permitted? >> >> for lesson <- lessons, let: lesson_counter do >> if lesson.active >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lesson.active&d=DwQFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=4-Z66M0GsRzg_A0w6SG_amg91N-pIPh7boj8KvVjZeo&e=>? >> do lesson_counter = lesson_counter + 1 >> endend >> >> Or this? >> >> for lesson <- lessons, let: lesson_counter do >> Enum.each(lesson.sessions, fn(_) -> lesson_counter = lesson_counter + 1 >> end)end >> >> If the above snippet with an anonymous function does not work, I think >> that may be quite surprising to newcomers as both `Enum.*` and `for` would >> work in languages which have conventional mutable variables (which they >> will likely be mentally using as a reference). >> >> Overall I don't feel this added complexity and potential for confusion is >> worth the benefits here. This brings no more expressive power to the >> language, and personally I don't have any issues with the "before" examples >> here using existing language features. >> Having said that, I'm not a big user of `for`, so the opinions of others >> may be more useful here! >> >> Cheers, >> Louis >> >> >> On Thu, 16 Dec 2021 at 15:02, José Valim <jose.va...@dashbit.co> wrote: >> >>> Note: This proposal contains images and rich text that may not display >>> correctly in your email. If so, you can read this proposal in a gist >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_josevalim_5c6735a4b90acc1bafdafec09acabe4f&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=7rvaZTayahGO6aWmmmJlru4EyCrc4JQqjvfXmNif-Mk&e=> >>> . >>> >>> There is prior art in languages like Common Lisp, Haskell, and even in >>> C# with LINQ on having very powerful comprehensions as part of the >>> language. While Elixir comprehensions are already very expressive, allowing >>> you to map, filter, reduce, and collect over multiple enumerables at >>> the same time, it is still not capable of expressing other constructs, such >>> as map_reduce. >>> >>> The challenge here is how to continue adding more expressive power to >>> comprehensions without making the API feel massive. That's why, 7 years >>> after v1.0, only two new options have been added to comprehensions, >>> :uniq and :reduce, to a total of 3 (:into, :uniq, and :reduce). >>> Imperative loops >>> >>> I have been on the record a couple times saying that, while many >>> problems are more cleanly solved with recursion, there is a category of >>> problems that are much more elegant with imperative loops. One of those >>> problems have been described in the "nested-data-structures-traversal" >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_josevalim_nested-2Ddata-2Dstructure-2Dtraversal&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=n2qVumP2N8sD8Ck37AGKz36JL81f-dth1EUAqmh72TA&e=> >>> repository, with solutions available in many different languages. Please >>> read the problem statement in said repository, as I will assume from now on >>> that you are familiar with it. >>> >>> Personally speaking, the most concise and clear solution is the Python >>> one, which I reproduce here: >>> >>> section_counter = 1lesson_counter = 1 >>> for section in sections: >>> if section["reset_lesson_position"]: >>> lesson_counter = 1 >>> >>> section["position"] = section_counter >>> section_counter += 1 >>> >>> for lesson in section["lessons"]: >>> lesson["position"] = lesson_counter >>> lesson_counter += 1 >>> >>> There are many things that make this solution clear: >>> >>> - Reassignment >>> - Mutability >>> - Sensitive whitespace >>> >>> Let's compare it with the Elixir solution I wrote and personally prefer >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_josevalim_nested-2Ddata-2Dstructure-2Dtraversal_blob_master_elixir_map-5Freduce.exs&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=RI8hAR35HBvRmWqKbbRspOUoAZfWNyYBJ2JArzIbPQs&e=>. >>> I am pasting an image below which highlights certain aspects: >>> >>> [image: Screenshot 2021-12-13 at 10 02 48] >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__user-2Dimages.githubusercontent.com_9582_145821890-2D6557ea21-2De61f-2D4813-2D8c54-2D53c4ea1a9438.png&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=HSWL6WlCQY6iSRUQvOTXuq4Oyn5BWmyTQ-jvzPQoKIM&e=> >>> >>> - >>> >>> Lack of reassignment: in Elixir, we can't reassign variables, we can >>> only rebind them. The difference is, when you do var = some_value >>> inside a if, for, etc, the value won't "leak" to the outer scope. >>> This implies two things in the snippet above: >>> 1. We need to use Enum.map_reduce/3 and pass the state in and out >>> (highlighted in red) >>> 2. When resetting the lesson counter, we need both sides of the >>> conditional (hihhlighted in yellow) >>> - >>> >>> Lack of mutability: even though we set the lesson counter inside the >>> inner map_reduce, we still need to update the lesson inside the >>> session (highlighted in green) >>> - >>> >>> Lack of sensitive whitespace: we have two additional lines with end >>> in them (highlighted in blue) >>> >>> As you can see, do-end blocks add very litte noise to the final solution >>> compared to sensitive whitespace. In fact, the only reason I brought it up >>> is so we can confidently discard it from the discussion from now on. And >>> also because there is zero chance of the language suddenly becoming >>> whitespace sensitive. >>> >>> There is also zero chance of us introducing reassignment and making >>> mutability first class in Elixir too. The reason for this is because we all >>> agree that, the majority of the time, lack of reassignment and lack of >>> mutability are features that make our code more readable and understandable >>> in the long term. The snippet above is one of the few examples where we are >>> on the wrong end of the trade-offs. >>> >>> Therefore, how can we move forward? >>> Comprehensions >>> >>> Comprehensions in Elixir have always been a syntax sugar to more complex >>> data-structure traversals. Do you want to have the cartesian product >>> between all points in x and y? You could write this: >>> >>> Enum.flat_map(x, fn i -> >>> Enum.map(y, fn j -> {i, j} end)end) >>> >>> Or with a comprehension: >>> >>> for i <- x, j <- y, do: {i, j} >>> >>> Or maybe you want to brute force your way into finding Pythagorean >>> Triples? >>> >>> Enum.flat_map(1..20, fn a -> >>> Enum.flat_map(1..20, fn b -> >>> 1..20 >>> |> Enum.filter(fn c -> a*a + b*b == c*c end) >>> |> Enum.map(fn c -> {a, b, c} end) >>> end)end) >>> >>> Or with a comprehension: >>> >>> for a <- 1..20, >>> b <- 1..20, >>> c <- 1..20, >>> a*a + b*b == c*c, >>> do: {a, b, c} >>> >>> There is no question the comprehensions are more concise and clearer, >>> once you understand their basic syntax elements (which are, at this point, >>> common to many languages). >>> >>> As mentioned in the introduction, we can express map, filter, reduce, >>> and collect inside comprehensions. But how can we represent map_reduce >>> in a clear and concise way? >>> The :map_reduce option >>> >>> Since we have :reduce in comprehensions, we could introduce :map_reduce. >>> The solution above would look like this: >>> >>> {sections, _acc} = >>> for section <- sections, map_reduce: {1, 1} do >>> {section_counter, lesson_counter} -> >>> lesson_counter = if section["reset_lesson_position"], do: 1, else: >>> lesson_counter >>> >>> {lessons, lesson_counter} = >>> for lesson <- section["lessons"], map_reduce: lesson_counter do >>> lesson_counter -> >>> {Map.put(lesson, "position", lesson_counter), lesson_counter + >>> 1} >>> end >>> >>> section = >>> section >>> |> Map.put("lessons", lessons) >>> |> Map.put("position", section_counter) >>> >>> {section, {section_counter + 1, lesson_counter}} >>> end >>> >>> While there is a bit less noise compared to the original solution, the >>> reduction of noise mostly happened by the removal of modules names and a >>> few tokens, such as fn, (, and ). In terms of implementation, there is >>> still a lot of book keeping required to manage the variables. Can we do >>> better? >>> Introducing :let >>> >>> Our goal is to declare variables that are automatically looped within >>> the comprehension. So let's introduce a new option that does exactly that: >>> :let. :let expects one or a tuple of variables that will be reused >>> across the comprehension. At the end, :let returns a tuple with the >>> comprehension elements and the let variables. >>> >>> Here is how the solution would look like: >>> >>> section_counter = 1lesson_counter = 1 >>> {sections, _} = >>> for section <- sections, >>> let: {section_counter, lesson_counter} do >>> lesson_counter = if section["reset_lesson_position"], do: 1, else: >>> lesson_counter >>> >>> {lessons, lesson_counter} = >>> for lesson <- section["lessons"], let: lesson_counter do >>> lesson = Map.put(lesson, "position", lesson_counter) >>> lesson_counter = lesson_counter + 1 >>> lesson >>> end >>> >>> section = >>> section >>> |> Map.put("lessons", lessons) >>> |> Map.put("position", section_counter) >>> >>> section_counter = section_counter + 1 >>> section >>> end >>> >>> The :let option automatically takes care of passing the variables >>> across the comprehension, considerably cutting down the noise, without >>> introducing any mutability into the language. At the end, for+:let >>> returns the result of the comprehension plus the :let variables wrapped >>> in a tuple. >>> Extensions >>> >>> Here are some extensions to the proposal above. Not all of them might be >>> available on the initial implementation. >>> Let initialization >>> >>> You can also initialize the variables within let for convenience: >>> >>> {sections, _} = >>> for section <- sections, >>> let: {section_counter = 1, lesson_counter = 1} do >>> >>> This should be available in the initial implementation. >>> :reduce vs :let >>> >>> With :let, :reduce becomes somewhat redundant. For example, >>> Enum.group_by/2 could be written as: >>> >>> for {k, v} <- Enum.reverse(list), reduce: %{} do >>> acc -> Map.update(acc, k, [v], &[v | &1])end >>> >>> with :let: >>> >>> {_, acc} = >>> for {k, v} <- Enum.reverse(list), let: acc = %{} do >>> acc = Map.update(acc, k, [v], &[v | &1]) >>> end >>> >>> The difference, however, is that :let returns the collection, while >>> :reduce does not. While the Elixir compiler could be smart enough to >>> optimize away building the collection in the :let case if we don't use >>> it, we may want to keep both :let and :reduce options for clarity. If >>> this is the case, I propose to align the syntaxes such that :reduce >>> uses the same semantics as :let. The only difference is the return type: >>> >>> for {k, v} <- Enum.reverse(list), reduce: acc = %{} do >>> acc = Map.update(acc, k, [v], &[v | &1])end >>> >>> This can be done in a backwards compatible fashion. >>> after >>> >>> When you look at our solution to the problem using let, we had to >>> introduce temporary variables in order to update our let variables: >>> >>> {lessons, lesson_counter} = >>> for lesson <- section["lessons"], let: lesson_counter do >>> lesson = Map.put(lesson, "position", lesson_counter) >>> lesson_counter = lesson_counter + 1 >>> lesson >>> end >>> >>> One extension is to add after to the comprehensions, which are computed >>> after the result is returned: >>> >>> {lessons, lesson_counter} = >>> for lesson <- section["lessons"], let: lesson_counter do >>> Map.put(lesson, "position", lesson_counter) >>> after >>> lesson_counter = lesson_counter + 1 >>> end >>> >>> This does not need to be part of the initial implementation. >>> Summary >>> >>> Feedback on the proposal and extensions is welcome! >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elixir-lang-core+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_elixir-2Dlang-2Dcore_CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD-5F-253DW-252BM9mSizFHJXFfqnQ-2540mail.gmail.com-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dfooter&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=St4TkXq_aqzXKmhDWcqAhlAryJ2Pcu8te1z1WQC3L-M&e=> >>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD_%3DW%2BM9mSizFHJXFfqnQ%40mail.gmail.com >>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_elixir-2Dlang-2Dcore_CAGnRm4JjyZ2EUcYm1TA747pP2pTgDAD-5F-253DW-252BM9mSizFHJXFfqnQ-2540mail.gmail.com&d=DwQFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=vhx95LgyiNVLl9R0jmAgrqx2u8MgCgBWzpuj0Tlk3b8&e=> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elixir-lang-core+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_elixir-2Dlang-2Dcore_CABu8xFAMx3hTfFbu-252B-252BN9XXnG5sEJ4zTSXdugSa96-2Du9uEGq-252B5g-2540mail.gmail.com-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dfooter&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=tSxdDRtNwUWvrRX8o4Evpxa-FvzcPfue-zsmJFqdy5I&e=> >> https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFAMx3hTfFbu%2B%2BN9XXnG5sEJ4zTSXdugSa96-u9uEGq%2B5g%40mail.gmail.com >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_elixir-2Dlang-2Dcore_CABu8xFAMx3hTfFbu-252B-252BN9XXnG5sEJ4zTSXdugSa96-2Du9uEGq-252B5g-2540mail.gmail.com&d=DwQFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=s8DPA-IUTYGBMqhvXYkAm0xDue05jiY2ymcX7TPq3Xc&e=> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_elixir-2Dlang-2Dcore_CAD3kWz99qSby-5F7PjBT-253D68ztzmX-2D2tq-252B-2DxV7RXL-5Fze4ZXsaFp7w-2540mail.gmail.com-3Futm-5Fmedium-3Demail-26utm-5Fsource-3Dfooter&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=CTKd6RYB_cGIZozBclEMFkS8HBaHr46sExxvgnqumSQ&e=> > https://groups.google.com/d/msgid/elixir-lang-core/CAD3kWz99qSby_7PjBT%3D68ztzmX-2tq%2B-xV7RXL_ze4ZXsaFp7w%40mail.gmail.com > <https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_elixir-2Dlang-2Dcore_CAD3kWz99qSby-5F7PjBT-253D68ztzmX-2D2tq-252B-2DxV7RXL-5Fze4ZXsaFp7w-2540mail.gmail.com&d=DwQFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=rmhwzhuTk1LyLPLZIrqkat6wS6r2qE3XZKnTTHGaxH8&m=i8qm3xa0UPPW40B0-cUGbTtJvXDo6Ahs7g6zY9e6RCo&s=7hNte96hyO1DB8iihl8Q4agL_7ZczvheCFv28E5dxB8&e=> > . > -- *Aleksei Matiushkin*, Software Engineer - R&D Office (+34) 935 679 834 8 Devonshire Square, London, EC2M 4PL, United Kingdom Torre Mapfre, Planta 22, Marina, 16-18, 08005 Barcelona, Spain *kantox.com <http://kantox.com/>* <http://www.linkedin.com/company/1871617> <http://www.linkedin.com/company/1871617>[image: LinkedIn] <https://www.linkedin.com/company/1871617> <https://twitter.com/kantox>[image: Twitter] <https://twitter.com/kantox> <http://www.youtube.com/user/kantoxfx>[image: YouTube] <https://www.youtube.com/user/kantoxfx> Kantox Limited is a UK private company with registered company number 07657495 and registered address at 8 Devonshire Square, London EC2M 4PL, United Kingdom. We are authorised with the UK Financial Conduct Authority (FCA) under the Payment Service Regulation 2017 as a Payments Institution (FRN 580343) for the provision of payment services and with HMRC as a Money Service Business Registration No.12641987. Kantox European Union, S.L. is a Spanish private company with tax ID number B67369371 and registered address at Torre Mapfre, Planta 22, Marina, 16-18, 08005 Barcelona, Spain. Kantox is authorized by the Bank of Spain, with registration number 6890, which is the supervisor of the Spanish banking system along with the European Central Bank. Additionally, we are supervised by SEPBLAC, the Supervisory Authority for the prevention of money laundering and terrorist financing in Spain. KANTOX is the Controller for the processing of data in accordance with the GDPR and LOPDGDD for the purpose of maintaining a commercial relationship. You may exercise your rights of access and rectification, portability, restriction and opposition by writing to KANTOX to the email: g...@kantox.com. You have your right to make a complaint at www.aepd.es. -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGF5_6fgNugS9sYdUjtc70Jk1GZ0kbW6V8C4FT-KNDCvy_rKFw%40mail.gmail.com.
