The problem with what we currently have is that it requires two passes. There's no one-pass solution available via composition.
I spent quite a bit of time trying to get Stream.group_by working (back in my streamz days) so we can compose things like this in a single pass. It was rough, because it results in a stream of streams which makes it very complicated. It requires one process per key, which leads to a lot complexity around supervision/linking/monitoring and is sort of in it's own league compared to the rest of Stream. Working with RethinkDB I've seen an alternative approach. Rather than have group_by produce a map of keys to lists, it produces a somewhat monadic result. Applying map to a grouped stream doesn't iterate over keys, instead it applies the function to each group of values. It continues to do so until you call ungroup, at which point the result is an ordinary map of keys to lists. I'll try to find some time to take a stab at that approach, just to see whether it is at all possible. On Thu, Jun 23, 2016, 5:26 AM Martin Svalin <[email protected]> wrote: > tors 23 juni 2016 kl 00:01 skrev Peter Hamilton <[email protected] > >: > >> Enum.group_map_reduce(list, group_key_fun, map_fun, initial_acc, reducer) >> >> Enum.group_map_reduce([1,1,2,3,5,5,8], &(&1), &(&1), 0, fn _, acc -> acc >> + 1 end) >> >> That would produce: >> >> %{1 => 2, 2 => 1, 3 => 1, 5 => 2, 8 => 1} >> > > The whole point here is to do some transformation on all unique values, > right? So we can group unique values and transform them? > > I feel like this is a straightforward `list |> Enum.group_by(& &1) |> > Enum.map(&transformation/1)`. Possibly piped to Map.new. > For the case quoted above: `[1,1,2,3,5,5,8] |> Enum.group_by(& &1) |> > Enum.map(fn {k, v} -> {k, Enum.count v} end) |> Map.new`. > > I feel like this is well-composed of what we already have in Enum. There > are a bunch these "pre-composed" functions in Enum, like `reverse_slice`, > `map_join`, `filter_map` etc. It's unclear to me when and how the need for > these pre-composed functions arise. Are they all performance optimisations > to have common compositions done in one pass? Is this particular case > common enough to be included as `group_map_reduce`? > > Anyway, I would advice against the name `merge` or `merge_by`, as it > carries the connotation of combining two enumerables, the way `Map.merge` > does. > > - Martin > > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CAAHw6C%2BM47dureeMaQNvKEM3jcag8PBwJWrqi7f%3D705i7%2BTe1g%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CAAHw6C%2BM47dureeMaQNvKEM3jcag8PBwJWrqi7f%3D705i7%2BTe1g%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAOMhEnxfaV%3DJm6MW2rFR6MRGMRM5iXfZ5ogB4QMgqqOTx7XaDQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
