Yes, I do use partition. The full flow is:
sequence
|> String.to_charlist
|> Enum.chunk(l, 1)
|> Flow.from_enumerable
|> Flow.partition
|> Flow.map(fn e -> Enum.chunk(e, k, 1) end)
|> Flow.map(
fn e ->
Enum.reduce(e, %{},
fn w, acc ->
Map.update(acc, w, 1, & &1 + 1)
end)
end)
|> Flow.flat_map(
fn e ->
Enum.reject(e, fn({_, n}) -> n < t end)
end)
|> Flow.map(fn({seq, _}) -> seq end)
|> Enum.to_list
On Wed, Nov 9, 2016 at 4:43 PM, José Valim <[email protected]>
wrote:
>
>
>> sequence
>> |> String.to_charlist
>> |> Enum.chunk(l, 1)
>> |> Flow.from_enumerable
>> |> Flow.map(fn e -> Enum.chunk(e, k, 1) end)
>>
>
> Do you call partition at some point in your flow? Otherwise it won't
> exploit parallelism if you have only one source. Also, if you need to chunk
> before you partition, you can chunk before calling from_enumerable:
>
> sequence
> |> String.to_charlist
> |> Stream.chunk(e, k, 1)
> |> Flow.from_enumerable
> |> ...
>
>
> I think it will be easy to add chunking to Flow because we can delegate to
> Stream but I just want to make sure I fully understand your use case and
> where parallelism is being introduced.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elixir-lang-core" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/elixir-lang-core/Avea6YFZLRQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/elixir-lang-core/CAGnRm4KpD-tf5p5sAS2nwZusd0reKdti-zL8-
> wzf%3DHjD8p%3D5qQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KpD-tf5p5sAS2nwZusd0reKdti-zL8-wzf%3DHjD8p%3D5qQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
Peter C. Marks
@PeterCMarks
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CA%2BKdhmg2EZt6SLgE8g_oH%2B-Jjpx075BmPOvfePSvjQZsitXVVg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
