Re: [akka-user] Flow.mapAsync and downstream demand

2018-02-23 Thread Jeff 
Actually found a much simpler solution using Source.unfoldAsync. 

On Thursday, February 22, 2018 at 6:54:54 PM UTC-8, Jeff wrote:
>
> My particular use case is a long poll where the id to be sent on the next 
> request is returned from the previous request.
>
> On Thursday, February 22, 2018 at 6:40:48 PM UTC-8, Konrad Malawski wrote:
>>
>> In general relying on exact timing of pulls is rather seen as an anti 
>> pattern…
>> Even if you do a custom stage, it’s very brittle, as any element put 
>> before your stage could cause pulls, simply because that’s how things work.
>> The same can be said about simply changed buffer sizes between stages — 
>> introducing an async boundary anywhere in the pipeline can lead 
>> to more buffer space being allocated, and thus, more pulls being issued.
>>
>> In other words — due to the fact that backpressure is connected through 
>> an entire pipeline (and has to, otherwise it would not do what it should),
>> you can not rely on exact timing of pulls, because changes in the 
>> pipeline may affect this.
>>
>> In your very specific case perhaps it’s doable to make a stage or 
>> pipeline that’ll do the right thing, with enforcing it with zips and loops,
>> but that’ll be specific and take some work and very precise wording of 
>> what you want to achieve (sample code?)
>>
>> -- 
>> Cheers,
>> Konrad 'ktoso ' Malawski
>> Akka  @ Lightbend 
>>
>> On February 23, 2018 at 7:05:18, Jeff (jknigh...@gmail.com) wrote:
>>
>> Currently, Flow.mapAsync and Flow.mapAsyncUnordered always pull upstream 
>> even if there is no demand from downstream and then buffer. However, there 
>> are situations where one might want to way for explicit downstream demand 
>> before pulling. For example, let's say that the next upstream item depends 
>> on the results of the previous item and that you need to feed that back 
>> into the upstream before pulling again. In its current form, Flow.mapAsync 
>> will pull before that feedback loop has been completed.  
>>
>> Is there any way to work around that other than writing a custom 
>> GraphStage? 
>>
>> Thanks
>> Jeff
>> --
>> >> Read the docs: http://akka.io/docs/
>> >> Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> >> Search the archives: https://groups.google.com/group/akka-user
>> ---
>> You received this message because you are subscribed to the Google Groups 
>> "Akka User List" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to akka-user+...@googlegroups.com.
>> To post to this group, send email to akka...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/akka-user.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Flow.mapAsync and downstream demand

2018-02-22 Thread Jeff 
My particular use case is a long poll where the id to be sent on the next 
request is returned from the previous request.

On Thursday, February 22, 2018 at 6:40:48 PM UTC-8, Konrad Malawski wrote:
>
> In general relying on exact timing of pulls is rather seen as an anti 
> pattern…
> Even if you do a custom stage, it’s very brittle, as any element put 
> before your stage could cause pulls, simply because that’s how things work.
> The same can be said about simply changed buffer sizes between stages — 
> introducing an async boundary anywhere in the pipeline can lead 
> to more buffer space being allocated, and thus, more pulls being issued.
>
> In other words — due to the fact that backpressure is connected through an 
> entire pipeline (and has to, otherwise it would not do what it should),
> you can not rely on exact timing of pulls, because changes in the pipeline 
> may affect this.
>
> In your very specific case perhaps it’s doable to make a stage or pipeline 
> that’ll do the right thing, with enforcing it with zips and loops,
> but that’ll be specific and take some work and very precise wording of 
> what you want to achieve (sample code?)
>
> -- 
> Cheers,
> Konrad 'ktoso ' Malawski
> Akka  @ Lightbend 
>
> On February 23, 2018 at 7:05:18, Jeff (jknigh...@gmail.com ) 
> wrote:
>
> Currently, Flow.mapAsync and Flow.mapAsyncUnordered always pull upstream 
> even if there is no demand from downstream and then buffer. However, there 
> are situations where one might want to way for explicit downstream demand 
> before pulling. For example, let's say that the next upstream item depends 
> on the results of the previous item and that you need to feed that back 
> into the upstream before pulling again. In its current form, Flow.mapAsync 
> will pull before that feedback loop has been completed.  
>
> Is there any way to work around that other than writing a custom 
> GraphStage? 
>
> Thanks
> Jeff
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: 
> http://doc.akka.io/docs/akka/current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups 
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to akka-user+...@googlegroups.com .
> To post to this group, send email to akka...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] Flow.mapAsync and downstream demand

2018-02-22 Thread Konrad “ktoso” Malawski 
In general relying on exact timing of pulls is rather seen as an anti
pattern…
Even if you do a custom stage, it’s very brittle, as any element put before
your stage could cause pulls, simply because that’s how things work.
The same can be said about simply changed buffer sizes between stages —
introducing an async boundary anywhere in the pipeline can lead
to more buffer space being allocated, and thus, more pulls being issued.

In other words — due to the fact that backpressure is connected through an
entire pipeline (and has to, otherwise it would not do what it should),
you can not rely on exact timing of pulls, because changes in the pipeline
may affect this.

In your very specific case perhaps it’s doable to make a stage or pipeline
that’ll do the right thing, with enforcing it with zips and loops,
but that’ll be specific and take some work and very precise wording of what
you want to achieve (sample code?)

-- 
Cheers,
Konrad 'ktoso ' Malawski
Akka  @ Lightbend 

On February 23, 2018 at 7:05:18, Jeff (jknight12...@gmail.com) wrote:

Currently, Flow.mapAsync and Flow.mapAsyncUnordered always pull upstream
even if there is no demand from downstream and then buffer. However, there
are situations where one might want to way for explicit downstream demand
before pulling. For example, let's say that the next upstream item depends
on the results of the previous item and that you need to feed that back
into the upstream before pulling again. In its current form, Flow.mapAsync
will pull before that feedback loop has been completed.

Is there any way to work around that other than writing a custom
GraphStage?

Thanks
Jeff
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups
"Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.