Hi,
Quick remark: operator chaining is only possible when the parallelism of the
upstream and downstream operators is the same. So having the same parallelism
is not the standard or desired way, it's the only way to achieve chaining.
Best,
Aljoscha
> On 16. Aug 2017, at 18:26, Karthik Deivasig
Thanks Urs for your inputs.
Yes we use AsyncIO operator for our webservice calls.
We were considering increasing the kafka partitions and increasing the
parallelism on the source to match the webservice operator. Wasn't quite
sure if this was the only way to achieve operator chaining. Thanks for
Hi Karthik,
maybe I'm misunderstanding, but there are a few things in your
description that seem strange to me:
- Your "slow" operator seems to be slow not because it's compute-heavy,
but because it's waiting for a response. Is AsyncIO (
https://ci.apache.org/projects/flink/flink-docs-release-1.3
Hi,
I have a use case where we read messages from a Kafka topic and invoke a
webservice. The web-service call can take a take couple of seconds and then
gives us back on avg 800KB of data. This data is set to another operator
which does the parsing and then it gets sent to sink which saves the
