Hi,
I think there are a couple of potential explanations for the increased latency.
Let me point out two of the most obvious that come to my mind:
1) A state size of 20 MB sounds like something that could (completely or to a
large extend) fit into some cache layer of a modern CPU, whereas 200
From: Biplob Biswas [mailto:revolutioni...@gmail.com]
Sent: יום ב 30 אוקטובר 2017 11:02
To: Sofer, Tovi [ICG-IT] <ts72...@imceu.eu.ssmb.com>
Cc: Narendra Joshi <narendr...@gmail.com>; user <user@flink.apache.org>
Subject: Re: state size effects latency
Hi Tovi,
This might s
ailto:narendr...@gmail.com]
> *Sent:* יום א 29 אוקטובר 2017 12:13
> *To:* Sofer, Tovi [ICG-IT] <ts72...@imceu.eu.ssmb.com>
> *Cc:* user <user@flink.apache.org>
> *Subject:* Re: state size effects latency
>
>
>
> We have also faced similar issues. The only thing that
s72...@imceu.eu.ssmb.com>
Cc: user <user@flink.apache.org>
Subject: Re: state size effects latency
We have also faced similar issues. The only thing that happens in sync when
using async snaphots is getting a persistent point in time picture which in
case of rocksdb backend is making symlinks
We have also faced similar issues. The only thing that happens in sync when
using async snaphots is getting a persistent point in time picture which in
case of rocksdb backend is making symlinks. That would linearly increase
with number of files to symlink but this should be negligible. We could
