with what's seen with state
processor API
Ok
Regards,
Alexis.
From: Peter Brucia
Sent: Freitag, 22. April 2022 15:31
To: Alexis Sarda-Espinosa
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
No
Sent from my iPhone
Ok
Regards,
Alexis.
From: Peter Brucia
Sent: Freitag, 22. April 2022 15:31
To: Alexis Sarda-Espinosa
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
No
Sent from my iPhone
s Sarda-Espinosa
Cc: ro...@apache.org ; user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Alexis,
Compaction isn't an all-at-once procedure. RocksDB is organized as a series of
levels, each 10x larger than the one below. There
ards,
> Alexis.
>
> -Original Message-
> From: Roman Khachatryan
> Sent: Mittwoch, 20. April 2022 10:37
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> processor API
>
>
Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
State Processor API works on a higher level and is not aware of any RocksDB
specifics (in fact, it can be used with any backend).
Regards,
Roman
On Tue, Apr 19, 2022
_
> From: Roman Khachatryan
> Sent: Tuesday, April 19, 2022 5:51 PM
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> processor API
>
> > I assume that when you say &quo
ds,
Alexis.
From: Roman Khachatryan
Sent: Tuesday, April 19, 2022 5:51 PM
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
> I assume that when you say "new states",
>
> -Original Message-----
> From: Roman Khachatryan
> Sent: Dienstag, 19. April 2022 11:48
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> processor API
>
> Hi Alexis,
>
> T
a-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Hi Alexis,
Thanks a lot for the information,
MANIFEST files list RocksDB column families (among other info); ever growing
size of these files might indicate that some new s
round?. My pipeline doesn't use timers,
> > although I guess Flink itself may use them. Is there any way I could get
> > some insight into which operator holds larger states?
> >
> > Regards,
> > Alexis.
> >
> > -Original Message-
> > From:
: Dienstag, 12. April 2022 15:39
To: ro...@apache.org
Cc: user@flink.apache.org
Subject: RE: RocksDB's state size discrepancy with what's seen with state
processor API
Thanks for all the pointers. The UI does show combined state for a chain, but
the only state descriptors inside that chain are the 3 I
in the state as well.
Regards,
Alexis.
-Original Message-
From: Roman Khachatryan
Sent: Dienstag, 12. April 2022 14:06
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
/shared folder contains ke
y use them. Is there any way I could get some
> insight into which operator holds larger states?
>
> Regards,
> Alexis.
>
> -Original Message-
> From: Roman Khachatryan
> Sent: Dienstag, 12. April 2022 12:37
> To: Alexis Sarda-Espinosa
> Cc: user@flink.apache.org
&
rom: Roman Khachatryan
Sent: Dienstag, 12. April 2022 12:37
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Hi Alexis,
Thanks a lot for sharing this. I think the program is correct.
Although it doesn't t
t;
>
> Regards,
>
> Alexis.
>
>
>
> From: Alexis Sarda-Espinosa
> Sent: Samstag, 9. April 2022 01:39
> To: ro...@apache.org
> Cc: user@flink.apache.org
> Subject: Re: RocksDB's state size discrepancy with what's seen with state
> processor API
>
Sarda-Espinosa
Sent: Samstag, 9. April 2022 01:39
To: ro...@apache.org
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processor API
Hi Roman,
Here's an example of a WindowReaderFunction:
public class StateReaderFunction extends
h
parallelism=4, this doesn't affect the result, does it?
Regards,
Alexis.
From: Roman Khachatryan
Sent: Friday, April 8, 2022 11:06 PM
To: Alexis Sarda-Espinosa
Cc: user@flink.apache.org
Subject: Re: RocksDB's state size discrepancy with what's seen with state
processo
Hi Alexis,
If I understand correctly, the provided StateProcessor program gives
you the number of stream elements per operator. However, you mentioned
that these operators have collection-type states (ListState and
MapState). That means that per one entry there can be an arbitrary
number of state
Hello,
I have a streaming job running on Flink 1.14.4 that uses managed state with
RocksDB with incremental checkpoints as backend. I've been monitoring a dev
environment that has been running for the last week and I noticed that state
size and end-to-end duration have been increasing
19 matches
Mail list logo
