The state size is not that huge. On the Flink UI when it showed the data sent as 4GB , the physical memory usage was close to 90GB ..
I will re-run by setting the Flushing options of RocksDB because I am facing this issue on 1.2.0 as well. Regards, Vinay Patil On Thu, Jun 29, 2017 at 9:03 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > Yup, I got that. I’m just wondering whether this occurs only with enabled > checkpointing or also when checkpointing is disabled. > > On 29. Jun 2017, at 17:31, Vinay Patil <vinay18.pa...@gmail.com> wrote: > > Hi Aljoscha, > > Yes I have tried with 1.2.1 and 1.3.0 , facing the same issue. > > The issue is not of Heap memory , it is of the Off-Heap memory that is > getting used ( please refer to the earlier snapshot I have attached in > which the graph keeps on growing ). > > > Regards, > Vinay Patil > > On Thu, Jun 29, 2017 at 8:55 PM, Aljoscha Krettek <aljos...@apache.org> > wrote: > >> Just a quick remark: Flink 1.3.0 and 1.2.1 always use FRocksDB, you >> shouldn’t manually specify that. >> >> On 29. Jun 2017, at 17:20, Vinay Patil <vinay18.pa...@gmail.com> wrote: >> >> Hi Gerry, >> >> Even I have faced this issue on 1.3.0 even by using FRocksDB and enabling >> incremental checkpointing. >> >> You can add FRocksDB dependency as shown here : >> https://github.com/apache/flink/pull/3704 >> >> We will have to set some RocksDB parameters to get this working. >> >> @Stefan or @Stephan : can you please help in resolving this issue >> >> Regards, >> Vinay Patil >> >> On Thu, Jun 29, 2017 at 6:01 PM, gerryzhou [via Apache Flink User Mailing >> List archive.] <ml+s2336050n1406...@n4.nabble.com> wrote: >> >>> Hi, Vinay, >>> I observed a similar problem in flink 1.3.0 with rocksdb. I wonder >>> how to use FRocksDB as you mentioned above. Thanks. >>> >>> ------------------------------ >>> If you reply to this email, your message will be added to the discussion >>> below: >>> http://apache-flink-user-mailing-list-archive.2336050.n4.nab >>> ble.com/Re-Checkpointing-with-RocksDB-as-statebackend-tp11752p14063.html >>> To start a new topic under Apache Flink User Mailing List archive., >>> email ml+s2336050n1...@n4.nabble.com >>> To unsubscribe from Apache Flink User Mailing List archive., click here >>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx> >>> . >>> NAML >>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>> >> >> >> > >
