[jira] [Comment Edited] (FLINK-24314) Always use memory state backend with RocksDB
[
https://issues.apache.org/jira/browse/FLINK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416430#comment-17416430
]
shiwuliang edited comment on FLINK-24314 at 9/17/21, 4:18 AM:
--
[~zlzhang0122]
by the way, do u mean that I should config like this?
!image-2021-09-17-12-16-46-277.png!
Seems like this config is little confused. I should config checkpoint dir in
rocksdbstatebackend and set this config in checkpointconfig again
was (Author: shiwuliang):
[~zlzhang0122]
by the way, do u mean that I should config like this?
!image-2021-09-17-12-16-46-277.png!
> Always use memory state backend with RocksDB
>
>
> Key: FLINK-24314
> URL: https://issues.apache.org/jira/browse/FLINK-24314
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: shiwuliang
>Priority: Major
> Attachments: image-2021-09-17-10-59-50-094.png,
> image-2021-09-17-12-03-58-285.png, image-2021-09-17-12-04-55-009.png,
> image-2021-09-17-12-16-46-277.png
>
>
> When I config to use `RocksDBStatebackend`,
>
> {code:java}
> //代码占位符
> RocksDBStateBackend rocksDBStateBackend = new
> RocksDBStateBackend("hdfs://");
> streamEnv.setStateBackend(rocksDBStateBackend);{code}
>
> there are some exception like this:
> !image-2021-09-17-10-59-50-094.png|width=1446,height=420!
>
> Seems like the RocksdbStatebackend will use FsStateBackend to store
> checkpoints. So it means that I have used FileSystemStateBackend, why does
> this exception occur?
>
> I use flink 1.13.0 and found a similar question like this at:
> [https://stackoverflow.com/questions/68314652/flink-state-backend-config-with-the-state-processor-api]
>
> I'm not sure if his question is same with mine.
> I want to know how can I solve this and if it is indeed a 1.13.0 bug, how can
> I bypass it besides upgrading?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-24314) Always use memory state backend with RocksDB
[
https://issues.apache.org/jira/browse/FLINK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416426#comment-17416426
]
zlzhang0122 edited comment on FLINK-24314 at 9/17/21, 3:54 AM:
---
In Flink 1.13, the storage of the state and checkpoint was separated, so you
should set statebackend and checkpoint location at the same time. In your case,
you just set the statebackend, so it use memory to store the checkpoint as
default and since the size is beyound the limit, it throw an exception.
was (Author: zlzhang0122):
In Flink 1.13, the storage of the state and checkpoint was separated, so you
should set statebackend and checkpoint location at the same time. In your case,
you just set the statebackend, so it use memory to store the checkpoint as
default and since the size is beyound the limit, it report an exception.
> Always use memory state backend with RocksDB
>
>
> Key: FLINK-24314
> URL: https://issues.apache.org/jira/browse/FLINK-24314
> Project: Flink
> Issue Type: Bug
>Affects Versions: 1.13.0
>Reporter: shiwuliang
>Priority: Major
> Attachments: image-2021-09-17-10-59-50-094.png
>
>
> When I config to use `RocksDBStatebackend`,
>
> {code:java}
> //代码占位符
> RocksDBStateBackend rocksDBStateBackend = new
> RocksDBStateBackend("hdfs://");
> streamEnv.setStateBackend(rocksDBStateBackend);{code}
>
> there are some exception like this:
> !image-2021-09-17-10-59-50-094.png|width=1446,height=420!
>
> Seems like the RocksdbStatebackend will use FsStateBackend to store
> checkpoints. So it means that I have used FileSystemStateBackend, why does
> this exception occur?
>
> I use flink 1.13.0 and found a similar question like this at:
> [https://stackoverflow.com/questions/68314652/flink-state-backend-config-with-the-state-processor-api]
>
> I'm not sure if his question is same with mine.
> I want to know how can I solve this and if it is indeed a 1.13.0 bug, how can
> I bypass it besides upgrading?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
