[jira] [Commented] (FLINK-19099) consume kafka message repeat

2020-09-01 Thread Aljoscha Krettek (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188410#comment-17188410
 ] 

Aljoscha Krettek commented on FLINK-19099:
--

Did you completely cancel your job and start it fresh? Without starting it from 
a savepoint or retained checkpoint?

If you want the Flink job to keep the read offset you need to take a savepoint 
before stopping the job or stop it with a savepoint and then restart the job 
later from that savepoint.

> consume kafka message repeat
> 
>
> Key: FLINK-19099
> URL: https://issues.apache.org/jira/browse/FLINK-19099
> Project: Flink
>  Issue Type: Bug
>  Components: API / DataStream, Connectors / Kafka
>Affects Versions: 1.11.0
>Reporter: zouwenlong
>Priority: Major
> Attachments: image-2020-08-31-19-23-59-038.png, 
> image-2020-08-31-19-25-09-548.png
>
>
> when taksmanager be killed ,my job consume some message , but the offset in 
> not commit,
> then restart it ,my job consume kafka message repeat,  I used checkpoint  and 
> set 5 seconds ,
> I think this is a very common problem,how to solve this problem?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (FLINK-19099) consume kafka message repeat

2020-08-31 Thread zouwenlong (Jira)


[ 
https://issues.apache.org/jira/browse/FLINK-19099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187653#comment-17187653
 ] 

zouwenlong commented on FLINK-19099:


[~danny0405] I used data stream,not SQL 

!image-2020-08-31-19-23-59-038.png!

> consume kafka message repeat
> 
>
> Key: FLINK-19099
> URL: https://issues.apache.org/jira/browse/FLINK-19099
> Project: Flink
>  Issue Type: Bug
>  Components: API / DataStream, Connectors / Kafka
>Affects Versions: 1.11.0
>Reporter: zouwenlong
>Priority: Major
> Attachments: image-2020-08-31-19-23-59-038.png
>
>
> when taksmanager be killed ,my job consume some message , but the offset in 
> not commit,
> then restart it ,my job consume kafka message repeat,  I used checkpoint  and 
> set 5 seconds ,
> I think this is a very common problem,how to solve this problem?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)