[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489962#comment-17489962 ] Matthias J. Sax commented on KAFKA-13289: - Great news! Thanks for sharing. Happy you got your

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487820#comment-17487820 ] Eugen Dück commented on KAFKA-13289: [~mjsax] Our Streams app has now been running in production for

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17474910#comment-17474910 ] Matthias J. Sax commented on KAFKA-13289: - {quote}However it seems the dropping of messages is

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17473235#comment-17473235 ] Eugen Dück commented on KAFKA-13289: {quote} bq. The app uses a 0ms join window with a 0ms That is

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17473149#comment-17473149 ] Matthias J. Sax commented on KAFKA-13289: - {quote}The app uses a 0ms join window with a 0ms

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17472424#comment-17472424 ] Eugen Dück commented on KAFKA-13289: I (accidentally) noticed that one way to reproduce "Skipping

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17471729#comment-17471729 ] Eugen Dück commented on KAFKA-13289: After reading

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17471710#comment-17471710 ] Eugen Dück commented on KAFKA-13289: Meanwhile I could reproduce the "Skipping record for expired

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469016#comment-17469016 ] Eugen Dück commented on KAFKA-13289: Yes, if you use Matthew's repo, which gives the "Skipping

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469010#comment-17469010 ] Matthias J. Sax commented on KAFKA-13289: - You can reproduce with the code in the github repo

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468913#comment-17468913 ] Eugen Dück commented on KAFKA-13289: [~mjsax]  I just retried with confluentinc/cp-kafka:7.0.1,

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468912#comment-17468912 ] Eugen Dück commented on KAFKA-13289: [~mjsax]  I tried setting `max.task.idle.ms` like so:

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468879#comment-17468879 ] Matthias J. Sax commented on KAFKA-13289: - {quote}However in the cases mentioned here, there is

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17464313#comment-17464313 ] Eugen Dück commented on KAFKA-13289: As the "Skipping record for expired segment" log indicates the

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17464274#comment-17464274 ] Eugen Dück commented on KAFKA-13289: As the "Skipping record for expired segment" log indicates the

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447678#comment-17447678 ] Eugen Dück commented on KAFKA-13289: Matthias To quote from the issue: > left/outer join result

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447676#comment-17447676 ] Eugen Dück commented on KAFKA-13289: Thanks Matthew for your comments! Too bad Kafka Streams didn't

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447566#comment-17447566 ] Matthias J. Sax commented on KAFKA-13289: - [~eugendueck] could your issue be related to

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447243#comment-17447243 ] Matthew Sheppard commented on KAFKA-13289: -- Hi Eugen, I'm unsure I'm afraid. The key problem I

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17447198#comment-17447198 ] Eugen Dück commented on KAFKA-13289: Matthias, Matthew If you have an opinion on whether you think

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432191#comment-17432191 ] Eugen Dück commented on KAFKA-13289: So the gaps are 100% reproducible. Up until 690 messages there

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431059#comment-17431059 ] Eugen Dück commented on KAFKA-13289: [~msheppard] You are right - my bad! I have changed the number

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431049#comment-17431049 ] Matthew Sheppard commented on KAFKA-13289: -- Hi Eugen - Can you point out for me specifically

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430275#comment-17430275 ] Eugen Dück commented on KAFKA-13289: We are running into similar issues: * lots of "Skipping record

[jira] [Commented] (KAFKA-13289) Bulk processing correctly ordered input data through a join with kafka-streams results in `Skipping record for expired segment`

[ https://issues.apache.org/jira/browse/KAFKA-13289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413408#comment-17413408 ] Matthias J. Sax commented on KAFKA-13289: - I guess that the repartition topic could be the