[jira] [Commented] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

[ https://issues.apache.org/jira/browse/SPARK-25005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637560#comment-16637560 ] Quentin Ambard commented on SPARK-25005: ok I see, great idea, and the consumer

[jira] [Commented] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

[ https://issues.apache.org/jira/browse/SPARK-25005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16637532#comment-16637532 ] Quentin Ambard commented on SPARK-25005: How do you make difference between data

[jira] [Created] (SPARK-25005) Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers)

Quentin Ambard created SPARK-25005: -- Summary: Structured streaming doesn't support kafka transaction (creating empty offset with abort & markers) Key: SPARK-25005 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Updated] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Ambard updated SPARK-24720: --- Component/s: (was: Structured Streaming) DStreams > kafka transaction c

[jira] [Commented] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563256#comment-16563256 ] Quentin Ambard commented on SPARK-24720: Maybe we could improve that. As we alre

[jira] [Commented] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562464#comment-16562464 ] Quentin Ambard commented on SPARK-24720: Something is wrong with the way the con

[jira] [Commented] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534838#comment-16534838 ] Quentin Ambard commented on SPARK-24720: I'm not sure how we could implement a r

[jira] [Updated] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Ambard updated SPARK-24720: --- Description: When kafka transactions are used, sending 1 message to kafka will result to 1

[jira] [Updated] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Ambard updated SPARK-24720: --- Description: When kafka transactions are used, sending 1 message to kafka will result to 1

[jira] [Updated] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Ambard updated SPARK-24720: --- External issue ID: (was: SPARK-17147) > kafka transaction creates Non-consecutive Offsets

[jira] [Created] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

Quentin Ambard created SPARK-24720: -- Summary: kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true Key: SPARK-24720 URL: https://issues.apac

[jira] [Updated] (SPARK-24720) kafka transaction creates Non-consecutive Offsets (due to transaction offset) making streaming fail when failOnDataLoss=true

[ https://issues.apache.org/jira/browse/SPARK-24720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Ambard updated SPARK-24720: --- External issue ID: SPARK-17147 > kafka transaction creates Non-consecutive Offsets (due to t