[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752478#comment-16752478 ] Apache Spark commented on SPARK-26718: -- User 'linehrr' has created a pull request for this issue:

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752420#comment-16752420 ] Ryne Yang commented on SPARK-26718: --- [~kabhwan], [~gsomogyi] PR submitted : 

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-25 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16752353#comment-16752353 ] Ryne Yang commented on SPARK-26718: --- [~kabhwan] cool, I will make a PR soon. thank you.  > structured

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-24 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751706#comment-16751706 ] Gabor Somogyi commented on SPARK-26718: --- +1 on [~kabhwan] suggestion > structured streaming

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-24 Thread Jungtaek Lim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751694#comment-16751694 ] Jungtaek Lim commented on SPARK-26718: -- [~linehrr] Thanks for the analysis. I think allowing

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-24 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751629#comment-16751629 ] Ryne Yang commented on SPARK-26718: --- A simple fix would be a if statement to check if the integer

[jira] [Commented] (SPARK-26718) structured streaming fetched wrong current offset from kafka

2019-01-24 Thread Ryne Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751625#comment-16751625 ] Ryne Yang commented on SPARK-26718: --- found the issue, it's the rateLimit calculation. if anyone set