[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15558550#comment-15558550
]
Cody Koeninger commented on SPARK-3146:
---
SPARK-4964 / the direct stream added a messageHandler.
>
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15139250#comment-15139250
]
Cody Koeninger commented on SPARK-3146:
---
I think this can be safely closed, given the messageHandler
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14258593#comment-14258593
]
Tathagata Das commented on SPARK-3146:
--
I am a little wary of adding any
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14258598#comment-14258598
]
Tathagata Das commented on SPARK-3146:
--
[~c...@koeninger.org] [~jerryshao] If either
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254100#comment-14254100
]
Cody Koeninger commented on SPARK-3146:
---
This is a real problem for production use,
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254123#comment-14254123
]
Hari Shreedharan commented on SPARK-3146:
-
For now, I am ok with just adding it to
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252113#comment-14252113
]
Hari Shreedharan commented on SPARK-3146:
-
I am a +1 for using something similar
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252146#comment-14252146
]
Cody Koeninger commented on SPARK-3146:
---
From my point of view, the interceptor
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252160#comment-14252160
]
Hari Shreedharan commented on SPARK-3146:
-
The difference between interceptor
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252210#comment-14252210
]
Cody Koeninger commented on SPARK-3146:
---
(1) is important because MessageAndMetadata
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252256#comment-14252256
]
Hari Shreedharan commented on SPARK-3146:
-
I understand what you mean by flatMap
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252299#comment-14252299
]
Cody Koeninger commented on SPARK-3146:
---
Yes, for the specific case of kafka,
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252319#comment-14252319
]
Hari Shreedharan commented on SPARK-3146:
-
If you look at the addDataWithCallback
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252397#comment-14252397
]
Cody Koeninger commented on SPARK-3146:
---
((K, V), (topicAndPartition, offset)) is
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252742#comment-14252742
]
Saisai Shao commented on SPARK-3146:
Hi all, thanks a lot for your comments. My
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195146#comment-14195146
]
Cody Koeninger commented on SPARK-3146:
---
I think this PR is an elegant way to solve
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119128#comment-14119128
]
Saisai Shao commented on SPARK-3146:
Hi [~tdas],
Sorry for late response, thanks a
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111614#comment-14111614
]
Tathagata Das commented on SPARK-3146:
--
I think this can be further generalized to
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103486#comment-14103486
]
Saisai Shao commented on SPARK-3146:
This issue can actually solve the problem
[
https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103520#comment-14103520
]
Apache Spark commented on SPARK-3146:
-
User 'jerryshao' has created a pull request for
20 matches
Mail list logo