[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14254100#comment-14254100 ] Cody Koeninger commented on SPARK-3146: --- This is a real problem for production use,

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252397#comment-14252397 ] Cody Koeninger commented on SPARK-3146: --- ((K, V), (topicAndPartition, offset)) is al

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252299#comment-14252299 ] Cody Koeninger commented on SPARK-3146: --- Yes, for the specific case of kafka, regard

[jira] [Commented] (SPARK-4122) Add library to write data back to Kafka

[ https://issues.apache.org/jira/browse/SPARK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252261#comment-14252261 ] Cody Koeninger commented on SPARK-4122: --- +1 for the idea of making this write from b

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252210#comment-14252210 ] Cody Koeninger commented on SPARK-3146: --- (1) is important because MessageAndMetadata

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252146#comment-14252146 ] Cody Koeninger commented on SPARK-3146: --- >From my point of view, the interceptor pat

[jira] [Updated] (SPARK-4229) Create hadoop configuration in a consistent way

[ https://issues.apache.org/jira/browse/SPARK-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-4229: -- Description: Some places use SparkHadoopUtil.get.conf, some create a new hadoop config. Prefer

[jira] [Created] (SPARK-4229) Create hadoop configuration in a consistent way

Cody Koeninger created SPARK-4229: - Summary: Create hadoop configuration in a consistent way Key: SPARK-4229 URL: https://issues.apache.org/jira/browse/SPARK-4229 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-4196) Streaming + checkpointing yields NotSerializableException for Hadoop Configuration from saveAsNewAPIHadoopFiles ?

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196641#comment-14196641 ] Cody Koeninger commented on SPARK-4196: --- Have you tried replacing JavaStreamingCont

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195146#comment-14195146 ] Cody Koeninger commented on SPARK-3146: --- I think this PR is an elegant way to solve

[jira] [Commented] (MESOS-123) Revisit the deploy scripts in src/deploy and update/improve them as necessary to make it very easy to start a new cluster

[ https://issues.apache.org/jira/browse/MESOS-123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14173810#comment-14173810 ] Cody Koeninger commented on MESOS-123: -- In the meantime, can we at least modify these

[jira] [Commented] (SPARK-3851) Support for reading parquet files with different but compatible schema

[ https://issues.apache.org/jira/browse/SPARK-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164191#comment-14164191 ] Cody Koeninger commented on SPARK-3851: --- So I have a couple of questions 1. Does i

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128682#comment-14128682 ] Cody Koeninger commented on SPARK-3462: --- Tested this on a cluster against unions of

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128451#comment-14128451 ] Cody Koeninger commented on SPARK-3462: --- Created a PR for feedback. https://github.

[jira] [Created] (SPARK-3462) parquet pushdown for unionAll

Cody Koeninger created SPARK-3462: - Summary: parquet pushdown for unionAll Key: SPARK-3462 URL: https://issues.apache.org/jira/browse/SPARK-3462 Project: Spark Issue Type: Improvement

<    1   2   3   4   5