[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252210#comment-14252210 ] Cody Koeninger commented on SPARK-3146: --- (1) is important because MessageAndMetadata

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252146#comment-14252146 ] Cody Koeninger commented on SPARK-3146: --- >From my point of view, the interceptor pat

[jira] [Updated] (SPARK-4229) Create hadoop configuration in a consistent way

[ https://issues.apache.org/jira/browse/SPARK-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cody Koeninger updated SPARK-4229: -- Description: Some places use SparkHadoopUtil.get.conf, some create a new hadoop config. Prefer

[jira] [Created] (SPARK-4229) Create hadoop configuration in a consistent way

Cody Koeninger created SPARK-4229: - Summary: Create hadoop configuration in a consistent way Key: SPARK-4229 URL: https://issues.apache.org/jira/browse/SPARK-4229 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-4196) Streaming + checkpointing yields NotSerializableException for Hadoop Configuration from saveAsNewAPIHadoopFiles ?

[ https://issues.apache.org/jira/browse/SPARK-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196641#comment-14196641 ] Cody Koeninger commented on SPARK-4196: --- Have you tried replacing JavaStreamingCont

[jira] [Commented] (SPARK-3146) Improve the flexibility of Spark Streaming Kafka API to offer user the ability to process message before storing into BM

[ https://issues.apache.org/jira/browse/SPARK-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195146#comment-14195146 ] Cody Koeninger commented on SPARK-3146: --- I think this PR is an elegant way to solve

[jira] [Commented] (SPARK-3851) Support for reading parquet files with different but compatible schema

[ https://issues.apache.org/jira/browse/SPARK-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164191#comment-14164191 ] Cody Koeninger commented on SPARK-3851: --- So I have a couple of questions 1. Does i

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128682#comment-14128682 ] Cody Koeninger commented on SPARK-3462: --- Tested this on a cluster against unions of

[jira] [Commented] (SPARK-3462) parquet pushdown for unionAll

[ https://issues.apache.org/jira/browse/SPARK-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128451#comment-14128451 ] Cody Koeninger commented on SPARK-3462: --- Created a PR for feedback. https://github.

[jira] [Created] (SPARK-3462) parquet pushdown for unionAll

Cody Koeninger created SPARK-3462: - Summary: parquet pushdown for unionAll Key: SPARK-3462 URL: https://issues.apache.org/jira/browse/SPARK-3462 Project: Spark Issue Type: Improvement

<    1   2   3   4   5