[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733935#comment-16733935 ] Kaspar Tint commented on SPARK-26359: - [~gsomogyi]: Yes, this ticket can be closed. The solution

[jira] [Commented] (SPARK-26396) Kafka consumer cache overflow since 2.4.x

[ https://issues.apache.org/jira/browse/SPARK-26396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725685#comment-16725685 ] Kaspar Tint commented on SPARK-26396: - Yes, issue can be closed. I do think the documentation about

[jira] [Commented] (SPARK-26396) Kafka consumer cache overflow since 2.4.x

[ https://issues.apache.org/jira/browse/SPARK-26396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725104#comment-16725104 ] Kaspar Tint commented on SPARK-26396: - So in case there are four groupid's all consuming same topic

[jira] [Comment Edited] (SPARK-26396) Kafka consumer cache overflow since 2.4.x

[ https://issues.apache.org/jira/browse/SPARK-26396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725050#comment-16725050 ] Kaspar Tint edited comment on SPARK-26396 at 12/19/18 2:39 PM: --- Any exact

[jira] [Commented] (SPARK-26396) Kafka consumer cache overflow since 2.4.x

[ https://issues.apache.org/jira/browse/SPARK-26396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725050#comment-16725050 ] Kaspar Tint commented on SPARK-26396: - Any exact formula to use for this when considering that the

[jira] [Created] (SPARK-26396) Kafka consumer cache overflow since 2.4.x

Kaspar Tint created SPARK-26396: --- Summary: Kafka consumer cache overflow since 2.4.x Key: SPARK-26396 URL: https://issues.apache.org/jira/browse/SPARK-26396 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721213#comment-16721213 ] Kaspar Tint commented on SPARK-26359: - Means we changed the configuration

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721107#comment-16721107 ] Kaspar Tint commented on SPARK-26359: - This seems to be related to the fact that we decided to try

[jira] [Updated] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaspar Tint updated SPARK-26359: Environment: Spark 2.4.0 deployed in standalone-client mode Checkpointing is done to S3 The Spark

[jira] [Updated] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaspar Tint updated SPARK-26359: Attachment: worker-redacted state-redacted redacted-offsets

[jira] [Created] (SPARK-26359) Spark checkpoint restore fails after query restart

Kaspar Tint created SPARK-26359: --- Summary: Spark checkpoint restore fails after query restart Key: SPARK-26359 URL: https://issues.apache.org/jira/browse/SPARK-26359 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718912#comment-16718912 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 12:26 PM: Does it

[jira] [Commented] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718912#comment-16718912 ] Kaspar Tint commented on SPARK-25136: - Does it make sense for our S3 case to be a separate issue

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718732#comment-16718732 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:18 AM: Yes...

[jira] [Commented] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718732#comment-16718732 ] Kaspar Tint commented on SPARK-25136: - Yes... we are sadly aware of this also. Slowly transitioning

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:09 AM: Looks

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:10 AM: Looks

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:09 AM: Looks

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:09 AM: Looks

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:08 AM: Looks

[jira] [Comment Edited] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint edited comment on SPARK-25136 at 12/12/18 10:08 AM: Looks

[jira] [Commented] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718719#comment-16718719 ] Kaspar Tint commented on SPARK-25136: - Looks like we bumped into a similar issue with S3

[jira] [Commented] (SPARK-10881) Unable to use custom log4j appender in spark executor

[ https://issues.apache.org/jira/browse/SPARK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477398#comment-16477398 ] Kaspar Tint commented on SPARK-10881: - This seems to be an issue with 2.2.X Spark also. We did not