[jira] [Created] (SPARK-44346) Python worker exited unexpectedly - java.io.EOFException on DataInputStream.readInt - cluster doesn't terminate

Dmitry Goldenberg created SPARK-44346: - Summary: Python worker exited unexpectedly - java.io.EOFException on DataInputStream.readInt - cluster doesn't terminate Key: SPARK-44346 URL:

[jira] [Commented] (SPARK-38557) What may be a cause for HDFSMetadataCommitter: Error while fetching MetaData and how to fix or work around this?

[ https://issues.apache.org/jira/browse/SPARK-38557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17507263#comment-17507263 ] Dmitry Goldenberg commented on SPARK-38557: --- Likely a DUP of

[jira] [Created] (SPARK-38557) What may be a cause for HDFSMetadataCommitter: Error while fetching MetaData and how to fix or work around this?

Dmitry Goldenberg created SPARK-38557: - Summary: What may be a cause for HDFSMetadataCommitter: Error while fetching MetaData and how to fix or work around this? Key: SPARK-38557 URL:

[jira] [Commented] (SPARK-36958) Reading of legacy timestamps from Parquet confusing in Spark 3, related config values don't seem working

[ https://issues.apache.org/jira/browse/SPARK-36958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426571#comment-17426571 ] Dmitry Goldenberg commented on SPARK-36958: --- [~hyukjin.kwon] Hi, so which setting out of the

[jira] [Created] (SPARK-36958) Reading of legacy timestamps from Parquet confusing in Spark 3, related config values don't seem working

Dmitry Goldenberg created SPARK-36958: - Summary: Reading of legacy timestamps from Parquet confusing in Spark 3, related config values don't seem working Key: SPARK-36958 URL:

[jira] [Comment Edited] (SPARK-8665) Update ALS documentation to include performance tips

[ https://issues.apache.org/jira/browse/SPARK-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044607#comment-17044607 ] Dmitry Goldenberg edited comment on SPARK-8665 at 2/25/20 4:04 PM: --- Any

[jira] [Commented] (SPARK-8665) Update ALS documentation to include performance tips

[ https://issues.apache.org/jira/browse/SPARK-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044607#comment-17044607 ] Dmitry Goldenberg commented on SPARK-8665: -- Any reason why this is marked as resolved?

[jira] [Resolved] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg resolved SPARK-27567. --- Resolution: Not A Bug In our case, issue was caused by someone changing the IP

[jira] [Commented] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855996#comment-16855996 ] Dmitry Goldenberg commented on SPARK-27567: --- OK, it appears that the IP address of the box in

[jira] [Reopened] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg reopened SPARK-27567: --- Issue not resolved, it appears intermittently. We have a QA box with Kafka installed

[jira] [Commented] (SPARK-8526) Provide a way to define custom metrics and custom metric sink in Spark

[ https://issues.apache.org/jira/browse/SPARK-8526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844761#comment-16844761 ] Dmitry Goldenberg commented on SPARK-8526: -- Any reason why this is marked as 'resolved'? What's

[jira] [Commented] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16829555#comment-16829555 ] Dmitry Goldenberg commented on SPARK-27567: --- Hi Liang-Chi, So you must be referring to this

[jira] [Updated] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

[ https://issues.apache.org/jira/browse/SPARK-27567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-27567: -- Description: Some of our consumers intermittently die with the stack traces I'm

[jira] [Created] (SPARK-27567) Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set'

Dmitry Goldenberg created SPARK-27567: - Summary: Spark Streaming consumers (from Kafka) intermittently die with 'SparkException: Couldn't find leaders for Set' Key: SPARK-27567 URL:

[jira] [Commented] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

[ https://issues.apache.org/jira/browse/SPARK-27529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16826072#comment-16826072 ] Dmitry Goldenberg commented on SPARK-27529: --- Hi Hyukjin, I could check although it seems this

[jira] [Updated] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

[ https://issues.apache.org/jira/browse/SPARK-27529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-27529: -- Description: We have a Spark Streaming consumer which at a certain point started

[jira] [Updated] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

[ https://issues.apache.org/jira/browse/SPARK-27529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-27529: -- Description: We have a Spark Streaming consumer which at a certain point started

[jira] [Updated] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

[ https://issues.apache.org/jira/browse/SPARK-27529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-27529: -- Description: We have a Spark Streaming consumer which at a certain point started

[jira] [Created] (SPARK-27529) Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException

Dmitry Goldenberg created SPARK-27529: - Summary: Spark Streaming consumer dies with kafka.common.OffsetOutOfRangeException Key: SPARK-27529 URL: https://issues.apache.org/jira/browse/SPARK-27529

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646419#comment-14646419 ] Dmitry Goldenberg commented on SPARK-9434: -- Thanks Cody and Sean. I'll take a

[jira] [Resolved] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg resolved SPARK-9434. -- Resolution: Not A Problem Documentation on checkpointing is available in the Spark doc

[jira] [Created] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

Dmitry Goldenberg created SPARK-9434: Summary: Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API Key: SPARK-9434

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645923#comment-14645923 ] Dmitry Goldenberg commented on SPARK-9434: -- Sean, I specifically did not want to

[jira] [Reopened] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg reopened SPARK-9434: -- Not resolved. Need how-to for resuming direct Kafka streaming consumers where they had

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645971#comment-14645971 ] Dmitry Goldenberg commented on SPARK-9434: -- [~tdas]: Could you weigh in on this

[jira] [Comment Edited] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645971#comment-14645971 ] Dmitry Goldenberg edited comment on SPARK-9434 at 7/29/15 12:30 PM:

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645992#comment-14645992 ] Dmitry Goldenberg commented on SPARK-9434: -- @Sean Owen: Per your point about

[jira] [Comment Edited] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645992#comment-14645992 ] Dmitry Goldenberg edited comment on SPARK-9434 at 7/29/15 12:50 PM:

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646021#comment-14646021 ] Dmitry Goldenberg commented on SPARK-9434: -- Great, thanks, Sean. Perhaps Cody

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646033#comment-14646033 ] Dmitry Goldenberg commented on SPARK-9434: -- Ah. I think this is making sense then

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646045#comment-14646045 ] Dmitry Goldenberg commented on SPARK-9434: -- Perhaps the Kafka user guide could

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14646047#comment-14646047 ] Dmitry Goldenberg commented on SPARK-9434: -- Yup, Cody, just connected the dots

[jira] [Commented] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645965#comment-14645965 ] Dmitry Goldenberg commented on SPARK-9434: -- [~sowen]: Putting this here was the

[jira] [Comment Edited] (SPARK-9434) Need how-to for resuming direct Kafka streaming consumers where they had left off before getting terminated, OR actual support for that mode in the Streaming API

[ https://issues.apache.org/jira/browse/SPARK-9434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645971#comment-14645971 ] Dmitry Goldenberg edited comment on SPARK-9434 at 7/29/15 12:30 PM:

[jira] [Created] (SPARK-8526) Provide a way to define custom metrics and custom metric sink in Spark

Dmitry Goldenberg created SPARK-8526: Summary: Provide a way to define custom metrics and custom metric sink in Spark Key: SPARK-8526 URL: https://issues.apache.org/jira/browse/SPARK-8526

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516241#comment-14516241 ] Dmitry Goldenberg commented on SPARK-7154: -- I've added this info to the 'mirror'

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516242#comment-14516242 ] Dmitry Goldenberg commented on SPARK-7154: -- Could this possibly be documented in

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514303#comment-14514303 ] Dmitry Goldenberg commented on SPARK-7154: -- Thanks, Sean. What is the right way

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14514314#comment-14514314 ] Dmitry Goldenberg commented on SPARK-7154: -- Trying to use --driver-java-options

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513303#comment-14513303 ] Dmitry Goldenberg commented on SPARK-7154: -- Not sure if this may be an HBase

[jira] [Comment Edited] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513217#comment-14513217 ] Dmitry Goldenberg edited comment on SPARK-7154 at 4/26/15 7:37 PM:

[jira] [Updated] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-7154: - Attachment: spark-1.3.1-local-build.txt This maven output includes [INFO] Including

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513218#comment-14513218 ] Dmitry Goldenberg commented on SPARK-7154: -- This is really a blocker for me, I

[jira] [Updated] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-7154: - Priority: Critical (was: Major) Spark distro appears to be pulling in incorrect

[jira] [Created] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

Dmitry Goldenberg created SPARK-7154: Summary: Spark distro appears to be pulling in incorrect protobuf classes Key: SPARK-7154 URL: https://issues.apache.org/jira/browse/SPARK-7154 Project:

[jira] [Commented] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14513208#comment-14513208 ] Dmitry Goldenberg commented on SPARK-7154: -- As a workaround, I just tried setting

[jira] [Updated] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-7154: - Attachment: in-spark-1.3.1-local-build.zip in-google-protobuf-2.5.0.zip

[jira] [Updated] (SPARK-7154) Spark distro appears to be pulling in incorrect protobuf classes

[ https://issues.apache.org/jira/browse/SPARK-7154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Goldenberg updated SPARK-7154: - Affects Version/s: (was: 1.3.1) 1.3.0 Spark distro appears to