[
https://issues.apache.org/jira/browse/SPARK-16244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-16244:
---
Description:
This streaming job has a very simple DAG. Each batch have only 1 job, and each
job has
Liyin Tang created SPARK-16244:
--
Summary: Failed job/stage couldn't stop JobGenerator immediately.
Key: SPARK-16244
URL: https://issues.apache.org/jira/browse/SPARK-16244
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-14230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15218672#comment-15218672
]
Liyin Tang commented on SPARK-14230:
[~davies], thanks for the response. If I understand it
Liyin Tang created SPARK-14230:
--
Summary: Config the start time (jitter) for streaming jobs
Key: SPARK-14230
URL: https://issues.apache.org/jira/browse/SPARK-14230
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang closed SPARK-14105.
--
Resolution: Won't Fix
Avoid serializing MessageAndMetadata.
Easy workaround:
{code}
val
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Comment: was deleted
(was: Avoid serializing MessageAndMetadata.
Easy workaround:
{code}
val
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211999#comment-15211999
]
Liyin Tang commented on SPARK-14105:
Thanks [~c...@koeninger.org] and [~sowen] for the review and
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Summary: Serialization issue for MessageAndMetadata in KafkaRDD (was:
Serialization issue for
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Description:
When using DISK or Memory to persistent KafkaDirectInputStream, it will
serialize the
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Description:
When using DISK or Memory to persistent KafkaDirectInputStream, it will
serialize the
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211147#comment-15211147
]
Liyin Tang commented on SPARK-14105:
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Description:
When using DISK or Memory to persistent KafkaDirectInputStream, it will
serialize the
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Comment: was deleted
(was: Copied from PR discussion:
===
That example is just for
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210864#comment-15210864
]
Liyin Tang commented on SPARK-14105:
Copied from PR discussion:
===
That example is just
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210734#comment-15210734
]
Liyin Tang edited comment on SPARK-14105 at 3/24/16 6:35 PM:
-
Add the min
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15210734#comment-15210734
]
Liyin Tang commented on SPARK-14105:
Add the min code example to reproduce this issue.
There is
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Description:
When using DISK or Memory to persistent KafkaDirectInputStream, it will
serialize the
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209594#comment-15209594
]
Liyin Tang commented on SPARK-14105:
I tested the PR with the same streaming job, and verified the
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Affects Version/s: 1.6.1
> Serialization issue for KafkaRDD
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Attachment: Screenshot 2016-03-23 09.04.51.png
The screenshot of KafkaRDD's serialization
>
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Component/s: (was: Spark Core)
> Serialization issue for KafkaRDD
>
[
https://issues.apache.org/jira/browse/SPARK-14105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-14105:
---
Affects Version/s: 1.5.2
Component/s: Streaming
Spark Core
>
Liyin Tang created SPARK-14105:
--
Summary: Serialization issue for KafkaRDD
Key: SPARK-14105
URL: https://issues.apache.org/jira/browse/SPARK-14105
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-13580:
---
Summary: Driver makes no progress when Executor's akka thread exits due to
OOM. (was: Driver makes
[
https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15172810#comment-15172810
]
Liyin Tang commented on SPARK-13580:
Thanks [~zsxwing] for the investigation! That's very helpful !
[
https://issues.apache.org/jira/browse/SPARK-13580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liyin Tang updated SPARK-13580:
---
Attachment: executor_jstack
driver_log.txt
driver_jstack.txt
>
Liyin Tang created SPARK-13580:
--
Summary: Driver makes no progress after failed to remove broadcast
on Executor
Key: SPARK-13580
URL: https://issues.apache.org/jira/browse/SPARK-13580
Project: Spark
27 matches
Mail list logo
