[jira] [Commented] (SPARK-1462) Examples of ML algorithms are using deprecated APIs

[ https://issues.apache.org/jira/browse/SPARK-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13965275#comment-13965275 ] Nan Zhu commented on SPARK-1462: mainly Vector, better to be Vector.dense Examples of ML

[jira] [Commented] (SPARK-1399) Reason for Stage Failure should be shown in UI

[ https://issues.apache.org/jira/browse/SPARK-1399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13970351#comment-13970351 ] Nan Zhu commented on SPARK-1399: made the PR: https://github.com/apache/spark/pull/421

[jira] [Assigned] (SPARK-1483) Rename minSplits to minPartitions in public APIs

[ https://issues.apache.org/jira/browse/SPARK-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu reassigned SPARK-1483: -- Assignee: Nan Zhu Rename minSplits to minPartitions in public APIs

[jira] [Updated] (SPARK-1519) support minPartitions parameter of wholeTextFiles() in pyspark

[ https://issues.apache.org/jira/browse/SPARK-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-1519: --- Description: though Scala implementation provides the parameter of minPartitions in wholeTextFiles, PySpark

[jira] [Commented] (SPARK-1483) Rename minSplits to minPartitions in public APIs

[ https://issues.apache.org/jira/browse/SPARK-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13971961#comment-13971961 ] Nan Zhu commented on SPARK-1483: made the PR: https://github.com/apache/spark/pull/430

[jira] [Resolved] (SPARK-1523) improve the readability of code in AkkaUtil

[ https://issues.apache.org/jira/browse/SPARK-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu resolved SPARK-1523. Resolution: Fixed Fix Version/s: 1.1.0 improve the readability of code in AkkaUtil

[jira] [Created] (SPARK-1556) jets3t dependency is outdated

Nan Zhu created SPARK-1556: -- Summary: jets3t dependency is outdated Key: SPARK-1556 URL: https://issues.apache.org/jira/browse/SPARK-1556 Project: Spark Issue Type: Bug Components: Spark

[jira] [Created] (SPARK-1600) flaky test case in streaming.CheckpointSuite

Nan Zhu created SPARK-1600: -- Summary: flaky test case in streaming.CheckpointSuite Key: SPARK-1600 URL: https://issues.apache.org/jira/browse/SPARK-1600 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1600) flaky test case in streaming.CheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-1600: --- Description: the case recovery with file input stream.recovery with file input stream sometimes fails

[jira] [Created] (SPARK-1603) flaky test case in StreamingContextSuite

Nan Zhu created SPARK-1603: -- Summary: flaky test case in StreamingContextSuite Key: SPARK-1603 URL: https://issues.apache.org/jira/browse/SPARK-1603 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-1600) flaky test case in streaming.CheckpointSuite

[ https://issues.apache.org/jira/browse/SPARK-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-1600: --- Component/s: Streaming Affects Version/s: 0.9.1 1.0.0

[jira] [Commented] (SPARK-1299) making comments of RDD.doCheckpoint consistent with its usage

[ https://issues.apache.org/jira/browse/SPARK-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981819#comment-13981819 ] Nan Zhu commented on SPARK-1299: addressed in https://github.com/apache/spark/pull/186

[jira] [Resolved] (SPARK-1299) making comments of RDD.doCheckpoint consistent with its usage

[ https://issues.apache.org/jira/browse/SPARK-1299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu resolved SPARK-1299. Resolution: Fixed Fix Version/s: 1.0.0 making comments of RDD.doCheckpoint consistent with its

[jira] [Resolved] (SPARK-1298) Remove duplicate partition id checking

[ https://issues.apache.org/jira/browse/SPARK-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu resolved SPARK-1298. Resolution: Fixed Fix Version/s: 1.0.0 Remove duplicate partition id checking

[jira] [Commented] (SPARK-1175) on shutting down a long running job, the cluster does not accept new jobs and gets hung

[ https://issues.apache.org/jira/browse/SPARK-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988940#comment-13988940 ] Nan Zhu commented on SPARK-1175: this should have been fixed in

[jira] [Resolved] (SPARK-1175) on shutting down a long running job, the cluster does not accept new jobs and gets hung

[ https://issues.apache.org/jira/browse/SPARK-1175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu resolved SPARK-1175. Resolution: Fixed Fix Version/s: 1.0.0 on shutting down a long running job, the cluster does not

[jira] [Assigned] (SPARK-1706) Allow multiple executors per worker in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu reassigned SPARK-1706: -- Assignee: Nan Zhu Allow multiple executors per worker in Standalone mode

[jira] [Commented] (SPARK-1706) Allow multiple executors per worker in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989165#comment-13989165 ] Nan Zhu commented on SPARK-1706: made the PR: https://github.com/apache/spark/pull/636

[jira] [Created] (SPARK-1715) Ensure actor is self-contained in DAGScheduler

Nan Zhu created SPARK-1715: -- Summary: Ensure actor is self-contained in DAGScheduler Key: SPARK-1715 URL: https://issues.apache.org/jira/browse/SPARK-1715 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-1686) Master switches thread when ElectedLeader

[ https://issues.apache.org/jira/browse/SPARK-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu reassigned SPARK-1686: -- Assignee: Nan Zhu Master switches thread when ElectedLeader

[jira] [Assigned] (SPARK-1603) flaky test case in StreamingContextSuite

[ https://issues.apache.org/jira/browse/SPARK-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu reassigned SPARK-1603: -- Assignee: Nan Zhu flaky test case in StreamingContextSuite

[jira] [Commented] (SPARK-1603) flaky test case in StreamingContextSuite

[ https://issues.apache.org/jira/browse/SPARK-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990209#comment-13990209 ] Nan Zhu commented on SPARK-1603: [~tdas], I checked the code, the fix itself should be

[jira] [Commented] (SPARK-1603) flaky test case in StreamingContextSuite

[ https://issues.apache.org/jira/browse/SPARK-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13990216#comment-13990216 ] Nan Zhu commented on SPARK-1603: Ah, just made a PR at the same time,

[jira] [Commented] (SPARK-1771) CoarseGrainedSchedulerBackend is not resilient to Akka restarts

[ https://issues.apache.org/jira/browse/SPARK-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995962#comment-13995962 ] Nan Zhu commented on SPARK-1771: [#Aaron Davidson], I think there are basically two ways

[jira] [Closed] (SPARK-1821) Document History Server

[ https://issues.apache.org/jira/browse/SPARK-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu closed SPARK-1821. -- Resolution: Implemented sorry, missed some documents Document History Server ---

[jira] [Created] (SPARK-1821) Document History Server

Nan Zhu created SPARK-1821: -- Summary: Document History Server Key: SPARK-1821 URL: https://issues.apache.org/jira/browse/SPARK-1821 Project: Spark Issue Type: Bug Components: Deploy

[jira] [Updated] (SPARK-1821) Document History Server

[ https://issues.apache.org/jira/browse/SPARK-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-1821: --- Issue Type: Improvement (was: Bug) Document History Server --- Key:

[jira] [Created] (SPARK-1976) misleading streaming document

Nan Zhu created SPARK-1976: -- Summary: misleading streaming document Key: SPARK-1976 URL: https://issues.apache.org/jira/browse/SPARK-1976 Project: Spark Issue Type: Bug Affects Versions: 1.0.0

[jira] [Commented] (SPARK-732) Recomputation of RDDs may result in duplicated accumulator updates

[ https://issues.apache.org/jira/browse/SPARK-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029503#comment-14029503 ] Nan Zhu commented on SPARK-732: --- I actually made a PR long time ago

[jira] [Commented] (SPARK-2039) Run hadoop output checks for all formats

[ https://issues.apache.org/jira/browse/SPARK-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031745#comment-14031745 ] Nan Zhu commented on SPARK-2039: PR: https://github.com/apache/spark/pull/1088 Run

[jira] [Commented] (SPARK-2038) Don't shadow conf variable in saveAsHadoop functions

[ https://issues.apache.org/jira/browse/SPARK-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031746#comment-14031746 ] Nan Zhu commented on SPARK-2038: PR: https://github.com/apache/spark/pull/1087 Don't

[jira] [Commented] (SPARK-2126) Move MapOutputTracker behind ShuffleManager interface

[ https://issues.apache.org/jira/browse/SPARK-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032319#comment-14032319 ] Nan Zhu commented on SPARK-2126: [~matei], how about assigning it to me? I'm interested in

[jira] [Commented] (SPARK-1471) Worker not recognize Driver state at standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033984#comment-14033984 ] Nan Zhu commented on SPARK-1471: I will fix it right now Worker not recognize Driver

[jira] [Commented] (SPARK-1471) Worker not recognize Driver state at standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033986#comment-14033986 ] Nan Zhu commented on SPARK-1471: this has been fixed by

[jira] [Commented] (SPARK-2038) Don't shadow conf variable in saveAsHadoop functions

[ https://issues.apache.org/jira/browse/SPARK-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034739#comment-14034739 ] Nan Zhu commented on SPARK-2038: Ah, I seethat's fine... Don't shadow conf variable

[jira] [Commented] (SPARK-2038) Don't shadow conf variable in saveAsHadoop functions

[ https://issues.apache.org/jira/browse/SPARK-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037739#comment-14037739 ] Nan Zhu commented on SPARK-2038: [~pwendell] Yeah, it's a good idea, just submit a new PR:

[jira] [Commented] (SPARK-2126) Move MapOutputTracker behind ShuffleManager interface

[ https://issues.apache.org/jira/browse/SPARK-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037746#comment-14037746 ] Nan Zhu commented on SPARK-2126: [~pwendell] Yes, [~markhamstra] just emailed me Yes, I

[jira] [Commented] (SPARK-2126) Move MapOutputTracker behind ShuffleManager interface

[ https://issues.apache.org/jira/browse/SPARK-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045514#comment-14045514 ] Nan Zhu commented on SPARK-2126: PR: https://github.com/apache/spark/pull/1240 Move

[jira] [Commented] (SPARK-1525) TaskSchedulerImpl should decrease availableCpus by spark.task.cpus not 1

[ https://issues.apache.org/jira/browse/SPARK-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14049054#comment-14049054 ] Nan Zhu commented on SPARK-1525: this is fixed by

[jira] [Commented] (SPARK-2294) TaskSchedulerImpl and TaskSetManager do not properly prioritize which tasks get assigned to an executor

[ https://issues.apache.org/jira/browse/SPARK-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053329#comment-14053329 ] Nan Zhu commented on SPARK-2294: PR: https://github.com/apache/spark/pull/1313

[jira] [Created] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

Nan Zhu created SPARK-2404: -- Summary: spark-submit and spark-class may overwrite the already defined SPARK_HOME Key: SPARK-2404 URL: https://issues.apache.org/jira/browse/SPARK-2404 Project: Spark

[jira] [Updated] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-2404: --- Description: in spark-class and spark-submit the SPARK_HOME is set to the present working directory,

[jira] [Commented] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055165#comment-14055165 ] Nan Zhu commented on SPARK-2404: PR https://github.com/apache/spark/pull/1331

[jira] [Updated] (SPARK-2404) spark-submit and spark-class may overwrite the already defined SPARK_HOME

[ https://issues.apache.org/jira/browse/SPARK-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-2404: --- Description: in spark-class and spark-submit the SPARK_HOME is set to the present working directory,

[jira] [Created] (SPARK-2459) the user should be able to configure the resources used by JDBC server

Nan Zhu created SPARK-2459: -- Summary: the user should be able to configure the resources used by JDBC server Key: SPARK-2459 URL: https://issues.apache.org/jira/browse/SPARK-2459 Project: Spark

[jira] [Updated] (SPARK-2459) the user should be able to configure the resources used by JDBC server

[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-2459: --- Description: I'm trying the jdbc server I found that the jdbc server always occupies all cores in the

[jira] [Commented] (SPARK-2459) the user should be able to configure the resources used by JDBC server

[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059636#comment-14059636 ] Nan Zhu commented on SPARK-2459: I discussed with [~liancheng], he is working on merging

[jira] [Comment Edited] (SPARK-2459) the user should be able to configure the resources used by JDBC server

[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059636#comment-14059636 ] Nan Zhu edited comment on SPARK-2459 at 7/12/14 4:35 AM: - I

[jira] [Commented] (SPARK-2459) the user should be able to configure the resources used by JDBC server

[ https://issues.apache.org/jira/browse/SPARK-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061981#comment-14061981 ] Nan Zhu commented on SPARK-2459: yeah, spark-submit may solve the problem... then we need

[jira] [Commented] (SPARK-1706) Allow multiple executors per worker in Standalone mode

[ https://issues.apache.org/jira/browse/SPARK-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062966#comment-14062966 ] Nan Zhu commented on SPARK-1706: Oh, the PR has been there for a whileI just rebased

[jira] [Commented] (SPARK-2454) Separate driver spark home from executor spark home

[ https://issues.apache.org/jira/browse/SPARK-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063402#comment-14063402 ] Nan Zhu commented on SPARK-2454: this will make sparkHome as an application-specific

[jira] [Commented] (SPARK-2454) Separate driver spark home from executor spark home

[ https://issues.apache.org/jira/browse/SPARK-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064085#comment-14064085 ] Nan Zhu commented on SPARK-2454: I see, it makes sense to me... Separate driver spark

[jira] [Commented] (SPARK-2454) Separate driver spark home from executor spark home

[ https://issues.apache.org/jira/browse/SPARK-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069479#comment-14069479 ] Nan Zhu commented on SPARK-2454: there is a related issue and fix

[jira] [Commented] (SPARK-2456) Scheduler refactoring

[ https://issues.apache.org/jira/browse/SPARK-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14073122#comment-14073122 ] Nan Zhu commented on SPARK-2456: maybe it's also related:

[jira] [Created] (SPARK-2855) pyspark test cases crashed for no reason

Nan Zhu created SPARK-2855: -- Summary: pyspark test cases crashed for no reason Key: SPARK-2855 URL: https://issues.apache.org/jira/browse/SPARK-2855 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2855) pyspark test cases crashed for no reason

[ https://issues.apache.org/jira/browse/SPARK-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113778#comment-14113778 ] Nan Zhu commented on SPARK-2855: [~joshrosen]? pyspark test cases crashed for no reason

[jira] [Commented] (SPARK-2855) pyspark test cases crashed for no reason

[ https://issues.apache.org/jira/browse/SPARK-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113776#comment-14113776 ] Nan Zhu commented on SPARK-2855: I guess they have fixed this.Jenkins side mistake?

[jira] [Commented] (SPARK-2855) pyspark test cases crashed for no reason

[ https://issues.apache.org/jira/browse/SPARK-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113965#comment-14113965 ] Nan Zhu commented on SPARK-2855: no https://github.com/apache/spark/pull/1313 search

[jira] [Commented] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14146714#comment-14146714 ] Nan Zhu commented on SPARK-3628: https://github.com/apache/spark/pull/2524 Don't apply

[jira] [Commented] (SPARK-2647) DAGScheduler plugs others when processing one JobSubmitted event

[ https://issues.apache.org/jira/browse/SPARK-2647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147378#comment-14147378 ] Nan Zhu commented on SPARK-2647: isn't it the expected behaviour as we keep DAGScheduler

[jira] [Commented] (SPARK-3759) SparkSubmitDriverBootstrapper should return exit code of driver process

[ https://issues.apache.org/jira/browse/SPARK-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164595#comment-14164595 ] Nan Zhu commented on SPARK-3759: just passed by for merged PR, shall the status of

[jira] [Commented] (SPARK-3767) Support wildcard in Spark properties

[ https://issues.apache.org/jira/browse/SPARK-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164605#comment-14164605 ] Nan Zhu commented on SPARK-3767: do you mean something like -DpropertyName=EXEID, where

[jira] [Commented] (SPARK-3835) Spark applications that are killed should show up as KILLED or CANCELLED in the Spark UI

[ https://issues.apache.org/jira/browse/SPARK-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165768#comment-14165768 ] Nan Zhu commented on SPARK-3835: this problem still exists? I once reported the same thing

[jira] [Commented] (SPARK-3835) Spark applications that are killed should show up as KILLED or CANCELLED in the Spark UI

[ https://issues.apache.org/jira/browse/SPARK-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165807#comment-14165807 ] Nan Zhu commented on SPARK-3835: ah, I see, didn't look at your description closely

[jira] [Commented] (SPARK-3835) Spark applications that are killed should show up as KILLED or CANCELLED in the Spark UI

[ https://issues.apache.org/jira/browse/SPARK-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165815#comment-14165815 ] Nan Zhu commented on SPARK-3835: no...it cannot capture kill -9 Spark applications that

[jira] [Commented] (SPARK-3795) Add scheduler hooks/heuristics for adding and removing executors

[ https://issues.apache.org/jira/browse/SPARK-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167131#comment-14167131 ] Nan Zhu commented on SPARK-3795: this is for YARN or standalone? Add scheduler

[jira] [Commented] (SPARK-2962) Suboptimal scheduling in spark

[ https://issues.apache.org/jira/browse/SPARK-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167144#comment-14167144 ] Nan Zhu commented on SPARK-2962: Hi, [~mrid...@yahoo-inc.com] I think this has been fixed

[jira] [Commented] (SPARK-1192) Around 30 parameters in Spark are used but undocumented and some are having confusing name

[ https://issues.apache.org/jira/browse/SPARK-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169663#comment-14169663 ] Nan Zhu commented on SPARK-1192: yes, I resubmitted

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171358#comment-14171358 ] Nan Zhu commented on SPARK-3736: if the worker itself timeout, the Master will remove the

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171360#comment-14171360 ] Nan Zhu commented on SPARK-3736: BTW, master will not send heartbeat to Worker proactively

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174053#comment-14174053 ] Nan Zhu commented on SPARK-3957: I agree with [~andrewor14], I was also thinking about

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174664#comment-14174664 ] Nan Zhu commented on SPARK-3957: After looking at the problem more closely, I think we

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174675#comment-14174675 ] Nan Zhu commented on SPARK-3957: BlockId can directly tell if the corresponding block is a

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174684#comment-14174684 ] Nan Zhu commented on SPARK-3957: [~andrewor14], why we didn't report broadcast variable

[jira] [Comment Edited] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174684#comment-14174684 ] Nan Zhu edited comment on SPARK-3957 at 10/17/14 3:04 AM: --

[jira] [Commented] (SPARK-3957) Broadcast variable memory usage not reflected in UI

[ https://issues.apache.org/jira/browse/SPARK-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174747#comment-14174747 ] Nan Zhu commented on SPARK-3957: Ok, when i work on executor tab, i rwslize that, we

[jira] [Created] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

Nan Zhu created SPARK-4004: -- Summary: add akka-persistence based recovery mechanism for Master (maybe Worker) Key: SPARK-4004 URL: https://issues.apache.org/jira/browse/SPARK-4004 Project: Spark

[jira] [Updated] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-4004: --- Description: Since we have upgraded akka version to 2.3.x we can utilize the features which are actually

[jira] [Commented] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176571#comment-14176571 ] Nan Zhu commented on SPARK-4004: I will post a design doc after Monday add

[jira] [Updated] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-4004: --- Description: Since we have upgraded akka version to 2.3.x we can utilize the features which are actually

[jira] [Created] (SPARK-4011) tighten the visibility of the members in Worker class

Nan Zhu created SPARK-4011: -- Summary: tighten the visibility of the members in Worker class Key: SPARK-4011 URL: https://issues.apache.org/jira/browse/SPARK-4011 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4012) Uncaught OOM in ContextCleaner

Nan Zhu created SPARK-4012: -- Summary: Uncaught OOM in ContextCleaner Key: SPARK-4012 URL: https://issues.apache.org/jira/browse/SPARK-4012 Project: Spark Issue Type: Bug Components: Spark

[jira] [Created] (SPARK-4067) refactor ExecutorUncaughtExceptionHandler as a general one as it is used like this

Nan Zhu created SPARK-4067: -- Summary: refactor ExecutorUncaughtExceptionHandler as a general one as it is used like this Key: SPARK-4067 URL: https://issues.apache.org/jira/browse/SPARK-4067 Project: Spark

[jira] [Commented] (SPARK-4238) Perform network-level retry of shuffle file fetches

[ https://issues.apache.org/jira/browse/SPARK-4238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200334#comment-14200334 ] Nan Zhu commented on SPARK-4238: it is related to

[jira] [Commented] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222973#comment-14222973 ] Nan Zhu commented on SPARK-3628: hmmmOK but for this case, shall I submit individual

[jira] [Created] (SPARK-4971) fix typo in BlockGenerator

Nan Zhu created SPARK-4971: -- Summary: fix typo in BlockGenerator Key: SPARK-4971 URL: https://issues.apache.org/jira/browse/SPARK-4971 Project: Spark Issue Type: Improvement Components:

[jira] [Commented] (SPARK-4971) fix typo in BlockGenerator

[ https://issues.apache.org/jira/browse/SPARK-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14258997#comment-14258997 ] Nan Zhu commented on SPARK-4971: didn't they say every PR should be bind with a JIRA? not

[jira] [Commented] (SPARK-4971) fix typo in BlockGenerator

[ https://issues.apache.org/jira/browse/SPARK-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259004#comment-14259004 ] Nan Zhu commented on SPARK-4971: I see...thanks for the explanation fix typo in

[jira] [Updated] (SPARK-5268) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[ https://issues.apache.org/jira/browse/SPARK-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-5268: --- Summary: CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent (was: ExecutorBackend exits

[jira] [Created] (SPARK-5268) ExecutorBackend exits for irrelevant DisassociatedEvent

Nan Zhu created SPARK-5268: -- Summary: ExecutorBackend exits for irrelevant DisassociatedEvent Key: SPARK-5268 URL: https://issues.apache.org/jira/browse/SPARK-5268 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5268) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[ https://issues.apache.org/jira/browse/SPARK-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-5268: --- Priority: Blocker (was: Major) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[jira] [Commented] (SPARK-5293) Enable Spark user applications to use different versions of Akka

[ https://issues.apache.org/jira/browse/SPARK-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289258#comment-14289258 ] Nan Zhu commented on SPARK-5293: shall we make this JIRA as an umbrella task, so that the

[jira] [Updated] (SPARK-5268) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[ https://issues.apache.org/jira/browse/SPARK-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-5268: --- Target Version/s: 1.2.1 CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[jira] [Closed] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu closed SPARK-4004. -- Resolution: Won't Fix I'd close the PR as I saw some discussions in https://github.com/apache/spark/pull/3825

[jira] [Comment Edited] (SPARK-4004) add akka-persistence based recovery mechanism for Master (maybe Worker)

[ https://issues.apache.org/jira/browse/SPARK-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14274299#comment-14274299 ] Nan Zhu edited comment on SPARK-4004 at 1/12/15 10:30 PM: -- I'd

[jira] [Commented] (SPARK-5153) flaky test of Reliable Kafka input stream with multiple topics

[ https://issues.apache.org/jira/browse/SPARK-5153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270964#comment-14270964 ] Nan Zhu commented on SPARK-5153: [~saisai_shao], yeah, I got up early to check whether the

[jira] [Commented] (SPARK-5174) Missing Document for starting multiple workers/supervisors in actor-based receiver

[ https://issues.apache.org/jira/browse/SPARK-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270955#comment-14270955 ] Nan Zhu commented on SPARK-5174: can anyone assign this to me, I'd submit a PR to fix this

[jira] [Commented] (SPARK-5175) bug in updating counters when starting multiple workers/supervisors in actor-based receiver

[ https://issues.apache.org/jira/browse/SPARK-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270957#comment-14270957 ] Nan Zhu commented on SPARK-5175: can anyone assign this to me, I'd like to submit a PR on

[jira] [Created] (SPARK-5175) bug in updating counters when starting multiple workers/supervisors in actor-based receiver

Nan Zhu created SPARK-5175: -- Summary: bug in updating counters when starting multiple workers/supervisors in actor-based receiver Key: SPARK-5175 URL: https://issues.apache.org/jira/browse/SPARK-5175

[jira] [Created] (SPARK-5724) misconfiguration in Akka system

Nan Zhu created SPARK-5724: -- Summary: misconfiguration in Akka system Key: SPARK-5724 URL: https://issues.apache.org/jira/browse/SPARK-5724 Project: Spark Issue Type: Bug Components:

  1   2   3   >