Devaraj K created SPARK-26737:
-
Summary: Executor/Task STDERR & STDOUT log urls are not correct in
Yarn deployment mode
Key: SPARK-26737
URL: https://issues.apache.org/jira/browse/SPARK-26737
Project:
Devaraj K created SPARK-26650:
-
Summary: Yarn Client throws 'ClassNotFoundException:
org.apache.hadoop.hbase.HBaseConfiguration'
Key: SPARK-26650
URL: https://issues.apache.org/jira/browse/SPARK-26650
[
https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652687#comment-16652687
]
Devaraj K commented on SPARK-24787:
---
It seems here the overhead is coming due the force call
Devaraj K created SPARK-25683:
-
Summary: Make AsyncEventQueue.lastReportTimestamp inital value as
the currentTime instead of 0
Key: SPARK-25683
URL: https://issues.apache.org/jira/browse/SPARK-25683
[
https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K resolved SPARK-25645.
---
Resolution: Duplicate
> Add provision to disable EventLoggingListener default flush/hsync/hflush
[
https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639079#comment-16639079
]
Devaraj K commented on SPARK-25645:
---
{code:java|title=with hflush(no hsync)|borderStyle=solid}
[
https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639046#comment-16639046
]
Devaraj K commented on SPARK-25645:
---
Thanks [~vanzin] for the jira pointer, I haven't tried just with
Devaraj K created SPARK-25645:
-
Summary: Add provision to disable EventLoggingListener default
flush/hsync/hflush for all events
Key: SPARK-25645
URL: https://issues.apache.org/jira/browse/SPARK-25645
[
https://issues.apache.org/jira/browse/SPARK-25645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16639005#comment-16639005
]
Devaraj K commented on SPARK-25645:
---
{code:java|title=Present Behavior(flushLogger=true for some
Devaraj K created SPARK-25637:
-
Summary: SparkException: Could not find CoarseGrainedScheduler
occurs during the application stop
Key: SPARK-25637
URL: https://issues.apache.org/jira/browse/SPARK-25637
Devaraj K created SPARK-25636:
-
Summary: spark-submit swallows the failure reason when there is an
error connecting to master
Key: SPARK-25636
URL: https://issues.apache.org/jira/browse/SPARK-25636
[
https://issues.apache.org/jira/browse/SPARK-25246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16615398#comment-16615398
]
Devaraj K commented on SPARK-25246:
---
I think it is not a problem, the behavior might be based on the
Devaraj K created SPARK-25009:
-
Summary: Standalone Cluster mode application submit is not working
Key: SPARK-25009
URL: https://issues.apache.org/jira/browse/SPARK-25009
Project: Spark
Issue
Devaraj K created SPARK-24129:
-
Summary: Add option to pass --build-arg's to docker-image-tool.sh
Key: SPARK-24129
URL: https://issues.apache.org/jira/browse/SPARK-24129
Project: Spark
Issue
Devaraj K created SPARK-24003:
-
Summary: Add support to provide spark.executor.extraJavaOptions in
terms of App Id and/or Executor Id's
Key: SPARK-24003
URL: https://issues.apache.org/jira/browse/SPARK-24003
[
https://issues.apache.org/jira/browse/SPARK-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377370#comment-16377370
]
Devaraj K commented on SPARK-22567:
---
Dup of SPARK-17928.
[~michaelmoss], can you check the PR
[
https://issues.apache.org/jira/browse/SPARK-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308802#comment-16308802
]
Devaraj K commented on SPARK-22404:
---
Thanks [~irashid] for the comment.
bq. can you provide a little
[
https://issues.apache.org/jira/browse/SPARK-14228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16286282#comment-16286282
]
Devaraj K commented on SPARK-14228:
---
[~KaiXinXIaoLei], Thanks for checking this. Is the issue you are
[
https://issues.apache.org/jira/browse/SPARK-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-22519:
--
Summary: Remove unnecessary stagingDirPath null check in
ApplicationMaster.cleanupStagingDir() (was:
[
https://issues.apache.org/jira/browse/SPARK-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-22519:
--
Priority: Trivial (was: Minor)
> Remove unnecessary stagingDirPath null check in
>
[
https://issues.apache.org/jira/browse/SPARK-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251925#comment-16251925
]
Devaraj K commented on SPARK-22519:
---
It is not an usual case, I have seen this NPE while working
Devaraj K created SPARK-22519:
-
Summary: ApplicationMaster.cleanupStagingDir() throws NPE when
SPARK_YARN_STAGING_DIR env var is not available
Key: SPARK-22519
URL: https://issues.apache.org/jira/browse/SPARK-22519
Devaraj K created SPARK-22404:
-
Summary: Provide an option to use unmanaged AM in yarn-client mode
Key: SPARK-22404
URL: https://issues.apache.org/jira/browse/SPARK-22404
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-22404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226015#comment-16226015
]
Devaraj K commented on SPARK-22404:
---
I am working on this, will update this jira with the proposal PR.
Devaraj K created SPARK-22172:
-
Summary: Worker hangs when the external shuffle service port is
already in use
Key: SPARK-22172
URL: https://issues.apache.org/jira/browse/SPARK-22172
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177253#comment-16177253
]
Devaraj K commented on SPARK-19417:
---
Thanks [~ckanich] for the test case.
[
https://issues.apache.org/jira/browse/SPARK-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121817#comment-16121817
]
Devaraj K commented on SPARK-18648:
---
[~FlamingMike], It has fixed as part of SPARK-21339, can you check
[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100079#comment-16100079
]
Devaraj K commented on SPARK-15142:
---
bq. That means there is no way to detect the new master while the
[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K resolved SPARK-15142.
---
Resolution: Duplicate
> Spark Mesos dispatcher becomes unusable when the Mesos master restarts
>
[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099535#comment-16099535
]
Devaraj K commented on SPARK-15142:
---
[~skonto] Thanks for showing interest on this. I have already
[
https://issues.apache.org/jira/browse/SPARK-21146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-21146:
--
Summary: Master/Worker should handle and shutdown when any thread gets
UncaughtException (was: Worker
[
https://issues.apache.org/jira/browse/SPARK-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K closed SPARK-21148.
-
Resolution: Duplicate
> Set SparkUncaughtExceptionHandler to the Master
>
Devaraj K created SPARK-21170:
-
Summary: Utils.tryWithSafeFinallyAndFailureCallbacks throws
IllegalArgumentException: Self-suppression not permitted
Key: SPARK-21170
URL:
Devaraj K created SPARK-21148:
-
Summary: Set SparkUncaughtExceptionHandler to the Master
Key: SPARK-21148
URL: https://issues.apache.org/jira/browse/SPARK-21148
Project: Spark
Issue Type:
Devaraj K created SPARK-21146:
-
Summary: Worker should handle and shutdown when any thread gets
UncaughtException
Key: SPARK-21146
URL: https://issues.apache.org/jira/browse/SPARK-21146
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942141#comment-15942141
]
Devaraj K commented on SPARK-15665:
---
[~samuel-soubeyran], This issue has been resolved, Please create
Devaraj K created SPARK-19689:
-
Summary: Job Details page doesn't show 'Tasks: Succeeded/Total'
progress bar text properly
Key: SPARK-19689
URL: https://issues.apache.org/jira/browse/SPARK-19689
Project:
[
https://issues.apache.org/jira/browse/SPARK-19689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-19689:
--
Attachment: Tasks Progress bar - Job Details Page.png
> Job Details page doesn't show 'Tasks:
[
https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-19354:
--
Description:
When we enable speculation, we can see there are multiple attempts running for
the same
[
https://issues.apache.org/jira/browse/SPARK-19377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-19377:
--
Description:
|143|10 |0 |SUCCESS|NODE_LOCAL |6 / x.xx.x.x
stdout
stderr
Devaraj K created SPARK-19377:
-
Summary: Killed tasks should have the status as KILLED
Key: SPARK-19377
URL: https://issues.apache.org/jira/browse/SPARK-19377
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838396#comment-15838396
]
Devaraj K commented on SPARK-19354:
---
bq. The question is, why the error during shutdown?
The shutdown
[
https://issues.apache.org/jira/browse/SPARK-19354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837346#comment-15837346
]
Devaraj K commented on SPARK-19354:
---
Thanks [~uncleGen] for the comment. Here the error occurred during
Devaraj K created SPARK-19354:
-
Summary: Killed tasks are getting marked as FAILED
Key: SPARK-19354
URL: https://issues.apache.org/jira/browse/SPARK-19354
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15786878#comment-15786878
]
Devaraj K commented on SPARK-15359:
---
Thanks [~yu2003w] for verifying this PR, I forgot to mention that
[
https://issues.apache.org/jira/browse/SPARK-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783821#comment-15783821
]
Devaraj K commented on SPARK-15359:
---
[~yu2003w], seems you are also facing the same issue which I
Devaraj K created SPARK-15665:
-
Summary: spark-submit --kill and --status are not working
Key: SPARK-15665
URL: https://issues.apache.org/jira/browse/SPARK-15665
Project: Spark
Issue Type: Bug
Devaraj K created SPARK-15560:
-
Summary: Queued/Supervise drivers waiting for retry drivers
disappear for kill command in Mesos mode
Key: SPARK-15560
URL: https://issues.apache.org/jira/browse/SPARK-15560
Devaraj K created SPARK-1:
-
Summary: Driver with --supervise option cannot be killed in Mesos
mode
Key: SPARK-1
URL: https://issues.apache.org/jira/browse/SPARK-1
Project: Spark
Devaraj K created SPARK-15359:
-
Summary: Mesos dispatcher should handle DRIVER_ABORTED status from
mesosDriver.run()
Key: SPARK-15359
URL: https://issues.apache.org/jira/browse/SPARK-15359
Project: Spark
Devaraj K created SPARK-15288:
-
Summary: Mesos dispatcher should handle gracefully when any thread
gets UncaughtException
Key: SPARK-15288
URL: https://issues.apache.org/jira/browse/SPARK-15288
Project:
[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15275978#comment-15275978
]
Devaraj K commented on SPARK-15142:
---
bq. Can you include the dispatcher logs?
I have attached the
[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-15142:
--
Attachment:
spark-devaraj-org.apache.spark.deploy.mesos.MesosClusterDispatcher-1-stobdtserver5.out
>
[
https://issues.apache.org/jira/browse/SPARK-15142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272056#comment-15272056
]
Devaraj K commented on SPARK-15142:
---
It is not running the queued applications after Mesos master comes
Devaraj K created SPARK-15142:
-
Summary: Spark Mesos dispatcher becomes unusable when the Mesos
master restarts
Key: SPARK-15142
URL: https://issues.apache.org/jira/browse/SPARK-15142
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15271875#comment-15271875
]
Devaraj K commented on SPARK-10713:
---
bq. However, on Mesos, SPARK_DIST_CLASSPATH is missing from
[
https://issues.apache.org/jira/browse/SPARK-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15270330#comment-15270330
]
Devaraj K commented on SPARK-10713:
---
[~daradib], I tried to reproduce it but seems SPARK_DIST_CLASSPATH
[
https://issues.apache.org/jira/browse/SPARK-13532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268670#comment-15268670
]
Devaraj K commented on SPARK-13532:
---
[~apivovarov], I tried to reproduce this issue but it seems
[
https://issues.apache.org/jira/browse/SPARK-15067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268659#comment-15268659
]
Devaraj K commented on SPARK-15067:
---
[~renatojdk], are you planning to create PR for this? If not
[
https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216090#comment-15216090
]
Devaraj K commented on SPARK-13063:
---
Thanks [~tgraves] for confirmation, I will create PR for this.
>
Devaraj K created SPARK-14234:
-
Summary: Executor crashes for TaskRunner thread interruption
Key: SPARK-14234
URL: https://issues.apache.org/jira/browse/SPARK-14234
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K resolved SPARK-13965.
---
Resolution: Duplicate
> TaskSetManager should kill the other running task attempts if any one task
[
https://issues.apache.org/jira/browse/SPARK-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-13965:
--
Summary: TaskSetManager should kill the other running task attempts if any
one task attempt succeeds
Devaraj K created SPARK-13965:
-
Summary: Driver should kill the other running task attempts if any
one task attempt succeeds for the same task
Key: SPARK-13965
URL: https://issues.apache.org/jira/browse/SPARK-13965
Devaraj K created SPARK-13621:
-
Summary: TestExecutor.scala needs to be moved to test package
Key: SPARK-13621
URL: https://issues.apache.org/jira/browse/SPARK-13621
Project: Spark
Issue Type:
Devaraj K created SPARK-13619:
-
Summary: Jobs page UI shows wrong number of failed tasks
Key: SPARK-13619
URL: https://issues.apache.org/jira/browse/SPARK-13619
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173224#comment-15173224
]
Devaraj K commented on SPARK-13117:
---
I think we can start the Jetty server with the default value as
[
https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15173218#comment-15173218
]
Devaraj K commented on SPARK-13117:
---
[~jaypanicker], the proposed PR does the same but there is a
[
https://issues.apache.org/jira/browse/SPARK-13445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated SPARK-13445:
--
Summary: Selecting "data" with window function does not work unless aliased
(using PARTITION BY)
[
https://issues.apache.org/jira/browse/SPARK-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15141340#comment-15141340
]
Devaraj K commented on SPARK-13061:
---
You have mentioned the id as 'Spark shell' in the issue
[
https://issues.apache.org/jira/browse/SPARK-13061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15140601#comment-15140601
]
Devaraj K commented on SPARK-13061:
---
[~avihoo], I am trying to reproduce the issue but I see the id
[
https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15138251#comment-15138251
]
Devaraj K commented on SPARK-13117:
---
Thanks [~jjordan].
> WebUI should use the local ip not 0.0.0.0
>
[
https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137320#comment-15137320
]
Devaraj K commented on SPARK-13117:
---
Thanks [~jjordan] for reporting. I would like to provide PR if you
[
https://issues.apache.org/jira/browse/SPARK-13016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137310#comment-15137310
]
Devaraj K commented on SPARK-13016:
---
I am working on this, I will provide PR for this. Thanks
>
[
https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121427#comment-15121427
]
Devaraj K commented on SPARK-13063:
---
Yes, the sub-directories under the staging dir are specific to
Devaraj K created SPARK-13063:
-
Summary: Make the SPARK YARN STAGING DIR as configurable
Key: SPARK-13063
URL: https://issues.apache.org/jira/browse/SPARK-13063
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121239#comment-15121239
]
Devaraj K commented on SPARK-13063:
---
If this default location is already used by another apps(like
[
https://issues.apache.org/jira/browse/SPARK-13063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15121523#comment-15121523
]
Devaraj K commented on SPARK-13063:
---
I don't think it would be good to store the files in the spark
[
https://issues.apache.org/jira/browse/SPARK-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069890#comment-15069890
]
Devaraj K commented on SPARK-1253:
--
Thanks [~srowen] for letting me know. I tried to reproduce it but
[
https://issues.apache.org/jira/browse/SPARK-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067962#comment-15067962
]
Devaraj K commented on SPARK-1253:
--
[~sandyr], Do you want to work on this issue? I would like to provide
[
https://issues.apache.org/jira/browse/SPARK-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15059602#comment-15059602
]
Devaraj K commented on SPARK-12316:
---
[~carlmartin], can you provide the stack trace for this error?
[
https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035721#comment-15035721
]
Devaraj K commented on SPARK-4117:
--
Thanks [~tgraves] for the pointer. I will provide PR for this to
[
https://issues.apache.org/jira/browse/SPARK-8276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K resolved SPARK-8276.
--
Resolution: Duplicate
Resolving it as duplicate of SPARK-8754, Please reopen it if you disagree.
> NPE
[
https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15026526#comment-15026526
]
Devaraj K commented on SPARK-4117:
--
For *ApplicationAttemptNotFoundException*, there is no explicit
[
https://issues.apache.org/jira/browse/SPARK-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023785#comment-15023785
]
Devaraj K commented on SPARK-4117:
--
In YARN, now AM commands(AM_RESYNC, AM_SHUTDOWN) are deprecated and
[
https://issues.apache.org/jira/browse/SPARK-8276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023790#comment-15023790
]
Devaraj K commented on SPARK-8276:
--
I think it is duplicate of SPARK-8754.
> NPE in
Devaraj K created SPARK-8754:
Summary: YarnClientSchedulerBackend doesn't stop gracefully in
failure conditions
Key: SPARK-8754
URL: https://issues.apache.org/jira/browse/SPARK-8754
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009352#comment-14009352
]
Devaraj K commented on SPARK-1936:
--
Pull Request added :
88 matches
Mail list logo
