[jira] [Created] (SPARK-20172) Event log without read permission should be filtered out before actually reading it

Saisai Shao created SPARK-20172: --- Summary: Event log without read permission should be filtered out before actually reading it Key: SPARK-20172 URL: https://issues.apache.org/jira/browse/SPARK-20172

[jira] [Commented] (SPARK-20128) MetricsSystem not always killed in SparkContext.stop()

[ https://issues.apache.org/jira/browse/SPARK-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946493#comment-15946493 ] Saisai Shao commented on SPARK-20128: - Sorry I cannot access the logs. What I could see from the link

[jira] [Comment Edited] (SPARK-20128) MetricsSystem not always killed in SparkContext.stop()

[ https://issues.apache.org/jira/browse/SPARK-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946457#comment-15946457 ] Saisai Shao edited comment on SPARK-20128 at 3/29/17 3:20 AM: -- Here the

[jira] [Commented] (SPARK-20128) MetricsSystem not always killed in SparkContext.stop()

[ https://issues.apache.org/jira/browse/SPARK-20128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946457#comment-15946457 ] Saisai Shao commented on SPARK-20128: - Here the exception is from MasterSource, which only exists in

[jira] [Commented] (SPARK-20079) Re registration of AM hangs spark cluster in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-20079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944629#comment-15944629 ] Saisai Shao commented on SPARK-20079: - What is the specific symptom you met? I believe there's bunch

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943289#comment-15943289 ] Saisai Shao commented on SPARK-19143: - Thanks [~tgraves], let me see how to propose a SPIP. > API in

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943303#comment-15943303 ] Saisai Shao commented on SPARK-19143: - [~tgraves], can I add you as a *SPIP Shepherd*? :) > API in

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943243#comment-15943243 ] Saisai Shao commented on SPARK-19143: - Attach a WIP branch

[jira] [Comment Edited] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942976#comment-15942976 ] Saisai Shao edited comment on SPARK-20059 at 3/27/17 10:02 AM: --- [~sowen],

[jira] [Commented] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15942976#comment-15942976 ] Saisai Shao commented on SPARK-20059: - [~sowen], it probably is not the same issue. In SPARK-20019,

[jira] [Updated] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-20059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20059: Description: {{HBaseCredentialProvider}} uses system classloader instead of child classloader,

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939867#comment-15939867 ] Saisai Shao commented on SPARK-19992: - Sorry I cannot give you valid suggestions without knowing your

[jira] [Commented] (SPARK-20050) Kafka 0.10 DirectStream doesn't commit last processed batch's offset when graceful shutdown

[ https://issues.apache.org/jira/browse/SPARK-20050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937845#comment-15937845 ] Saisai Shao commented on SPARK-20050: - I think you could register a commit callback function in

[jira] [Commented] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937841#comment-15937841 ] Saisai Shao commented on SPARK-20058: - I have a PR for this bug very very long ago

[jira] [Comment Edited] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937841#comment-15937841 ] Saisai Shao edited comment on SPARK-20058 at 3/23/17 7:02 AM: -- I have a PR

[jira] [Commented] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936255#comment-15936255 ] Saisai Shao commented on SPARK-20058: - Please subscribe this spark user mail list and set the

[jira] [Comment Edited] (SPARK-20058) the running application status changed from running to waiting when a master is down and it change to another standy by master

[ https://issues.apache.org/jira/browse/SPARK-20058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936255#comment-15936255 ] Saisai Shao edited comment on SPARK-20058 at 3/22/17 1:03 PM: -- Please

[jira] [Created] (SPARK-20059) HbaseCredentialProvider uses wrong classloader

Saisai Shao created SPARK-20059: --- Summary: HbaseCredentialProvider uses wrong classloader Key: SPARK-20059 URL: https://issues.apache.org/jira/browse/SPARK-20059 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936241#comment-15936241 ] Saisai Shao edited comment on SPARK-19992 at 3/22/17 12:51 PM: --- Oh, I see.

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936241#comment-15936241 ] Saisai Shao commented on SPARK-19992: - Oh, I see. Check the code again, looks like "/*" cannot be

[jira] [Commented] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932088#comment-15932088 ] Saisai Shao commented on SPARK-19941: - I think this scenario is quite similar to container

[jira] [Created] (SPARK-19995) Using real user to connect HiveMetastore in HiveClientImpl

Saisai Shao created SPARK-19995: --- Summary: Using real user to connect HiveMetastore in HiveClientImpl Key: SPARK-19995 URL: https://issues.apache.org/jira/browse/SPARK-19995 Project: Spark

[jira] [Comment Edited] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929547#comment-15929547 ] Saisai Shao edited comment on SPARK-19992 at 3/17/17 7:48 AM: -- Looks like I

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929547#comment-15929547 ] Saisai Shao commented on SPARK-19992: - Looks like I guess wrong from the url you provided. If you're

[jira] [Commented] (SPARK-19992) spark-submit on deployment-mode cluster

[ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929544#comment-15929544 ] Saisai Shao commented on SPARK-19992: - Are you using HDP environment, if so I guess need to

[jira] [Commented] (SPARK-19143) API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

[ https://issues.apache.org/jira/browse/SPARK-19143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904497#comment-15904497 ] Saisai Shao commented on SPARK-19143: - Hi all, I wrote a rough design doc based on the comments

[jira] [Comment Edited] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900821#comment-15900821 ] Saisai Shao edited comment on SPARK-19812 at 3/8/17 7:42 AM: - [~tgraves], I'm

[jira] [Comment Edited] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900821#comment-15900821 ] Saisai Shao edited comment on SPARK-19812 at 3/8/17 7:40 AM: - [~tgraves], I'm

[jira] [Commented] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900821#comment-15900821 ] Saisai Shao commented on SPARK-19812: - [~tgraves], I'm not quite sure what you mean here? bq. The

[jira] [Commented] (SPARK-19802) Remote History Server

[ https://issues.apache.org/jira/browse/SPARK-19802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893580#comment-15893580 ] Saisai Shao commented on SPARK-19802: - Spark's {{ApplicationHistoryProvider}} is pluggable, user

[jira] [Commented] (SPARK-19750) Spark UI http -> https redirect error

[ https://issues.apache.org/jira/browse/SPARK-19750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887204#comment-15887204 ] Saisai Shao commented on SPARK-19750: - This issue was found by [~yeshavora], credits to her. > Spark

[jira] [Created] (SPARK-19750) Spark UI http -> https redirect error

Saisai Shao created SPARK-19750: --- Summary: Spark UI http -> https redirect error Key: SPARK-19750 URL: https://issues.apache.org/jira/browse/SPARK-19750 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882027#comment-15882027 ] Saisai Shao commented on SPARK-19688: - According to my test, "spark.yarn.credentials.file" will be

[jira] [Updated] (SPARK-19707) Improve the invalid path check for sc.addJar

[ https://issues.apache.org/jira/browse/SPARK-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19707: Summary: Improve the invalid path check for sc.addJar (was: Improve the invalid path handling for

[jira] [Created] (SPARK-19707) Improve the invalid path handling for sc.addJar

Saisai Shao created SPARK-19707: --- Summary: Improve the invalid path handling for sc.addJar Key: SPARK-19707 URL: https://issues.apache.org/jira/browse/SPARK-19707 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879893#comment-15879893 ] Saisai Shao commented on SPARK-19688: - I see. So what issue did you encounter when you restart the

[jira] [Commented] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879883#comment-15879883 ] Saisai Shao commented on SPARK-19688: - [~j.devaraj], when you say Spark application is restarted, are

[jira] [Commented] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879564#comment-15879564 ] Saisai Shao commented on SPARK-19688: - I see, so we should exclude this configuration in checkpoint

[jira] [Updated] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19688: Component/s: DStreams > Spark on Yarn Credentials File set to different application directory >

[jira] [Commented] (SPARK-19688) Spark on Yarn Credentials File set to different application directory

[ https://issues.apache.org/jira/browse/SPARK-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877587#comment-15877587 ] Saisai Shao commented on SPARK-19688: - Can you please elaborate the problem you met, otherwise it is

[jira] [Commented] (SPARK-19649) Spark YARN client throws exception if job succeeds and max-completed-applications=0

[ https://issues.apache.org/jira/browse/SPARK-19649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874101#comment-15874101 ] Saisai Shao commented on SPARK-19649: - MapReduce could delegate to history server to query the state

[jira] [Commented] (SPARK-19649) Spark YARN client throws exception if job succeeds and max-completed-applications=0

[ https://issues.apache.org/jira/browse/SPARK-19649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874098#comment-15874098 ] Saisai Shao commented on SPARK-19649: - >From my understanding, this looks like a deliberately made

[jira] [Commented] (SPARK-19588) Allow putting keytab file to HDFS location specified in spark.yarn.keytab

[ https://issues.apache.org/jira/browse/SPARK-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865434#comment-15865434 ] Saisai Shao commented on SPARK-19588: - Putting on HDFS still requires downloading to local disk for

[jira] [Commented] (SPARK-19579) spark-submit fails to run Kafka Stream python script

[ https://issues.apache.org/jira/browse/SPARK-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865360#comment-15865360 ] Saisai Shao commented on SPARK-19579: - Only this specific part is not supported, all the that Python

[jira] [Closed] (SPARK-19579) spark-submit fails to run Kafka Stream python script

[ https://issues.apache.org/jira/browse/SPARK-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-19579. --- Resolution: Won't Fix > spark-submit fails to run Kafka Stream python script >

[jira] [Comment Edited] (SPARK-19579) spark-submit fails to run Kafka Stream python script

[ https://issues.apache.org/jira/browse/SPARK-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865360#comment-15865360 ] Saisai Shao edited comment on SPARK-19579 at 2/14/17 8:33 AM: -- Only this

[jira] [Commented] (SPARK-19579) spark-submit fails to run Kafka Stream python script

[ https://issues.apache.org/jira/browse/SPARK-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865346#comment-15865346 ] Saisai Shao commented on SPARK-19579: - The support of Python API is rejected by the community, so

[jira] [Commented] (SPARK-19579) spark-submit fails to run Kafka Stream python script

[ https://issues.apache.org/jira/browse/SPARK-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865330#comment-15865330 ] Saisai Shao commented on SPARK-19579: - If you're using Python API to writing Streaming application,

[jira] [Commented] (SPARK-19038) Can't find keytab file when using Hive catalog

[ https://issues.apache.org/jira/browse/SPARK-19038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864983#comment-15864983 ] Saisai Shao commented on SPARK-19038: - I think the issue you met is the same as this JIRA mentioned,

[jira] [Commented] (SPARK-19579) spark-submit fails to run Kafka Stream python script

[ https://issues.apache.org/jira/browse/SPARK-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864811#comment-15864811 ] Saisai Shao commented on SPARK-19579: - Spark Streaming Kafka Python API doesn't support Kafka 0.10

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863371#comment-15863371 ] Saisai Shao edited comment on SPARK-16742 at 2/13/17 9:34 AM: -- The proposed

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863371#comment-15863371 ] Saisai Shao commented on SPARK-16742: - The proposed solution is quite different from what existed in

[jira] [Created] (SPARK-19545) Compilation error with method not found when build against Hadoop 2.6.0.

Saisai Shao created SPARK-19545: --- Summary: Compilation error with method not found when build against Hadoop 2.6.0. Key: SPARK-19545 URL: https://issues.apache.org/jira/browse/SPARK-19545 Project:

[jira] [Updated] (SPARK-19306) Fix inconsistent state in DiskBlockObjectWriter when exception occurred

[ https://issues.apache.org/jira/browse/SPARK-19306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19306: Summary: Fix inconsistent state in DiskBlockObjectWriter when exception occurred (was: Fix

[jira] [Created] (SPARK-19306) Fix inconsistent state in DiskBlockObjectWrite when exception occurred

Saisai Shao created SPARK-19306: --- Summary: Fix inconsistent state in DiskBlockObjectWrite when exception occurred Key: SPARK-19306 URL: https://issues.apache.org/jira/browse/SPARK-19306 Project: Spark

[jira] [Commented] (SPARK-19179) spark.yarn.access.namenodes description is wrong

[ https://issues.apache.org/jira/browse/SPARK-19179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820371#comment-15820371 ] Saisai Shao commented on SPARK-19179: - Thanks [~tgraves] to point out the left thing, let me handle

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814001#comment-15814001 ] Saisai Shao commented on SPARK-19090: - Are you using SparkConf API to set configuration in

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813973#comment-15813973 ] Saisai Shao commented on SPARK-19090: - Spark shell is a real spark *application*. The underlying

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813941#comment-15813941 ] Saisai Shao commented on SPARK-19090: - {code} ./bin/spark-shell --master yarn-client --conf

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813801#comment-15813801 ] Saisai Shao commented on SPARK-19090: - I also tested with Spark 1.5.0, I don't see an issue here, the

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813782#comment-15813782 ] Saisai Shao commented on SPARK-19090: - I tested with Spark 2.0 and latest master (2.2.0-SNAPSHOT),

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813704#comment-15813704 ] Saisai Shao commented on SPARK-19090: - Thanks for your elaboration, would you please tell which

[jira] [Commented] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803754#comment-15803754 ] Saisai Shao commented on SPARK-19090: - "spark.executor.cores" has nothing to do with dynamic

[jira] [Commented] (SPARK-19038) Can't find keytab file when using Hive catalog

[ https://issues.apache.org/jira/browse/SPARK-19038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803722#comment-15803722 ] Saisai Shao commented on SPARK-19038: - Please see the comment I made in

[jira] [Commented] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15796834#comment-15796834 ] Saisai Shao commented on SPARK-19033: - Thanks a lot [~tgraves], I will think about how to better

[jira] [Commented] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15787031#comment-15787031 ] Saisai Shao commented on SPARK-19033: - Ping [~vanzin], I found that you made this change, would you

[jira] [Updated] (SPARK-19033) HistoryServer still uses old ACLs even if ACLs are updated

[ https://issues.apache.org/jira/browse/SPARK-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19033: Summary: HistoryServer still uses old ACLs even if ACLs are updated (was: HistoryServer will

[jira] [Created] (SPARK-19033) HistoryServer will honor old ACLs even if ACLs are updated

Saisai Shao created SPARK-19033: --- Summary: HistoryServer will honor old ACLs even if ACLs are updated Key: SPARK-19033 URL: https://issues.apache.org/jira/browse/SPARK-19033 Project: Spark

[jira] [Created] (SPARK-19021) Generailize HDFSCredentialProvider to support non HDFS security FS

Saisai Shao created SPARK-19021: --- Summary: Generailize HDFSCredentialProvider to support non HDFS security FS Key: SPARK-19021 URL: https://issues.apache.org/jira/browse/SPARK-19021 Project: Spark

[jira] [Updated] (SPARK-19021) Generailize HDFSCredentialProvider to support non HDFS security FS

[ https://issues.apache.org/jira/browse/SPARK-19021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-19021: Priority: Minor (was: Major) > Generailize HDFSCredentialProvider to support non HDFS security FS

[jira] [Created] (SPARK-18975) Add an API to remove SparkListener from SparkContext

Saisai Shao created SPARK-18975: --- Summary: Add an API to remove SparkListener from SparkContext Key: SPARK-18975 URL: https://issues.apache.org/jira/browse/SPARK-18975 Project: Spark Issue

[jira] [Commented] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746877#comment-15746877 ] Saisai Shao commented on SPARK-18840: - [~vanzin], is it necessary to fix it in old version (2.0/1.6),

[jira] [Comment Edited] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744370#comment-15744370 ] Saisai Shao edited comment on SPARK-18840 at 12/13/16 9:30 AM: --- This

[jira] [Updated] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-18840: Priority: Minor (was: Major) > HDFSCredentialProvider throws exception in non-HDFS security

[jira] [Commented] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

[ https://issues.apache.org/jira/browse/SPARK-18840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744370#comment-15744370 ] Saisai Shao commented on SPARK-18840: - This problem also existed in branch 1.6, but the fix is a

[jira] [Created] (SPARK-18840) HDFSCredentialProvider throws exception in non-HDFS security environment

Saisai Shao created SPARK-18840: --- Summary: HDFSCredentialProvider throws exception in non-HDFS security environment Key: SPARK-18840 URL: https://issues.apache.org/jira/browse/SPARK-18840 Project:

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736957#comment-15736957 ] Saisai Shao commented on SPARK-13955: - Yes, forgot to mention this zip file doesn't support nested

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734207#comment-15734207 ] Saisai Shao commented on SPARK-13955: - Can you please check the runtime environment of launching

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734184#comment-15734184 ] Saisai Shao commented on SPARK-13955: - Do you have spark-yarn_2.11 jar in your archive? > Spark in

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734071#comment-15734071 ] Saisai Shao commented on SPARK-13955: - IIRC {{spark.yarn.archive}} should be worked, I tried

[jira] [Commented] (SPARK-18743) StreamingContext.textFileStream(directory) has no events shown in Web UI

[ https://issues.apache.org/jira/browse/SPARK-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15728710#comment-15728710 ] Saisai Shao commented on SPARK-18743: - >From my understanding, {{FileInputDStream}} is not a record

[jira] [Commented] (SPARK-18554) leader master lost the leadership, when the slave become master, the perivious app's state display as waitting

[ https://issues.apache.org/jira/browse/SPARK-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695947#comment-15695947 ] Saisai Shao commented on SPARK-18554: - Nothing blocked actually, just no one reviewed that PR, also

[jira] [Commented] (SPARK-18554) leader master lost the leadership, when the slave become master, the perivious app's state display as waitting

[ https://issues.apache.org/jira/browse/SPARK-18554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15694563#comment-15694563 ] Saisai Shao commented on SPARK-18554: - I think this is a known issue and I have a fix for it long

[jira] [Comment Edited] (SPARK-18225) job will miss when driver removed by master in spark streaming

[ https://issues.apache.org/jira/browse/SPARK-18225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15635846#comment-15635846 ] Saisai Shao edited comment on SPARK-18225 at 11/4/16 10:01 AM: --- I think the

[jira] [Commented] (SPARK-18225) job will miss when driver removed by master in spark streaming

[ https://issues.apache.org/jira/browse/SPARK-18225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15635846#comment-15635846 ] Saisai Shao commented on SPARK-18225: - I think the problem [~liujianhui] trying to address is that

[jira] [Commented] (SPARK-18225) job will miss when driver removed by master in spark streaming

[ https://issues.apache.org/jira/browse/SPARK-18225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631915#comment-15631915 ] Saisai Shao commented on SPARK-18225: - I simply doubt that kill button provided on the web UI does

[jira] [Created] (SPARK-17999) Add getPreferredLocations for KafkaSourceRDD

Saisai Shao created SPARK-17999: --- Summary: Add getPreferredLocations for KafkaSourceRDD Key: SPARK-17999 URL: https://issues.apache.org/jira/browse/SPARK-17999 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17984) Add support for numa aware feature

[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587459#comment-15587459 ] Saisai Shao commented on SPARK-17984: - NUMA should be supported by most commodity servers as well as

[jira] [Commented] (SPARK-17787) spark submit throws error while using kafka Appender log4j:ERROR Could not instantiate class [kafka.producer.KafkaLog4jAppender]. java.lang.ClassNotFoundException: kaf

[ https://issues.apache.org/jira/browse/SPARK-17787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15568020#comment-15568020 ] Saisai Shao commented on SPARK-17787: - I guess you don't have jars on the node where AM/Executors

[jira] [Closed] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao closed SPARK-17814. --- Resolution: Won't Fix > spark submit arguments are truncated in yarn-cluster mode >

[jira] [Comment Edited] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567964#comment-15567964 ] Saisai Shao edited comment on SPARK-17814 at 10/12/16 7:57 AM: --- By checking

[jira] [Comment Edited] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567964#comment-15567964 ] Saisai Shao edited comment on SPARK-17814 at 10/12/16 7:56 AM: --- By checking

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567964#comment-15567964 ] Saisai Shao commented on SPARK-17814: - By checking the yarn code, I think you might run into YARN's

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567933#comment-15567933 ] Saisai Shao commented on SPARK-17814: - Which version of YARN do you use? > spark submit arguments

[jira] [Commented] (SPARK-17814) spark submit arguments are truncated in yarn-cluster mode

[ https://issues.apache.org/jira/browse/SPARK-17814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567931#comment-15567931 ] Saisai Shao commented on SPARK-17814: - This looks like a YARN problem, after tracking the code path,

[jira] [Commented] (SPARK-17846) A bad state of Running Applications with spark standalone HA

[ https://issues.apache.org/jira/browse/SPARK-17846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567662#comment-15567662 ] Saisai Shao commented on SPARK-17846: - I think this issue should be the same as SPARK-14262. > A bad

[jira] [Created] (SPARK-17686) Propose to print Scala version in "spark-submit --version" command

Saisai Shao created SPARK-17686: --- Summary: Propose to print Scala version in "spark-submit --version" command Key: SPARK-17686 URL: https://issues.apache.org/jira/browse/SPARK-17686 Project: Spark

[jira] [Created] (SPARK-17678) Spark 1.6 Scala-2.11 repl doesn't honor "spark.replClassServer.port"

Saisai Shao created SPARK-17678: --- Summary: Spark 1.6 Scala-2.11 repl doesn't honor "spark.replClassServer.port" Key: SPARK-17678 URL: https://issues.apache.org/jira/browse/SPARK-17678 Project: Spark

[jira] [Commented] (SPARK-17637) Packed scheduling for Spark tasks across executors

[ https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515836#comment-15515836 ] Saisai Shao commented on SPARK-17637: - [~zhanzhang] would you mind sharing more details about your

[jira] [Comment Edited] (SPARK-17624) Flaky test? StateStoreSuite maintenance

[ https://issues.apache.org/jira/browse/SPARK-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15512237#comment-15512237 ] Saisai Shao edited comment on SPARK-17624 at 9/22/16 5:36 AM: -- I cannot

<    1   2   3   4   5   6   7   8   9   10   >