[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564562#comment-16564562 ] Saisai Shao edited comment on SPARK-24615 at 8/1/18 12:35 AM:

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564562#comment-16564562 ] Saisai Shao commented on SPARK-24615: - Leveraging dynamic allocation to tear down an

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Target Version/s: 2.2.3, 2.3.2, 2.4.0 > SHS filters wrongly some applications due to permission ch

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Priority: Blocker (was: Major) > SHS filters wrongly some applications due to permission check >

[jira] [Commented] (SPARK-25030) SparkSubmit.doSubmit will not return result if the mainClass submitted creates a Timer()

[ https://issues.apache.org/jira/browse/SPARK-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570995#comment-16570995 ] Saisai Shao commented on SPARK-25030: - I would like to see more about this issue. >

[jira] [Commented] (SPARK-22634) Update Bouncy castle dependency

[ https://issues.apache.org/jira/browse/SPARK-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572542#comment-16572542 ] Saisai Shao commented on SPARK-22634: - [~srowen] I'm wondering if it is possible to

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Fix Version/s: 2.3.2 > SHS filters wrongly some applications due to permission check > ---

[jira] [Assigned] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24948: --- Assignee: Marco Gaido > SHS filters wrongly some applications due to permission check > ---

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Fix Version/s: 2.2.3 > SHS filters wrongly some applications due to permission check > ---

[jira] [Commented] (SPARK-25084) "distribute by" on multiple columns may lead to codegen issue

[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575772#comment-16575772 ] Saisai Shao commented on SPARK-25084: - I'm already preparing new RC4. If this is not

[jira] [Commented] (SPARK-25084) "distribute by" on multiple columns may lead to codegen issue

[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575774#comment-16575774 ] Saisai Shao commented on SPARK-25084: - Is this a regression or just a bug existed in

[jira] [Commented] (SPARK-25084) "distribute by" on multiple columns may lead to codegen issue

[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575778#comment-16575778 ] Saisai Shao commented on SPARK-25084: - I see. Unfortunately I've cut the RC4, if it

[jira] [Commented] (SPARK-25020) Unable to Perform Graceful Shutdown in Spark Streaming with Hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-25020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577846#comment-16577846 ] Saisai Shao commented on SPARK-25020: - Yes, we also met the same issue. This is more

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585321#comment-16585321 ] Saisai Shao commented on SPARK-25114: - What's the ETA of this issue [~jiangxb1987]?

[jira] [Updated] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23679: Component/s: YARN > uiWebUrl show inproper URL when running on YARN >

[jira] [Commented] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585473#comment-16585473 ] Saisai Shao commented on SPARK-23679: - Let me fix this issue. This is mainly a RM HA

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585482#comment-16585482 ] Saisai Shao commented on SPARK-25144: - Does this exist in master branch? > distinct

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25144: Target Version/s: 2.2.3, 2.3.2 > distinct on Dataset leads to exception due to Managed memory leak

[jira] [Commented] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587062#comment-16587062 ] Saisai Shao commented on SPARK-25131: - This is by design, not a bug. > Event logs m

[jira] [Commented] (SPARK-25221) [DEPLOY] Consistent trailing whitespace treatment of conf values

[ https://issues.apache.org/jira/browse/SPARK-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593124#comment-16593124 ] Saisai Shao commented on SPARK-25221: - I'm going to remove the target version, I don

[jira] [Updated] (SPARK-25221) [DEPLOY] Consistent trailing whitespace treatment of conf values

[ https://issues.apache.org/jira/browse/SPARK-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25221: Target Version/s: (was: 2.3.2, 2.4.0) > [DEPLOY] Consistent trailing whitespace treatment of con

[jira] [Updated] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24785: Fix Version/s: 2.4.0 > Making sure REPL prints Spark UI info and then Welcome message > --

[jira] [Commented] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598110#comment-16598110 ] Saisai Shao commented on SPARK-25206: - What is the status of this JIRA, are we going

[jira] [Commented] (SPARK-25135) insert datasource table may all null when select from view on parquet

[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598111#comment-16598111 ] Saisai Shao commented on SPARK-25135: - What's the ETA of this issue [~yumwang]? > i

[jira] [Commented] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598115#comment-16598115 ] Saisai Shao commented on SPARK-25206: - I see, if it is not going to be merged, let's

[jira] [Resolved] (SPARK-25183) Spark HiveServer2 registers shutdown hook with JVM, not ShutdownHookManager; race conditions can arise

[ https://issues.apache.org/jira/browse/SPARK-25183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-25183. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22186 [https://gith

[jira] [Assigned] (SPARK-25183) Spark HiveServer2 registers shutdown hook with JVM, not ShutdownHookManager; race conditions can arise

[ https://issues.apache.org/jira/browse/SPARK-25183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-25183: --- Assignee: Steve Loughran > Spark HiveServer2 registers shutdown hook with JVM, not Shutdown

[jira] [Assigned] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24615: --- Assignee: (was: Saisai Shao) > Accelerator-aware task scheduling for Spark > --

[jira] [Created] (SPARK-24062) SASL encryption cannot be worked in ThriftServer

Saisai Shao created SPARK-24062: --- Summary: SASL encryption cannot be worked in ThriftServer Key: SPARK-24062 URL: https://issues.apache.org/jira/browse/SPARK-24062 Project: Spark Issue Type: Bu

[jira] [Resolved] (SPARK-24062) SASL encryption cannot be worked in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24062. - Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.4.0 2.

[jira] [Commented] (SPARK-24062) SASL encryption cannot be worked in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453521#comment-16453521 ] Saisai Shao commented on SPARK-24062: - Issue resolved by pull request 21138 [https://

[jira] [Commented] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455621#comment-16455621 ] Saisai Shao commented on SPARK-23830: - I agree [~emaynard]. > Spark on YARN in clust

[jira] [Created] (SPARK-24110) Avoid calling UGI loginUserFromKeytab in ThriftServer

Saisai Shao created SPARK-24110: --- Summary: Avoid calling UGI loginUserFromKeytab in ThriftServer Key: SPARK-24110 URL: https://issues.apache.org/jira/browse/SPARK-24110 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23830. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21168 [https://githu

[jira] [Assigned] (SPARK-23830) Spark on YARN in cluster deploy mode fail with NullPointerException when a Spark application is a Scala class not object

[ https://issues.apache.org/jira/browse/SPARK-23830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23830: --- Assignee: Eric Maynard > Spark on YARN in cluster deploy mode fail with NullPointerException

[jira] [Resolved] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23688. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21152 [https://githu

[jira] [Assigned] (SPARK-23688) Refactor tests away from rate source

[ https://issues.apache.org/jira/browse/SPARK-23688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23688: --- Assignee: Jungtaek Lim > Refactor tests away from rate source >

[jira] [Assigned] (SPARK-24110) Avoid calling UGI loginUserFromKeytab in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24110: --- Assignee: Saisai Shao > Avoid calling UGI loginUserFromKeytab in ThriftServer >

[jira] [Resolved] (SPARK-24110) Avoid calling UGI loginUserFromKeytab in ThriftServer

[ https://issues.apache.org/jira/browse/SPARK-24110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24110. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21178 [https://githu

[jira] [Resolved] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

[ https://issues.apache.org/jira/browse/SPARK-24136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24136. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21207 [https://githu

[jira] [Assigned] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

[ https://issues.apache.org/jira/browse/SPARK-24136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24136: --- Assignee: Arun Mahadevan > MemoryStreamDataReader.next should skip sleeping if record is ava

[jira] [Commented] (SPARK-24174) Expose Hadoop config as part of /environment API

[ https://issues.apache.org/jira/browse/SPARK-24174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16463575#comment-16463575 ] Saisai Shao commented on SPARK-24174: - I believe Hadoop web UI already expose such co

[jira] [Assigned] (SPARK-24188) /api/v1/version not working

[ https://issues.apache.org/jira/browse/SPARK-24188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24188: --- Assignee: Marcelo Vanzin > /api/v1/version not working > --- > >

[jira] [Resolved] (SPARK-24188) /api/v1/version not working

[ https://issues.apache.org/jira/browse/SPARK-24188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24188. - Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull req

[jira] [Created] (SPARK-24219) Improve the docker build script to avoid copying everything in example

Saisai Shao created SPARK-24219: --- Summary: Improve the docker build script to avoid copying everything in example Key: SPARK-24219 URL: https://issues.apache.org/jira/browse/SPARK-24219 Project: Spark

[jira] [Resolved] (SPARK-24182) Improve error message for client mode when AM fails

[ https://issues.apache.org/jira/browse/SPARK-24182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24182. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21243 [https://githu

[jira] [Assigned] (SPARK-24182) Improve error message for client mode when AM fails

[ https://issues.apache.org/jira/browse/SPARK-24182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24182: --- Assignee: Marcelo Vanzin > Improve error message for client mode when AM fails > ---

[jira] [Assigned] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24241: --- Assignee: Kent Yao > Do not fail fast when dynamic resource allocation enabled with 0 execut

[jira] [Updated] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24241: Fix Version/s: 2.4.0 > Do not fail fast when dynamic resource allocation enabled with 0 executor >

[jira] [Resolved] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24241. - Resolution: Fixed > Do not fail fast when dynamic resource allocation enabled with 0 executor > -

[jira] [Commented] (SPARK-24241) Do not fail fast when dynamic resource allocation enabled with 0 executor

[ https://issues.apache.org/jira/browse/SPARK-24241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16475480#comment-16475480 ] Saisai Shao commented on SPARK-24241: - Issue resolved by pull request 21290 https://g

[jira] [Created] (SPARK-24377) Make --py-files work in non pyspark application

Saisai Shao created SPARK-24377: --- Summary: Make --py-files work in non pyspark application Key: SPARK-24377 URL: https://issues.apache.org/jira/browse/SPARK-24377 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-23991) data loss when allocateBlocksToBatch

[ https://issues.apache.org/jira/browse/SPARK-23991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-23991: --- Assignee: Gabor Somogyi > data loss when allocateBlocksToBatch > --

[jira] [Resolved] (SPARK-23991) data loss when allocateBlocksToBatch

[ https://issues.apache.org/jira/browse/SPARK-23991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-23991. - Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by pull re

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496187#comment-16496187 ] Saisai Shao commented on SPARK-18673: - I created a PR for this issue [https://github

[jira] [Commented] (SPARK-24448) File not found on the address SparkFiles.get returns on standalone cluster

[ https://issues.apache.org/jira/browse/SPARK-24448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497548#comment-16497548 ] Saisai Shao commented on SPARK-24448: - Does it only happen in standalone cluster mod

[jira] [Commented] (SPARK-24355) Improve Spark shuffle server responsiveness to non-ChunkFetch requests

[ https://issues.apache.org/jira/browse/SPARK-24355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497626#comment-16497626 ] Saisai Shao commented on SPARK-24355: - Do we have a test result before and after? >

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497931#comment-16497931 ] Saisai Shao commented on SPARK-18673: - Thanks Steve, looking forward to your inputs.

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499707#comment-16499707 ] Saisai Shao commented on SPARK-20202: - What is our plan to to fix this issue, are we

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16501225#comment-16501225 ] Saisai Shao commented on SPARK-20202: - OK, for the 1st, I've already started working

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505608#comment-16505608 ] Saisai Shao commented on SPARK-24487: - What's the usage and purpose to integrate Rab

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Priority: Major (was: Blocker) > Kerberos Ticket Renewal is failing in long running Spark job > -

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Component/s: (was: Spark Core) YARN > Kerberos Ticket Renewal is failing in l

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in long running Spark job

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Component/s: Spark Core > Kerberos Ticket Renewal is failing in long running Spark job > -

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505760#comment-16505760 ] Saisai Shao commented on SPARK-24487: - You can add it in your own package with DataS

[jira] [Updated] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24493: Summary: Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3 (was: Kerberos Ticket Ren

[jira] [Commented] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505768#comment-16505768 ] Saisai Shao commented on SPARK-24493: - Adding more background, the issue is happened

[jira] [Comment Edited] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505768#comment-16505768 ] Saisai Shao edited comment on SPARK-24493 at 6/8/18 6:56 AM: -

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505771#comment-16505771 ] Saisai Shao commented on SPARK-23534: - Spark with Hadoop 3 will be failed in token r

[jira] [Commented] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505783#comment-16505783 ] Saisai Shao commented on SPARK-23151: - I will convert this as a subtask of SPARK-235

[jira] [Updated] (SPARK-23151) Provide a distribution of Spark with Hadoop 3.0

[ https://issues.apache.org/jira/browse/SPARK-23151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23151: Issue Type: Sub-task (was: Dependency upgrade) Parent: SPARK-23534 > Provide a distributi

[jira] [Commented] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16505786#comment-16505786 ] Saisai Shao commented on SPARK-24487: - I'm sure Spark community will not merge this

[jira] [Resolved] (SPARK-24487) Add support for RabbitMQ.

[ https://issues.apache.org/jira/browse/SPARK-24487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24487. - Resolution: Won't Fix > Add support for RabbitMQ. > - > >

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506023#comment-16506023 ] Saisai Shao commented on SPARK-23534: - Yes, it is a Hadoop 2.8+ issue, but we don't

[jira] [Created] (SPARK-24518) Using Hadoop credential provider API to store password

Saisai Shao created SPARK-24518: --- Summary: Using Hadoop credential provider API to store password Key: SPARK-24518 URL: https://issues.apache.org/jira/browse/SPARK-24518 Project: Spark Issue Ty

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Environment: (was: Current Spark configs password in a plaintext way, like putting in the conf

[jira] [Updated] (SPARK-24518) Using Hadoop credential provider API to store password

[ https://issues.apache.org/jira/browse/SPARK-24518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24518: Description: Current Spark configs password in a plaintext way, like putting in the configuration

[jira] [Commented] (SPARK-24493) Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3

[ https://issues.apache.org/jira/browse/SPARK-24493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517720#comment-16517720 ] Saisai Shao commented on SPARK-24493: - This BUG is fixed in HDFS-12670, Spark should

[jira] [Created] (SPARK-24615) Accelerator aware task scheduling for Spark

Saisai Shao created SPARK-24615: --- Summary: Accelerator aware task scheduling for Spark Key: SPARK-24615 URL: https://issues.apache.org/jira/browse/SPARK-24615 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Description: In the machine learning area, accelerator card (GPU, FPGA, TPU) is predominant compa

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Description: In the machine learning area, accelerator card (GPU, FPGA, TPU) is predominant compa

[jira] [Updated] (SPARK-24615) Accelerator aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24615: Labels: SPIP (was: ) > Accelerator aware task scheduling for Spark >

[jira] [Commented] (SPARK-24374) SPIP: Support Barrier Scheduling in Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519934#comment-16519934 ] Saisai Shao commented on SPARK-24374: - Hi [~mengxr] [~jiangxb1987] SPARK-24615 would

[jira] [Created] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

Saisai Shao created SPARK-24646: --- Summary: Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes Key: SPARK-24646 URL: https://issues.apache.org/jira/browse/SPARK-24646 Project: Spark

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523022#comment-16523022 ] Saisai Shao commented on SPARK-24646: - Hi [~vanzin], here is a specific example: We

[jira] [Resolved] (SPARK-24418) Upgrade to Scala 2.11.12

[ https://issues.apache.org/jira/browse/SPARK-24418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24418. - Resolution: Fixed Issue resolved by pull request 21495 [https://github.com/apache/spark/pull/214

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24530: Target Version/s: 2.1.3, 2.2.2, 2.3.2, 2.4.0 (was: 2.4.0) > Sphinx doesn't render autodoc_docstri

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529352#comment-16529352 ] Saisai Shao commented on SPARK-24530: - [~hyukjin.kwon], Spark 2.1.3 and 2.2.2 are on

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532143#comment-16532143 ] Saisai Shao commented on SPARK-24530: - Hi [~hyukjin.kwon] what is the current status

[jira] [Commented] (SPARK-24535) Fix java version parsing in SparkR on Windows

[ https://issues.apache.org/jira/browse/SPARK-24535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532145#comment-16532145 ] Saisai Shao commented on SPARK-24535: - Hi [~felixcheung] what's the current status o

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729357#comment-16729357 ] Saisai Shao commented on SPARK-25299: - [~jealous] Can we have a doc about this propo

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730084#comment-16730084 ] Saisai Shao commented on SPARK-20415: - Have you tried latest version of Spark, does

[jira] [Commented] (SPARK-26516) zeppelin with spark on mesos: environment variable setting

[ https://issues.apache.org/jira/browse/SPARK-26516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732587#comment-16732587 ] Saisai Shao commented on SPARK-26516: - Questions should go to user@spark mail list.

[jira] [Updated] (SPARK-26513) Trigger GC on executor node idle

[ https://issues.apache.org/jira/browse/SPARK-26513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-26513: Fix Version/s: (was: 3.0.0) > Trigger GC on executor node idle > -

[jira] [Resolved] (SPARK-26516) zeppelin with spark on mesos: environment variable setting

[ https://issues.apache.org/jira/browse/SPARK-26516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-26516. - Resolution: Invalid > zeppelin with spark on mesos: environment variable setting > -

[jira] [Updated] (SPARK-26457) Show hadoop configurations in HistoryServer environment tab

[ https://issues.apache.org/jira/browse/SPARK-26457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-26457: Priority: Minor (was: Major) > Show hadoop configurations in HistoryServer environment tab >

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733740#comment-16733740 ] Saisai Shao commented on SPARK-26512: - Please list the problems you saw, any log or

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10

[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734072#comment-16734072 ] Saisai Shao commented on SPARK-26512: - This seems like a Netty version problem, nett

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750872#comment-16750872 ] Saisai Shao commented on SPARK-24615: - I'm really sorry about the delay. Due to some

[jira] [Commented] (SPARK-19179) spark.yarn.access.namenodes description is wrong

[ https://issues.apache.org/jira/browse/SPARK-19179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15820371#comment-15820371 ] Saisai Shao commented on SPARK-19179: - Thanks [~tgraves] to point out the left thing,

<    1   2   3   4   5   6   7   8   9   10   >