[jira] [Updated] (SPARK-13704) TaskSchedulerImpl.createTaskSetManager can be expensive, and result in lost executors due to blocked heartbeats

[ https://issues.apache.org/jira/browse/SPARK-13704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-13704: Description: In some cases, TaskSchedulerImpl.createTaskSetManager can be expensive. For example,

[jira] [Commented] (SPARK-30586) NPE in LiveRDDDistribution (AppStatusListener)

[ https://issues.apache.org/jira/browse/SPARK-30586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036734#comment-17036734 ] Saisai Shao commented on SPARK-30586: - We also met the same issue. Seems like the co

[jira] [Comment Edited] (SPARK-30586) NPE in LiveRDDDistribution (AppStatusListener)

[ https://issues.apache.org/jira/browse/SPARK-30586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036734#comment-17036734 ] Saisai Shao edited comment on SPARK-30586 at 2/14/20 7:13 AM:

[jira] [Resolved] (SPARK-29112) Expose more details when ApplicationMaster reporter faces a fatal exception

[ https://issues.apache.org/jira/browse/SPARK-29112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-29112. - Fix Version/s: 3.0.0 Assignee: Lantao Jin Resolution: Fixed Issue resolved by pu

[jira] [Updated] (SPARK-29114) Dataset.coalesce(10) throw ChunkFetchFailureException when original Dataset size is big

[ https://issues.apache.org/jira/browse/SPARK-29114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-29114: Priority: Major (was: Blocker) > Dataset.coalesce(10) throw ChunkFetchFailureException when origi

[jira] [Commented] (SPARK-29038) SPIP: Support Spark Materialized View

[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927265#comment-16927265 ] Saisai Shao commented on SPARK-29038: - [~cltlfcjin] I think we need a SPIP review an

[jira] [Commented] (SPARK-29038) SPIP: Support Spark Materialized View

[ https://issues.apache.org/jira/browse/SPARK-29038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927263#comment-16927263 ] Saisai Shao commented on SPARK-29038: - IIUC, I think the key difference between MV a

[jira] [Reopened] (SPARK-19147) netty throw NPE

[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reopened SPARK-19147: - > netty throw NPE > --- > > Key: SPARK-19147 > URL: http

[jira] [Commented] (SPARK-19147) netty throw NPE

[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923014#comment-16923014 ] Saisai Shao commented on SPARK-19147: - Yes, we also met the similar issue when execu

[jira] [Commented] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException

[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918386#comment-16918386 ] Saisai Shao commented on SPARK-28340: - My simple concern is that there may be other

[jira] [Commented] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException

[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918306#comment-16918306 ] Saisai Shao commented on SPARK-28340: - We also saw a bunch of exceptions in our prod

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Attachment: D18F4.png 95330.png 91ADA.png > Spark's UnsafeShuffleW

[jira] [Updated] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

[ https://issues.apache.org/jira/browse/SPARK-28849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28849: Description: Spark's {{UnsafeShuffleWriter}} may run into infinite loop when calling {{transferTo

[jira] [Created] (SPARK-28849) Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally

Saisai Shao created SPARK-28849: --- Summary: Spark's UnsafeShuffleWriter may run into infinite loop in transferTo occasionally Key: SPARK-28849 URL: https://issues.apache.org/jira/browse/SPARK-28849 Proje

[jira] [Assigned] (SPARK-28475) Add regex MetricFilter to GraphiteSink

[ https://issues.apache.org/jira/browse/SPARK-28475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-28475: --- Assignee: Nick Karpov > Add regex MetricFilter to GraphiteSink > --

[jira] [Updated] (SPARK-28475) Add regex MetricFilter to GraphiteSink

[ https://issues.apache.org/jira/browse/SPARK-28475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28475: Priority: Minor (was: Major) > Add regex MetricFilter to GraphiteSink > -

[jira] [Resolved] (SPARK-28475) Add regex MetricFilter to GraphiteSink

[ https://issues.apache.org/jira/browse/SPARK-28475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-28475. - Resolution: Resolved Fix Version/s: 3.0.0 > Add regex MetricFilter to GraphiteSink >

[jira] [Commented] (SPARK-28475) Add regex MetricFilter to GraphiteSink

[ https://issues.apache.org/jira/browse/SPARK-28475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898749#comment-16898749 ] Saisai Shao commented on SPARK-28475: - This is resolved via https://github.com/apach

[jira] [Assigned] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-28106: --- Assignee: angerszhu > Spark SQL add jar with wrong hdfs path, SparkContext still add it to

[jira] [Resolved] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-28106. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24909 [https://gith

[jira] [Updated] (SPARK-28106) Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar path ,and cause Task Failed

[ https://issues.apache.org/jira/browse/SPARK-28106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-28106: Component/s: Spark Core > Spark SQL add jar with wrong hdfs path, SparkContext still add it to jar

[jira] [Assigned] (SPARK-28202) [Core] [Test] Avoid noises of system props in SparkConfSuite

[ https://issues.apache.org/jira/browse/SPARK-28202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-28202: --- Assignee: ShuMing Li > [Core] [Test] Avoid noises of system props in SparkConfSuite > -

[jira] [Resolved] (SPARK-28202) [Core] [Test] Avoid noises of system props in SparkConfSuite

[ https://issues.apache.org/jira/browse/SPARK-28202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-28202. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24998 [https://gith

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876070#comment-16876070 ] Saisai Shao commented on SPARK-25299: - Better to post a pdf version [~mcheah] :). >

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16874703#comment-16874703 ] Saisai Shao commented on SPARK-25299: - Votes were passed, so what is our plan for co

[jira] [Updated] (SPARK-25299) Use remote storage for persisting shuffle data

[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25299: Labels: SPIP (was: ) > Use remote storage for persisting shuffle data > -

[jira] [Created] (SPARK-27996) Spark UI redirect will be failed behind the https reverse proxy

Saisai Shao created SPARK-27996: --- Summary: Spark UI redirect will be failed behind the https reverse proxy Key: SPARK-27996 URL: https://issues.apache.org/jira/browse/SPARK-27996 Project: Spark

[jira] [Commented] (SPARK-15348) Hive ACID

[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844750#comment-16844750 ] Saisai Shao commented on SPARK-15348: - No, it doesn't support hive ACID, it has its

[jira] [Commented] (SPARK-15348) Hive ACID

[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844597#comment-16844597 ] Saisai Shao commented on SPARK-15348: - I think delta lake project is exactly what yo

[jira] [Comment Edited] (SPARK-15348) Hive ACID

[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16844597#comment-16844597 ] Saisai Shao edited comment on SPARK-15348 at 5/21/19 7:38 AM:

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750872#comment-16750872 ] Saisai Shao commented on SPARK-24615: - I'm really sorry about the delay. Due to some

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10

[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734072#comment-16734072 ] Saisai Shao commented on SPARK-26512: - This seems like a Netty version problem, nett

[jira] [Commented] (SPARK-26512) Spark 2.4.0 is not working with Hadoop 2.8.3 in windows 10?

[ https://issues.apache.org/jira/browse/SPARK-26512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733740#comment-16733740 ] Saisai Shao commented on SPARK-26512: - Please list the problems you saw, any log or

[jira] [Updated] (SPARK-26457) Show hadoop configurations in HistoryServer environment tab

[ https://issues.apache.org/jira/browse/SPARK-26457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-26457: Priority: Minor (was: Major) > Show hadoop configurations in HistoryServer environment tab >

[jira] [Resolved] (SPARK-26516) zeppelin with spark on mesos: environment variable setting

[ https://issues.apache.org/jira/browse/SPARK-26516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-26516. - Resolution: Invalid > zeppelin with spark on mesos: environment variable setting > -

[jira] [Updated] (SPARK-26513) Trigger GC on executor node idle

[ https://issues.apache.org/jira/browse/SPARK-26513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-26513: Fix Version/s: (was: 3.0.0) > Trigger GC on executor node idle > -

[jira] [Commented] (SPARK-26516) zeppelin with spark on mesos: environment variable setting

[ https://issues.apache.org/jira/browse/SPARK-26516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732587#comment-16732587 ] Saisai Shao commented on SPARK-26516: - Questions should go to user@spark mail list.

[jira] [Commented] (SPARK-20415) SPARK job hangs while writing DataFrame to HDFS

[ https://issues.apache.org/jira/browse/SPARK-20415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16730084#comment-16730084 ] Saisai Shao commented on SPARK-20415: - Have you tried latest version of Spark, does

[jira] [Commented] (SPARK-25299) Use remote storage for persisting shuffle data

[ https://issues.apache.org/jira/browse/SPARK-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729357#comment-16729357 ] Saisai Shao commented on SPARK-25299: - [~jealous] Can we have a doc about this propo

[jira] [Updated] (SPARK-25501) Kafka delegation token support

[ https://issues.apache.org/jira/browse/SPARK-25501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25501: Labels: SPIP (was: ) > Kafka delegation token support > -- > >

[jira] [Updated] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25206: Labels: Parquet correctness known_issue (was: Parquet correctness) > wrong records are returned w

[jira] [Assigned] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24615: --- Assignee: (was: Saisai Shao) > Accelerator-aware task scheduling for Spark > --

[jira] [Assigned] (SPARK-25183) Spark HiveServer2 registers shutdown hook with JVM, not ShutdownHookManager; race conditions can arise

[ https://issues.apache.org/jira/browse/SPARK-25183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-25183: --- Assignee: Steve Loughran > Spark HiveServer2 registers shutdown hook with JVM, not Shutdown

[jira] [Resolved] (SPARK-25183) Spark HiveServer2 registers shutdown hook with JVM, not ShutdownHookManager; race conditions can arise

[ https://issues.apache.org/jira/browse/SPARK-25183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-25183. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22186 [https://gith

[jira] [Commented] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598115#comment-16598115 ] Saisai Shao commented on SPARK-25206: - I see, if it is not going to be merged, let's

[jira] [Commented] (SPARK-25135) insert datasource table may all null when select from view on parquet

[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598111#comment-16598111 ] Saisai Shao commented on SPARK-25135: - What's the ETA of this issue [~yumwang]? > i

[jira] [Commented] (SPARK-25206) wrong records are returned when Hive metastore schema and parquet schema are in different letter cases

[ https://issues.apache.org/jira/browse/SPARK-25206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598110#comment-16598110 ] Saisai Shao commented on SPARK-25206: - What is the status of this JIRA, are we going

[jira] [Updated] (SPARK-24785) Making sure REPL prints Spark UI info and then Welcome message

[ https://issues.apache.org/jira/browse/SPARK-24785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24785: Fix Version/s: 2.4.0 > Making sure REPL prints Spark UI info and then Welcome message > --

[jira] [Updated] (SPARK-25221) [DEPLOY] Consistent trailing whitespace treatment of conf values

[ https://issues.apache.org/jira/browse/SPARK-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25221: Target Version/s: (was: 2.3.2, 2.4.0) > [DEPLOY] Consistent trailing whitespace treatment of con

[jira] [Commented] (SPARK-25221) [DEPLOY] Consistent trailing whitespace treatment of conf values

[ https://issues.apache.org/jira/browse/SPARK-25221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593124#comment-16593124 ] Saisai Shao commented on SPARK-25221: - I'm going to remove the target version, I don

[jira] [Commented] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587062#comment-16587062 ] Saisai Shao commented on SPARK-25131: - This is by design, not a bug. > Event logs m

[jira] [Updated] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-25144: Target Version/s: 2.2.3, 2.3.2 > distinct on Dataset leads to exception due to Managed memory leak

[jira] [Commented] (SPARK-25144) distinct on Dataset leads to exception due to Managed memory leak detected

[ https://issues.apache.org/jira/browse/SPARK-25144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585482#comment-16585482 ] Saisai Shao commented on SPARK-25144: - Does this exist in master branch? > distinct

[jira] [Commented] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585473#comment-16585473 ] Saisai Shao commented on SPARK-23679: - Let me fix this issue. This is mainly a RM HA

[jira] [Updated] (SPARK-23679) uiWebUrl show inproper URL when running on YARN

[ https://issues.apache.org/jira/browse/SPARK-23679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23679: Component/s: YARN > uiWebUrl show inproper URL when running on YARN >

[jira] [Commented] (SPARK-25114) RecordBinaryComparator may return wrong result when subtraction between two words is divisible by Integer.MAX_VALUE

[ https://issues.apache.org/jira/browse/SPARK-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585321#comment-16585321 ] Saisai Shao commented on SPARK-25114: - What's the ETA of this issue [~jiangxb1987]?

[jira] [Commented] (SPARK-25020) Unable to Perform Graceful Shutdown in Spark Streaming with Hadoop 2.8

[ https://issues.apache.org/jira/browse/SPARK-25020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16577846#comment-16577846 ] Saisai Shao commented on SPARK-25020: - Yes, we also met the same issue. This is more

[jira] [Commented] (SPARK-25084) "distribute by" on multiple columns may lead to codegen issue

[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575778#comment-16575778 ] Saisai Shao commented on SPARK-25084: - I see. Unfortunately I've cut the RC4, if it

[jira] [Commented] (SPARK-25084) "distribute by" on multiple columns may lead to codegen issue

[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575774#comment-16575774 ] Saisai Shao commented on SPARK-25084: - Is this a regression or just a bug existed in

[jira] [Commented] (SPARK-25084) "distribute by" on multiple columns may lead to codegen issue

[ https://issues.apache.org/jira/browse/SPARK-25084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575772#comment-16575772 ] Saisai Shao commented on SPARK-25084: - I'm already preparing new RC4. If this is not

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Fix Version/s: 2.2.3 > SHS filters wrongly some applications due to permission check > ---

[jira] [Assigned] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24948: --- Assignee: Marco Gaido > SHS filters wrongly some applications due to permission check > ---

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Fix Version/s: 2.3.2 > SHS filters wrongly some applications due to permission check > ---

[jira] [Commented] (SPARK-22634) Update Bouncy castle dependency

[ https://issues.apache.org/jira/browse/SPARK-22634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572542#comment-16572542 ] Saisai Shao commented on SPARK-22634: - [~srowen] I'm wondering if it is possible to

[jira] [Commented] (SPARK-25030) SparkSubmit.doSubmit will not return result if the mainClass submitted creates a Timer()

[ https://issues.apache.org/jira/browse/SPARK-25030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570995#comment-16570995 ] Saisai Shao commented on SPARK-25030: - I would like to see more about this issue. >

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Priority: Blocker (was: Major) > SHS filters wrongly some applications due to permission check >

[jira] [Updated] (SPARK-24948) SHS filters wrongly some applications due to permission check

[ https://issues.apache.org/jira/browse/SPARK-24948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24948: Target Version/s: 2.2.3, 2.3.2, 2.4.0 > SHS filters wrongly some applications due to permission ch

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564562#comment-16564562 ] Saisai Shao commented on SPARK-24615: - Leveraging dynamic allocation to tear down an

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564562#comment-16564562 ] Saisai Shao edited comment on SPARK-24615 at 8/1/18 12:35 AM:

[jira] [Comment Edited] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563730#comment-16563730 ] Saisai Shao edited comment on SPARK-24615 at 7/31/18 2:16 PM:

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563730#comment-16563730 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves], I think eval() might unnecessa

[jira] [Resolved] (SPARK-24975) Spark history server REST API /api/v1/version returns error 404

[ https://issues.apache.org/jira/browse/SPARK-24975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24975. - Resolution: Duplicate > Spark history server REST API /api/v1/version returns error 404 > --

[jira] [Updated] (SPARK-24957) Decimal arithmetic can lead to wrong values using codegen

[ https://issues.apache.org/jira/browse/SPARK-24957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24957: Target Version/s: 2.4.0 > Decimal arithmetic can lead to wrong values using codegen >

[jira] [Commented] (SPARK-24957) Decimal arithmetic can lead to wrong values using codegen

[ https://issues.apache.org/jira/browse/SPARK-24957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561334#comment-16561334 ] Saisai Shao commented on SPARK-24957: - Not necessary to mark as blocker, we still ha

[jira] [Updated] (SPARK-24932) Allow update mode for streaming queries with join

[ https://issues.apache.org/jira/browse/SPARK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24932: Target Version/s: (was: 2.3.2) > Allow update mode for streaming queries with join > ---

[jira] [Commented] (SPARK-24932) Allow update mode for streaming queries with join

[ https://issues.apache.org/jira/browse/SPARK-24932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561331#comment-16561331 ] Saisai Shao commented on SPARK-24932: - I'm going to remove the target version for th

[jira] [Updated] (SPARK-24964) Please add OWASP Dependency Check to all comonent builds(pom.xml)

[ https://issues.apache.org/jira/browse/SPARK-24964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24964: Target Version/s: (was: 2.3.2, 2.4.0, 3.0.0, 2.3.3) > Please add OWASP Dependency Check to all

[jira] [Commented] (SPARK-24964) Please add OWASP Dependency Check to all comonent builds(pom.xml)

[ https://issues.apache.org/jira/browse/SPARK-24964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561327#comment-16561327 ] Saisai Shao commented on SPARK-24964: - I'm going to remove the set target version, u

[jira] [Commented] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556760#comment-16556760 ] Saisai Shao commented on SPARK-24867: - I see, thanks! Please let me know when the JI

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555080#comment-16555080 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] [~irashid] thanks a lot for you

[jira] [Commented] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554986#comment-16554986 ] Saisai Shao commented on SPARK-24867: - [~smilegator] what's the ETA of this issue?

[jira] [Assigned] (SPARK-24297) Change default value for spark.maxRemoteBlockSizeFetchToMem to be < 2GB

[ https://issues.apache.org/jira/browse/SPARK-24297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24297: --- Assignee: Imran Rashid > Change default value for spark.maxRemoteBlockSizeFetchToMem to be

[jira] [Resolved] (SPARK-24297) Change default value for spark.maxRemoteBlockSizeFetchToMem to be < 2GB

[ https://issues.apache.org/jira/browse/SPARK-24297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24297. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21474 [https://gith

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16553679#comment-16553679 ] Saisai Shao commented on SPARK-24615: - [~Tagar] yes! > Accelerator-aware task sched

[jira] [Resolved] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24594. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21635 [https://gith

[jira] [Assigned] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24594: --- Assignee: Attila Zsolt Piros > Introduce metrics for YARN executor allocation problems > -

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552233#comment-16552233 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] what you mentioned above is als

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550266#comment-16550266 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] I'm still not sure how to handl

[jira] [Commented] (SPARK-24723) Discuss necessary info and access in barrier mode + YARN

[ https://issues.apache.org/jira/browse/SPARK-24723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550182#comment-16550182 ] Saisai Shao commented on SPARK-24723: - Hi [~mengxr], I don't think YARN has such fea

[jira] [Assigned] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24195: --- Assignee: Li Yuanjian > sc.addFile for local:/ path is broken > ---

[jira] [Resolved] (SPARK-24195) sc.addFile for local:/ path is broken

[ https://issues.apache.org/jira/browse/SPARK-24195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24195. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21533 [https://gith

[jira] [Commented] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550167#comment-16550167 ] Saisai Shao commented on SPARK-24307: - Issue resolved by pull request 21440 [https:/

[jira] [Resolved] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24307. - Resolution: Fixed Assignee: Imran Rashid > Support sending messages over 2GB from memory >

[jira] [Updated] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-24307: Fix Version/s: 2.4.0 > Support sending messages over 2GB from memory > ---

  1   2   3   4   5   6   7   8   9   10   >