[jira] [Commented] (SPARK-24137) [K8s] Mount temporary directories in emptydir volumes

[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459900#comment-16459900 ] Yinan Li commented on SPARK-24137: -- Yeah, {{LocalDirectoryMountConfigurationStep}} was missed in the

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459892#comment-16459892 ] Yinan Li commented on SPARK-24135: -- I think it's fine detecting and deleting the executor pods that

[jira] [Commented] (SPARK-23074) Dataframe-ified zipwithindex

[ https://issues.apache.org/jira/browse/SPARK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460087#comment-16460087 ] Ruslan Dautkhanov commented on SPARK-23074: --- [~rxin], monotonically_increasing_id wouldn't

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459954#comment-16459954 ] Erik Erlandson commented on SPARK-24135: I think it makes sense to detect these failure states. 

[jira] [Commented] (SPARK-23074) Dataframe-ified zipwithindex

[ https://issues.apache.org/jira/browse/SPARK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460116#comment-16460116 ] Reynold Xin commented on SPARK-23074: - It should give you the same ordering - that's the

[jira] [Commented] (SPARK-23074) Dataframe-ified zipwithindex

[ https://issues.apache.org/jira/browse/SPARK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459979#comment-16459979 ] Reynold Xin commented on SPARK-23074: - Can your problem be solved by monotonically_increasing_id,

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460047#comment-16460047 ] Matt Cheah commented on SPARK-24135: _> But I'm not sure how much this buys us because very likely

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460066#comment-16460066 ] Yinan Li commented on SPARK-24135: -- I agree that we should add detection for initialization errors. But

[jira] [Comment Edited] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460066#comment-16460066 ] Yinan Li edited comment on SPARK-24135 at 5/1/18 7:53 PM: -- I agree that we

[jira] [Assigned] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-6305: - Assignee: (was: Marcelo Vanzin) > Add support for log4j 2.x to Spark >

[jira] [Assigned] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-6305: - Assignee: Marcelo Vanzin > Add support for log4j 2.x to Spark >

[jira] [Commented] (SPARK-23686) Make better usage of org.apache.spark.ml.util.Instrumentation

[ https://issues.apache.org/jira/browse/SPARK-23686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460164#comment-16460164 ] Joseph K. Bradley commented on SPARK-23686: --- [~yogeshgarg] made the good point that we should

[jira] [Commented] (SPARK-23798) The CreateArray and ConcatArray should return the default array type when no children provided

[ https://issues.apache.org/jira/browse/SPARK-23798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460329#comment-16460329 ] Takeshi Yamamuro commented on SPARK-23798: -- That pr was to make the behaviour along with the

[jira] [Updated] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23458: -- Description: Sometimes we have UT failures with the following stacktrace: {code:java}

[jira] [Commented] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460336#comment-16460336 ] Dongjoon Hyun commented on SPARK-23458: --- cc [~smilegator] > Flaky test: OrcQuerySuite >

[jira] [Comment Edited] (SPARK-23074) Dataframe-ified zipwithindex

[ https://issues.apache.org/jira/browse/SPARK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460137#comment-16460137 ] Ruslan Dautkhanov edited comment on SPARK-23074 at 5/1/18 8:56 PM: ---

[jira] [Resolved] (SPARK-24138) Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-24138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24138. --- Resolution: Duplicate > Flaky test:

[jira] [Updated] (SPARK-15750) Constructing FPGrowth fails when no numPartitions specified in pyspark

[ https://issues.apache.org/jira/browse/SPARK-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15750: -- Shepherd: Joseph K. Bradley > Constructing FPGrowth fails when no numPartitions

[jira] [Assigned] (SPARK-15750) Constructing FPGrowth fails when no numPartitions specified in pyspark

[ https://issues.apache.org/jira/browse/SPARK-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-15750: - Assignee: Jeff Zhang > Constructing FPGrowth fails when no numPartitions

[jira] [Commented] (SPARK-23074) Dataframe-ified zipwithindex

[ https://issues.apache.org/jira/browse/SPARK-23074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460137#comment-16460137 ] Ruslan Dautkhanov commented on SPARK-23074: --- That's . When we have a "record" that spans

[jira] [Commented] (SPARK-23798) The CreateArray and ConcatArray should return the default array type when no children provided

[ https://issues.apache.org/jira/browse/SPARK-23798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460213#comment-16460213 ] Sunitha Kambhampati commented on SPARK-23798: - ArrayType.defaultConcreteType is

[jira] [Commented] (SPARK-14146) Imported implicits can't be found in Spark REPL in some cases

[ https://issues.apache.org/jira/browse/SPARK-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460239#comment-16460239 ] Aaron S. Hawley commented on SPARK-14146: - Should be resolved in Scala after this was merged:

[jira] [Commented] (SPARK-24138) Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-24138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460335#comment-16460335 ] Dongjoon Hyun commented on SPARK-24138: --- I see. Sure. [~smilegator]. Actually, we observed this

[jira] [Assigned] (SPARK-23925) High-order function: repeat(element, count) → array

[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23925: Assignee: Apache Spark > High-order function: repeat(element, count) → array >

[jira] [Assigned] (SPARK-23925) High-order function: repeat(element, count) → array

[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23925: Assignee: (was: Apache Spark) > High-order function: repeat(element, count) → array >

[jira] [Commented] (SPARK-23925) High-order function: repeat(element, count) → array

[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460174#comment-16460174 ] Apache Spark commented on SPARK-23925: -- User 'pepinoflo' has created a pull request for this issue:

[jira] [Commented] (SPARK-24138) Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-24138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460266#comment-16460266 ] Xiao Li commented on SPARK-24138: - cc [~dongjoon] Could you take a look at this? > Flaky test:

[jira] [Reopened] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

[ https://issues.apache.org/jira/browse/SPARK-23197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-23197: - > Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle" >

[jira] [Commented] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

[ https://issues.apache.org/jira/browse/SPARK-23197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460275#comment-16460275 ] Xiao Li commented on SPARK-23197: - [~tdas] We are still hitting the same issue in the latest build tests.

[jira] [Comment Edited] (SPARK-23686) Make better usage of org.apache.spark.ml.util.Instrumentation

[ https://issues.apache.org/jira/browse/SPARK-23686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460164#comment-16460164 ] Joseph K. Bradley edited comment on SPARK-23686 at 5/1/18 9:52 PM: ---

[jira] [Created] (SPARK-24138) Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite

Xiao Li created SPARK-24138: --- Summary: Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite Key: SPARK-24138 URL: https://issues.apache.org/jira/browse/SPARK-24138 Project: Spark

[jira] [Updated] (SPARK-24138) Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-24138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24138: Shepherd: Wenchen Fan > Flaky test: org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite >

[jira] [Comment Edited] (SPARK-23686) Make better usage of org.apache.spark.ml.util.Instrumentation

[ https://issues.apache.org/jira/browse/SPARK-23686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460164#comment-16460164 ] Joseph K. Bradley edited comment on SPARK-23686 at 5/2/18 12:21 AM:

[jira] [Commented] (SPARK-24142) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460525#comment-16460525 ] Bruce Robbins commented on SPARK-24142: --- I opened another Jira on this a few days ago, but it was

[jira] [Resolved] (SPARK-24142) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-24142. --- Resolution: Duplicate > Add interpreted execution to SortPrefix expression >

[jira] [Created] (SPARK-24144) monotonically_increasing_id on streaming dataFrames

Hemant Bhanawat created SPARK-24144: --- Summary: monotonically_increasing_id on streaming dataFrames Key: SPARK-24144 URL: https://issues.apache.org/jira/browse/SPARK-24144 Project: Spark

[jira] [Updated] (SPARK-24144) monotonically_increasing_id on streaming dataFrames

[ https://issues.apache.org/jira/browse/SPARK-24144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Bhanawat updated SPARK-24144: Priority: Major (was: Minor) > monotonically_increasing_id on streaming dataFrames >

[jira] [Updated] (SPARK-24119) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-24119: - Issue Type: Sub-task (was: Task) Parent: SPARK-23580 > Add interpreted

[jira] [Updated] (SPARK-24142) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24142: -- Affects Version/s: (was: 2.3.0) 2.4.0 > Add interpreted execution

[jira] [Commented] (SPARK-24142) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460532#comment-16460532 ] Bruce Robbins commented on SPARK-24142: --- [~maropu] I don't seem to have the Jira authority (or

[jira] [Created] (SPARK-24143) filter empty blocks when convert mapstatus to (blockId, size) pair

jin xing created SPARK-24143: Summary: filter empty blocks when convert mapstatus to (blockId, size) pair Key: SPARK-24143 URL: https://issues.apache.org/jira/browse/SPARK-24143 Project: Spark

[jira] [Updated] (SPARK-24143) filter empty blocks when convert mapstatus to (blockId, size) pair

[ https://issues.apache.org/jira/browse/SPARK-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-24143: - Issue Type: Bug (was: New Feature) > filter empty blocks when convert mapstatus to (blockId, size) pair

[jira] [Assigned] (SPARK-24143) filter empty blocks when convert mapstatus to (blockId, size) pair

[ https://issues.apache.org/jira/browse/SPARK-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24143: Assignee: Apache Spark > filter empty blocks when convert mapstatus to (blockId, size)

[jira] [Commented] (SPARK-24143) filter empty blocks when convert mapstatus to (blockId, size) pair

[ https://issues.apache.org/jira/browse/SPARK-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460548#comment-16460548 ] Apache Spark commented on SPARK-24143: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24143) filter empty blocks when convert mapstatus to (blockId, size) pair

[ https://issues.apache.org/jira/browse/SPARK-24143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24143: Assignee: (was: Apache Spark) > filter empty blocks when convert mapstatus to

[jira] [Commented] (SPARK-24142) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460555#comment-16460555 ] Takeshi Yamamuro commented on SPARK-24142: -- I think it is ok as it is, resolved as duplicate. >

[jira] [Created] (SPARK-24142) Add interpreted execution to SortPrefix expression

Bruce Robbins created SPARK-24142: - Summary: Add interpreted execution to SortPrefix expression Key: SPARK-24142 URL: https://issues.apache.org/jira/browse/SPARK-24142 Project: Spark Issue

[jira] [Commented] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

[ https://issues.apache.org/jira/browse/SPARK-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460529#comment-16460529 ] Apache Spark commented on SPARK-24131: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-23489) Flaky Test: HiveExternalCatalogVersionsSuite

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Affects Version/s: 2.2.1 2.3.0 > Flaky Test:

[jira] [Updated] (SPARK-23489) Flaky Test: HiveExternalCatalogVersionsSuite

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Summary: Flaky Test: HiveExternalCatalogVersionsSuite (was: HiveExternalCatalogVersionsSuite

[jira] [Commented] (SPARK-24119) Add interpreted execution to SortPrefix expression

[ https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460527#comment-16460527 ] Bruce Robbins commented on SPARK-24119: --- [~maropu] Ahh... we crossed paths and I opened a second

[jira] [Assigned] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23489: Assignee: Apache Spark > HiveExternalCatalogVersionsSuite flaky test >

[jira] [Assigned] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23489: Assignee: (was: Apache Spark) > HiveExternalCatalogVersionsSuite flaky test >

[jira] [Commented] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460528#comment-16460528 ] Apache Spark commented on SPARK-23489: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-24137) [K8s] Mount temporary directories in emptydir volumes

[ https://issues.apache.org/jira/browse/SPARK-24137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460554#comment-16460554 ] Anirudh Ramanathan commented on SPARK-24137: SGTM. Let's try and share as much code as

[jira] [Created] (SPARK-24139) Fix Flaky Tests

Dongjoon Hyun created SPARK-24139: - Summary: Fix Flaky Tests Key: SPARK-24139 URL: https://issues.apache.org/jira/browse/SPARK-24139 Project: Spark Issue Type: Umbrella Components:

[jira] [Created] (SPARK-24140) Flaky test: KMeansClusterSuite.task size should be small in both training and prediction

Dongjoon Hyun created SPARK-24140: - Summary: Flaky test: KMeansClusterSuite.task size should be small in both training and prediction Key: SPARK-24140 URL: https://issues.apache.org/jira/browse/SPARK-24140

[jira] [Updated] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Description: I saw this error in an unrelated PR. It seems to me a bad configuration in the

[jira] [Assigned] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

[ https://issues.apache.org/jira/browse/SPARK-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24141: Assignee: Apache Spark > Fix bug in CoarseGrainedSchedulerBackend.killExecutors >

[jira] [Updated] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Component/s: SQL > HiveExternalCatalogVersionsSuite flaky test >

[jira] [Resolved] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

[ https://issues.apache.org/jira/browse/SPARK-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24131. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21203

[jira] [Resolved] (SPARK-24089) DataFrame.write().mode(SaveMode.Append).insertInto(TABLE)

[ https://issues.apache.org/jira/browse/SPARK-24089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kumar resolved SPARK-24089. --- Resolution: Not A Problem Thanks for the clarification.  >

[jira] [Created] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

wuyi created SPARK-24141: Summary: Fix bug in CoarseGrainedSchedulerBackend.killExecutors Key: SPARK-24141 URL: https://issues.apache.org/jira/browse/SPARK-24141 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-23458) Flaky test: OrcQuerySuite

[ https://issues.apache.org/jira/browse/SPARK-23458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460336#comment-16460336 ] Dongjoon Hyun edited comment on SPARK-23458 at 5/2/18 12:36 AM: We

[jira] [Updated] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Description: I saw this error in an unrelated PR. It seems to me a bad configuration in the

[jira] [Commented] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

[ https://issues.apache.org/jira/browse/SPARK-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460404#comment-16460404 ] Apache Spark commented on SPARK-24141: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24141) Fix bug in CoarseGrainedSchedulerBackend.killExecutors

[ https://issues.apache.org/jira/browse/SPARK-24141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24141: Assignee: (was: Apache Spark) > Fix bug in

[jira] [Commented] (SPARK-24116) SparkSQL inserting overwrite table has inconsistent behavior regarding HDFS trash

[ https://issues.apache.org/jira/browse/SPARK-24116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460472#comment-16460472 ] Rui Li commented on SPARK-24116: [~hyukjin.kwon], sorry for the late response. For example, assume we

[jira] [Updated] (SPARK-24139) Fix Flaky Tests

[ https://issues.apache.org/jira/browse/SPARK-24139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24139: -- Description: As of today, master and branch-2.3 branches suffer from several flaky tests. This

[jira] [Assigned] (SPARK-24131) Add majorMinorVersion API to PySpark for determining Spark versions

[ https://issues.apache.org/jira/browse/SPARK-24131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24131: Assignee: Liang-Chi Hsieh > Add majorMinorVersion API to PySpark for determining Spark

[jira] [Commented] (SPARK-24116) SparkSQL inserting overwrite table has inconsistent behavior regarding HDFS trash

[ https://issues.apache.org/jira/browse/SPARK-24116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460473#comment-16460473 ] Hyukjin Kwon commented on SPARK-24116: -- [~lirui], mind if I ask a reproducer please? > SparkSQL

[jira] [Updated] (SPARK-23415) BufferHolderSparkSubmitSuite is flaky

[ https://issues.apache.org/jira/browse/SPARK-23415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23415: -- Description: The test suite fails due to 60-second timeout sometimes. {code} Error Message

[jira] [Updated] (SPARK-23489) HiveExternalCatalogVersionsSuite flaky test

[ https://issues.apache.org/jira/browse/SPARK-23489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23489: -- Issue Type: Bug (was: Task) > HiveExternalCatalogVersionsSuite flaky test >

[jira] [Created] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

Xiaodong DENG created SPARK-24134: - Summary: A missing full-stop in doc "Tuning Spark" Key: SPARK-24134 URL: https://issues.apache.org/jira/browse/SPARK-24134 Project: Spark Issue Type:

[jira] [Updated] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG updated SPARK-24134: -- Attachment: 39468206-778e3d8a-4d64-11e8-8a92-38464952b54b.png > A missing full-stop in doc

[jira] [Assigned] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24134: Assignee: Apache Spark > A missing full-stop in doc "Tuning Spark" >

[jira] [Commented] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459664#comment-16459664 ] Apache Spark commented on SPARK-24134: -- User 'XD-DENG' has created a pull request for this issue:

[jira] [Updated] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG updated SPARK-24134: -- Attachment: screenshot.png > A missing full-stop in doc "Tuning Spark" >

[jira] [Updated] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG updated SPARK-24134: -- Attachment: (was: 39468206-778e3d8a-4d64-11e8-8a92-38464952b54b.png) > A missing full-stop

[jira] [Assigned] (SPARK-24134) A missing full-stop in doc "Tuning Spark"

[ https://issues.apache.org/jira/browse/SPARK-24134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24134: Assignee: (was: Apache Spark) > A missing full-stop in doc "Tuning Spark" >

[jira] [Resolved] (SPARK-23342) Add ORC configuration tests for ORC data source

[ https://issues.apache.org/jira/browse/SPARK-23342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-23342. --- Resolution: Not A Problem I close this issue as a `Not A Problem`. > Add ORC configuration

[jira] [Assigned] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

[ https://issues.apache.org/jira/browse/SPARK-24061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-24061: Assignee: Wang Yanlin > [SS]TypedFilter is not supported in Continuous Processing >

[jira] [Created] (SPARK-24132) Instruments improvement for classification

Lu Wang created SPARK-24132: --- Summary: Instruments improvement for classification Key: SPARK-24132 URL: https://issues.apache.org/jira/browse/SPARK-24132 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-23993) Support DESC FORMATTED table_name column_name

[ https://issues.apache.org/jira/browse/SPARK-23993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Volodymyr Glushak resolved SPARK-23993. --- Resolution: Duplicate Fix Version/s: 2.3.0 Duplicate of SPARK-17642. Thanks

[jira] [Created] (SPARK-24133) Reading Parquet files containing large strings can fail with java.lang.ArrayIndexOutOfBoundsException

Ala Luszczak created SPARK-24133: Summary: Reading Parquet files containing large strings can fail with java.lang.ArrayIndexOutOfBoundsException Key: SPARK-24133 URL:

[jira] [Commented] (SPARK-24132) Instruments improvement for classification

[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459477#comment-16459477 ] Apache Spark commented on SPARK-24132: -- User 'ludatabricks' has created a pull request for this

[jira] [Assigned] (SPARK-24132) Instruments improvement for classification

[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24132: Assignee: Apache Spark > Instruments improvement for classification >

[jira] [Assigned] (SPARK-24132) Instruments improvement for classification

[ https://issues.apache.org/jira/browse/SPARK-24132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24132: Assignee: (was: Apache Spark) > Instruments improvement for classification >

[jira] [Resolved] (SPARK-24061) [SS]TypedFilter is not supported in Continuous Processing

[ https://issues.apache.org/jira/browse/SPARK-24061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-24061. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21136

[jira] [Assigned] (SPARK-24133) Reading Parquet files containing large strings can fail with java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24133: Assignee: (was: Apache Spark) > Reading Parquet files containing large strings can

[jira] [Assigned] (SPARK-24133) Reading Parquet files containing large strings can fail with java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24133: Assignee: Apache Spark > Reading Parquet files containing large strings can fail with >

[jira] [Commented] (SPARK-24133) Reading Parquet files containing large strings can fail with java.lang.ArrayIndexOutOfBoundsException

[ https://issues.apache.org/jira/browse/SPARK-24133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459562#comment-16459562 ] Apache Spark commented on SPARK-24133: -- User 'ala' has created a pull request for this issue:

[jira] [Commented] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459754#comment-16459754 ] Matt Cheah commented on SPARK-24135: [~foxish] [~eje] [~liyinan926] wanted to get feedback on this -

[jira] [Assigned] (SPARK-23941) Mesos task failed on specific spark app name

[ https://issues.apache.org/jira/browse/SPARK-23941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23941: -- Assignee: bounkong khamphousone > Mesos task failed on specific spark app name >

[jira] [Resolved] (SPARK-23941) Mesos task failed on specific spark app name

[ https://issues.apache.org/jira/browse/SPARK-23941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23941. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0

[jira] [Created] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

Matt Cheah created SPARK-24135: -- Summary: [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size Key: SPARK-24135 URL:

[jira] [Updated] (SPARK-22885) ML test for StructuredStreaming: spark.ml.tuning

[ https://issues.apache.org/jira/browse/SPARK-22885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-22885: -- Shepherd: Joseph K. Bradley > ML test for StructuredStreaming: spark.ml.tuning >

[jira] [Assigned] (SPARK-22885) ML test for StructuredStreaming: spark.ml.tuning

[ https://issues.apache.org/jira/browse/SPARK-22885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-22885: - Assignee: Weichen Xu > ML test for StructuredStreaming: spark.ml.tuning >

[jira] [Created] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

Arun Mahadevan created SPARK-24136: -- Summary: MemoryStreamDataReader.next should skip sleeping if record is available Key: SPARK-24136 URL: https://issues.apache.org/jira/browse/SPARK-24136 Project:

[jira] [Assigned] (SPARK-24136) MemoryStreamDataReader.next should skip sleeping if record is available

[ https://issues.apache.org/jira/browse/SPARK-24136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24136: Assignee: (was: Apache Spark) > MemoryStreamDataReader.next should skip sleeping if

  1   2   >