[jira] [Comment Edited] (SPARK-24382) Spark Structured Streaming aggregation on old timestamp data

[ https://issues.apache.org/jira/browse/SPARK-24382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523227#comment-16523227 ] Richard Yu edited comment on SPARK-24382 at 6/26/18 5:34 AM: - [~karthikus]

[jira] [Commented] (SPARK-24382) Spark Structured Streaming aggregation on old timestamp data

[ https://issues.apache.org/jira/browse/SPARK-24382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523227#comment-16523227 ] Richard Yu commented on SPARK-24382: [~karthikus] Yes, the old dates would in fact matter, primarily

[jira] [Assigned] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24658: Assignee: (was: Apache Spark) > Remove workaround for ANTLR bug >

[jira] [Assigned] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24658: Assignee: Apache Spark > Remove workaround for ANTLR bug >

[jira] [Commented] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523223#comment-16523223 ] Apache Spark commented on SPARK-24658: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-24658) Remove workaround for ANTLR bug

Yuming Wang created SPARK-24658: --- Summary: Remove workaround for ANTLR bug Key: SPARK-24658 URL: https://issues.apache.org/jira/browse/SPARK-24658 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-24657) SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleanup resource when finished the merge join

[ https://issues.apache.org/jira/browse/SPARK-24657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-24657: - Description: In my sql, It join three tables, and all these tables are small table (about

[jira] [Updated] (SPARK-24657) SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleanup resource when finished the merge join

[ https://issues.apache.org/jira/browse/SPARK-24657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-24657: - Description: In my sql, It join three tables, and all these tables are small table (about

[jira] [Created] (SPARK-24657) SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleaning resource when finished the merge join

Joshuawangzj created SPARK-24657: Summary: SortMergeJoin may cause SparkOutOfMemory in execution memory because of not cleaning resource when finished the merge join Key: SPARK-24657 URL:

[jira] [Issue Comment Deleted] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish updated SPARK-18681: --- Comment: was deleted (was: [~michael] I see the same issue in 2.3.1. I have hive partitioned table with

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523064#comment-16523064 ] Li Yuanjian commented on SPARK-24630: - cc [~zsxwing] and [~tdas] We have some practice over

[jira] [Resolved] (SPARK-24636) Type Coercion of Arrays for array_join Function

[ https://issues.apache.org/jira/browse/SPARK-24636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24636. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21620

[jira] [Assigned] (SPARK-24636) Type Coercion of Arrays for array_join Function

[ https://issues.apache.org/jira/browse/SPARK-24636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-24636: Assignee: Marek Novotny > Type Coercion of Arrays for array_join Function >

[jira] [Resolved] (SPARK-24418) Upgrade to Scala 2.11.12

[ https://issues.apache.org/jira/browse/SPARK-24418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24418. - Resolution: Fixed Issue resolved by pull request 21495

[jira] [Assigned] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23776: Assignee: Bruce Robbins > pyspark-sql tests should display build instructions when

[jira] [Resolved] (SPARK-23776) pyspark-sql tests should display build instructions when components are missing

[ https://issues.apache.org/jira/browse/SPARK-23776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23776. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21628

[jira] [Commented] (SPARK-18681) Throw Filtering is supported only on partition keys of type string exception

[ https://issues.apache.org/jira/browse/SPARK-18681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523049#comment-16523049 ] Harish commented on SPARK-18681: [~michael] I see the same issue in 2.3.1. I have hive partitioned table

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523023#comment-16523023 ] vaquar khan commented on SPARK-24631: - Why pull request? Please close jira no issue found. >

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523022#comment-16523022 ] Saisai Shao commented on SPARK-24646: - Hi [~vanzin], here is a specific example: We have a

[jira] [Resolved] (SPARK-24552) Task attempt numbers are reused when stages are retried

[ https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-24552. Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.4.0

[jira] [Resolved] (SPARK-24652) Strange ALS Implementation for Implicit Feedback

[ https://issues.apache.org/jira/browse/SPARK-24652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Lam resolved SPARK-24652. --- Resolution: Not A Problem > Strange ALS Implementation for Implicit Feedback >

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522946#comment-16522946 ] Marcelo Vanzin commented on SPARK-24646: I'm not sure I understand what is the problem here. Can

[jira] [Commented] (SPARK-23858) Need to apply pyarrow adjustments to complex types with DateType/TimestampType

[ https://issues.apache.org/jira/browse/SPARK-23858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522916#comment-16522916 ] Bryan Cutler commented on SPARK-23858: -- [~semanticbeeng] sorry, there aren't failing tests I can

[jira] [Commented] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522877#comment-16522877 ] Apache Spark commented on SPARK-24654: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24654: Assignee: Sean Owen (was: Apache Spark) > Update, fix LICENSE and NOTICE, and

[jira] [Assigned] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24654: Assignee: Apache Spark (was: Sean Owen) > Update, fix LICENSE and NOTICE, and

[jira] [Created] (SPARK-24656) SparkML Transformers and Estimators with multiple columns

Michael Dreibelbis created SPARK-24656: -- Summary: SparkML Transformers and Estimators with multiple columns Key: SPARK-24656 URL: https://issues.apache.org/jira/browse/SPARK-24656 Project: Spark

[jira] [Created] (SPARK-24655) [K8S] Custom Docker Image Expectations and Documentation

Matt Cheah created SPARK-24655: -- Summary: [K8S] Custom Docker Image Expectations and Documentation Key: SPARK-24655 URL: https://issues.apache.org/jira/browse/SPARK-24655 Project: Spark Issue

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24631: Assignee: Apache Spark > Cannot up cast column from bigint to smallint as it may

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24631: Assignee: (was: Apache Spark) > Cannot up cast column from bigint to smallint as it

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522871#comment-16522871 ] Apache Spark commented on SPARK-24631: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522869#comment-16522869 ] Bryan Cutler commented on SPARK-24579: -- I left some comments on the shared doc, overall sounds

[jira] [Resolved] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-24648. --- Resolution: Fixed Assignee: Stacy Kerkela Fix Version/s: 2.4.0 >

[jira] [Created] (SPARK-24654) Update, fix LICENSE and NOTICE, and specialize for source vs binary

Sean Owen created SPARK-24654: - Summary: Update, fix LICENSE and NOTICE, and specialize for source vs binary Key: SPARK-24654 URL: https://issues.apache.org/jira/browse/SPARK-24654 Project: Spark

[jira] [Created] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

Marcelo Vanzin created SPARK-24653: -- Summary: Flaky test "JoinSuite.test SortMergeJoin (with spill)" Key: SPARK-24653 URL: https://issues.apache.org/jira/browse/SPARK-24653 Project: Spark

[jira] [Updated] (SPARK-24651) Add ability to write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Summary: Add ability to write null values while writing JSON (was: Write null values while

[jira] [Updated] (SPARK-24651) Write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Description: Hello,  Spark is configured to ignore the null values when writing JSON based off

[jira] [Created] (SPARK-24652) Strange ALS Implementation for Implicit Feedback

Jerry Lam created SPARK-24652: - Summary: Strange ALS Implementation for Implicit Feedback Key: SPARK-24652 URL: https://issues.apache.org/jira/browse/SPARK-24652 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24651) Write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Summary: Write null values while writing JSON (was: Write null values when writing JSON) >

[jira] [Updated] (SPARK-24651) Write null values when writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Liem updated SPARK-24651: - Description: Hello,  Spark is configured to ignore the null values when writing JSON based off

[jira] [Created] (SPARK-24651) Write null values when writing JSON

Matthew Liem created SPARK-24651: Summary: Write null values when writing JSON Key: SPARK-24651 URL: https://issues.apache.org/jira/browse/SPARK-24651 Project: Spark Issue Type: New Feature

[jira] [Assigned] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22357: Assignee: (was: Apache Spark) > SparkContext.binaryFiles ignore minPartitions

[jira] [Commented] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522754#comment-16522754 ] Apache Spark commented on SPARK-22357: -- User 'bomeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-22357) SparkContext.binaryFiles ignore minPartitions parameter

[ https://issues.apache.org/jira/browse/SPARK-22357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22357: Assignee: Apache Spark > SparkContext.binaryFiles ignore minPartitions parameter >

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522661#comment-16522661 ] Xiangrui Meng commented on SPARK-24530: --- [~dongjoon] [~hyukjin.kwon] Could you report your system,

[jira] [Commented] (SPARK-24324) Pandas Grouped Map UserDefinedFunction mixes column labels

[ https://issues.apache.org/jira/browse/SPARK-24324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522617#comment-16522617 ] Apache Spark commented on SPARK-24324: -- User 'BryanCutler' has created a pull request for this

[jira] [Created] (SPARK-24650) GroupingSet

Mihir Sahu created SPARK-24650: -- Summary: GroupingSet Key: SPARK-24650 URL: https://issues.apache.org/jira/browse/SPARK-24650 Project: Spark Issue Type: Improvement Components: SQL

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Target Version/s: 2.4.0 > Sink Should Return OffsetSeqs For ProgressReporting >

[jira] [Commented] (SPARK-21478) Unpersist a DF also unpersists related DFs

[ https://issues.apache.org/jira/browse/SPARK-21478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522563#comment-16522563 ] Xiao Li commented on SPARK-21478: - [~roberto.mirizzi] The issue has been resolved by 

[jira] [Commented] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522562#comment-16522562 ] Xiao Li commented on SPARK-19765: - The reported issue has been resolved by 

[jira] [Resolved] (SPARK-21607) Can dropTempView function add a param like dropTempView(viewName: String, dropSelfOnly: Boolean)

[ https://issues.apache.org/jira/browse/SPARK-21607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21607. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 2.4.0 > Can dropTempView

[jira] [Commented] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522557#comment-16522557 ] Joseph K. Bradley commented on SPARK-24632: --- CC [~yanboliang], [~holden.ka...@gmail.com]: You

[jira] [Commented] (SPARK-21607) Can dropTempView function add a param like dropTempView(viewName: String, dropSelfOnly: Boolean)

[ https://issues.apache.org/jira/browse/SPARK-21607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522559#comment-16522559 ] Xiao Li commented on SPARK-21607: - This is resolved by https://issues.apache.org/jira/browse/SPARK-24596

[jira] [Assigned] (SPARK-24596) Non-cascading Cache Invalidation

[ https://issues.apache.org/jira/browse/SPARK-24596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24596: --- Assignee: Maryann Xue > Non-cascading Cache Invalidation > > >

[jira] [Resolved] (SPARK-24596) Non-cascading Cache Invalidation

[ https://issues.apache.org/jira/browse/SPARK-24596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24596. - Resolution: Fixed Target Version/s: 2.4.0 > Non-cascading Cache Invalidation >

[jira] [Updated] (SPARK-24632) Allow 3rd-party libraries to use pyspark.ml abstractions for Java wrappers for persistence

[ https://issues.apache.org/jira/browse/SPARK-24632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-24632: -- Description: This is a follow-up for [SPARK-17025], which allowed users to implement

[jira] [Updated] (SPARK-24649) SparkUDF.unapply is not backwards compatable

[ https://issues.apache.org/jira/browse/SPARK-24649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon H.K. Fitch updated SPARK-24649: -- Priority: Minor (was: Major) > SparkUDF.unapply is not backwards compatable >

[jira] [Assigned] (SPARK-24633) arrays_zip function's code generator splits input processing incorrectly

[ https://issues.apache.org/jira/browse/SPARK-24633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24633: --- Assignee: Marco Gaido > arrays_zip function's code generator splits input processing

[jira] [Resolved] (SPARK-24633) arrays_zip function's code generator splits input processing incorrectly

[ https://issues.apache.org/jira/browse/SPARK-24633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24633. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21621

[jira] [Updated] (SPARK-24649) SparkUDF.unapply is not backwards compatable

[ https://issues.apache.org/jira/browse/SPARK-24649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simeon H.K. Fitch updated SPARK-24649: -- Description: The shape of the `ScalaUDF` case class changed in 2.3.0. A secondary

[jira] [Assigned] (SPARK-24533) typesafe has rebranded to lightbend. change the build/mvn endpoint from downloads.typesafe.com to downloads.lightbend.com

[ https://issues.apache.org/jira/browse/SPARK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24533: Assignee: Apache Spark > typesafe has rebranded to lightbend. change the build/mvn

[jira] [Commented] (SPARK-24533) typesafe has rebranded to lightbend. change the build/mvn endpoint from downloads.typesafe.com to downloads.lightbend.com

[ https://issues.apache.org/jira/browse/SPARK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522349#comment-16522349 ] Apache Spark commented on SPARK-24533: -- User 'redsanket' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24533) typesafe has rebranded to lightbend. change the build/mvn endpoint from downloads.typesafe.com to downloads.lightbend.com

[ https://issues.apache.org/jira/browse/SPARK-24533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24533: Assignee: (was: Apache Spark) > typesafe has rebranded to lightbend. change the

[jira] [Created] (SPARK-24649) SparkUDF.unapply is not backwards compatable

Simeon H.K. Fitch created SPARK-24649: - Summary: SparkUDF.unapply is not backwards compatable Key: SPARK-24649 URL: https://issues.apache.org/jira/browse/SPARK-24649 Project: Spark Issue

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522339#comment-16522339 ] Hyukjin Kwon commented on SPARK-23710: -- That was a rather initial work having some concerns and

[jira] [Assigned] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24594: Assignee: (was: Apache Spark) > Introduce metrics for YARN executor allocation

[jira] [Commented] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522330#comment-16522330 ] Apache Spark commented on SPARK-24594: -- User 'attilapiros' has created a pull request for this

[jira] [Assigned] (SPARK-24594) Introduce metrics for YARN executor allocation problems

[ https://issues.apache.org/jira/browse/SPARK-24594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24594: Assignee: Apache Spark > Introduce metrics for YARN executor allocation problems >

[jira] [Commented] (SPARK-23710) Upgrade Hive to 2.3.2

[ https://issues.apache.org/jira/browse/SPARK-23710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522319#comment-16522319 ] Darek commented on SPARK-23710: --- The work was done in [PR20659|https://github.com/apache/spark/pull/20659]

[jira] [Commented] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522251#comment-16522251 ] Apache Spark commented on SPARK-24648: -- User 'dbkerkela' has created a pull request for this issue:

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Affects Version/s: (was: 2.4.0) 2.3.1 > Sink Should Return OffsetSeqs

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Fix Version/s: 2.4.0 > Sink Should Return OffsetSeqs For ProgressReporting >

[jira] [Assigned] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24648: Assignee: (was: Apache Spark) > SQLMetrics counters are not thread safe >

[jira] [Assigned] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24648: Assignee: Apache Spark > SQLMetrics counters are not thread safe >

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Description: To be able to track data lineage for Structured Streaming (I intend to implement this

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Vayda updated SPARK-24647: --- Description: To be able to track data lineage for Structured Streaming (I intend to implement this

[jira] [Commented] (SPARK-24648) SQLMetrics counters are not thread safe

[ https://issues.apache.org/jira/browse/SPARK-24648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1654#comment-1654 ] Stacy Kerkela commented on SPARK-24648: --- I have a PR prepared for this. > SQLMetrics counters are

[jira] [Created] (SPARK-24648) SQLMetrics counters are not thread safe

Stacy Kerkela created SPARK-24648: - Summary: SQLMetrics counters are not thread safe Key: SPARK-24648 URL: https://issues.apache.org/jira/browse/SPARK-24648 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaclav Kosar updated SPARK-24647: - Description: To be able to track data lineage for Structured Streaming (I intend to implement

[jira] [Commented] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522218#comment-16522218 ] Vaclav Kosar commented on SPARK-24647: -- [~c...@koeninger.org] and [~jlaskowski] you may be

[jira] [Created] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

Vaclav Kosar created SPARK-24647: Summary: Sink Should Return OffsetSeqs For ProgressReporting Key: SPARK-24647 URL: https://issues.apache.org/jira/browse/SPARK-24647 Project: Spark Issue

[jira] [Commented] (SPARK-9775) Query Mesos for number of CPUs to set default parallelism

[ https://issues.apache.org/jira/browse/SPARK-9775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522064#comment-16522064 ] Maxim Gekk commented on SPARK-9775: --- Please, change another related methods like proposed in the PR: 

[jira] [Commented] (SPARK-22150) PeriodicCheckpointer fails with FileNotFoundException in case of dependant RDDs

[ https://issues.apache.org/jira/browse/SPARK-22150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522015#comment-16522015 ] Sergey Zhemzhitsky commented on SPARK-22150: Just a kind remainder... >

[jira] [Commented] (SPARK-22184) GraphX fails in case of insufficient memory and checkpoints enabled

[ https://issues.apache.org/jira/browse/SPARK-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16522017#comment-16522017 ] Sergey Zhemzhitsky commented on SPARK-22184: Just a kind remainder... > GraphX fails in

[jira] [Updated] (SPARK-24628) Typos of the example code in docs/mllib-data-types.md

[ https://issues.apache.org/jira/browse/SPARK-24628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weizhe Huang updated SPARK-24628: - Summary: Typos of the example code in docs/mllib-data-types.md (was: The example given to

[jira] [Assigned] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24646: Assignee: (was: Apache Spark) > Support wildcard '*' for to

[jira] [Commented] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521929#comment-16521929 ] Apache Spark commented on SPARK-24646: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

[ https://issues.apache.org/jira/browse/SPARK-24646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24646: Assignee: Apache Spark > Support wildcard '*' for to

[jira] [Created] (SPARK-24646) Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes

Saisai Shao created SPARK-24646: --- Summary: Support wildcard '*' for to spark.yarn.dist.forceDownloadSchemes Key: SPARK-24646 URL: https://issues.apache.org/jira/browse/SPARK-24646 Project: Spark

[jira] [Commented] (SPARK-21917) Remote http(s) resources is not supported in YARN mode

[ https://issues.apache.org/jira/browse/SPARK-21917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521910#comment-16521910 ] Apache Spark commented on SPARK-21917: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Resolved] (SPARK-24327) Verify and normalize a partition column name based on the JDBC resolved schema

[ https://issues.apache.org/jira/browse/SPARK-24327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24327. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.4.0 > Verify and