[jira] [Commented] (SPARK-27981) Remove `Illegal reflective access` warning for `java.nio.Bits.unaligned()`

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951625#comment-16951625 ] Dongjoon Hyun commented on SPARK-27981: --- For now, `java.nio.DirectByteBuffer` is an intentional

[jira] [Commented] (SPARK-29451) Some queries with divisions in SQL windows are failling in Thrift

2019-10-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951622#comment-16951622 ] Yuming Wang commented on SPARK-29451: - It works for me. {code:sql} CREATE OR REPLACE TEMPORARY VIEW

[jira] [Commented] (SPARK-27259) Processing Compressed HDFS files with spark failing with error: "java.lang.IllegalArgumentException: requirement failed: length (-1) cannot be negative" from spark 2.2

2019-10-14 Thread Praneet Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951608#comment-16951608 ] Praneet Sharma commented on SPARK-27259: Submitted a PR for this issue: 

[jira] [Resolved] (SPARK-29454) Reduce unsafeProjection call times when read parquet file

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29454. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26106

[jira] [Assigned] (SPARK-29454) Reduce unsafeProjection call times when read parquet file

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29454: --- Assignee: Yang Jie > Reduce unsafeProjection call times when read parquet file >

[jira] [Commented] (SPARK-29465) Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode.

2019-10-14 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951581#comment-16951581 ] Sandeep Katta commented on SPARK-29465: --- I see as per the comments in the [code

[jira] [Commented] (SPARK-29437) CSV Writer should escape 'escapechar' when it exists in the data

2019-10-14 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951525#comment-16951525 ] Ankit Raj Boudh commented on SPARK-29437: - Hi [~kretes], This is the correct behaviour, as per

[jira] [Assigned] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-10614: - Assignee: Marcelo Masiero Vanzin > SystemClock uses non-monotonic time in its wait

[jira] [Resolved] (SPARK-10614) SystemClock uses non-monotonic time in its wait logic

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-10614. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26058

[jira] [Commented] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951492#comment-16951492 ] Dongjoon Hyun commented on SPARK-29461: --- Thank you for filing a JIRA, [~rkalhans]. Thank you for

[jira] [Updated] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29461: -- Affects Version/s: (was: 2.4.4) (was: 2.4.0)

[jira] [Updated] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29461: -- Issue Type: Improvement (was: Bug) > Spark dataframe writer does not expose metrics for JDBC

[jira] [Updated] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29461: -- Component/s: (was: Spark Core) SQL > Spark dataframe writer does not

[jira] [Updated] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29461: -- Priority: Minor (was: Major) > Spark dataframe writer does not expose metrics for JDBC

[jira] [Resolved] (SPARK-29412) refine the document of v2 session catalog config

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29412. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26071

[jira] [Created] (SPARK-29472) Mechanism for Excluding Jars at Launch for YARN

2019-10-14 Thread Abhishek Modi (Jira)
Abhishek Modi created SPARK-29472: - Summary: Mechanism for Excluding Jars at Launch for YARN Key: SPARK-29472 URL: https://issues.apache.org/jira/browse/SPARK-29472 Project: Spark Issue

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-14 Thread fengchaoge (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951437#comment-16951437 ] fengchaoge commented on SPARK-26570: It's just a snapshot of an instant, and it's actually getting

[jira] [Commented] (SPARK-24915) Calling SparkSession.createDataFrame with schema can throw exception

2019-10-14 Thread Joachim Hereth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951417#comment-16951417 ] Joachim Hereth commented on SPARK-24915: this is fixed by

[jira] [Reopened] (SPARK-16483) Unifying struct fields and columns

2019-10-14 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas reopened SPARK-16483: -- Though this has been bulk closed, I still think it's a valuable potential improvement to

[jira] [Resolved] (SPARK-29347) External Row should be JSON serializable

2019-10-14 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-29347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-29347. --- Fix Version/s: 3.0.0 Resolution: Fixed > External Row should be JSON

[jira] [Created] (SPARK-29471) "TaskResultLost (result lost from block manager)" error message is misleading in case result fetch is caused by client-side issues

2019-10-14 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-29471: -- Summary: "TaskResultLost (result lost from block manager)" error message is misleading in case result fetch is caused by client-side issues Key: SPARK-29471 URL:

[jira] [Updated] (SPARK-29471) "TaskResultLost (result lost from block manager)" error message is misleading in case result fetch is caused by client-side network connectivity issues

2019-10-14 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-29471: --- Summary: "TaskResultLost (result lost from block manager)" error message is misleading in case

[jira] [Created] (SPARK-29470) Update plugins to latest versions

2019-10-14 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29470: - Summary: Update plugins to latest versions Key: SPARK-29470 URL: https://issues.apache.org/jira/browse/SPARK-29470 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-27623) Provider org.apache.spark.sql.avro.AvroFileFormat could not be instantiated

2019-10-14 Thread Tom Tang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951375#comment-16951375 ] Tom Tang edited comment on SPARK-27623 at 10/14/19 9:42 PM: I found the same

[jira] [Comment Edited] (SPARK-27623) Provider org.apache.spark.sql.avro.AvroFileFormat could not be instantiated

2019-10-14 Thread Tom Tang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951375#comment-16951375 ] Tom Tang edited comment on SPARK-27623 at 10/14/19 9:42 PM: I found the same

[jira] [Comment Edited] (SPARK-27623) Provider org.apache.spark.sql.avro.AvroFileFormat could not be instantiated

2019-10-14 Thread Tom Tang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951375#comment-16951375 ] Tom Tang edited comment on SPARK-27623 at 10/14/19 9:41 PM: I found the same

[jira] [Commented] (SPARK-27623) Provider org.apache.spark.sql.avro.AvroFileFormat could not be instantiated

2019-10-14 Thread Tom Tang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951375#comment-16951375 ] Tom Tang commented on SPARK-27623: -- I found the same issue with spark 2.4.3, and when I fall back to

[jira] [Assigned] (SPARK-29469) Avoid retries by RetryingBlockFetcher when ExternalBlockStoreClient is closed

2019-10-14 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-29469: --- Assignee: L. C. Hsieh > Avoid retries by RetryingBlockFetcher when

[jira] [Resolved] (SPARK-29463) move v2 commands to a new file

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29463. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26111

[jira] [Created] (SPARK-29469) Avoid retries by RetryingBlockFetcher when ExternalBlockStoreClient is closed

2019-10-14 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-29469: --- Summary: Avoid retries by RetryingBlockFetcher when ExternalBlockStoreClient is closed Key: SPARK-29469 URL: https://issues.apache.org/jira/browse/SPARK-29469 Project:

[jira] [Created] (SPARK-29468) Floating point literals produce incorrect SQL

2019-10-14 Thread Jose Torres (Jira)
Jose Torres created SPARK-29468: --- Summary: Floating point literals produce incorrect SQL Key: SPARK-29468 URL: https://issues.apache.org/jira/browse/SPARK-29468 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-29467) dev/merge_spark_pr.py fails on CAPTCHA

2019-10-14 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-29467: Description: so, i was merging a PR and when i tried to update the associated jira, it failed

[jira] [Updated] (SPARK-29467) dev/merge_spark_pr.py fails on CAPTCHA

2019-10-14 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-29467: Description: so, i was merging a PR and when i tried to update the associated jira, it failed

[jira] [Created] (SPARK-29467) dev/merge_spark_pr.py fails on CAPTCHA

2019-10-14 Thread Shane Knapp (Jira)
Shane Knapp created SPARK-29467: --- Summary: dev/merge_spark_pr.py fails on CAPTCHA Key: SPARK-29467 URL: https://issues.apache.org/jira/browse/SPARK-29467 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-29417) Resource Scheduling - add TaskContext.resource java api

2019-10-14 Thread Xiangrui Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-29417. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26083

[jira] [Resolved] (SPARK-25152) Enable Spark on Kubernetes R Integration Tests

2019-10-14 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp resolved SPARK-25152. - Assignee: Ilan Filonenko Resolution: Fixed merged in to master > Enable Spark on

[jira] [Resolved] (SPARK-29442) Set `default` mode should override the existing mode

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29442. --- Fix Version/s: 3.0.0 Assignee: Dongjoon Hyun Resolution: Fixed This is

[jira] [Updated] (SPARK-29442) Set `default` mode should override the existing mode

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29442: -- Affects Version/s: (was: 2.4.4) (was: 2.3.4) > Set `default`

[jira] [Created] (SPARK-29466) Show `Duration` for running drivers in Standalone master web UI

2019-10-14 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29466: - Summary: Show `Duration` for running drivers in Standalone master web UI Key: SPARK-29466 URL: https://issues.apache.org/jira/browse/SPARK-29466 Project: Spark

[jira] [Updated] (SPARK-29465) Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode.

2019-10-14 Thread Vishwas Nalka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishwas Nalka updated SPARK-29465: -- Description:  I'm trying to restrict the ports used by spark app which is launched in yarn

[jira] [Created] (SPARK-29465) Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode.

2019-10-14 Thread Vishwas Nalka (Jira)
Vishwas Nalka created SPARK-29465: - Summary: Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode. Key: SPARK-29465 URL: https://issues.apache.org/jira/browse/SPARK-29465 Project:

[jira] [Updated] (SPARK-29465) Unable to configure SPARK UI (spark.ui.port) in spark yarn cluster mode.

2019-10-14 Thread Vishwas Nalka (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishwas Nalka updated SPARK-29465: -- Description:  I'm trying to restrict the ports used by spark app which is launched in yarn

[jira] [Commented] (SPARK-29464) PySpark ML should expose Params.clear() to unset a user supplied Param

2019-10-14 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951241#comment-16951241 ] Huaxin Gao commented on SPARK-29464: I will work on this. Thanks! > PySpark ML should expose

[jira] [Commented] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951228#comment-16951228 ] Dongjoon Hyun commented on SPARK-10848: --- Hi, [~purijatin]. If you are sure the schema, there is a

[jira] [Comment Edited] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951228#comment-16951228 ] Dongjoon Hyun edited comment on SPARK-10848 at 10/14/19 6:23 PM: - Hi,

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-14 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951223#comment-16951223 ] L. C. Hsieh commented on SPARK-26570: - First from the stacktrace, I was thinking it is possible that

[jira] [Updated] (SPARK-29464) PySpark ML should expose Params.clear() to unset a user supplied Param

2019-10-14 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-29464: - Description: PySpark ML currently has a private {{_clear()}} method that will unset a param.

[jira] [Created] (SPARK-29464) PySpark ML should expose Params.clear() to unset a user supplied Param

2019-10-14 Thread Bryan Cutler (Jira)
Bryan Cutler created SPARK-29464: Summary: PySpark ML should expose Params.clear() to unset a user supplied Param Key: SPARK-29464 URL: https://issues.apache.org/jira/browse/SPARK-29464 Project:

[jira] [Resolved] (SPARK-29428) Can't persist/set None-valued param

2019-10-14 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-29428. -- Resolution: Not A Problem > Can't persist/set None-valued param >

[jira] [Commented] (SPARK-29428) Can't persist/set None-valued param

2019-10-14 Thread Bryan Cutler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951212#comment-16951212 ] Bryan Cutler commented on SPARK-29428: -- The usage of {{None}} in pyspark ml is a bit confusing in

[jira] [Assigned] (SPARK-29107) Add window.sql - Part 1

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29107: - Assignee: Dylan Guedes > Add window.sql - Part 1 > --- > >

[jira] [Resolved] (SPARK-29107) Add window.sql - Part 1

2019-10-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29107. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25816

[jira] [Comment Edited] (SPARK-21492) Memory leak in SortMergeJoin

2019-10-14 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951165#comment-16951165 ] Min Shen edited comment on SPARK-21492 at 10/14/19 5:13 PM: Want to further

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-10-14 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951165#comment-16951165 ] Min Shen commented on SPARK-21492: -- Want to further clarify the scope of the fix in PR 

[jira] [Commented] (SPARK-21492) Memory leak in SortMergeJoin

2019-10-14 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951154#comment-16951154 ] Min Shen commented on SPARK-21492: -- We have deployed the latest version of the PR in 

[jira] [Commented] (SPARK-29206) Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads

2019-10-14 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951128#comment-16951128 ] Min Shen commented on SPARK-29206: -- It appears that simply by making sure the number of Netty server

[jira] [Resolved] (SPARK-29435) Spark 3 doesnt work with older shuffle service

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29435. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26095

[jira] [Assigned] (SPARK-29435) Spark 3 doesnt work with older shuffle service

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29435: --- Assignee: Sandeep Katta > Spark 3 doesnt work with older shuffle service >

[jira] [Resolved] (SPARK-29381) Add 'private' _XXXParams classes for classification & regression

2019-10-14 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29381. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26103

[jira] [Assigned] (SPARK-29381) Add 'private' _XXXParams classes for classification & regression

2019-10-14 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29381: Assignee: Huaxin Gao > Add 'private' _XXXParams classes for classification & regression

[jira] [Updated] (SPARK-29381) Add 'private' _XXXParams classes for classification & regression

2019-10-14 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-29381: - Priority: Minor (was: Major) > Add 'private' _XXXParams classes for classification &

[jira] [Resolved] (SPARK-29369) Accept strings without `interval` prefix in casting to intervals

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29369. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26079

[jira] [Assigned] (SPARK-29369) Accept strings without `interval` prefix in casting to intervals

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29369: --- Assignee: Maxim Gekk > Accept strings without `interval` prefix in casting to intervals >

[jira] [Created] (SPARK-29463) move v2 commands to a new file

2019-10-14 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29463: --- Summary: move v2 commands to a new file Key: SPARK-29463 URL: https://issues.apache.org/jira/browse/SPARK-29463 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-29279) DataSourceV2: merge SHOW NAMESPACES and SHOW DATABASES code path

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29279. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26006

[jira] [Assigned] (SPARK-29279) DataSourceV2: merge SHOW NAMESPACES and SHOW DATABASES code path

2019-10-14 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29279: --- Assignee: Terry Kim > DataSourceV2: merge SHOW NAMESPACES and SHOW DATABASES code path >

[jira] [Created] (SPARK-29462) The data type of "array()" should be array

2019-10-14 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-29462: -- Summary: The data type of "array()" should be array Key: SPARK-29462 URL: https://issues.apache.org/jira/browse/SPARK-29462 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2019-10-14 Thread Alexandre Gattiker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950932#comment-16950932 ] Alexandre Gattiker edited comment on SPARK-17914 at 10/14/19 11:37 AM:

[jira] [Comment Edited] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2019-10-14 Thread Alexandre Gattiker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950932#comment-16950932 ] Alexandre Gattiker edited comment on SPARK-17914 at 10/14/19 11:36 AM:

[jira] [Comment Edited] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2019-10-14 Thread Alexandre Gattiker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950932#comment-16950932 ] Alexandre Gattiker edited comment on SPARK-17914 at 10/14/19 11:36 AM:

[jira] [Comment Edited] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2019-10-14 Thread Alexandre Gattiker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950932#comment-16950932 ] Alexandre Gattiker edited comment on SPARK-17914 at 10/14/19 11:36 AM:

[jira] [Commented] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2019-10-14 Thread Alexandre Gattiker (Jira)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950932#comment-16950932 ] Alexandre Gattiker commented on SPARK-17914: As reported by other commenters, the issue is

[jira] [Updated] (SPARK-29274) Should not coerce decimal type to double type when it's join column

2019-10-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29274: Summary: Should not coerce decimal type to double type when it's join column (was: Can not

[jira] [Commented] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950808#comment-16950808 ] Jungtaek Lim commented on SPARK-29461: -- I'm taking a look at this. > Spark dataframe writer does

[jira] [Updated] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29461: Target Version/s: (was: 2.4.0) > Spark dataframe writer does not expose metrics for JDBC writer

[jira] [Created] (SPARK-29461) Spark dataframe writer does not expose metrics for JDBC writer

2019-10-14 Thread ROHIT KALHANS (Jira)
ROHIT KALHANS created SPARK-29461: - Summary: Spark dataframe writer does not expose metrics for JDBC writer Key: SPARK-29461 URL: https://issues.apache.org/jira/browse/SPARK-29461 Project: Spark

[jira] [Commented] (SPARK-29294) Update Kafka to a verison that supports Scala 2.13

2019-10-14 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950785#comment-16950785 ] Gabor Somogyi commented on SPARK-29294: --- [~srowen] I'm intended to upgrade kafka version to 2.4

[jira] [Commented] (SPARK-29323) Add tooltip for The Executors Tab's column names in the Spark history server Page

2019-10-14 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950756#comment-16950756 ] ABHISHEK KUMAR GUPTA commented on SPARK-29323: -- [~liucht-inspur] Let me know if new sub 

[jira] [Commented] (SPARK-29460) Improve tooltip for Job Tab

2019-10-14 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950749#comment-16950749 ] pavithra ramachandran commented on SPARK-29460: --- i shall work on this > Improve tooltip

[jira] [Created] (SPARK-29460) Improve tooltip for Job Tab

2019-10-14 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-29460: Summary: Improve tooltip for Job Tab Key: SPARK-29460 URL: https://issues.apache.org/jira/browse/SPARK-29460 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-29377) parity between scala ml tuning and python ml tuning

2019-10-14 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-29377: Assignee: Huaxin Gao > parity between scala ml tuning and python ml tuning >

[jira] [Resolved] (SPARK-29377) parity between scala ml tuning and python ml tuning

2019-10-14 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-29377. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26057

[jira] [Updated] (SPARK-29323) Add tooltip for The Executors Tab's column names in the Spark history server Page

2019-10-14 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ABHISHEK KUMAR GUPTA updated SPARK-29323: - Attachment: image.png > Add tooltip for The Executors Tab's column names in the

[jira] [Created] (SPARK-29459) Spark takes lot of time to initialize when the network is not connected to internet.

2019-10-14 Thread Raman (Jira)
Raman created SPARK-29459: - Summary: Spark takes lot of time to initialize when the network is not connected to internet. Key: SPARK-29459 URL: https://issues.apache.org/jira/browse/SPARK-29459 Project:

[jira] [Commented] (SPARK-29323) Add tooltip for The Executors Tab's column names in the Spark history server Page

2019-10-14 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950743#comment-16950743 ] ABHISHEK KUMAR GUPTA commented on SPARK-29323: -- This require to handle for Spark Live UI