[jira] [Created] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-31819: - Summary: Add a workaround for Java 8u251+ and update integration test cases Key: SPARK-31819 URL: https://issues.apache.org/jira/browse/SPARK-31819 Project: Spark

[jira] [Resolved] (SPARK-31673) Enhance QueryExecution.debugFile to take an additional explain mode parameter.

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31673. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28493

[jira] [Assigned] (SPARK-31673) Enhance QueryExecution.debugFile to take an additional explain mode parameter.

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31673: Assignee: Dilip Biswal > Enhance QueryExecution.debugFile to take an additional explain

[jira] [Resolved] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31792. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28609

[jira] [Assigned] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31792: Assignee: Xingcan Cui > Introduce the structured streaming UI in the Web UI page >

[jira] [Resolved] (SPARK-31806) Check reading date/timestamp from Parquet: plain, dictionary enc, Spark version

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31806. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28630

[jira] [Assigned] (SPARK-31806) Check reading date/timestamp from Parquet: plain, dictionary enc, Spark version

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31806: --- Assignee: Maxim Gekk > Check reading date/timestamp from Parquet: plain, dictionary enc,

[jira] [Resolved] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31810. -- Fix Version/s: 3.0.0 Assignee: Prakhar Jain Resolution: Fixed Fixed in

[jira] [Commented] (SPARK-31800) Unable to disable Kerberos when submitting jobs to Kubernetes

2020-05-25 Thread Devaraj Kavali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116420#comment-17116420 ] Devaraj Kavali commented on SPARK-31800: Seems there has been considerable changes between 2.4.*

[jira] [Resolved] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31808. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28633

[jira] [Assigned] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31808: - Assignee: Hyukjin Kwon > Make struct function's output name and class name pretty >

[jira] [Resolved] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31818. --- Resolution: Fixed Issue resolved by pull request 28636

[jira] [Commented] (SPARK-31771) Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116130#comment-17116130 ] Apache Spark commented on SPARK-31771: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-31771) Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116129#comment-17116129 ] Apache Spark commented on SPARK-31771: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Updated] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

2020-05-25 Thread Sunny Jain (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunny Jain updated SPARK-31814: --- Description: Hi,   We are trying to convert a column with string datatype to date type using

[jira] [Updated] (SPARK-31771) Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31771: Parent: SPARK-31408 Issue Type: Sub-task (was: Bug) > Disable Narrow TextStyle for

[jira] [Updated] (SPARK-7768) Make user-defined type (UDT) API public

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-7768: --- Target Version/s: 3.1.0 (was: 3.0.0) > Make user-defined type (UDT) API public >

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Attachment: (was: pushdown.png) > Pushdown IsNotNull to file scan if possible >

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Attachment: pushdown.png > Pushdown IsNotNull to file scan if possible >

[jira] [Commented] (SPARK-31629) "py4j.protocol.Py4JJavaError: An error occurred while calling o90.save" in pyspark 2.3.1

2020-05-25 Thread appleyuchi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116094#comment-17116094 ] appleyuchi commented on SPARK-31629: it has been solved,due to slave node can not access the

[jira] [Resolved] (SPARK-30352) DataSourceV2: Add CURRENT_CATALOG function

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30352. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27006

[jira] [Resolved] (SPARK-31734) add weight support in ClusteringEvaluator

2020-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31734. -- Fix Version/s: 3.1.0 Assignee: Huaxin Gao Resolution: Fixed Resolved by

[jira] [Assigned] (SPARK-30352) DataSourceV2: Add CURRENT_CATALOG function

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30352: --- Assignee: Kent Yao > DataSourceV2: Add CURRENT_CATALOG function >

[jira] [Updated] (SPARK-31734) add weight support in ClusteringEvaluator

2020-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-31734: - Priority: Minor (was: Major) > add weight support in ClusteringEvaluator >

[jira] [Commented] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116006#comment-17116006 ] Apache Spark commented on SPARK-31818: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31818: Assignee: Apache Spark (was: Maxim Gekk) > Failure on pushing down filters with

[jira] [Commented] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17116004#comment-17116004 ] Apache Spark commented on SPARK-31818: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31818: Assignee: Maxim Gekk (was: Apache Spark) > Failure on pushing down filters with

[jira] [Updated] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31818: --- Description: When spark.sql.datetime.java8API.enabled is set to true, filters pushed down with

[jira] [Created] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31818: -- Summary: Failure on pushing down filters with java.time.Instant values in ORC Key: SPARK-31818 URL: https://issues.apache.org/jira/browse/SPARK-31818 Project: Spark

[jira] [Created] (SPARK-31817) Pass-through of Kerberos credentials from Spark SQL to a jdbc source

2020-05-25 Thread Luis Lozano Coira (Jira)
Luis Lozano Coira created SPARK-31817: - Summary: Pass-through of Kerberos credentials from Spark SQL to a jdbc source Key: SPARK-31817 URL: https://issues.apache.org/jira/browse/SPARK-31817

[jira] [Commented] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115922#comment-17115922 ] Apache Spark commented on SPARK-31337: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Assigned] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31337: Assignee: (was: Apache Spark) > Support MS Sql Kerberos login in JDBC connector >

[jira] [Assigned] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31337: Assignee: Apache Spark > Support MS Sql Kerberos login in JDBC connector >

[jira] [Commented] (SPARK-31696) Support spark.kubernetes.driver.service.annotation

2020-05-25 Thread Canbin Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115873#comment-17115873 ] Canbin Zheng commented on SPARK-31696: -- Thanks [~dongjoon]! Since not familiar with Prometheus, I

[jira] [Commented] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115855#comment-17115855 ] Gabor Somogyi commented on SPARK-31337: --- [~GeetBhalshankar] security is super sensitive area and

[jira] [Comment Edited] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115820#comment-17115820 ] Mohammad Islam edited comment on SPARK-31812 at 5/25/20, 8:45 AM: -- If

[jira] [Updated] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Islam updated SPARK-31812: --- Fix Version/s: (was: 2.4.7) > Spark to support the auto cancelation of delegation token

[jira] [Created] (SPARK-31816) Create high level description about JDBC connection providers for developers

2020-05-25 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31816: - Summary: Create high level description about JDBC connection providers for developers Key: SPARK-31816 URL: https://issues.apache.org/jira/browse/SPARK-31816

[jira] [Assigned] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-31811: --- Assignee: (was: Yuming Wang) > Pushdown IsNotNull to file scan if possible >

[jira] [Created] (SPARK-31815) Support Hive Kerberos login in JDBC connector

2020-05-25 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31815: - Summary: Support Hive Kerberos login in JDBC connector Key: SPARK-31815 URL: https://issues.apache.org/jira/browse/SPARK-31815 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-05-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115836#comment-17115836 ] Gabor Somogyi commented on SPARK-12312: --- Not yet planned but we can take a look at this when the

[jira] [Created] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

2020-05-25 Thread Sunny Jain (Jira)
Sunny Jain created SPARK-31814: -- Summary: Null in Date conversion from yyMMddHHmmss for specific date and time Key: SPARK-31814 URL: https://issues.apache.org/jira/browse/SPARK-31814 Project: Spark

[jira] [Commented] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115820#comment-17115820 ] Mohammad Islam commented on SPARK-31812: I have a patch and I would like to contribute. Can

[jira] [Created] (SPARK-31813) Cannot write snappy-compressed text files

2020-05-25 Thread Ondrej Kokes (Jira)
Ondrej Kokes created SPARK-31813: Summary: Cannot write snappy-compressed text files Key: SPARK-31813 URL: https://issues.apache.org/jira/browse/SPARK-31813 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
Mohammad Islam created SPARK-31812: -- Summary: Spark to support the auto cancelation of delegation token when an Application completes Key: SPARK-31812 URL: https://issues.apache.org/jira/browse/SPARK-31812

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Description: We should Pushdown {{IsNotNull}} to file scan if possible. For example: {code:sql}

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Attachment: default.png pushdown.png > Pushdown IsNotNull to file scan if

[jira] [Created] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-31811: --- Summary: Pushdown IsNotNull to file scan if possible Key: SPARK-31811 URL: https://issues.apache.org/jira/browse/SPARK-31811 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31810: Assignee: (was: Apache Spark) > Fix recoverPartitions test in DDLSuite >

[jira] [Commented] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115795#comment-17115795 ] Apache Spark commented on SPARK-31810: -- User 'prakharjain09' has created a pull request for this

[jira] [Assigned] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31810: Assignee: Apache Spark > Fix recoverPartitions test in DDLSuite >

[jira] [Updated] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Prakhar Jain (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakhar Jain updated SPARK-31810: - Component/s: Tests > Fix recoverPartitions test in DDLSuite >

[jira] [Updated] (SPARK-31809) Infer IsNotNull for all children of NullIntolerant expressions

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31809: Description: We should infer {{IsNotNull}} for all children of {{NullIntolerant}} expressions.

[jira] [Created] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Prakhar Jain (Jira)
Prakhar Jain created SPARK-31810: Summary: Fix recoverPartitions test in DDLSuite Key: SPARK-31810 URL: https://issues.apache.org/jira/browse/SPARK-31810 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-31809) Infer IsNotNull for all children of NullIntolerant expressions

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31809: Attachment: default.png infer.png > Infer IsNotNull for all children of

[jira] [Created] (SPARK-31809) Infer IsNotNull for all children of NullIntolerant expressions

2020-05-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-31809: --- Summary: Infer IsNotNull for all children of NullIntolerant expressions Key: SPARK-31809 URL: https://issues.apache.org/jira/browse/SPARK-31809 Project: Spark

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31751: - Description: This is an issue that have caused us so many data errors.  1) using spark ( with

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31751: - Flags: (was: Important) > spark serde property path overwrites table property location >

[jira] [Resolved] (SPARK-31758) Incorrect timestamp parsing from JSON

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31758. -- Resolution: Cannot Reproduce > Incorrect timestamp parsing from JSON >

[jira] [Commented] (SPARK-31758) Incorrect timestamp parsing from JSON

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115786#comment-17115786 ] Hyukjin Kwon commented on SPARK-31758: -- This was fixed in the current master: {code} >>> schema =

[jira] [Commented] (SPARK-31763) DataFrame.inputFiles() not Available

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115783#comment-17115783 ] Hyukjin Kwon commented on SPARK-31763: -- Are you interested in opening a PR for this? >

[jira] [Updated] (SPARK-31760) Simplification Based on Containment

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31760: - Labels: starter (was: ) > Simplification Based on Containment >

[jira] [Updated] (SPARK-31770) spark double count issue when table is recreated from hive

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31770: - Priority: Major (was: Critical) > spark double count issue when table is recreated from hive >

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31751: - Priority: Major (was: Critical) > spark serde property path overwrites table property location

[jira] [Resolved] (SPARK-31772) Json schema reading is not consistent between int and string types

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31772. -- Resolution: Not A Problem It works by design. The last JSON does not match with the type and

[jira] [Resolved] (SPARK-31773) getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(pa

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31773. -- Resolution: Duplicate > getting the Caused by: >

[jira] [Resolved] (SPARK-31774) getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(pa

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31774. -- Resolution: Incomplete > getting the Caused by: >

[jira] [Commented] (SPARK-31774) getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(p

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115777#comment-17115777 ] Hyukjin Kwon commented on SPARK-31774: -- Spark 2.2.x is EOL. Please try 2.4.5 or preview 3.0 as

[jira] [Resolved] (SPARK-31779) Redefining struct inside array incorrectly wraps child fields in array

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31779. -- Resolution: Not A Problem > Redefining struct inside array incorrectly wraps child fields in

[jira] [Resolved] (SPARK-31789) cancel

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31789. -- Resolution: Invalid > cancel > -- > > Key: SPARK-31789 >

[jira] [Resolved] (SPARK-31795) Stream Data with API to ServiceNow

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31795. -- Resolution: Invalid > Stream Data with API to ServiceNow > --

[jira] [Resolved] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31796. -- Resolution: Invalid > Download or Install Spark On windows >

[jira] [Commented] (SPARK-31795) Stream Data with API to ServiceNow

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115766#comment-17115766 ] Hyukjin Kwon commented on SPARK-31795: -- It's best to ask questions into the mailing list. Let's

[jira] [Updated] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31796: - Target Version/s: (was: 2.4.5) > Download or Install Spark On windows >

[jira] [Commented] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115765#comment-17115765 ] Hyukjin Kwon commented on SPARK-31796: -- For questions, it's best to interact with the mailing list.

[jira] [Updated] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31796: - Fix Version/s: (was: 2.4.5) > Download or Install Spark On windows >

[jira] [Commented] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115755#comment-17115755 ] Apache Spark commented on SPARK-31808: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31808: Assignee: Apache Spark > Make struct function's output name and class name pretty >

[jira] [Commented] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115753#comment-17115753 ] Apache Spark commented on SPARK-31808: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31808: Assignee: (was: Apache Spark) > Make struct function's output name and class name