[jira] [Commented] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115765#comment-17115765 ] Hyukjin Kwon commented on SPARK-31796: -- For questions, it's best to interact with t

[jira] [Updated] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31796: - Fix Version/s: (was: 2.4.5) > Download or Install Spark On windows > ---

[jira] [Updated] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31796: - Target Version/s: (was: 2.4.5) > Download or Install Spark On windows > --

[jira] [Commented] (SPARK-31795) Stream Data with API to ServiceNow

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115766#comment-17115766 ] Hyukjin Kwon commented on SPARK-31795: -- It's best to ask questions into the mailing

[jira] [Resolved] (SPARK-31796) Download or Install Spark On windows

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31796. -- Resolution: Invalid > Download or Install Spark On windows > -

[jira] [Resolved] (SPARK-31795) Stream Data with API to ServiceNow

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31795. -- Resolution: Invalid > Stream Data with API to ServiceNow > --

[jira] [Resolved] (SPARK-31789) cancel

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31789. -- Resolution: Invalid > cancel > -- > > Key: SPARK-31789 > U

[jira] [Resolved] (SPARK-31779) Redefining struct inside array incorrectly wraps child fields in array

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31779. -- Resolution: Not A Problem > Redefining struct inside array incorrectly wraps child fields in a

[jira] [Commented] (SPARK-31774) getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(p

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115777#comment-17115777 ] Hyukjin Kwon commented on SPARK-31774: -- Spark 2.2.x is EOL. Please try 2.4.5 or pre

[jira] [Resolved] (SPARK-31774) getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(pa

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31774. -- Resolution: Incomplete > getting the Caused by: > org.apache.spark.sql.catalyst.errors.packag

[jira] [Resolved] (SPARK-31773) getting the Caused by: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, at org.apache.spark.sql.catalyst.errors.package$.attachTree(pa

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31773. -- Resolution: Duplicate > getting the Caused by: > org.apache.spark.sql.catalyst.errors.package

[jira] [Resolved] (SPARK-31772) Json schema reading is not consistent between int and string types

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31772. -- Resolution: Not A Problem It works by design. The last JSON does not match with the type and i

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31751: - Priority: Major (was: Critical) > spark serde property path overwrites table property location

[jira] [Updated] (SPARK-31770) spark double count issue when table is recreated from hive

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31770: - Priority: Major (was: Critical) > spark double count issue when table is recreated from hive >

[jira] [Updated] (SPARK-31760) Simplification Based on Containment

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31760: - Labels: starter (was: ) > Simplification Based on Containment > ---

[jira] [Commented] (SPARK-31763) DataFrame.inputFiles() not Available

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115783#comment-17115783 ] Hyukjin Kwon commented on SPARK-31763: -- Are you interested in opening a PR for this

[jira] [Commented] (SPARK-31758) Incorrect timestamp parsing from JSON

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115786#comment-17115786 ] Hyukjin Kwon commented on SPARK-31758: -- This was fixed in the current master: {cod

[jira] [Resolved] (SPARK-31758) Incorrect timestamp parsing from JSON

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31758. -- Resolution: Cannot Reproduce > Incorrect timestamp parsing from JSON > ---

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31751: - Flags: (was: Important) > spark serde property path overwrites table property location > -

[jira] [Updated] (SPARK-31751) spark serde property path overwrites table property location

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31751: - Description: This is an issue that have caused us so many data errors.  1) using spark ( with h

[jira] [Created] (SPARK-31809) Infer IsNotNull for all children of NullIntolerant expressions

2020-05-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-31809: --- Summary: Infer IsNotNull for all children of NullIntolerant expressions Key: SPARK-31809 URL: https://issues.apache.org/jira/browse/SPARK-31809 Project: Spark

[jira] [Updated] (SPARK-31809) Infer IsNotNull for all children of NullIntolerant expressions

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31809: Attachment: default.png infer.png > Infer IsNotNull for all children of NullIntole

[jira] [Created] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Prakhar Jain (Jira)
Prakhar Jain created SPARK-31810: Summary: Fix recoverPartitions test in DDLSuite Key: SPARK-31810 URL: https://issues.apache.org/jira/browse/SPARK-31810 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-31809) Infer IsNotNull for all children of NullIntolerant expressions

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31809: Description: We should infer {{IsNotNull}} for all children of {{NullIntolerant}} expressions. Fo

[jira] [Updated] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Prakhar Jain (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakhar Jain updated SPARK-31810: - Component/s: Tests > Fix recoverPartitions test in DDLSuite > --

[jira] [Assigned] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31810: Assignee: Apache Spark > Fix recoverPartitions test in DDLSuite > ---

[jira] [Assigned] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31810: Assignee: (was: Apache Spark) > Fix recoverPartitions test in DDLSuite >

[jira] [Commented] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115795#comment-17115795 ] Apache Spark commented on SPARK-31810: -- User 'prakharjain09' has created a pull req

[jira] [Created] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-31811: --- Summary: Pushdown IsNotNull to file scan if possible Key: SPARK-31811 URL: https://issues.apache.org/jira/browse/SPARK-31811 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Description: We should Pushdown {{IsNotNull}} to file scan if possible. For example: {code:sql} CR

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Attachment: default.png pushdown.png > Pushdown IsNotNull to file scan if possible

[jira] [Created] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
Mohammad Islam created SPARK-31812: -- Summary: Spark to support the auto cancelation of delegation token when an Application completes Key: SPARK-31812 URL: https://issues.apache.org/jira/browse/SPARK-31812

[jira] [Created] (SPARK-31813) Cannot write snappy-compressed text files

2020-05-25 Thread Ondrej Kokes (Jira)
Ondrej Kokes created SPARK-31813: Summary: Cannot write snappy-compressed text files Key: SPARK-31813 URL: https://issues.apache.org/jira/browse/SPARK-31813 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115820#comment-17115820 ] Mohammad Islam commented on SPARK-31812: I have a patch and I would like to cont

[jira] [Commented] (SPARK-12312) JDBC connection to Kerberos secured databases fails on remote executors

2020-05-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115836#comment-17115836 ] Gabor Somogyi commented on SPARK-12312: --- Not yet planned but we can take a look at

[jira] [Created] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

2020-05-25 Thread Sunny Jain (Jira)
Sunny Jain created SPARK-31814: -- Summary: Null in Date conversion from yyMMddHHmmss for specific date and time Key: SPARK-31814 URL: https://issues.apache.org/jira/browse/SPARK-31814 Project: Spark

[jira] [Created] (SPARK-31815) Support Hive Kerberos login in JDBC connector

2020-05-25 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31815: - Summary: Support Hive Kerberos login in JDBC connector Key: SPARK-31815 URL: https://issues.apache.org/jira/browse/SPARK-31815 Project: Spark Issue Type: S

[jira] [Assigned] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-31811: --- Assignee: (was: Yuming Wang) > Pushdown IsNotNull to file scan if possible > --

[jira] [Created] (SPARK-31816) Create high level description about JDBC connection providers for developers

2020-05-25 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-31816: - Summary: Create high level description about JDBC connection providers for developers Key: SPARK-31816 URL: https://issues.apache.org/jira/browse/SPARK-31816 Projec

[jira] [Updated] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Islam updated SPARK-31812: --- Fix Version/s: (was: 2.4.7) > Spark to support the auto cancelation of delegation token

[jira] [Comment Edited] (SPARK-31812) Spark to support the auto cancelation of delegation token when an Application completes

2020-05-25 Thread Mohammad Islam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115820#comment-17115820 ] Mohammad Islam edited comment on SPARK-31812 at 5/25/20, 8:45 AM:

[jira] [Commented] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115855#comment-17115855 ] Gabor Somogyi commented on SPARK-31337: --- [~GeetBhalshankar] security is super sens

[jira] [Commented] (SPARK-31696) Support spark.kubernetes.driver.service.annotation

2020-05-25 Thread Canbin Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115873#comment-17115873 ] Canbin Zheng commented on SPARK-31696: -- Thanks [~dongjoon]! Since not familiar with

[jira] [Assigned] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31337: Assignee: (was: Apache Spark) > Support MS Sql Kerberos login in JDBC connector > ---

[jira] [Assigned] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31337: Assignee: Apache Spark > Support MS Sql Kerberos login in JDBC connector > --

[jira] [Commented] (SPARK-31337) Support MS Sql Kerberos login in JDBC connector

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17115922#comment-17115922 ] Apache Spark commented on SPARK-31337: -- User 'gaborgsomogyi' has created a pull req

[jira] [Created] (SPARK-31817) Pass-through of Kerberos credentials from Spark SQL to a jdbc source

2020-05-25 Thread Luis Lozano Coira (Jira)
Luis Lozano Coira created SPARK-31817: - Summary: Pass-through of Kerberos credentials from Spark SQL to a jdbc source Key: SPARK-31817 URL: https://issues.apache.org/jira/browse/SPARK-31817 Projec

[jira] [Created] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31818: -- Summary: Failure on pushing down filters with java.time.Instant values in ORC Key: SPARK-31818 URL: https://issues.apache.org/jira/browse/SPARK-31818 Project: Spark

[jira] [Updated] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-31818: --- Description: When spark.sql.datetime.java8API.enabled is set to true, filters pushed down with java

[jira] [Assigned] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31818: Assignee: Maxim Gekk (was: Apache Spark) > Failure on pushing down filters with java.tim

[jira] [Assigned] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31818: Assignee: Apache Spark (was: Maxim Gekk) > Failure on pushing down filters with java.tim

[jira] [Commented] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116004#comment-17116004 ] Apache Spark commented on SPARK-31818: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116006#comment-17116006 ] Apache Spark commented on SPARK-31818: -- User 'MaxGekk' has created a pull request f

[jira] [Resolved] (SPARK-30352) DataSourceV2: Add CURRENT_CATALOG function

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30352. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27006 [https://gith

[jira] [Updated] (SPARK-31734) add weight support in ClusteringEvaluator

2020-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-31734: - Priority: Minor (was: Major) > add weight support in ClusteringEvaluator >

[jira] [Resolved] (SPARK-31734) add weight support in ClusteringEvaluator

2020-05-25 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31734. -- Fix Version/s: 3.1.0 Assignee: Huaxin Gao Resolution: Fixed Resolved by https:

[jira] [Assigned] (SPARK-30352) DataSourceV2: Add CURRENT_CATALOG function

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30352: --- Assignee: Kent Yao > DataSourceV2: Add CURRENT_CATALOG function > -

[jira] [Commented] (SPARK-31629) "py4j.protocol.Py4JJavaError: An error occurred while calling o90.save" in pyspark 2.3.1

2020-05-25 Thread appleyuchi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116094#comment-17116094 ] appleyuchi commented on SPARK-31629: it has been solved,due to slave node can not ac

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Attachment: (was: pushdown.png) > Pushdown IsNotNull to file scan if possible > --

[jira] [Updated] (SPARK-31811) Pushdown IsNotNull to file scan if possible

2020-05-25 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31811: Attachment: pushdown.png > Pushdown IsNotNull to file scan if possible > -

[jira] [Updated] (SPARK-7768) Make user-defined type (UDT) API public

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-7768: --- Target Version/s: 3.1.0 (was: 3.0.0) > Make user-defined type (UDT) API public > ---

[jira] [Updated] (SPARK-31771) Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31771: Parent: SPARK-31408 Issue Type: Sub-task (was: Bug) > Disable Narrow TextStyle for dateti

[jira] [Updated] (SPARK-31814) Null in Date conversion from yyMMddHHmmss for specific date and time

2020-05-25 Thread Sunny Jain (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunny Jain updated SPARK-31814: --- Description: Hi,   We are trying to convert a column with string datatype to date type using below

[jira] [Commented] (SPARK-31771) Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116129#comment-17116129 ] Apache Spark commented on SPARK-31771: -- User 'yaooqinn' has created a pull request

[jira] [Commented] (SPARK-31771) Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116130#comment-17116130 ] Apache Spark commented on SPARK-31771: -- User 'yaooqinn' has created a pull request

[jira] [Resolved] (SPARK-31818) Failure on pushing down filters with java.time.Instant values in ORC

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31818. --- Resolution: Fixed Issue resolved by pull request 28636 [https://github.com/apache/spark/pull

[jira] [Resolved] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31808. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28633 [https://

[jira] [Assigned] (SPARK-31808) Make struct function's output name and class name pretty

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31808: - Assignee: Hyukjin Kwon > Make struct function's output name and class name pretty > ---

[jira] [Commented] (SPARK-31800) Unable to disable Kerberos when submitting jobs to Kubernetes

2020-05-25 Thread Devaraj Kavali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116420#comment-17116420 ] Devaraj Kavali commented on SPARK-31800: Seems there has been considerable chang

[jira] [Resolved] (SPARK-31810) Fix recoverPartitions test in DDLSuite

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31810. -- Fix Version/s: 3.0.0 Assignee: Prakhar Jain Resolution: Fixed Fixed in https:/

[jira] [Resolved] (SPARK-31806) Check reading date/timestamp from Parquet: plain, dictionary enc, Spark version

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31806. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28630 [https://gith

[jira] [Assigned] (SPARK-31806) Check reading date/timestamp from Parquet: plain, dictionary enc, Spark version

2020-05-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31806: --- Assignee: Maxim Gekk > Check reading date/timestamp from Parquet: plain, dictionary enc, Sp

[jira] [Resolved] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31792. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 28609 [https://gi

[jira] [Assigned] (SPARK-31792) Introduce the structured streaming UI in the Web UI page

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31792: Assignee: Xingcan Cui > Introduce the structured streaming UI in the Web UI page > --

[jira] [Assigned] (SPARK-31673) Enhance QueryExecution.debugFile to take an additional explain mode parameter.

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31673: Assignee: Dilip Biswal > Enhance QueryExecution.debugFile to take an additional explain m

[jira] [Resolved] (SPARK-31673) Enhance QueryExecution.debugFile to take an additional explain mode parameter.

2020-05-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31673. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28493 [https://gi

[jira] [Created] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-31819: - Summary: Add a workaround for Java 8u251+ and update integration test cases Key: SPARK-31819 URL: https://issues.apache.org/jira/browse/SPARK-31819 Project: Spark

[jira] [Commented] (SPARK-31786) Exception on submitting Spark-Pi to Kubernetes 1.17.3

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116460#comment-17116460 ] Dongjoon Hyun commented on SPARK-31786: --- Okay. I'll create a PR for that, [~maver1

[jira] [Assigned] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31819: Assignee: (was: Apache Spark) > Add a workaround for Java 8u251+ and update integrati

[jira] [Commented] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116461#comment-17116461 ] Apache Spark commented on SPARK-31819: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31819: Assignee: Apache Spark > Add a workaround for Java 8u251+ and update integration test cas

[jira] [Commented] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116462#comment-17116462 ] Apache Spark commented on SPARK-31819: -- User 'dongjoon-hyun' has created a pull req

[jira] [Commented] (SPARK-31794) Incorrect distribution with repartitionByRange and repartition column expression

2020-05-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116470#comment-17116470 ] Jungtaek Lim commented on SPARK-31794: -- http://spark.apache.org/docs/3.0.0-preview2

[jira] [Created] (SPARK-31820) Flaky JavaBeanDeserializationSuite

2020-05-25 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-31820: -- Summary: Flaky JavaBeanDeserializationSuite Key: SPARK-31820 URL: https://issues.apache.org/jira/browse/SPARK-31820 Project: Spark Issue Type: Bug Comp

[jira] [Updated] (SPARK-31819) Add a workaround for Java 8u251+ and update integration test cases

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31819: -- Priority: Blocker (was: Major) > Add a workaround for Java 8u251+ and update integration test

[jira] [Updated] (SPARK-31819) Add a workaround for Java 8u251+/K8s 1.17 and update integration test cases

2020-05-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31819: -- Summary: Add a workaround for Java 8u251+/K8s 1.17 and update integration test cases (was: Ad

[jira] [Assigned] (SPARK-31820) Flaky JavaBeanDeserializationSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31820: Assignee: (was: Apache Spark) > Flaky JavaBeanDeserializationSuite >

[jira] [Assigned] (SPARK-31820) Flaky JavaBeanDeserializationSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-31820: Assignee: Apache Spark > Flaky JavaBeanDeserializationSuite > ---

[jira] [Commented] (SPARK-31820) Flaky JavaBeanDeserializationSuite

2020-05-25 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17116494#comment-17116494 ] Apache Spark commented on SPARK-31820: -- User 'MaxGekk' has created a pull request f