[jira] [Updated] (SPARK-36831) Read/write dataframes with ANSI intervals from/to CSV files

2021-09-29 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36831: --- Description: Implement writing and reading ANSI intervals (year-month and day-time

[jira] [Commented] (SPARK-36831) Read/write dataframes with ANSI intervals from/to CSV files

2021-09-29 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422007#comment-17422007 ] Kousuke Saruta commented on SPARK-36831: [~maxgekk] Are you already working on this? > R

[jira] [Resolved] (SPARK-36882) Support ILIKE API on Python

2021-09-29 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36882. Fix Version/s: 3.3.0 Assignee: Leona Yoda Resolution: Fixed Issue

[jira] [Updated] (SPARK-36865) Add PySpark API document of session_window

2021-09-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36865: --- Summary: Add PySpark API document of session_window (was: Add PySpark API document

[jira] [Updated] (SPARK-36865) Add PySpark API document of session_window

2021-09-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36865: --- Description: There is no PySpark API document of session_window. The docstring

[jira] [Updated] (SPARK-36865) Add PySpark API document for session_window

2021-09-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36865: --- Description: There is no PySpark API document for session_window. The docstring

[jira] [Updated] (SPARK-36865) Add PySpark API document for session_window

2021-09-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36865: --- Summary: Add PySpark API document for session_window (was: The layout of PySpark API

[jira] [Created] (SPARK-36865) The layout of PySpark API document for session_window is broken

2021-09-27 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36865: -- Summary: The layout of PySpark API document for session_window is broken Key: SPARK-36865 URL: https://issues.apache.org/jira/browse/SPARK-36865 Project: Spark

[jira] [Commented] (SPARK-36860) Create the external hive table for HBase failed

2021-09-26 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420483#comment-17420483 ] Kousuke Saruta commented on SPARK-36860: [~yimo_yym] Spark doesn't support creating Hive table

[jira] [Updated] (SPARK-36808) Upgrade Kafka to 2.8.1

2021-09-20 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36808: --- Issue Type: Improvement (was: Bug) > Upgrade Kafka to 2.

[jira] [Resolved] (SPARK-36683) Support secant and cosecant

2021-09-20 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36683. Fix Version/s: 3.3.0 Assignee: Yuto Akutsu Resolution: Fixed Issue

[jira] [Created] (SPARK-36808) Upgrade Kafka to 2.8.1

2021-09-20 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36808: -- Summary: Upgrade Kafka to 2.8.1 Key: SPARK-36808 URL: https://issues.apache.org/jira/browse/SPARK-36808 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36777) Move Java 17 on GitHub Actions from EA to LTS release

2021-09-16 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36777: --- Issue Type: Improvement (was: Bug) > Move Java 17 on GitHub Actions from EA to LTS rele

[jira] [Created] (SPARK-36777) Move Java 17 on GitHub Actions from EA to LTS release

2021-09-16 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36777: -- Summary: Move Java 17 on GitHub Actions from EA to LTS release Key: SPARK-36777 URL: https://issues.apache.org/jira/browse/SPARK-36777 Project: Spark

[jira] [Resolved] (SPARK-36751) octet_length/bit_length API is not implemented on Scala/Python/R

2021-09-15 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36751. Fix Version/s: 3.3.0 Assignee: Leona Yoda Resolution: Fixed Issue

[jira] [Updated] (SPARK-35930) Upgrade kinesis-client to 1.14.4

2021-09-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-35930: --- Priority: Major (was: Minor) > Upgrade kinesis-client to 1.1

[jira] [Commented] (SPARK-35930) Upgrade kinesis-client to 1.14.4

2021-09-13 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414656#comment-17414656 ] Kousuke Saruta commented on SPARK-35930: [~holden] Yes, I didn't think it's a common case so I

[jira] [Updated] (SPARK-36729) Upgrade Netty from 4.1.63 to 4.1.68

2021-09-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36729: --- Priority: Minor (was: Major) > Upgrade Netty from 4.1.63 to 4.1

[jira] [Created] (SPARK-36729) Upgrade Netty from 4.1.63 to 4.1.68

2021-09-12 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36729: -- Summary: Upgrade Netty from 4.1.63 to 4.1.68 Key: SPARK-36729 URL: https://issues.apache.org/jira/browse/SPARK-36729 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-28620) Double type returned for float type in Beeline/JDBC

2021-09-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413602#comment-17413602 ] Kousuke Saruta commented on SPARK-28620: [~yumwang] The reason is ColumnValue.floatValue treats

[jira] [Created] (SPARK-36725) Ensure HiveThriftServer2Suites to stop Thrift JDBC server on exit

2021-09-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36725: -- Summary: Ensure HiveThriftServer2Suites to stop Thrift JDBC server on exit Key: SPARK-36725 URL: https://issues.apache.org/jira/browse/SPARK-36725 Project: Spark

[jira] [Created] (SPARK-36724) Support timestamp_ntz as a type of time column for SessionWindow

2021-09-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36724: -- Summary: Support timestamp_ntz as a type of time column for SessionWindow Key: SPARK-36724 URL: https://issues.apache.org/jira/browse/SPARK-36724 Project: Spark

[jira] [Updated] (SPARK-36723) day-time interval types should respect daylight saving time correctly

2021-09-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36723: --- Description: In the current master, day-time interval types handle 24 hours as 1 day

[jira] [Updated] (SPARK-36723) day-time interval types should respect daylight saving time correctly

2021-09-11 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36723: --- Description: In the current master, day-time interval types handle 24 hours as 1 day

[jira] [Created] (SPARK-36723) day-time interval types should respect daylight saving time correctly

2021-09-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36723: -- Summary: day-time interval types should respect daylight saving time correctly Key: SPARK-36723 URL: https://issues.apache.org/jira/browse/SPARK-36723 Project

[jira] [Commented] (SPARK-36694) Unable to build Spark on Azure DevOps with ubuntu-latest

2021-09-08 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412272#comment-17412272 ] Kousuke Saruta commented on SPARK-36694: [~LvffY] Could you make sure that Java8 is used

[jira] [Commented] (SPARK-36683) Support secent and cosecent

2021-09-08 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17411808#comment-17411808 ] Kousuke Saruta commented on SPARK-36683: [~YActs] This issue is still open so let's not set Fix

[jira] [Updated] (SPARK-36683) Support secent and cosecent

2021-09-08 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36683: --- Fix Version/s: (was: 3.3.0) > Support secent and cosec

[jira] [Created] (SPARK-36675) Support ScriptTransformation for timestamp_ntz

2021-09-06 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36675: -- Summary: Support ScriptTransformation for timestamp_ntz Key: SPARK-36675 URL: https://issues.apache.org/jira/browse/SPARK-36675 Project: Spark Issue

[jira] [Commented] (SPARK-36672) Using value from map in grouping sets result org.apache.spark.sql.AnalysisException

2021-09-06 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410441#comment-17410441 ] Kousuke Saruta commented on SPARK-36672: [~jhu] After SPARK-33308 was resolved, this issue seems

[jira] [Resolved] (SPARK-36603) Use WeakReference over SoftReference in LevelDB to avoid retaining iterator refs

2021-09-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36603. Fix Version/s: 3.1.3 3.2.0 Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-36635) spark-sql do NOT support that select name expression as string type now

2021-09-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36635. Resolution: Not A Problem [~weixiuli] I'll close this. If you can justify that it should

[jira] [Updated] (SPARK-36660) Cotangent is not supported by Dataframe

2021-09-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36660: --- Issue Type: Improvement (was: Bug) > Cotangent is not supported by Datafr

[jira] [Updated] (SPARK-36664) Log time spent waiting for cluster resources

2021-09-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36664: --- Issue Type: Improvement (was: Bug) > Log time spent waiting for cluster resour

[jira] [Commented] (SPARK-36668) py4j.protocol.Py4JJavaError: java.util.NoSuchElementException: None.get

2021-09-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410126#comment-17410126 ] Kousuke Saruta commented on SPARK-36668: [~ruzi]I meant that 2.4.8 is the last release of 2.4.x

[jira] [Commented] (SPARK-36668) py4j.protocol.Py4JJavaError: java.util.NoSuchElementException: None.get

2021-09-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410112#comment-17410112 ] Kousuke Saruta commented on SPARK-36668: [~ruzi] No more 2.4.x will be released. Does this issue

[jira] [Commented] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409539#comment-17409539 ] Kousuke Saruta commented on SPARK-36639: Issue resolved in https://github.com/apache/spark/pull

[jira] [Resolved] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36639. Assignee: Kousuke Saruta Resolution: Fixed > SQL sequence function with inter

[jira] [Updated] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36639: --- Fix Version/s: 3.1.3 3.2.0 > SQL sequence function with interval retu

[jira] [Commented] (SPARK-36639) SQL sequence function with interval returns unexpected error in latest versions

2021-09-01 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17408267#comment-17408267 ] Kousuke Saruta commented on SPARK-36639: [~amstel8] Thank you for reporting. SPARK-31980 seems

[jira] [Updated] (SPARK-36605) Upgrade Jackson to 2.12.5

2021-08-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36605: --- Description: Recently, Jackson 2.12.5 was released and it seems to be expected as the last

[jira] [Updated] (SPARK-36605) Upgrade Jackson to 2.12.5

2021-08-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36605: --- Description: Recently, Jackson 2.12.5 was released and it seems to be expected as the last

[jira] [Resolved] (SPARK-36509) Executors don't get rescheduled in standalone mode when worker dies

2021-08-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36509. Target Version/s: 3.2.0, 3.1.3, 3.0.4 Assignee: Kousuke Saruta

[jira] [Updated] (SPARK-36605) Upgrade Jackson to 2.12.5

2021-08-28 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36605: --- Description: Recently, Jackson 2.12.5 was released and it seems to be expected as the last

[jira] [Created] (SPARK-36605) Upgrade Jackson to 2.12.5

2021-08-28 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36605: -- Summary: Upgrade Jackson to 2.12.5 Key: SPARK-36605 URL: https://issues.apache.org/jira/browse/SPARK-36605 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36601) Reading a valid JSON file causes a NPE

2021-08-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17405993#comment-17405993 ] Kousuke Saruta commented on SPARK-36601: [~lyytinen] This issue seems resolved in SPARK-36379

[jira] [Updated] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36398: --- Fix Version/s: 3.1.3 3.2.0 > Redact sensitive information in Sp

[jira] [Resolved] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36398. Target Version/s: 3.2.0, 3.1.3 Resolution: Fixed Issue resolved in https

[jira] [Updated] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36398: --- Target Version/s: (was: 3.2.0, 3.1.3) > Redact sensitive information in Spark Thr

[jira] [Reopened] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta reopened SPARK-36398: > Redact sensitive information in Spark Thrift Server

[jira] [Commented] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404322#comment-17404322 ] Kousuke Saruta commented on SPARK-36400: Issue resolved in https://github.com/apache/spark/pull

[jira] [Updated] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36400: --- Fix Version/s: (was: 3.1.2) 3.1.3 > Redact sensitive informat

[jira] [Updated] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36400: --- Fix Version/s: (was: 3.3.0) 3.1.2 > Redact sensitive informat

[jira] [Updated] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36400: --- Fix Version/s: (was: 3.1.3) 3.3.0 > Redact sensitive informat

[jira] [Updated] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-25 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36398: --- Fix Version/s: (was: 3.1.3) (was: 3.2.0) > Redact sensit

[jira] [Commented] (SPARK-36572) Removal of netty-3.9.9.Final.jar dependent code and use only netty-all-4.1.47.Final.jar in Spark 2.4.x

2021-08-24 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403798#comment-17403798 ] Kousuke Saruta commented on SPARK-36572: [~sshukla05] Sorry, 2.4.8 is the last release of 2.4.x

[jira] [Commented] (SPARK-36429) JacksonParser should throw exception when data type unsupported.

2021-08-22 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402951#comment-17402951 ] Kousuke Saruta commented on SPARK-36429: Issue resolved in https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-36544) Special strings for dates are no longer automatically converted without the "date" keyword

2021-08-20 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402366#comment-17402366 ] Kousuke Saruta commented on SPARK-36544: [~laurikoobas] The behavior was changed. See SPARK

[jira] [Commented] (SPARK-36548) Throwing NoClassDefFoundError for Logging$class

2021-08-20 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402364#comment-17402364 ] Kousuke Saruta commented on SPARK-36548: [~sadagopan] It was wrongly exposed before. Logging

[jira] [Resolved] (SPARK-36398) Redact sensitive information in Spark Thrift Server log

2021-08-17 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36398. Fix Version/s: 3.1.3 3.2.0 Assignee: Kousuke Saruta

[jira] [Resolved] (SPARK-36400) Redact sensitive information in Spark Thrift Server UI

2021-08-17 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36400. Fix Version/s: 3.1.3 3.2.0 Assignee: Kousuke Saruta

[jira] [Created] (SPARK-36512) Fix UISeleniumSuite in sql/hive-thriftserver

2021-08-14 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36512: -- Summary: Fix UISeleniumSuite in sql/hive-thriftserver Key: SPARK-36512 URL: https://issues.apache.org/jira/browse/SPARK-36512 Project: Spark Issue Type

[jira] [Created] (SPARK-36507) Remove/Replace missing links to AMP Camp materials from index.md

2021-08-12 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36507: -- Summary: Remove/Replace missing links to AMP Camp materials from index.md Key: SPARK-36507 URL: https://issues.apache.org/jira/browse/SPARK-36507 Project: Spark

[jira] [Created] (SPARK-36491) Make from_json/to_json to handle timestamp_ntz type properly

2021-08-12 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36491: -- Summary: Make from_json/to_json to handle timestamp_ntz type properly Key: SPARK-36491 URL: https://issues.apache.org/jira/browse/SPARK-36491 Project: Spark

[jira] [Updated] (SPARK-36490) Make from_csv/to_csv to handle timestamp_ntz type properly

2021-08-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36490: --- Summary: Make from_csv/to_csv to handle timestamp_ntz type properly (was: Make from_csv

[jira] [Created] (SPARK-36490) Make from_csv/to_csv to handle timestamp w/o timezone properly

2021-08-12 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36490: -- Summary: Make from_csv/to_csv to handle timestamp w/o timezone properly Key: SPARK-36490 URL: https://issues.apache.org/jira/browse/SPARK-36490 Project: Spark

[jira] [Created] (SPARK-36441) Downloading lintr dependencies fail on GA

2021-08-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36441: -- Summary: Downloading lintr dependencies fail on GA Key: SPARK-36441 URL: https://issues.apache.org/jira/browse/SPARK-36441 Project: Spark Issue Type

[jira] [Resolved] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36426. Resolution: Invalid I re-triggered the failed GA job and it finally succeeded so I'll

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Description: GA is failing and the root cause seems the latest release of pyzmq, which

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Description: GA is failing and the root cause seems the latest release of pyzmq, which

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Description: GA is failing and the root cause seems the latest release of pyzmq. https

[jira] [Updated] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36426: --- Affects Version/s: 3.2.0 > Pin pyzmq to 2.2

[jira] [Created] (SPARK-36426) Pin pyzmq to 2.22.0

2021-08-05 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36426: -- Summary: Pin pyzmq to 2.22.0 Key: SPARK-36426 URL: https://issues.apache.org/jira/browse/SPARK-36426 Project: Spark Issue Type: Bug Components

[jira] [Commented] (SPARK-35870) Upgrade Jetty to 9.4.42

2021-08-04 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393277#comment-17393277 ] Kousuke Saruta commented on SPARK-35870: Hi [~this]. v2.4.8 is the last release of brnch-2.4

[jira] [Updated] (SPARK-36408) Upgrade json4s to 4.0.3

2021-08-03 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36408: --- Issue Type: Improvement (was: Bug) > Upgrade json4s to 4.

[jira] [Created] (SPARK-36408) Upgrade json4s to 4.0.3

2021-08-03 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36408: -- Summary: Upgrade json4s to 4.0.3 Key: SPARK-36408 URL: https://issues.apache.org/jira/browse/SPARK-36408 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36382) Remove noisy footer from the summary table for metrics

2021-08-02 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36382: --- Summary: Remove noisy footer from the summary table for metrics (was: Remove unnecesssary

[jira] [Created] (SPARK-36382) Remove unnecesssary footer from the summary table for metrics

2021-08-02 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36382: -- Summary: Remove unnecesssary footer from the summary table for metrics Key: SPARK-36382 URL: https://issues.apache.org/jira/browse/SPARK-36382 Project: Spark

[jira] [Created] (SPARK-36371) Support raw string literal

2021-07-31 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36371: -- Summary: Support raw string literal Key: SPARK-36371 URL: https://issues.apache.org/jira/browse/SPARK-36371 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-36318) Update docs about mapping of ANSI interval types to Java/Scala/SQL types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-36318. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved in https://github.com/apache

[jira] [Created] (SPARK-36323) Support ANSI interval literals for TimeWindow

2021-07-27 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36323: -- Summary: Support ANSI interval literals for TimeWindow Key: SPARK-36323 URL: https://issues.apache.org/jira/browse/SPARK-36323 Project: Spark Issue Type

[jira] [Updated] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34619: --- Affects Version/s: 3.2.0 > Update the Spark SQL guide about day-time and year-mo

[jira] [Updated] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34619: --- Affects Version/s: (was: 3.3.0) > Update the Spark SQL guide about day-time and y

[jira] [Updated] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-34619: --- Affects Version/s: (was: 3.2.0) 3.3.0 > Update the Spark

[jira] [Resolved] (SPARK-34619) Update the Spark SQL guide about day-time and year-month interval types

2021-07-27 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-34619. Fix Version/s: 3.2.0 Assignee: Max Gekk Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36256) Upgrade lz4-java to 1.8.0

2021-07-22 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36256: -- Summary: Upgrade lz4-java to 1.8.0 Key: SPARK-36256 URL: https://issues.apache.org/jira/browse/SPARK-36256 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36244) Upgrade zstd-jni to 1.5.0-3 to avoid a bug about buffer size calculation

2021-07-21 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36244: -- Summary: Upgrade zstd-jni to 1.5.0-3 to avoid a bug about buffer size calculation Key: SPARK-36244 URL: https://issues.apache.org/jira/browse/SPARK-36244 Project

[jira] [Commented] (SPARK-35815) Allow delayThreshold for watermark to be represented as ANSI day-time/year-month interval literals

2021-07-19 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383781#comment-17383781 ] Kousuke Saruta commented on SPARK-35815: I don't think so, and I'll work on this soon. > Al

[jira] [Updated] (SPARK-36208) SparkScriptTransformation should support ANSI interval types

2021-07-19 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36208: --- Summary: SparkScriptTransformation should support ANSI interval types

[jira] [Updated] (SPARK-36208) SparkScriptTransformation

2021-07-19 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36208: --- Parent: SPARK-27790 Issue Type: Sub-task (was: Bug) > SparkScriptTransformat

[jira] [Created] (SPARK-36208) SparkScriptTransformation

2021-07-19 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36208: -- Summary: SparkScriptTransformation Key: SPARK-36208 URL: https://issues.apache.org/jira/browse/SPARK-36208 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-36171) Upgrade GenJavadoc to 0.18

2021-07-15 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36171: -- Summary: Upgrade GenJavadoc to 0.18 Key: SPARK-36171 URL: https://issues.apache.org/jira/browse/SPARK-36171 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36170) Change quoted interval literal (interval constructor) to be converted to ANSI interval types

2021-07-15 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36170: -- Summary: Change quoted interval literal (interval constructor) to be converted to ANSI interval types Key: SPARK-36170 URL: https://issues.apache.org/jira/browse/SPARK-36170

[jira] [Created] (SPARK-36129) Upgrade commons-compress to 1.21 to deal with CVEs

2021-07-13 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36129: -- Summary: Upgrade commons-compress to 1.21 to deal with CVEs Key: SPARK-36129 URL: https://issues.apache.org/jira/browse/SPARK-36129 Project: Spark Issue

[jira] [Updated] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36110: --- Description: SBT 1.5.5 was released, which includes 16 imrovements/bug fixes. https

[jira] [Updated] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36110: --- Issue Type: Improvement (was: Bug) > Upgrade SBT to 1.

[jira] [Created] (SPARK-36110) Upgrade SBT to 1.5.5

2021-07-12 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36110: -- Summary: Upgrade SBT to 1.5.5 Key: SPARK-36110 URL: https://issues.apache.org/jira/browse/SPARK-36110 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36081) Update the document about the behavior change of trimming characters for cast

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36081: --- Description: sql-migration-guide.md mentions about the behavior of cast like as follows

[jira] [Updated] (SPARK-36081) Update the document about the behavior change of trimming characters for cast

2021-07-12 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36081: --- Summary: Update the document about the behavior change of trimming characters for cast

[jira] [Updated] (SPARK-36081) The implementation of cast doesn't comply with its specification

2021-07-10 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-36081: --- Description: sql-migration-guide.md mentions about the behavior of cast like as follows

<    1   2   3   4   5   6   7   8   9   10   >