[jira] [Created] (SPARK-49128) Support custom History Server UI title

2024-08-06 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49128: - Summary: Support custom History Server UI title Key: SPARK-49128 URL: https://issues.apache.org/jira/browse/SPARK-49128 Project: Spark Issue Type: Sub

[jira] [Assigned] (SPARK-49126) Move `spark.history.ui.maxApplications` config definition to `History.scala`

2024-08-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49126: - Assignee: Dongjoon Hyun > Move `spark.history.ui.maxApplications` config definition

[jira] [Created] (SPARK-49126) Move `spark.history.ui.maxApplications` config definition to `History.scala`

2024-08-06 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49126: - Summary: Move `spark.history.ui.maxApplications` config definition to `History.scala` Key: SPARK-49126 URL: https://issues.apache.org/jira/browse/SPARK-49126

[jira] [Resolved] (SPARK-49098) Support Table Options for Insert SQL

2024-08-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49098. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47591 [https

[jira] [Assigned] (SPARK-49098) Support Table Options for Insert SQL

2024-08-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49098: - Assignee: Szehon Ho > Support Table Options for Insert

[jira] [Resolved] (SPARK-49116) Fix `InvalidDefaultArgInFrom` in Python/R binding Dockerfiles

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49116. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47617 [https

[jira] [Assigned] (SPARK-49014) Bump Apache Avro to 1.12.0

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49014: - Assignee: Fokko Driesprong > Bump Apache Avro to 1.1

[jira] [Updated] (SPARK-49014) Bump Apache Avro to 1.12.0

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49014: -- Target Version/s: (was: 4.0.0) > Bump Apache Avro to 1.1

[jira] [Resolved] (SPARK-49014) Bump Apache Avro to 1.12.0

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49014. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47498 [https

[jira] [Updated] (SPARK-49014) Bump Apache Avro to 1.12.0

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49014: -- Component/s: Build (was: Input/Output) > Bump Apache Avro to 1.1

[jira] [Resolved] (SPARK-49097) Add Python3 environment detection for the `build_error_docs` method in `build_api_decs.rb`

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49097. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47590 [https

[jira] [Assigned] (SPARK-49097) Add Python3 environment detection for the `build_error_docs` method in `build_api_decs.rb`

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49097: - Assignee: Wei Guo > Add Python3 environment detection for the `build_error_d

[jira] [Resolved] (ORC-1753) Use Avro 1.12.0 in bench module

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1753. Fix Version/s: 2.0.2 Resolution: Fixed Issue resolved by pull request 1996 [https

[jira] [Created] (SPARK-49117) Fix `docker-image-tool.sh` up-to-date

2024-08-05 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49117: - Summary: Fix `docker-image-tool.sh` up-to-date Key: SPARK-49117 URL: https://issues.apache.org/jira/browse/SPARK-49117 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-49117) Fix `docker-image-tool.sh` to be up-to-date

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49117: -- Summary: Fix `docker-image-tool.sh` to be up-to-date (was: Fix `docker-image-tool.sh` up-to

[jira] [Assigned] (SPARK-49116) Fix `InvalidDefaultArgInFrom` in Python/R binding Dockerfiles

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49116: - Assignee: Dongjoon Hyun > Fix `InvalidDefaultArgInFrom` in Python/R bind

[jira] [Created] (SPARK-49116) Fix `InvalidDefaultArgInFrom` in Python/R binding Dockerfiles

2024-08-05 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49116: - Summary: Fix `InvalidDefaultArgInFrom` in Python/R binding Dockerfiles Key: SPARK-49116 URL: https://issues.apache.org/jira/browse/SPARK-49116 Project: Spark

[jira] [Assigned] (ORC-1753) Use Avro 1.12.0 in bench module

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1753: -- Assignee: Dongjoon Hyun > Use Avro 1.12.0 in bench mod

[jira] [Created] (ORC-1753) Use Avro 1.12.0 in bench module

2024-08-05 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1753: -- Summary: Use Avro 1.12.0 in bench module Key: ORC-1753 URL: https://issues.apache.org/jira/browse/ORC-1753 Project: ORC Issue Type: Test Components

[jira] [Updated] (ORC-1697) Fix IllegalArgumentException when reading json timestamp type in benchmark

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1697: --- Issue Type: Test (was: Bug) > Fix IllegalArgumentException when reading json timestamp type

[jira] [Updated] (ORC-1752) Fix NumberFormatException when reading json timestamp type in benchmark

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1752: --- Issue Type: Test (was: Bug) > Fix NumberFormatException when reading json timestamp type in benchm

[jira] [Updated] (ORC-1752) Fix NumberFormatException when reading json timestamp type in benchmark

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1752: --- Fix Version/s: (was: 2.1.0) > Fix NumberFormatException when reading json timestamp type

[jira] [Assigned] (ORC-1752) Fix NumberFormatException when reading json timestamp type in benchmark

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1752: -- Assignee: dzcxzl > Fix NumberFormatException when reading json timestamp type in benchm

[jira] [Resolved] (ORC-1752) Fix NumberFormatException when reading json timestamp type in benchmark

2024-08-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1752. Fix Version/s: 2.1.0 2.0.2 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-49108) Add `submit_pi.sh` REST API example

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49108: -- Summary: Add `submit_pi.sh` REST API example (was: Add `submit_pi.sh` example via REST API

[jira] [Updated] (SPARK-49108) Add `submit_pi.sh` example via REST API

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49108: -- Summary: Add `submit_pi.sh` example via REST API (was: Add `SparkPi` example to use REST API

[jira] [Updated] (ORC-1697) Fix IllegalArgumentException when reading json timestamp type in benchmark

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1697: --- Issue Type: Bug (was: Improvement) > Fix IllegalArgumentException when reading json timestamp type

[jira] [Updated] (ORC-1697) Fix IllegalArgumentException when reading json timestamp type in benchmark

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1697: --- Fix Version/s: (was: 2.1.0) > Fix IllegalArgumentException when reading json timestamp type

[jira] [Assigned] (ORC-1697) Fix IllegalArgumentException when reading json timestamp type in benchmark

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1697: -- Assignee: dzcxzl > Fix IllegalArgumentException when reading json timestamp type in benchm

[jira] [Resolved] (ORC-1697) Fix IllegalArgumentException when reading json timestamp type in benchmark

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1697. Fix Version/s: 2.1.0 2.0.2 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-49106) Documented Prometheus endpoints

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49106. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47219 [https

[jira] [Created] (SPARK-49106) Documented Prometheus endpoints

2024-08-04 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49106: - Summary: Documented Prometheus endpoints Key: SPARK-49106 URL: https://issues.apache.org/jira/browse/SPARK-49106 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-48731) Upgrade `docker/build-push-action` to v6

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48731. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47112 [https

[jira] [Resolved] (SPARK-48949) SPJ: Runtime Partition Filtering

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48949. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47426 [https

[jira] [Resolved] (SPARK-49105) Upgrade ojdbc11 to 23.5.0.24.07 and OracleDatabaseOnDocker docker image tag to oracle-free:23.5-slim

2024-08-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49105. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47597 [https

[jira] [Created] (SPARK-49104) Document `JWSFilter` usage in Spark UI and REST API and rename parameter to `secretKey`

2024-08-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49104: - Summary: Document `JWSFilter` usage in Spark UI and REST API and rename parameter to `secretKey` Key: SPARK-49104 URL: https://issues.apache.org/jira/browse/SPARK-49104

[jira] [Resolved] (SPARK-49103) Support `spark.master.rest.filters`

2024-08-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49103. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47595 [https

[jira] [Assigned] (SPARK-49103) Support `spark.master.rest.filters`

2024-08-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49103: - Assignee: Dongjoon Hyun > Support `spark.master.rest.filt

[jira] [Created] (SPARK-49103) Support `spark.master.rest.filters`

2024-08-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49103: - Summary: Support `spark.master.rest.filters` Key: SPARK-49103 URL: https://issues.apache.org/jira/browse/SPARK-49103 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-49080) Upgrade mssql-jdbc to 12.8.0.jre11

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49080. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47569 [https

[jira] [Resolved] (SPARK-49094) ignoreCorruptFiles file source option is partially supported for orc format

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49094. --- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution

[jira] [Assigned] (SPARK-49094) ignoreCorruptFiles file source option is partially supported for orc format

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49094: - Assignee: Kent Yao > ignoreCorruptFiles file source option is partially supported

[jira] [Updated] (SPARK-49094) ignoreCorruptFiles file source option is partially supported for orc format

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49094: -- Issue Type: Improvement (was: Bug) > ignoreCorruptFiles file source option is partia

[jira] [Updated] (SPARK-49094) ignoreCorruptFiles file source option is partially supported for orc format

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49094: -- Issue Type: Bug (was: Improvement) > ignoreCorruptFiles file source option is partia

[jira] [Assigned] (SPARK-49090) Support `JWSFilter`

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49090: - Assignee: Dongjoon Hyun > Support `JWSFil

[jira] [Resolved] (SPARK-49090) Support `JWSFilter`

2024-08-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49090. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47575 [https

[jira] [Created] (SPARK-49090) Support `JWSFilter`

2024-08-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49090: - Summary: Support `JWSFilter` Key: SPARK-49090 URL: https://issues.apache.org/jira/browse/SPARK-49090 Project: Spark Issue Type: Sub-task

Re: [NOTICE] Progress of 3.5.2-RC5

2024-08-01 Thread Dongjoon Hyun
Thank you for summarizing them and leading the release, Kent. :) Dongjoon. On Wed, Jul 31, 2024 at 10:39 PM Kent Yao wrote: > Hi dev, > > Since version 3.5.2-RC4, we have received several reports regarding > correctness issues, some of which are still unresolved. We will need a > few days to ad

[jira] [Commented] (SPARK-48950) Corrupt data from parquet scans

2024-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870285#comment-17870285 ] Dongjoon Hyun commented on SPARK-48950: --- According to the Apache Spark guide

[jira] [Updated] (SPARK-48950) Corrupt data from parquet scans

2024-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-48950: -- Target Version/s: 3.5.3 > Corrupt data from parquet sc

[jira] [Assigned] (SPARK-49052) Add SparkOperator class and tests

2024-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49052: - Assignee: Zhou JIANG > Add SparkOperator class and te

[jira] [Resolved] (SPARK-49052) Add SparkOperator class and tests

2024-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49052. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-49052) Add SparkOperator class and tests

2024-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49052: -- Summary: Add SparkOperator class and tests (was: Add SparkOperator main entry point class

[jira] [Commented] (SPARK-49030) Self join of a CTE seems non-deterministic

2024-08-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870269#comment-17870269 ] Dongjoon Hyun commented on SPARK-49030: --- Thank you for reporting, [~jihoo

[jira] [Resolved] (SPARK-49066) Reduce the effective scope of the configuration `spark.hadoop.hadoop.security.key.provider.path` in test scenarios

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49066. --- Fix Version/s: 4.0.0 3.5.2 Resolution: Fixed > Reduce

[jira] [Updated] (SPARK-49066) Reduce the effective scope of the configuration `spark.hadoop.hadoop.security.key.provider.path` in test scenarios

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49066: -- Issue Type: Test (was: Improvement) > Reduce the effective scope of the configurat

[jira] [Updated] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44638: -- Fix Version/s: 3.5.2 > Unable to read from JDBC data sources when using custom sch

[jira] [Resolved] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44638. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47551 [https

[jira] [Assigned] (SPARK-44638) Unable to read from JDBC data sources when using custom schema containing varchar

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44638: - Assignee: Kent Yao > Unable to read from JDBC data sources when using custom sch

[jira] [Commented] (SPARK-18134) SQL: MapType in Group BY and Joins not working

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869944#comment-17869944 ] Dongjoon Hyun commented on SPARK-18134: --- You can file a working PR to the Ap

Apache ORC 2.0.2 Release?

2024-07-31 Thread Dongjoon Hyun
Hi, All. Apache ORC 2.0.2 is scheduled on 2024-08-15. I'd like to volunteer for the 2.0.2 release manager. In this release, there are a few important patches requested by other ASF communities. In addition, please ping me if you have more patches to deliver as a part of Apache ORC 2.0.2. https://

[jira] [Commented] (ORC-1751) [C++] Syntax error in ThirdpartyToolchain

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869935#comment-17869935 ] Dongjoon Hyun commented on ORC-1751: Thank you for reporting, [~luffyZ]. I also

[jira] [Commented] (SPARK-48950) Corrupt data from parquet scans

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869931#comment-17869931 ] Dongjoon Hyun commented on SPARK-48950: --- Thank you, [~Tom_Newton]. Accordin

[jira] [Commented] (SPARK-49000) Aggregation with DISTINCT gives wrong results when dealing with literals

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869930#comment-17869930 ] Dongjoon Hyun commented on SPARK-49000: --- Thank you, [~uros-db]. > Aggr

[jira] [Commented] (SPARK-49000) Aggregation with DISTINCT gives wrong results when dealing with literals

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869925#comment-17869925 ] Dongjoon Hyun commented on SPARK-49000: --- When I use `spark-sq`, Apache S

[jira] [Updated] (SPARK-49000) Aggregation with DISTINCT gives wrong results when dealing with literals

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49000: -- Affects Version/s: 3.4.3 3.3.4 3.5.1

[jira] [Updated] (SPARK-49000) Aggregation with DISTINCT gives wrong results when dealing with literals

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49000: -- Labels: correctness pull-request-available (was: pull-request-available) > Aggregation w

[jira] [Resolved] (SPARK-49064) Upgrade Kafka to 3.8.0

2024-07-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49064. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47540 [https

[jira] [Comment Edited] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869533#comment-17869533 ] Dongjoon Hyun edited comment on SPARK-49055 at 7/30/24 5:4

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869533#comment-17869533 ] Dongjoon Hyun commented on SPARK-49055: --- Please see the discuss and conclusio

[jira] [Closed] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-49055. - > Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS > envir

[jira] [Resolved] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49055. --- Resolution: Not A Problem After investigating with [~LuciferYang], we concluded that there

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869525#comment-17869525 ] Dongjoon Hyun commented on SPARK-49055: --- There is no functional change in Ha

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869523#comment-17869523 ] Dongjoon Hyun commented on SPARK-49055: --- To [~LuciferYang], this is a false a

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869522#comment-17869522 ] Dongjoon Hyun commented on SPARK-49055: --- This seems to be a new feature of Ha

[jira] [Updated] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Environment: (was: MacOS on AppleSilicon) > Investigate OrcEncryptionSu

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869521#comment-17869521 ] Dongjoon Hyun commented on SPARK-49055: --- Here is the Hadoop code. - h

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869519#comment-17869519 ] Dongjoon Hyun commented on SPARK-49055: --- Ack. BTW, this is not a failure, i

[jira] [Commented] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17869520#comment-17869520 ] Dongjoon Hyun commented on SPARK-49055: --- Does it mean any functional failur

[jira] [Updated] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Summary: Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Environment: MacOS on AppleSilicon (was: MacOS) > Fix OrcEncryptionSuite failure on

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on AppleSilicon MacOS environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Summary: Fix OrcEncryptionSuite failure on AppleSilicon MacOS environment (was: Fix

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Environment: MacOS > Fix OrcEncryptionSuite failure on Mac environm

[jira] [Updated] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49055: -- Reporter: Yang Jie (was: Dongjoon Hyun) > Fix OrcEncryptionSuite failure on Mac environm

[jira] [Created] (SPARK-49055) Fix OrcEncryptionSuite failure on Mac environment

2024-07-29 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-49055: - Summary: Fix OrcEncryptionSuite failure on Mac environment Key: SPARK-49055 URL: https://issues.apache.org/jira/browse/SPARK-49055 Project: Spark Issue

[jira] [Updated] (SPARK-46058) Add separate flag for privateKeyPassword

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46058: -- Summary: Add separate flag for privateKeyPassword (was: [CORE] Add separate flag for

[jira] [Updated] (SPARK-45730) Make ReloadingX509TrustManagerSuite less flaky

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45730: -- Summary: Make ReloadingX509TrustManagerSuite less flaky (was: [CORE] Make

[jira] [Updated] (SPARK-45544) Integrate SSL support into TransportContext

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45544: -- Summary: Integrate SSL support into TransportContext (was: [CORE] Integrate SSL support into

[jira] [Updated] (SPARK-46132) Support key password for JKS keys for RPC SSL

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46132: -- Summary: Support key password for JKS keys for RPC SSL (was: [CORE] Support key password for

[jira] [Updated] (SPARK-45541) Add SSLFactory

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45541: -- Summary: Add SSLFactory (was: [CORE] Add SSLFactory) > Add SSLFact

[jira] [Updated] (SPARK-45431) Document SSL RPC feature

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45431: -- Summary: Document SSL RPC feature (was: [DOCS] Document SSL RPC feature) > Document SSL

[jira] [Updated] (SPARK-45429) Add helper classes for RPC SSL communication

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45429: -- Summary: Add helper classes for RPC SSL communication (was: [CORE] Add helper classes for

[jira] [Updated] (SPARK-45464) Fix yarn distribution build

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45464: -- Summary: Fix yarn distribution build (was: [CORE] Fix yarn distribution build) > Fix y

[jira] [Updated] (SPARK-45377) Handle InputStream in NettyLogger

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45377: -- Summary: Handle InputStream in NettyLogger (was: [CORE] Handle InputStream in NettyLogger

[jira] [Updated] (SPARK-45408) Add RPC SSL settings to TransportConf

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45408: -- Summary: Add RPC SSL settings to TransportConf (was: [CORE] Add RPC SSL settings to

[jira] [Updated] (SPARK-45378) Add convertToNettyForSsl to ManagedBuffer

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45378: -- Summary: Add convertToNettyForSsl to ManagedBuffer (was: [CORE] Add convertToNettyForSsl to

[jira] [Updated] (SPARK-45426) Add support for a ReloadingTrustManager

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45426: -- Summary: Add support for a ReloadingTrustManager (was: [CORE] Add support for a

[jira] [Updated] (SPARK-45427) Add RPC SSL settings to SSLOptions and SparkTransportConf

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45427: -- Summary: Add RPC SSL settings to SSLOptions and SparkTransportConf (was: [CORE] Add RPC SSL

[jira] [Updated] (SPARK-45375) Mark connection as timedOut in TransportClient.close

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45375: -- Summary: Mark connection as timedOut in TransportClient.close (was: [CORE] Mark connection

[jira] [Updated] (SPARK-45374) Add test keys for SSL functionality

2024-07-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45374: -- Summary: Add test keys for SSL functionality (was: [CORE] Add test keys for SSL

<    3   4   5   6   7   8   9   10   11   12   >