[jira] [Created] (SPARK-43647) Maven test failed in ClientE2ETestSuite

2023-05-22 Thread Yang Jie (Jira)
Yang Jie created SPARK-43647: Summary: Maven test failed in ClientE2ETestSuite Key: SPARK-43647 URL: https://issues.apache.org/jira/browse/SPARK-43647 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-05-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724795#comment-17724795 ] Yang Jie commented on SPARK-43646: -- friendly ping [~rangadi] , do you have any suggestions for these

[jira] [Created] (SPARK-43646) Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options`

2023-05-22 Thread Yang Jie (Jira)
Yang Jie created SPARK-43646: Summary: Maven test failed: `from_protobuf_messageClassName` and `from_protobuf_messageClassName_options` Key: SPARK-43646 URL: https://issues.apache.org/jira/browse/SPARK-43646

[jira] [Created] (SPARK-43609) Upgrade Netty from 4.1.89 to 4.1.92

2023-05-21 Thread Yang Jie (Jira)
Yang Jie created SPARK-43609: Summary: Upgrade Netty from 4.1.89 to 4.1.92 Key: SPARK-43609 URL: https://issues.apache.org/jira/browse/SPARK-43609 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43590) Make `CheckConnectJvmClientCompatibility` to compare client and protobuf

2023-05-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-43590: Summary: Make `CheckConnectJvmClientCompatibility` to compare client and protobuf Key: SPARK-43590 URL: https://issues.apache.org/jira/browse/SPARK-43590 Project: Spark

[jira] [Created] (SPARK-43588) Upgrade ASM to 9.5

2023-05-18 Thread Yang Jie (Jira)
Yang Jie created SPARK-43588: Summary: Upgrade ASM to 9.5 Key: SPARK-43588 URL: https://issues.apache.org/jira/browse/SPARK-43588 Project: Spark Issue Type: Improvement Components:

[jira] [Updated] (SPARK-43588) Upgrade ASM to 9.5

2023-05-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43588: - Description: ASM 9.5 is for Java 21   https://asm.ow2.io/versions.html was:ASM 9.5 is for Java 21

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

2023-05-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Priority: Minor (was: Major) > There will be many invalid tasks when `Range.numSlices` >

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

2023-05-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Description: For example, start a spark shell with `--master "local[100]"`, then run

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

2023-05-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Attachment: image-2023-05-19-13-01-19-589.png > There will be many invalid tasks when `Range.numSlices`

[jira] [Updated] (SPARK-43586) There will be many invalid tasks when `Range.numSlices` > `Range.numElements`

2023-05-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43586: - Summary: There will be many invalid tasks when `Range.numSlices` > `Range.numElements` (was: there

[jira] [Created] (SPARK-43586) there will be many invalid tasks when `Range.numSlices` > `Range.numElements`

2023-05-18 Thread Yang Jie (Jira)
Yang Jie created SPARK-43586: Summary: there will be many invalid tasks when `Range.numSlices` > `Range.numElements` Key: SPARK-43586 URL: https://issues.apache.org/jira/browse/SPARK-43586 Project: Spark

[jira] [Assigned] (SPARK-43022) protobuf functions

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43022: Assignee: Yang Jie > protobuf functions > -- > > Key:

[jira] [Resolved] (SPARK-43022) protobuf functions

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43022. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 40654

[jira] [Resolved] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43535. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41184

[jira] [Assigned] (SPARK-43535) Adjust the ImportOrderChecker rule to resolve long-standing import issues

2023-05-17 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43535: Assignee: BingKun Pan > Adjust the ImportOrderChecker rule to resolve long-standing import

[jira] [Created] (SPARK-43537) Upgrade the asm deps in the tools module to 9.4

2023-05-17 Thread Yang Jie (Jira)
Yang Jie created SPARK-43537: Summary: Upgrade the asm deps in the tools module to 9.4 Key: SPARK-43537 URL: https://issues.apache.org/jira/browse/SPARK-43537 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-43461) Skip compiling useless files when making distribution

2023-05-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43461. -- Resolution: Fixed Issue resolved by pull request 41141 https://github.com/apache/spark/pull/41141 >

[jira] [Assigned] (SPARK-43461) Skip compiling useless files when making distribution

2023-05-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43461: Assignee: Yuming Wang > Skip compiling useless files when making distribution >

[jira] [Updated] (SPARK-43495) Upgrade RoaringBitmap to 0.9.44

2023-05-14 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43495: - Description: * [https://github.com/RoaringBitmap/RoaringBitmap/releases/tag/0.9.40] *

[jira] [Created] (SPARK-43495) Upgrade RoaringBitmap to 0.9.44

2023-05-14 Thread Yang Jie (Jira)
Yang Jie created SPARK-43495: Summary: Upgrade RoaringBitmap to 0.9.44 Key: SPARK-43495 URL: https://issues.apache.org/jira/browse/SPARK-43495 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-43494) Directly call `replicate()` instead of reflection in `SparkHadoopUtil#createFile`

2023-05-14 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43494: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Directly call `replicate()`

[jira] [Created] (SPARK-43494) Directly call `replicate()` instead of reflection in `SparkHadoopUtil#createFile`

2023-05-14 Thread Yang Jie (Jira)
Yang Jie created SPARK-43494: Summary: Directly call `replicate()` instead of reflection in `SparkHadoopUtil#createFile` Key: SPARK-43494 URL: https://issues.apache.org/jira/browse/SPARK-43494 Project:

[jira] [Commented] (SPARK-43265) Move Error framework to a common utils module

2023-05-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17721708#comment-17721708 ] Yang Jie commented on SPARK-43265: -- What are the future plans for the common-utils module? [~amaliujia] 

[jira] [Resolved] (SPARK-43437) Upgrade Arrow to 12.0.0

2023-05-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43437. -- Resolution: Duplicate > Upgrade Arrow to 12.0.0 > --- > > Key:

[jira] [Created] (SPARK-43437) Upgrade Arrow to 12.0.0

2023-05-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-43437: Summary: Upgrade Arrow to 12.0.0 Key: SPARK-43437 URL: https://issues.apache.org/jira/browse/SPARK-43437 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43436) Upgrade rocksdbjni to 8.1.1.1

2023-05-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-43436: Summary: Upgrade rocksdbjni to 8.1.1.1 Key: SPARK-43436 URL: https://issues.apache.org/jira/browse/SPARK-43436 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-42579) Extend function.lit() to match Literal.apply()

2023-05-09 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-42579. -- Resolution: Fixed > Extend function.lit() to match Literal.apply() >

[jira] [Updated] (SPARK-42514) Scala Client add partition transforms functions

2023-05-08 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42514: - Epic Link: SPARK-42554 > Scala Client add partition transforms functions >

[jira] [Created] (SPARK-43294) Upgrade zstd-jni to 1.5.5-2

2023-04-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-43294: Summary: Upgrade zstd-jni to 1.5.5-2 Key: SPARK-43294 URL: https://issues.apache.org/jira/browse/SPARK-43294 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-43292) ArtifactManagerSuite can't run using maven

2023-04-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43292: - Description: run {code:java} build/mvn clean install -DskipTests -Phive  build/mvn test -pl

[jira] [Updated] (SPARK-43292) ArtifactManagerSuite can't run using maven

2023-04-26 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43292: - Description: run {code:java} build/mvn clean install -DskipTests -Phive  build/mvn test -pl

[jira] [Created] (SPARK-43292) ArtifactManagerSuite can't run using maven

2023-04-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-43292: Summary: ArtifactManagerSuite can't run using maven Key: SPARK-43292 URL: https://issues.apache.org/jira/browse/SPARK-43292 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-43279) Cleanup unused members from `SparkHadoopUtil`

2023-04-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-43279: Summary: Cleanup unused members from `SparkHadoopUtil` Key: SPARK-43279 URL: https://issues.apache.org/jira/browse/SPARK-43279 Project: Spark Issue Type:

[jira] [Created] (SPARK-43277) Clean up deprecation hadoop api usage in Yarn module

2023-04-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-43277: Summary: Clean up deprecation hadoop api usage in Yarn module Key: SPARK-43277 URL: https://issues.apache.org/jira/browse/SPARK-43277 Project: Spark Issue Type:

[jira] [Created] (SPARK-43272) Replace reflection w/ direct calling for `SparkHadoopUtil#createFile`

2023-04-25 Thread Yang Jie (Jira)
Yang Jie created SPARK-43272: Summary: Replace reflection w/ direct calling for `SparkHadoopUtil#createFile` Key: SPARK-43272 URL: https://issues.apache.org/jira/browse/SPARK-43272 Project: Spark

[jira] [Comment Edited] (SPARK-43246) Make `CheckConnectJvmClientCompatibility` filter the check on `private[packageName]` scope member as default

2023-04-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715592#comment-17715592 ] Yang Jie edited comment on SPARK-43246 at 4/24/23 6:12 AM: --- As

[jira] [Commented] (SPARK-43246) Make `CheckConnectJvmClientCompatibility` filter the check on `private[packageName]` scope member as default

2023-04-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715592#comment-17715592 ] Yang Jie commented on SPARK-43246: -- As [https://github.com/apache/spark/pull/40898/files#r1174782564]  

[jira] [Updated] (SPARK-43246) Make `CheckConnectJvmClientCompatibility` filter the check on `private[packageName]` scope member as default

2023-04-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43246: - Epic Link: SPARK-42554 > Make `CheckConnectJvmClientCompatibility` filter the check on >

[jira] [Created] (SPARK-43246) Make `CheckConnectJvmClientCompatibility` filter the check on `private[packageName]` scope member as default

2023-04-24 Thread Yang Jie (Jira)
Yang Jie created SPARK-43246: Summary: Make `CheckConnectJvmClientCompatibility` filter the check on `private[packageName]` scope member as default Key: SPARK-43246 URL:

[jira] [Updated] (SPARK-43222) Remove check of `isHadoop3`

2023-04-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43222: - Component/s: (was: YARN) > Remove check of `isHadoop3` > --- > >

[jira] [Updated] (SPARK-43222) Remove check of `isHadoop3`

2023-04-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43222: - Summary: Remove check of `isHadoop3` (was: Remove check of `VersionUtils.isHadoop3`) > Remove check

[jira] [Created] (SPARK-43222) Remove check of `VersionUtils.isHadoop3`

2023-04-20 Thread Yang Jie (Jira)
Yang Jie created SPARK-43222: Summary: Remove check of `VersionUtils.isHadoop3` Key: SPARK-43222 URL: https://issues.apache.org/jira/browse/SPARK-43222 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-43215) Remove `ResourceRequestHelper#isYarnResourceTypesAvailable`

2023-04-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43215. -- Resolution: Duplicate > Remove `ResourceRequestHelper#isYarnResourceTypesAvailable` >

[jira] [Resolved] (SPARK-43216) Refactor `ResourceRequestHelper ` to no longer use reflection

2023-04-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43216. -- Resolution: Duplicate > Refactor `ResourceRequestHelper ` to no longer use reflection >

[jira] [Created] (SPARK-43216) Refactor `ResourceRequestHelper ` to no longer use reflection

2023-04-20 Thread Yang Jie (Jira)
Yang Jie created SPARK-43216: Summary: Refactor `ResourceRequestHelper ` to no longer use reflection Key: SPARK-43216 URL: https://issues.apache.org/jira/browse/SPARK-43216 Project: Spark Issue

[jira] [Created] (SPARK-43215) Remove `ResourceRequestHelper#isYarnResourceTypesAvailable`

2023-04-20 Thread Yang Jie (Jira)
Yang Jie created SPARK-43215: Summary: Remove `ResourceRequestHelper#isYarnResourceTypesAvailable` Key: SPARK-43215 URL: https://issues.apache.org/jira/browse/SPARK-43215 Project: Spark Issue

[jira] [Updated] (SPARK-43184) Resume using enumeration to compare `NodeState.DECOMMISSIONING`

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43184: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Resume using enumeration to

[jira] [Updated] (SPARK-43185) Inline `hadoop-client` related properties in `pom.xml`

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43185: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Inline `hadoop-client` related

[jira] [Updated] (SPARK-43186) Remove workaround for FileSinkDesc

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43186: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Remove workaround for

[jira] [Updated] (SPARK-43187) Remove workaround for MiniKdc's BindException

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43187: - Parent: SPARK-43197 Issue Type: Sub-task (was: Test) > Remove workaround for MiniKdc's

[jira] [Updated] (SPARK-43191) Replace reflection w/ direct calling for Hadoop CallerContext

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43191: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Replace reflection w/ direct

[jira] [Updated] (SPARK-43193) Remove workaround for HADOOP-12074

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43193: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Remove workaround for

[jira] [Updated] (SPARK-43195) Remove unnecessary serializable wrapper in HadoopFSUtils

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43195: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Remove unnecessary serializable

[jira] [Updated] (SPARK-43196) Replace reflection w/ direct calling for `ContainerLaunchContext#setTokensConf`

2023-04-19 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43196: - Parent: SPARK-43197 Issue Type: Sub-task (was: Improvement) > Replace reflection w/ direct

[jira] [Created] (SPARK-43197) Clean up the code written for compatibility with Hadoop 2

2023-04-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-43197: Summary: Clean up the code written for compatibility with Hadoop 2 Key: SPARK-43197 URL: https://issues.apache.org/jira/browse/SPARK-43197 Project: Spark Issue

[jira] [Created] (SPARK-43196) Replace reflection w/ direct calling for `ContainerLaunchContext#setTokensConf`

2023-04-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-43196: Summary: Replace reflection w/ direct calling for `ContainerLaunchContext#setTokensConf` Key: SPARK-43196 URL: https://issues.apache.org/jira/browse/SPARK-43196 Project:

[jira] [Created] (SPARK-43185) Inline `hadoop-client` related properties in `pom.xml`

2023-04-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-43185: Summary: Inline `hadoop-client` related properties in `pom.xml` Key: SPARK-43185 URL: https://issues.apache.org/jira/browse/SPARK-43185 Project: Spark Issue Type:

[jira] [Created] (SPARK-43184) Resume using enumeration to compare `NodeState.DECOMMISSIONING`

2023-04-19 Thread Yang Jie (Jira)
Yang Jie created SPARK-43184: Summary: Resume using enumeration to compare `NodeState.DECOMMISSIONING` Key: SPARK-43184 URL: https://issues.apache.org/jira/browse/SPARK-43184 Project: Spark

[jira] [Updated] (SPARK-43173) `write jdbc` in `ClientE2ETestSuite` will test fail with out `-Phive`

2023-04-18 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43173: - Description: both ``` build/mvn clean install -Dtest=none

[jira] [Created] (SPARK-43173) `write jdbc` in `ClientE2ETestSuite` will test fail with out `-Phive`

2023-04-18 Thread Yang Jie (Jira)
Yang Jie created SPARK-43173: Summary: `write jdbc` in `ClientE2ETestSuite` will test fail with out `-Phive` Key: SPARK-43173 URL: https://issues.apache.org/jira/browse/SPARK-43173 Project: Spark

[jira] [Created] (SPARK-43169) Update mima's previousSparkVersion to 3.4.0

2023-04-17 Thread Yang Jie (Jira)
Yang Jie created SPARK-43169: Summary: Update mima's previousSparkVersion to 3.4.0 Key: SPARK-43169 URL: https://issues.apache.org/jira/browse/SPARK-43169 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43121) Use `BytesWritable.copyBytes` instead of manual copy in `HiveInspectors`

2023-04-13 Thread Yang Jie (Jira)
Yang Jie created SPARK-43121: Summary: Use `BytesWritable.copyBytes` instead of manual copy in `HiveInspectors` Key: SPARK-43121 URL: https://issues.apache.org/jira/browse/SPARK-43121 Project: Spark

[jira] [Created] (SPARK-43104) Set `shadeTestJar` of protobuf module to false

2023-04-12 Thread Yang Jie (Jira)
Yang Jie created SPARK-43104: Summary: Set `shadeTestJar` of protobuf module to false Key: SPARK-43104 URL: https://issues.apache.org/jira/browse/SPARK-43104 Project: Spark Issue Type:

[jira] [Created] (SPARK-43102) Upgrade commons-compress to 1.23.0

2023-04-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-43102: Summary: Upgrade commons-compress to 1.23.0 Key: SPARK-43102 URL: https://issues.apache.org/jira/browse/SPARK-43102 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-42380) Upgrade maven to 3.9.1

2023-04-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42380: - Summary: Upgrade maven to 3.9.1 (was: Upgrade maven to 3.9.0) > Upgrade maven to 3.9.1 >

[jira] [Created] (SPARK-43093) Test case "Add a directory when spark.sql.legacy.addSingleFileInAddFile set to false" should use random directories for testing

2023-04-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-43093: Summary: Test case "Add a directory when spark.sql.legacy.addSingleFileInAddFile set to false" should use random directories for testing Key: SPARK-43093 URL:

[jira] [Created] (SPARK-43092) Cleanup unsuppoerted function `dropDuplicatesWithinWatermark` from `Dataset`

2023-04-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-43092: Summary: Cleanup unsuppoerted function `dropDuplicatesWithinWatermark` from `Dataset` Key: SPARK-43092 URL: https://issues.apache.org/jira/browse/SPARK-43092 Project: Spark

[jira] [Created] (SPARK-43090) Move withTable from RemoteSparkSession to SQLHelper

2023-04-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-43090: Summary: Move withTable from RemoteSparkSession to SQLHelper Key: SPARK-43090 URL: https://issues.apache.org/jira/browse/SPARK-43090 Project: Spark Issue Type:

[jira] [Created] (SPARK-43080) Upgrade zstd-jni to 1.5.5-1

2023-04-10 Thread Yang Jie (Jira)
Yang Jie created SPARK-43080: Summary: Upgrade zstd-jni to 1.5.5-1 Key: SPARK-43080 URL: https://issues.apache.org/jira/browse/SPARK-43080 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-43023) Add switch catalog testing scenario for CatalogSuite

2023-04-03 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-43023: - Summary: Add switch catalog testing scenario for CatalogSuite (was: Enhanced CatalogMetadata API test)

[jira] [Updated] (SPARK-42519) Add more WriteTo tests after Scala Client session config is supported

2023-04-03 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42519: - Epic Link: SPARK-42554 > Add more WriteTo tests after Scala Client session config is supported >

[jira] [Updated] (SPARK-42519) Add more WriteTo tests after Scala Client session config is supported

2023-04-03 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42519: - Labels: SPARK-42554 (was: ) > Add more WriteTo tests after Scala Client session config is supported >

[jira] [Resolved] (SPARK-42810) Add a test catalog like InMemoryTableCatalog for SimpleSparkConnectService

2023-04-03 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-42810. -- Resolution: Duplicate > Add a test catalog like InMemoryTableCatalog for SimpleSparkConnectService >

[jira] [Updated] (SPARK-42519) Add more WriteTo tests after Scala Client session config is supported

2023-04-03 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42519: - Labels: (was: SPARK-42554) > Add more WriteTo tests after Scala Client session config is supported >

[jira] [Created] (SPARK-43023) Enhanced CatalogMetadata API test

2023-04-03 Thread Yang Jie (Jira)
Yang Jie created SPARK-43023: Summary: Enhanced CatalogMetadata API test Key: SPARK-43023 URL: https://issues.apache.org/jira/browse/SPARK-43023 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43022) protobuf functions

2023-04-03 Thread Yang Jie (Jira)
Yang Jie created SPARK-43022: Summary: protobuf functions Key: SPARK-43022 URL: https://issues.apache.org/jira/browse/SPARK-43022 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-43008) Upgrade jodatime to 2.12.5

2023-04-03 Thread Yang Jie (Jira)
Yang Jie created SPARK-43008: Summary: Upgrade jodatime to 2.12.5 Key: SPARK-43008 URL: https://issues.apache.org/jira/browse/SPARK-43008 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-43007) Upgrade rocksdbjni to 8.0.0

2023-04-02 Thread Yang Jie (Jira)
Yang Jie created SPARK-43007: Summary: Upgrade rocksdbjni to 8.0.0 Key: SPARK-43007 URL: https://issues.apache.org/jira/browse/SPARK-43007 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-42974) Restore `Utils#createTempDir` use `ShutdownHookManager.registerShutdownDeleteDir` to cleanup tempDir

2023-03-30 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42974: - Issue Type: Improvement (was: Bug) > Restore `Utils#createTempDir` use >

[jira] [Updated] (SPARK-42974) Restore `Utils#createTempDir` use `ShutdownHookManager.registerShutdownDeleteDir` to cleanup tempDir

2023-03-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42974: - Summary: Restore `Utils#createTempDir` use `ShutdownHookManager.registerShutdownDeleteDir` to cleanup

[jira] [Created] (SPARK-42974) Restoring `Utils#createTempDir` using `ShutdownHookManager.registerShutdownDeleteDir` cleanup temporary files

2023-03-29 Thread Yang Jie (Jira)
Yang Jie created SPARK-42974: Summary: Restoring `Utils#createTempDir` using `ShutdownHookManager.registerShutdownDeleteDir` cleanup temporary files Key: SPARK-42974 URL:

[jira] [Created] (SPARK-42971) When processing the WorkDirCleanup event, if appDirs is empty, should print workdir

2023-03-29 Thread Yang Jie (Jira)
Yang Jie created SPARK-42971: Summary: When processing the WorkDirCleanup event, if appDirs is empty, should print workdir Key: SPARK-42971 URL: https://issues.apache.org/jira/browse/SPARK-42971

[jira] [Comment Edited] (SPARK-41359) Use `PhysicalDataType` instead of DataType in UnsafeRow

2023-03-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706248#comment-17706248 ] Yang Jie edited comment on SPARK-41359 at 3/29/23 7:14 AM: --- [~cloud_fan] this

[jira] [Commented] (SPARK-41359) Use `PhysicalDataType` instead of DataType in UnsafeRow

2023-03-29 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706248#comment-17706248 ] Yang Jie commented on SPARK-41359: -- [~cloud_fan] this ticket is created by me, but not resoloved by me,

[jira] [Created] (SPARK-42958) Refactor `CheckConnectJvmClientCompatibility` to compare client and avro

2023-03-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-42958: Summary: Refactor `CheckConnectJvmClientCompatibility` to compare client and avro Key: SPARK-42958 URL: https://issues.apache.org/jira/browse/SPARK-42958 Project: Spark

[jira] [Created] (SPARK-42956) avro functions

2023-03-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-42956: Summary: avro functions Key: SPARK-42956 URL: https://issues.apache.org/jira/browse/SPARK-42956 Project: Spark Issue Type: Improvement Components: Connect

[jira] [Commented] (SPARK-42382) Upgrade `cyclonedx-maven-plugin` to 2.7.5

2023-03-28 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705862#comment-17705862 ] Yang Jie commented on SPARK-42382: -- OK > Upgrade `cyclonedx-maven-plugin` to 2.7.5 >

[jira] [Created] (SPARK-42934) Testing OrcEncryptionSuite using maven is always skipped

2023-03-27 Thread Yang Jie (Jira)
Yang Jie created SPARK-42934: Summary: Testing OrcEncryptionSuite using maven is always skipped Key: SPARK-42934 URL: https://issues.apache.org/jira/browse/SPARK-42934 Project: Spark Issue Type:

[jira] [Created] (SPARK-42930) Change the access scope of `ProtobufSerDe` related implementations to `private[spark]`

2023-03-27 Thread Yang Jie (Jira)
Yang Jie created SPARK-42930: Summary: Change the access scope of `ProtobufSerDe` related implementations to `private[spark]` Key: SPARK-42930 URL: https://issues.apache.org/jira/browse/SPARK-42930

[jira] [Created] (SPARK-42927) Make `o.a.spark.util.Iterators#size` as `private[spark]`

2023-03-26 Thread Yang Jie (Jira)
Yang Jie created SPARK-42927: Summary: Make `o.a.spark.util.Iterators#size` as `private[spark]` Key: SPARK-42927 URL: https://issues.apache.org/jira/browse/SPARK-42927 Project: Spark Issue Type:

[jira] [Updated] (SPARK-42921) SQLQueryTestSuite test failed with `SPARK_ANSI_SQL_MODE=true`

2023-03-24 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42921: - Issue Type: Bug (was: Improvement) > SQLQueryTestSuite test failed with `SPARK_ANSI_SQL_MODE=true` >

[jira] [Created] (SPARK-42921) SQLQueryTestSuite test failed with `SPARK_ANSI_SQL_MODE=true`

2023-03-24 Thread Yang Jie (Jira)
Yang Jie created SPARK-42921: Summary: SQLQueryTestSuite test failed with `SPARK_ANSI_SQL_MODE=true` Key: SPARK-42921 URL: https://issues.apache.org/jira/browse/SPARK-42921 Project: Spark Issue

[jira] [Created] (SPARK-42913) Upgrade Hadoop to 3.3.5

2023-03-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-42913: Summary: Upgrade Hadoop to 3.3.5 Key: SPARK-42913 URL: https://issues.apache.org/jira/browse/SPARK-42913 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-42901) Should move message StorageLevel from `base.proto` to a separate file

2023-03-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42901: - Summary: Should move message StorageLevel from `base.proto` to a separate file (was: Should move

[jira] [Updated] (SPARK-42901) Should move message StorageLevel from `base.proto` into a separate file

2023-03-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42901: - Summary: Should move message StorageLevel from `base.proto` into a separate file (was: Should move

[jira] [Updated] (SPARK-42901) Should move message StorageLevel from into a separate file

2023-03-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42901: - Summary: Should move message StorageLevel from into a separate file (was: Should move StorageLevel

[jira] [Updated] (SPARK-42901) Should move message StorageLevel from into a separate file

2023-03-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42901: - Description: [https://github.com/apache/spark/pull/40510] introduce `message StorageLevel` to

[jira] [Updated] (SPARK-42901) Should move message StorageLevel from `base.proto` to a separate file

2023-03-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42901: - Summary: Should move message StorageLevel from `base.proto` to a separate file (was: Should move

[jira] [Updated] (SPARK-42901) Should move StorageLevel into a separate file

2023-03-22 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-42901: - Summary: Should move StorageLevel into a separate file (was: Move StorageLevel into a separate file to

<    6   7   8   9   10   11   12   13   14   15   >