Re: [DISCUSS] Support spark.ml on Spark Connect

Awesome progress! On Tue, Jan 21, 2025 at 8:15 AM Bobby wrote: > Hi all, > > Thank you for reviewing and merging PR #48791 > , which enables running > spark.ml on Connect starting from Spark 4.0. With this PR, it becomes > extremely easy to support thi

Re: [DISCUSS] Support spark.ml on Spark Connect

Hi all, Thank you for reviewing and merging PR #48791 , which enables running spark.ml on Connect starting from Spark 4.0. With this PR, it becomes extremely easy to support third-party Spark ML libraries on Connect. I also want to highlight PR #49503 <

How do I repackage org.spark-project.hive-exec-1.2.1.spark2

I need to modify some codes in hive-exec, how do I re build org.spark-project.hive-exec-1.2.1.spark2.jar?

Re: [FYI] Known `Spark Connect` Test Suite Flakiness

Thank you, Paddy. Dongjoon. On Mon, Jan 20, 2025 at 2:32 AM Paddy Xu wrote: > I have worked on tests related to “interrupt”. Not sure about SPARK-50888: > > My findings: > 1. These test failures only occur in the GitHub CI. > 2. The failure is due to the thread pool we created in CI having on

RE: [FYI] Known `Spark Connect` Test Suite Flakiness

I have worked on tests related to “interrupt”. Not sure about SPARK-50888: My findings: 1. These test failures only occur in the GitHub CI. 2. The failure is due to the thread pool we created in CI having only two threads, while our tests require three concurrent threads to run. To workaround th