Given our recent discussion on using spark connect as a stable API, this will be another positive step.
HTH Mich Talebzadeh, Architect | Data Science | Financial Crime | Forensic Analysis | GDPR view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> On Tue, 21 Jan 2025 at 07:44, Martin Grund <mar...@databricks.com> wrote: > Awesome progress! > > On Tue, Jan 21, 2025 at 8:15 AM Bobby <wbo4...@gmail.com> wrote: > >> Hi all, >> >> Thank you for reviewing and merging PR #48791 >> <https://github.com/apache/spark/pull/48791>, which enables running >> spark.ml on Connect starting from Spark 4.0. With this PR, it becomes >> extremely easy to support third-party Spark ML libraries on Connect. >> >> I also want to highlight PR #49503 >> <https://github.com/apache/spark/pull/49503>, which adds support for the >> Connect ML backend. This PR allows users to leverage GPU-accelerated >> spark-ml libraries to speed up the built-in spark.ml algorithms without >> requiring any changes to user code. >> >> You can find the full story at SPARK-50812 >> <https://issues.apache.org/jira/browse/SPARK-50812>. >> >> Thank you again for your feedback and for merging these PRs! >> >> Best Regards, >> Bobby Wang >> >> Mich Talebzadeh <mich.talebza...@gmail.com> 于2024年11月13日周三 21:17写道： >> >>> OK I added a comment to PR >>> >>> HTH, >>> >>> Mich Talebzadeh, >>> >>> Architect | Data Engineer | Data Science | Financial Crime >>> PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial >>> College London <https://en.wikipedia.org/wiki/Imperial_College_London> >>> London, United Kingdom >>> >>> >>> view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>> >>> >>> *Disclaimer:* The information provided is correct to the best of my >>> knowledge but of course cannot be guaranteed . It is essential to note >>> that, as with any advice, quote "one test result is worth one-thousand >>> expert opinions (Werner >>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>> >>> >>> On Wed, 13 Nov 2024 at 09:17, Bobby <wbo4...@gmail.com> wrote: >>> >>>> Hi all, >>>> >>>> Finally, I got the draft PR up >>>> https://github.com/apache/spark/pull/48791 last week, but I would like >>>> to send it out here and hope to get your feedback and reviews. >>>> >>>> Thx very much. >>>> >>>> Mich Talebzadeh <mich.talebza...@gmail.com> 于2024年10月15日周二 15:47写道： >>>> >>>>> +1 >>>>> >>>>> It will be a desirable feature >>>>> >>>>> Mich Talebzadeh, >>>>> >>>>> Architect | Data Engineer | Data Science | Financial Crime >>>>> PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial >>>>> College London <https://en.wikipedia.org/wiki/Imperial_College_London> >>>>> >>>>> London, United Kingdom >>>>> >>>>> >>>>> view my Linkedin profile >>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>> >>>>> >>>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>>> >>>>> >>>>> >>>>> *Disclaimer:* The information provided is correct to the best of my >>>>> knowledge but of course cannot be guaranteed . It is essential to note >>>>> that, as with any advice, quote "one test result is worth one-thousand >>>>> expert opinions (Werner >>>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>>>> >>>>> >>>>> On Tue, 15 Oct 2024 at 08:24, Bobby <wbo4...@gmail.com> wrote: >>>>> >>>>>> Thank you for your kind response. I will prepare a formal PR for >>>>>> Spark. >>>>>> >>>>>> >>>>>> >>>>>> Niranjan Jayakar <n...@databricks.com> 于2024年10月11日周五 22:45写道： >>>>>> >>>>>>> +1 >>>>>>> >>>>>>> On Thu, Oct 10, 2024 at 5:28 PM Xiao Li <gatorsm...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Thank you for working on this! >>>>>>>> >>>>>>>> Xiao >>>>>>>> >>>>>>>> Martin Grund <mar...@databricks.com.invalid> 于2024年10月10日周四 >>>>>>>> 03:01写道： >>>>>>>> >>>>>>>>> >>>>>>>>> Hi Bobby, >>>>>>>>> >>>>>>>>> Awesome to see the proposal! I'm very much looking forward to the >>>>>>>>> contributions! >>>>>>>>> >>>>>>>>> Martin >>>>>>>>> >>>>>>>>> On Thu, Oct 10, 2024 at 4:16 AM Ángel < >>>>>>>>> angel.alvarez.pas...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> You have my vote (btw, great idea, ML is so sexy nowadays 😉) >>>>>>>>>> >>>>>>>>>> El jue, 10 oct 2024 a las 3:19, Bobby (<wbo4...@gmail.com>) >>>>>>>>>> escribió: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I'd like to start a discussion about support spark.ml on >>>>>>>>>>> Connect. With this feature, Users don't need to change their code >>>>>>>>>>> to run >>>>>>>>>>> Spark ML cases on Connect. >>>>>>>>>>> >>>>>>>>>>> Please refer to the JIRA: >>>>>>>>>>> https://issues.apache.org/jira/browse/SPARK-49907 >>>>>>>>>>> >>>>>>>>>>> Thx, >>>>>>>>>>> Bobby Wang >>>>>>>>>>> >>>>>>>>>>
