Slightly biased, but per my conversations - this would be awesome to have! On Mon, Jun 19, 2023 at 09:43 Abdeali Kothari <abdealikoth...@gmail.com> wrote:
> I would definitely use it - is it's available :) > > On Mon, 19 Jun 2023, 21:56 Jacek Laskowski, <ja...@japila.pl> wrote: > >> Hi Allison and devs, >> >> Although I was against this idea at first sight (probably because I'm a >> Scala dev), I think it could work as long as there are people who'd be >> interested in such an API. Were there any? I'm just curious. I've seen no >> emails requesting it. >> >> I also doubt that Python devs would like to work on new data sources but >> support their wishes wholeheartedly :) >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> "The Internals Of" Online Books <https://books.japila.pl/> >> Follow me on https://twitter.com/jaceklaskowski >> >> <https://twitter.com/jaceklaskowski> >> >> >> On Fri, Jun 16, 2023 at 6:14 AM Allison Wang >> <allison.w...@databricks.com.invalid> wrote: >> >>> Hi everyone, >>> >>> I would like to start a discussion on “Python Data Source API”. >>> >>> This proposal aims to introduce a simple API in Python for Data Sources. >>> The idea is to enable Python developers to create data sources without >>> having to learn Scala or deal with the complexities of the current data >>> source APIs. The goal is to make a Python-based API that is simple and easy >>> to use, thus making Spark more accessible to the wider Python developer >>> community. This proposed approach is based on the recently introduced >>> Python user-defined table functions with extensions to support data sources. >>> >>> *SPIP Doc*: >>> https://docs.google.com/document/d/1oYrCKEKHzznljYfJO4kx5K_Npcgt1Slyfph3NEk7JRU/edit?usp=sharing >>> >>> *SPIP JIRA*: https://issues.apache.org/jira/browse/SPARK-44076 >>> >>> Looking forward to your feedback. >>> >>> Thanks, >>> Allison >>> >>
