+1 (non-binding) I like the idea, especially for UDF to understand the FILE semantic.
An extended idea is if it makes sense to even support a folder/directory as a collection of FILEs. I see many use cases have a dataset (e.g., images) in a folder, and if spark can understand that's a collection of FILEs it would be great to handle their life cycles. Best, Yicong On 2026/09/08 22:03:52 Gengliang Wang wrote: > +1 > > On Tue, Sep 8, 2026 at 2:59 PM Hyukjin Kwon <[email protected]> wrote: > > > +1 > > > > On 2026/09/08 20:34:38 Xiao Li wrote: > > > +1 from me. This fills a real gap in Spark. Modeling files as either > > binary > > > blobs or opaque paths is awkward, especially as unstructured data > > workloads > > > grow. > > > > > > A first-class FILE type, aligned with Parquet and with lazy content > > > loading, feels like the right abstraction. I support moving this forward. > > > > > > Xiao > > > > > > Burak Yavuz <[email protected]> 于2026年9月8日周二 12:49写道： > > > > > > > I'll kick off the vote with a +1 (non-binding) > > > > > > > > Thanks, > > > > Burak > > > > > > > > On Tue, Sep 8, 2026 at 3:47 PM Burak Yavuz <[email protected]> wrote: > > > > > > > >> Hi Spark devs, > > > >> > > > >> I would like to start a vote on introducing FileType for handling > > > >> unstructured data. > > > >> > > > >> The SPIP document: > > > >> > > > >> > > https://docs.google.com/document/d/1pPof896ZwcZ-2Yn-YhC4TbyYWn1umiQJykGDkAxzCWc/edit?tab=t.0#heading=h.m1700lw4wsoj > > > >> > > > >> Discussion thread: > > > >> https://lists.apache.org/thread/6f83qcpj1ox40jfxtottqlhjbhhxwmhf > > > >> > > > >> JIRA Ticket: > > > >> https://issues.apache.org/jira/browse/SPARK-59132 > > > >> > > > >> The vote will be open for at least 72 hours, and passes if a majority > > +1 > > > >> PMC > > > >> votes are cast, with a minimum of 3 +1 votes. > > > >> Please vote: > > > >> [ ] +1: Accept the proposal as an official SPIP > > > >> [ ] +0 > > > >> [ ] -1: I don't think this is a good idea because ... > > > >> > > > >> Best regards, > > > >> Burak Yavuz > > > >> > > > >> > > > >> > > > > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > > > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
