Hello Lorenzo! Thank you for your input. I am happy to move forward as suggested.
Cheng, feel free to pull the issues I’ve created under an umbrella / roadmap. Look forward to it! As for Lance QuickStart doc [1] that I am working on, I will update it to indicate that union read and batch queries are on the roadmap. And that vector search only works with pylance, also pointing to roadmap in case if we decide to support vector search/indexing through Flink SQL/fluss API. The QuickStart doc can be updated once those features are complete. Best regards Keith Lee Ref [1] https://github.com/apache/fluss/pull/2716 On Mon, 2 Mar 2026 at 10:48, Lorenzo Affetti via dev <[email protected]> wrote: > Hello! Thanks for wrapping this up! > > I do understand both Cheng and Keith. > For sure Lance support should be on par with other lake formats. If > something is not supported, there should be a concrete reason why (apart > from a lack of resources :) ). > > Still, input from the Lance community would be essential for > understanding evolution areas of the support itself. > > For this item, I would take an approach similar to what Mehul did for > Iceberg support. > I think there is a lack of a roadmap for Lance support in 2026. > > Having a roadmap doesn't actually mean we will accomplish everything, but, > it signals that we understand the problem space and have an idea of the > sequence of actions to take. > > @cheng, I think you are the de-facto owner of the Lance module. > Would it make sense to dedicate some of our resources to discuss this via > Slack and start drafting a roadmap? > > On Sun, Mar 1, 2026 at 2:11 PM Keith Lee <[email protected]> > wrote: > > > Hello Cheng, > > > > Good call. I agree that gathering input from Lance community will be > > beneficial to inform integration of features such as vector search, > vector > > indexing and hybrid search. > > > > However, the issues I’ve outlined only meant to cover the scope of > bringing > > current fluss lance integration up to parity to other lakehouses like > > paimon or iceberg e.g. batch or union read without lance feature such as > > vector search. As such, I believe these can be decoupled and we can have > a > > separate effort, gathering input from lance community and FIP proposal > for > > integrating vector search into feature such as union read. > > > > Let me know what your thoughts are on this. Thank you! > > > > Best regards > > Keith Lee > > > > > > On Sun, 1 Mar 2026 at 10:20, Cheng Wang <[email protected]> wrote: > > > > > Hello Keith, > > > > > > > > > Regarding our plan to implement union read for Lance using Flink, might > > it > > > be beneficial to first gather input from the Lance community? > > Understanding > > > the primary scenarios where union read would help in the machine > learning > > > scenario, along with the most popular execution engine in Lance > > ecosystem, > > > could ensure we're building the right integration to maximize its > > adoption. > > > > > > > > > > > > > > > Regards, > > > Cheng Wang > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------ Original ------------------ > > > From: > > > "dev" > > > < > > > [email protected]>; > > > Date: Sat, Feb 28, 2026 11:20 PM > > > To: "dev"<[email protected]>; > > > Cc: "Cheng Wang"<[email protected]>;"forwardxu"< > > > [email protected]>; > > > Subject: Re: Analysis of Lance storage format support > > > > > > > > > > > > This is extremely helpful, thanks for putting this together. > > > > > > Maybe we can create an umbrella ticket on GitHub to keep track on these > > and > > > open individual tasks, for tracking. > > > > > > Best, > > > Giannis > > > > > > On Sat, 28 Feb 2026 at 3:52 PM, Keith Lee <[email protected] > > > > > > wrote: > > > > > > > Hello, > > > > > > > > As discussed on community sync yesterday on analysing where we are > > at > > > the > > > > moment in terms of Lance format support. > > > > Here are my findings as part of working on Lance QuickStart > > > documentation > > > > [1]. Lance lake tiering works in general, however there are some > > gaps > > > that > > > > to be addressed to bring Lance format support in parity with > Paimon > > / > > > > Iceberg. > > > > > > > > - (Merged) Support for Arrow FixedSizeList to enable pylance > native > > > vector > > > > search [2] > > > > - (In progress) Support Flink SQL Union Read query against Lance > > > table [3] > > > > - (Open) Support Flink SQL batch query against Lance table [4] > > > > - (Blocked) Primary Key table support - I believe this is still > > > blocking on > > > > Lance format support for delete API [5] > > > > > > > > Finally there is also a gap in the ability of performing vector > > > search on > > > > hot data / via union read. After discussion with Mehul, native > > vector > > > > indexing on hot data in Fluss would be a separate, bigger effort > > that > > > we > > > > can evolve towards if there's demand for it. > > > > > > > > Appreciate feedback here from Cheng, Forward and anyone else with > > > > familiarity around this area as I have only started dipping my > toes > > > into > > > > Lance. > > > > > > > > *Additionally, if anyone wants to help contributing in this area, > > > please > > > > reach out. * > > > > > > > > Best regards > > > > Keith Lee > > > > > > > > Reference > > > > [1] https://github.com/apache/fluss/pull/2716 > > > > [2] https://github.com/apache/fluss/issues/2706 > > > > [3] https://github.com/apache/fluss/issues/2715 > > > > [4] https://github.com/apache/fluss/issues/2751 > > > > [5] https://github.com/lance-format/lance/issues/3961 > > > > > > > > > -- > Lorenzo Affetti > Senior Software Engineer @ Flink Team > Ververica <http://www.ververica.com> >
