StreamFusion doesn't change any flink code ,just move the jar to the lib and add a configuration flag to enable this feature. and after dig the details of it, the project current design is very good.so i advice put it as sub project. 1. for the planner: StreamFusion just by matches to replace Physical Plan and ExecPlan to implement Native Operator replace 2. for the runtime: it reuse the checkpointing and tolerance. 3. for the stateful operator: it refers risingwave/arroyo some design and use multiget/multiset to optimize the i/o performance 4. for production: the development is very quickly by AI. and feature/issue will be supported quickly.
Ran Tao <[email protected]> 于2026年10月15日周四 15:18写道： > Hi Jacky, > > Thanks for sharing StreamFusion! This looks like an interesting project. > > I'm curious about how StreamFusion integrates with Flink's existing planner > and runtime, especially regarding stateful streaming, checkpointing, and > fault tolerance. > > Also, has StreamFusion been used in production yet? It would be great to > hear about any real use cases and experiences. > > Best Regards, > Ran Tao > https://github.com/taoran92 > > > Jacky Lau <[email protected]> 于2026年10月15日周四 11:14写道： > > > Hi guys: > > I want to introduce all of you a flink naive engine StreamFusion > which > > is based on DataFusion, refers to Risingwave, Arroyo, DataFusion.It has > > good performance and supports lots of sql operator. > > Will it be move it to flink subproject like > > flink-cdc/flink-kubernetes-operator. Anyone has a good advice? > > Query Memory, off Memory, on Disk, off Disk, on > > q0 1.69× 1.39× 1.36× 1.40× > > q1† 1.40× 1.40× 1.38× 1.41× > > q2 1.26× 1.08× 1.11× 1.08× > > q3 1.03× 1.16× 1.08× 1.14× > > q4 1.83× 1.60× 8.92× 15.11× > > q5 1.41× 1.23× 4.38× 4.38× > > q7 1.30× 1.47× 7.41× 11.93× > > q8 1.27× 1.12× 2.33× 2.43× > > q9 1.22× 1.80× 17.83× 67.95× > > q10† 1.45× 1.94× 1.42× 1.40× > > q11 1.47× 1.50× 9.84× 10.03× > > q12 1.09× 1.13× 1.45× 1.33× > > q13 1.20× 1.14× 1.15× 1.22× > > q14† 1.47× 1.25× 1.49× 1.53× > > q15† 1.47× 1.22× 1.61× 1.32× > > q16† 1.17× 1.41× 1.49× 1.28× > > q17† 1.20× 1.18× 1.92× 1.65× > > q18 1.07× 1.37× 6.01× 8.92× > > q19 0.99× 2.82× 2.44× 5.09× > > q20 1.22× 1.56× 37.43× 68.20× > > q21† 1.23× 1.31× 1.26× 1.27× > > q22 1.33× 1.25× 1.30× 1.22× > > q23 1.69× 3.32× 3.15× 12.10× > > geomean 1.31× 1.44× 2.75× 3.41× > > > > † The benchmark's existing opt-in expression variants are used for these > > headline cells: q1 uses approximate decimal arithmetic; q10, q14–q17, and > > q21 use native datetime or regex/case behavior that can differ from Flink > > at documented edge cases. > > > > [1] https://github.com/datafusion-contrib/StreamFusion > > [2] https://streamfusion.tech/ . > > >
