> 3. Launcher placement - Here I'd go the opposite way of the current doc. > Currently, every deployment shape keeps its bootstrap in the Flink > distribution, and that convention is what decouples operator version from > Flink version. My opinion is that the MiniCluster launcher should follow > it. In practice, the launcher is close to `flink run -t local`, only adding > HA-aware resubmission and pod exit-code semantics on top, so it is core > code either way. I'd also say it makes sense to ship it in the distribution > (ideally with a bin/ script, so the same setup runs locally too) because > this means the operator depends on no MiniCluster Java API at all, and it > can manage the deployment in the same unified way as the other shapes.
I really like this idea, thanks - that's much better than what I'd originally thought of. It simplifies a few things, such as migration between MiniCluster and Deployment by letting you just use the same image for both as-is. I'll have a proper think through what it would need and update the doc. Do you think a Flink launcher would need a separate/dedicated FLIP to get broader scrutiny? Kind regards D -- dalelane.co.uk On Tuesday, 25 August 2026 at 14:10, Dennis-Mircea Ciupitu <[email protected]> wrote: > Hi Dale, > > My response to the open points is the following: > > 1. MiniCluster vs 2 containers in one pod - Robert's numbers and > argument make it clear to me. So, in order to ensure the lowest possible > resource consumption, we need to go with the MiniCluster approach and not > rely on something intermediary that does not reach this end goal and is > only better suited from the implementation/design perspective. > 2. Separate CRD vs a third mode - Because the MiniCluster approach is > the way to go, I'd say that we should have an implementation that > integrates with it in the best way possible. For me, I see a separate CRD > as the best way to proceed here, as MiniCluster is different from many > points of view from a normal Flink cluster, and by going with a third mode > I think we just complicate ourselves and we introduce too much blast > radius. > 3. Launcher placement - Here I'd go the opposite way of the current doc. > Currently, every deployment shape keeps its bootstrap in the Flink > distribution, and that convention is what decouples operator version from > Flink version. My opinion is that the MiniCluster launcher should follow > it. In practice, the launcher is close to `flink run -t local`, only adding > HA-aware resubmission and pod exit-code semantics on top, so it is core > code either way. I'd also say it makes sense to ship it in the distribution > (ideally with a bin/ script, so the same setup runs locally too) because > this means the operator depends on no MiniCluster Java API at all, and it > can manage the deployment in the same unified way as the other shapes. > 4. Promoting the MiniCluster - Here I say if the entrypoint lands in > Flink core, then I'd follow Gyula's and Robert's guidance on this side, if > there is something that can be done further here in order to promote it > properly. > 5. Resource and memory mapping - Deferring the exact ratios to > implementation is fine with me. > > > Best regards, > Dennis > > > On Mon, Aug 24, 2026 at 2:00 PM Dale Lane <[email protected]> > wrote: > > > > Whether it should be a different CRD I think should > > > depend on the overlap with the FlinkDeployment. If > > > we are exposing essentially the same spec (podTemplate, > > > jobSpec, flinkConfig, ingress..) then it feels like we > > > should reuse the CRD and simply introduce a new > > > deployment mode next to native and standalone. > > > > @Gyula, @Dennis - How would you like to proceed with this question? > > > > I can absolutely see arguments for/against both approaches. In the end I > > leaned more towards a separate CRD, partly from an API purity perspective > > (feeling that the separate topology deserved a separate API, and avoids > > describing the pod spec and status as "jobManager" or "taskManager" when > > it's both) and partly because it dodged the automated topology migration. > > > > But maybe we don't mind the "a MiniCluster includes a Job Manager so we > > can just describe it as jobManager (and ignore the taskManager references)" > > API smell. It could change the manual migration path I describe to > > transition jobs between MiniCluster and full distributed clusters to just > > flipping a mode value. And we can always reject/block such a mode switch in > > the meantime unless/until the Operator is ready to support topology > > transition/migration. > > > > Kind regards > > > > Dale > > -- > > dalelane.co.uk > > > > Sent with Proton Mail secure email. > > > > On Thursday, 20 August 2026 at 10:47, Gyula Fóra <[email protected]> > > wrote: > > > > > Thanks @Robert Metzger for the explanation, > > > > > > Overall I think this is a good idea and hopefully we can reuse most of > > > the controller flow that we already have. > > > It would be nice to share the spec/status as much as reasonably possible. > > > > > > Whether it should be a different CRD I think should depend on the > > > overlap with the FlinkDeployment. If we are exposing essentially the > > > same spec (podTemplate, jobSpec, flinkConfig, ingress..) then it feels > > > like we should reuse the CRD and simply introduce a new deployment > > > mode next to native and standalone. > > > > > > Cheers > > > Gyula > > > > > > On Thu, Aug 20, 2026 at 11:26 AM Dale Lane > > > <[email protected]> wrote: > > > > > > > > This bit that I added to the FLIP is perhaps worthy of a bit more > > scrutiny, as it's based on my assumptions (misunderstandings?) of how the > > FKO project works. > > > > > > > > In the section "Promoting MiniCluster to a stable API" I asserted: > > > > > > > > > For the Flink Kubernetes Operator project to provide a launcher > > > > > as an example project would depend on the core Flink project > > > > > promoting some MiniCluster components to @PublicEvolving ... > > > > > This is not a technical requirement, as the current > > > > > proof-of-concept demonstrates that the existing API is sufficient > > > > > - the promotion recommended here is of MiniCluster’s existing > > > > > submission surface. The requirement is about the Operator project > > > > > managing risk by building upon a stable contract. ... > > > > > > > > Is that really a requirement or have I just invented that? :) > > > > > > > > Looking more closely at the existing code, I can see we already have > > plenty of uses of flink-runtime classes that don't have @Public / > > @PublicEvolving annotations, so my usage of flink-runtime classes like the > > MiniCluster and MiniClusterConfiguration wouldn't be without precedent. > > > > > > > > While I still think it'd be lovely to build this feature solely on > > stable APIs, maybe I'm creating an unreasonably high bar to clear by > > framing it this so strongly. > > > > > > > > What do you think? What are the norms here? > > > > > > > > Kind regards > > > > > > > > Dale > > > > -- > > > > dalelane.co.uk > > > > > > > > > > > > On Wednesday, 5 August 2026 at 16:02, Dale Lane < > > [email protected]> wrote: > > > > > > > > > I'd like to start a discussion on > > > > > FLIP-XXX : Running Flink jobs in MiniCluster using the Kubernetes > > Operator > > > > > > > https://docs.google.com/document/d/1dtGjPYcsBkx1vxHPs1QnDtPxeH_Acz_pl8gx4b1BLB4/edit?usp=sharing > > > > > > > > > > The aim of the FLIP is to extend the Flink Kubernetes Operator to > > offer a single-pod, light-weight deployment option for low-throughput jobs. > > > > > > > > > > From the motivation: > > > > > A single-pod, self-contained Flink job that starts fast and needs no > > multi-pod coordination could be a good fit for low-throughput jobs that > > aren't suitable for session clusters because they need isolation. > > > > > > > > > > Looking forward to feedback, both on the general motivation (Have > > you seen a need for very small lightweight Flink jobs where fast > > crash-consistent resume is good enough without a full distributed Flink > > cluster?) and the suggested implementation approach (Do you think a new > > custom resource kind is the best way to represent this capability?) > > > > > > > > > > Kind regards > > > > > > > > > > Dale > > > > > -- > > > > > dalelane.co.uk > > > > > > > > > > >
