Good suggestion. You may write a proposal first according to your thoughts.
Best Regards --------------- David Linkedin: https://www.linkedin.com/in/davidzollo --------------- On Fri, Dec 5, 2025 at 4:13 PM Doyeon Kim <[email protected]> wrote: > Hi, I’d like to discuss a proposal to decouple the engine storage layer > from Hazelcast IMap. > > Currently, many parts of the engine directly depend on Hazelcast IMap. To > reduce this tight coupling and make it easier to introduce alternative > storage implementations (e.g., RocksDB) in the future, I’m considering > introducing a MapStorage abstraction layer. > > The basic idea is: > > - Introduce a MapStorage<K, V> interface > - Provide a Hazelcast-based implementation (IMapStorage) > - Use a MapFactory to create MapStorage instances > - Let services depend on MapStorage instead of Hazelcast IMap directly > > In addition, for lifecycle management, I’m considering a context-based > approach: > > - Introduce a service-level context that holds: > - NodeEngine > - A storage factory for MapStorage > - Initialize this context once during SeaTunnelServer initialization > - Inject this context into services instead of passing NodeEngine > directly > > The goal is to further reduce direct coupling between the service layer and > Hazelcast-specific APIs. > > Before proceeding with this refactoring, I’d appreciate feedback on: > > 1. Whether introducing a MapStorage abstraction makes sense in SeaTunnel > Engine. > 2. What would be the preferred way to manage its lifecycle. > 3. Any concerns about introducing this level of abstraction. > > If the direction looks good, I’ll follow up with a concrete implementation > PR. > Thanks! > > Best regards, Doyeon (GitHub ID : dybyte) >
