Shoot I have a personal appointment that day I can’t move at that time. We could schedule a special session for this or move the community meeting to a different day. I could do Monday or Tuesday that week at 9:00 am PDT.
> On Sep 21, 2021, at 8:14 AM, Anthony Baker <bak...@vmware.com> wrote: > > I know we have folks spread across different time zones, would a 9AM PST / > 4PM UTC discussion work? Yes, we will post the recording afterwards but that > makes it hard to ask live questions :-) > > Anthony > > >> On Sep 21, 2021, at 12:14 AM, Alberto Gomez <alberto.go...@est.tech> wrote: >> >> I think this is a great initiative. Not only are you giving a reminder about >> the need to implement new features in a modular and pluggable way, but you >> are also providing a real example with an already implemented not in the >> best way feature, to show the steps to follow in the right direction for >> this and future features. >> >> I would be interested in attending to a live walkthrough over the details of >> the changes. >> >> Thanks, >> >> Alberto >> ________________________________ >> From: Jacob Barrett <jabarr...@vmware.com> >> Sent: Tuesday, September 21, 2021 3:04 AM >> To: dev@geode.apache.org <dev@geode.apache.org> >> Subject: Re: [DISCUSS] Modularizing new WAN TX Batching (and Modularization >> Efforts in General) >> >> Unfortunately I can’t do meetings that early. The earliest I could do that >> day is 9am PDT. >> >>>> On Sep 20, 2021, at 3:34 PM, Anthony Baker <bak...@vmware.com> wrote: >>> >>> The approach makes sense to me. The idea of a stable core + extensibility >>> is a common and successful pattern in many OSS projects. >>> >>> @Jake, you want to discuss at the next Geode Community meeting in Oct? >>> >>> Anthony >>> >>> >>>> On Sep 20, 2021, at 1:48 PM, Jacob Barrett <jabarr...@vmware.com> wrote: >>>> >>>> Devs, >>>> >>>> We need to be doing a better job with implementing new features in a >>>> modular and plugable way. We have had discussions in the past but we >>>> haven’t been the best at sticking to it. Most recently we began work on a >>>> modified version of WAN that supports transactional batching. Rather than >>>> implement it in a plugable model we modified the existing implementation >>>> with a new property that changes the internal behavior of the >>>> implementation. This is a clear smell that what we have should be plugable >>>> and modularized. I am not suggesting that we run out and define clear >>>> public SPIs for everything or come up with some complicated plan to >>>> re-architect everything into modules but rather that we take small steps. >>>> I will argue that when we are adding functionality to a core service that >>>> is the point we should consider steps to break it up into clear module >>>> components. Think to yourself, what would it take for me to implement this >>>> new functionality as its own module, meaning its own jar and Gradle >>>> sub-project. As you begin to develop the solution you may find you need >>>> some clean interfaces for it to extend from the core, that might be the >>>> start of an internal SPI. You may find that some concrete classes you >>>> would have normally modified just need to be extended with a few protected >>>> methods to implement alternative logic. >>>> >>>> I think we should focus efforts on extracting an interface to plug in >>>> different WAN gateway implementations so that existing implementations >>>> aren’t modified with new behavior. With proper interface extraction we can >>>> more easily unit test around WAN gateways. By keeping implementations >>>> small we can more easily test them in isolation. Making them all plugable >>>> allows distributions of Geode to deliver specific implementations they >>>> would like to support without impacting the existing implementations. It >>>> also frees Geode to release new experimental or beta implementations of >>>> WAN gateways without impacting the existing implementations rather than >>>> delaying releases waiting for modified WAN gateways to be production ready >>>> and fully tested. >>>> >>>> In looking at the geode-wan module one might notice that it was already >>>> designed to be plugable. Unfortunately it isn’t that easy. This SPI was >>>> originally there to provide a way for Geode to be donated initially >>>> without WAN support. It turns out that most of the core to WAN is actually >>>> still in geode-core and only some of the “secret sauce” was moved into >>>> geode-wan. The bulk of the geode-core code for WAN is actually in support >>>> of the region queues for WAN and AEQ, so moving it into geod-wan would >>>> have cut off AEQ. While it would be possible to utilize this SPI for >>>> providing alternative gateway implementations it is very high level,so >>>> much of the alternative implementations would be duplications, and it >>>> doesn’t allow for both implementations to sit side by side at runtime. I >>>> would actually suggest we eliminate this public SPI in favor of just the >>>> geode-wan core module that it is and eventually migrate the region queue >>>> code into its own module as well, but these are for another day. >>>> >>>> Looking closer at the WAN gateways themselves there is mostly a pluggable >>>> interface already there with the existing interfaces. I spent a little >>>> time pulling apart an internal SPI and it was quite easy. With a small >>>> modification to gfsh and cache xml to specify that alternative >>>> implementation by name is all that needs to be done immediately to >>>> configure an alternative. Without extracting too many of the common >>>> implementation out into its own module just a few of the classes in >>>> geode-core can be modified to provide empty implementation of key >>>> overridable plug-in points for the TX batching implementation. The result >>>> is that the TX batching module can be added to the classpath and be >>>> configured as a gateway sender without injecting any of the TX batching >>>> specific code into geode-core or geode-wan. Deeper details can be found at >>>> the end of this discussion. >>>> >>>> This solution isn’t perfect but it is a step in the right direction. The >>>> more we continue to extract and extend the closer we will get to a true >>>> pluggable architecture. The key to being successful will be to keep the >>>> changes to the core components small and the interfaces between modules >>>> internal. If we all make an effort towards this, both in our code and in >>>> our reviews of others code, we can keep the efforts small and manageable. >>>> >>>> I would love feedback and thoughts on this suggestion. >>>> >>>> -Jake >>>> >>> >
