Very nicely put, I like the separation between API and SPI. Lets start first by laying out all the entities in Helix.
I see pros and cons in maintaining compatibility. But if we know that current api's are broken I prefer getting rid of them instead of carrying it over and always having to maintain them. Also as long we are compatible with physical layout on zookeeper, most of the compatibility issues should be resolved at compile time. Will start another thread to discuss the data model and high level api's. thanks, kishore G On Sat, Feb 22, 2014 at 10:58 AM, Sandeep Nayak <[email protected]> wrote: > Hey guys, > > I prefer option 2, its clear and does not have package specific rules > i.e API breakage is acceptable in certain areas whereas others we need > to keep it as is. Its going to be too difficult for new contributors > to be on-boarded if this ruleset needs to be followed. > > My suggestion is to consider two sets > > (a) API: Which clients/users of Helix will stick to and use. This API > needs to be clear i.e. users should know what is API boundary and what > is leaking into implementation. If Helix contract to its users is > clear, users bypass it (ideally we don't allow them to) at their own > risk. > > (b) SPI: This API is what Helix contributors will adhere to for > subsystems in Helix. SPI will be the contract for someone who is > changing a core subsystem like for example Helix uses zookeeper, lets > say tomorrow it needs to get replaced then the contract from Helix to > the zookeeper layer if abstracted through an SPI would allow other > adapters to be added to replace zookeeper and Helix code does not need > to change when it talks to the new adapters. > > As regards the steps listed, here are my thoughts > > 1) Remove APIs that we're confident no one uses or can use (the alerting > stuff) > [Sandeep] What about the APIs which we are not confident about? Why > keep carrying forward APIs which possibly or in some cases definitely > need to change because they were not clear/clean but cannot be changed > with the fear of breaking backward compatibility? Besides controlling > and tracking such APIs will be a management overhead. > > 2) Keep the interfaces for the most common currently used APIs > (HelixManagerFactory, HelixManager, HelixAdmin, ClusterSetup, command > line, REST), but use our new implementations underneath > (HelixConnection, HelixController, HelixParticipant, HelixSpectator, > HelixAdministrator). This means there will be breakages for people who > used the implementation classes directly, but the changes would be > minor. > [Sandeep] The problem I see here is that some of the existing APIs > need to be changed, point in case HelixAdmin needs to throw correct > exceptions rather than HelixException which is a runtime exception. If > we want to fix it we will need to break API compatibility. > > 3) Provide adapters between common new and old APIs > (HelixConnectionAdapter is an example) > [Sandeep] Regardless of what approach is taken, this is a good idea > wherever possible. > > 4) Maintain the model package as-is > > Sandeep > > On Sat, Feb 22, 2014 at 10:09 AM, Kanak Biscuitwala <[email protected]> > wrote: > > The biggest problem is that we pretty much have to backport bug fixes > forever if we completely break compatibility. Here's what I think we could > do: > > > > 1) Remove APIs that we're confident no one uses or can use (the alerting > stuff) > > 2) Keep the interfaces for the most common currently used APIs > (HelixManagerFactory, HelixManager, HelixAdmin, ClusterSetup, command line, > REST), but use our new implementations underneath (HelixConnection, > HelixController, HelixParticipant, HelixSpectator, HelixAdministrator). > This means there will be breakages for people who used the implementation > classes directly, but the changes would be minor. > > 3) Provide adapters between common new and old APIs > (HelixConnectionAdapter is an example) > > 4) Maintain the model package as-is > > > > This way, there's a clear transition path from one branch to another and > we can eventually end support for the old branch. > > ________________________________ > >> Date: Sat, 22 Feb 2014 07:33:10 -0800 > >> Subject: Helix V2 0.7.0 > >> From: [email protected] > >> To: [email protected]; [email protected] > >> > >> Hi, > >> > >> We have made lot of good changes in 0.7.0 and improved the api's. > >> However, I think it is not easy/intuitive for a new user. One of the > >> problems with 0.7.0 code is that we tried to maintain complete backward > >> compatibility with respect to both logical api's and physical layout on > >> zookeeper. > >> > >> Trying to maintain the logical api backward compatibility has > >> definitely caused some pain and did not allow us to do the right thing > >> in 0.7.0 and it has made our code base huge. The reasoning here is - > >> when we built Helix ((almost 3 years back), we did not anticipate Helix > >> being used in other systems. So our main focus was minimal code and to > >> make sure it works for the use case we had. We did not gather much > >> feedback from users. > >> > >> However, we are seeing the usage grow and while everyone agrees that > >> the high level concepts are good, it is apparent that api's are making > >> people shy away from Helix. I would even say some of the terminologies > >> are confusing until you spend quite some time with Helix. > >> > >> I want to see what others think about this. > >> > >> We have two options going forward > >> > >> Option1: Continue to maintain backward compatibility and improving the > api's > >> Option2: Break the api compatibility and call it Helix V2. We redesign > >> our api's and make it more intuitive and easier/flexible to use. > >> > >> I think the core functionality and design is great and don't see much > >> change needed in the architecture (Do let us know if you think we need > >> any change). What is lacking is documentation and a simple set of api's > >> that are intuitive. > >> > >> While Option 1 is great for existing users, I prefer Option2. We will > >> redesign the 0.7.0 api's without maintaining backward compatibility. > >> Lot of work has already been done in 0.7.0, so we are not that far. > >> This also gives chance to the community to contribute and provide > >> suggestions/feedback/ideas. > >> > >> For existing users, we will continue to maintain 0.6.2 and continue to > >> make critical bug fixes. But no new features will be added. > >> > >> Thoughts ? > >> > >> thanks, > >> Kishore G >
