All - I think that I will assume that the #3 approach described here will be the path forward. I also think that this refactoring effort deserves a KIP to collect the approach and set the expectations clearly.
I will also start a thread for 0.13.0/1.0.0 Planning which will include this refactoring. We will need to determine the other features and bug fixes that are vital in the next release while trying to keep the scope as tight as possible. thanks, --larry On Tue, Apr 11, 2017 at 4:37 PM, larry mccay <[email protected]> wrote: > Good to hear, Jeffrey. > > If you and/or your customers have had to extend any of the provider > classes or anything else for our pluggable infrastucture it would be great > to get some insights into what classes are being referenced. That way we > can get some adapter classes created and tests to insure that we don't > break our users. > > Thanks! > > --larry > > On Tue, Apr 11, 2017 at 1:34 PM, Jeffrey Rodriguez <[email protected]> > wrote: > >> Hi folks, >> >> It is quite exciting reaching this milestone on the project. I've >> been following closely the discussions on the details for the new release >> 1.0.0 and would like to volunteer to help. Please count on me in any way >> that may be useful to the project. >> >> Best Regards, >> Jeffrey E Rodriguez >> >> On Tue, Apr 11, 2017 at 7:00 AM, larry mccay <[email protected]> >> wrote: >> >>> So, in #2 you don't really mean an additional patch but instead one just >>> for the refactoring branch. >>> >>> I'm thinking that there may be a valid #3 in which we branch for an >>> 0.13.0 >>> release and refactor master. >>> If the refactoring of master - which is intended for a 1.0.0 branch - >>> takes >>> longer than anticipated then we release from 0.13.0. >>> This will require double commits and separate patches while it lasts. >>> >>> Ideally, the refactoring goes well and we never ship 0.13.0 and 1.0.0 >>> supersedes it. >>> >>> On Mon, Apr 10, 2017 at 12:25 PM, Sandeep More <[email protected]> >>> wrote: >>> >>> > Hello Larry, >>> > >>> > About the logistics: >>> > I like the branching idea for refactoring, by doing that we can keep >>> > master stable while we get tests (Knox, 3rd party, samples) to work. >>> > Regarding merging patches, we could, as you pointed out hold off on the >>> > patches for some short time, else just submit them to master or >>> > refactoring-branch and then merge the branch to master. >>> > >>> > IMO adding patches to both the branch and master might cause a bit of >>> > complication during final merge (and might screwup the git commit >>> history). >>> > The problem would be that most of the patches would be on packages >>> > "org.apache.hadoop.gateway" which would be affected by refactoring. >>> > >>> > >>> > When a patch needed to be merged we could either: >>> > 1. Submit the patch on master and merge the master and branch manually >>> and >>> > regularly - which would be a bit tedious. >>> > 2. Or, we could ask the community to submit an additional patch for the >>> > branch and merge the branch into master after refactoring is done. >>> Since, >>> > master hasn't moved merging in this case would be much cleaner and >>> easier. >>> > >>> > I like the #2 approach as it would prevent unintentional errors and the >>> > patch authors will be properly attributed, the downside is that there >>> would >>> > be some additional work on part of the patch authors :( >>> > >>> > Hopefully this was not confusing :) >>> > >>> > About the insights, it would be great to get some info on the classes >>> that >>> > are extended or customized ! >>> > >>> > Best, >>> > Sandeep >>> > >>> > >>> > >>> > >>> > On Sat, Apr 8, 2017 at 10:28 AM, larry mccay <[email protected]> >>> wrote: >>> > >>> >> Hey Sandeep - >>> >> >>> >> Sorry for the late response was on the road. >>> >> Glad to hear you are in favor of the refactoring. >>> >> >>> >> I think we need to determine the best way forward with that work in >>> terms >>> >> of git branches. >>> >> Given that we generally create release line branches from master we >>> need >>> >> to make sure to not destabilize it while we work on the refactoring. >>> >> >>> >> It may make sense to create a separate branch for the refactoring >>> work to >>> >> keep master on existing package names for now. >>> >> Then once we have the refactoring working reasonably well we can merge >>> >> into master. >>> >> >>> >> I imagine that patches would need to be rationalized across branches >>> >> during that time however. >>> >> This may require separate patches for each branch until the merge >>> happens. >>> >> If we try and concentrate on the refactoring work for a couple weeks >>> >> before we push many patches this pain can be minimized. >>> >> >>> >> We will also need to get some insights into as many custom user >>> >> extensions that may exist as possible. >>> >> This is the only way we will know exactly what adapter classes we will >>> >> need to provide. >>> >> >>> >> Thoughts on these logistics would certainly be welcomed! >>> >> >>> >> thanks, >>> >> >>> >> --larry >>> >> >>> >> On Sun, Apr 2, 2017 at 10:31 AM, Sandeep More <[email protected]> >>> >> wrote: >>> >> >>> >>> +1 for 1.0.0 release. >>> >>> >>> >>> About the package names, that seems to be a big refactoring change >>> and a >>> >>> needed one IMO (could be now or near future), with a lot of UnitTest >>> and >>> >>> other tests breaking (pure Knox tests, custom tests and third party >>> tests) >>> >>> as you rightly pointed out. >>> >>> >>> >>> I really like the idea of Shim classes to to avoid the test failures >>> and >>> >>> support the custom extensions. >>> >>> Would love to know what folks think about it. >>> >>> >>> >>> Thank you for kicking off the discussion on 1.0.0 release Larry ! >>> >>> >>> >>> Best, >>> >>> Sandeep >>> >>> >>> >>> >>> >>> On Sat, Apr 1, 2017 at 6:50 PM, larry mccay <[email protected]> >>> wrote: >>> >>> >>> >>>> All @devs and @users - >>> >>>> >>> >>>> With the 0.12.0 release behind us which concentrated on KIP-4 to >>> improve >>> >>>> the knoxshell SDK and DSL capabilities and maturity as well as >>> other bug >>> >>>> fixes and improvements, I think we should seriously consider a 1.0.0 >>> >>>> release for Apache Knox. >>> >>>> >>> >>>> We have always tried to maintain backward compatibility across the >>> >>>> releases >>> >>>> anyway - so this will not be adding any additional burden. >>> >>>> >>> >>>> The one aspect that we do need to close on is the package names >>> used by >>> >>>> the >>> >>>> project. >>> >>>> >>> >>>> Currently, we use a base of org.apache.hadoop.gateway for all of our >>> >>>> java >>> >>>> code. We should at least consider changing this to something like >>> >>>> org.apache.knox.gateway. >>> >>>> >>> >>>> At the same time, we need to consider the backward compatibility >>> >>>> aspects of >>> >>>> this change and how it would effect a number of consumers: >>> >>>> >>> >>>> * folks that have extended existing abstract classes or implemented >>> >>>> interfaces >>> >>>> * folks that are running tests suites that have explicit classnames >>> in >>> >>>> configuration, etc >>> >>>> * existing deployments that may upgrade to a 1.0.0 release and have >>> >>>> {GATEWAY_HOME}/data/deployments directories full of descriptors >>> with >>> >>>> the >>> >>>> current KnoxLdapRealm classname and package >>> >>>> * any others? >>> >>>> >>> >>>> It would likely require some testing to ensure that our unit tests >>> and >>> >>>> any >>> >>>> known system/functional tests that are running outside of dev >>> >>>> environments >>> >>>> pass. Which may require some shim classes in the old packages for >>> known >>> >>>> extension points and classes. >>> >>>> >>> >>>> A 1.0.0 release is an exciting milestone and I look forward to >>> seeing it >>> >>>> happen for our community! >>> >>>> >>> >>>> Any thoughts, concerns or ideas? >>> >>>> >>> >>>> thanks! >>> >>>> >>> >>>> --larry >>> >>>> >>> >>> >>> >>> >>> >> >>> > >>> >> >> >
