If it's marked Experimental in the javadocs, I think it may be fine. -- Christopher L Tubbs II http://gravatar.com/ctubbsii
On Fri, Mar 28, 2014 at 4:50 PM, Sean Busbey <[email protected]> wrote: > If the new interface is not in the public API, then I think adding it > (without deprecating MAC) is fine. > > That way it can evolve if needed and we can add it to the public API on a > later release. > > > On Fri, Mar 28, 2014 at 3:39 PM, Christopher <[email protected]> wrote: > >> But... without more time to fully develop the requirements for the >> interface, with a few implementations, it's probably going to change >> anyway. I think even adding the interface could complicate the >> follow-on work. But... *shrug*.... maybe you can have guarantees that >> the interface will stay as is (same package, same methods, same name, >> etc.)? >> >> -- >> Christopher L Tubbs II >> http://gravatar.com/ctubbsii >> >> >> On Fri, Mar 28, 2014 at 3:14 PM, Josh Elser <[email protected]> wrote: >> > Not even the addition of a new interface, Christopher? I'd very much like >> > to have an interface that we can get in 1.6.0 at a minimum. I wouldn't >> even >> > push for any deprecation of what's currently in place. >> > On Mar 28, 2014 10:02 AM, "Christopher" <[email protected]> wrote: >> > >> >> I don't think any of this should be done for 1.6.0, but I like the >> >> idea of creating a separate cluster interface for testing. I think it >> >> should be integrated into the accumulo-maven-plugin, also. I think the >> >> idea should be hammered out, and tested as a separate thing, to >> >> experiment with the options, and provided as a complete feature for >> >> the next major release. If it would change packaging dependencies, it >> >> shouldn't even be done for 1.6.x bugfix releases. >> >> >> >> -- >> >> Christopher L Tubbs II >> >> http://gravatar.com/ctubbsii >> >> >> >> >> >> On Fri, Mar 28, 2014 at 12:24 PM, Josh Elser <[email protected]> >> wrote: >> >> > Oh, I like that idea, Bill & Sean. >> >> > >> >> > Package: org.apache.accumulo.cluster >> >> > Public API: org.apache.accumulo.cluster.AccumuloCluster >> >> > MAC: org.apache.accumulo.cluster.mini.MiniAccumuloCluster (implements >> >> > AccumuloCluster, allows for backwards compat) >> >> > Yarn: org.apache.accumulo.cluster.yarn >> >> > Docker: ... >> >> > Mesos: ... >> >> > >> >> > etc etc etc. >> >> > >> >> > One question in my mind, do we keep the maven module >> >> 'accumulo-minicluster'? >> >> > I would imagine that if we struck the 'mini' portion from 1.6 that >> would >> >> > create some confusion. Would it be worth the indirection to rename >> >> > accumulo-minicluster to accumulo-cluster and then create a new >> >> > accumulo-minicluster module that depends on accumulo-minicluster (but >> >> > contains no code itself) to preserve the 1.4 and 1.5 poms to generally >> >> work >> >> > with a version bump? I'm not sure if Maven would be happy with that >> or do >> >> > what I think it "should". >> >> > >> >> > >> >> > On 3/28/14, 6:26 AM, Bill Havanki wrote: >> >> >> >> >> >> I've been watching the conversation on the side, but I wanted to >> mention >> >> >> that it seems the focus isn't so much on "mini" clusters anymore. >> You're >> >> >> thinking of programmatic cluster management, whether one node or >> many. >> >> The >> >> >> idea of a basic cluster management interface, with MAC as an >> >> >> implementation, is promising. A package name of just "cluster" could >> >> work. >> >> >> >> >> >> Carry on :) >> >> >> >> >> >> Bill H >> >> >> >> >> >> >> >> >> On Fri, Mar 28, 2014 at 12:39 AM, Sean Busbey >> >> >> <[email protected]>wrote: >> >> >> >> >> >>> If you decide to go the mapred/mapreduce way, you could go with the >> >> >>> package >> >> >>> name "mini". >> >> >>> >> >> >>> alternatively, we can do a multi-stage change out >> >> >>> >> >> >>> 1) 1.6.x: introduce TestAccumuloCluster interface, @deprecate >> >> >>> MiniAccumuloCluster class and make it implement TestAccumuloCluster >> >> >>> >> >> >>> 2) 1.6 + major: change MiniAccumuloCluster to an interface that >> extends >> >> >>> TestAccumuloCluster, @deprecate TestAccumuloCluster >> >> >>> >> >> >>> 3) 1.6 + 2 major: remove TestAccumuloCluster >> >> >>> >> >> >>> Or just go with TestAccumuloCluster as the interface, have >> >> >>> MiniAccumuloCluster as the local pseudo distributed implementation, >> and >> >> >>> then call your new one something like YarnAccumuloCluster. >> >> >>> >> >> >>> In that case we could use the deprecation cycle to move the MAC >> class >> >> out >> >> >>> of the public api. >> >> >>> >> >> >>> >> >> >>> On Thu, Mar 27, 2014 at 6:48 PM, Josh Elser <[email protected]> >> >> wrote: >> >> >>> >> >> >>>> Thoughts on if this would be an acceptable change for 1.6.0 to >> >> alleviate >> >> >>>> future cruft? >> >> >>>> >> >> >>>> Suggestions on the new package and/or class name would be greatly >> >> >>>> appreciated over "NewMiniAccumuloC*". >> >> >>>> >> >> >>>> >> >> >>>> On 3/26/14, 3:37 PM, Josh Elser wrote: >> >> >>>> >> >> >>>>> Those who are interested: check out >> >> >>>>> https://github.com/joshelser/accumulo/commit/ >> >> >>>>> 9f63cf32559ab514a69ff2c6b02acef9c9cbb4e8 >> >> >>>>> >> >> >>>>> >> >> >>>>> tl;dr I could create some real interfaces for the cluster and >> config, >> >> >>>>> which are "hidden" under the covers by the 1.4 and 1.5 >> >> >>>>> MiniAccumuloCluster and MiniAccumuloConfig classes. This >> de-couples >> >> the >> >> >>>>> default implementation, gives us the ability to hide >> "implementation >> >> >>>>> details" if wanted, and moves us towards some factory methods >> instead >> >> >>>>> of >> >> >>>>> calling a class directly. >> >> >>>>> >> >> >>>>> Thoughts? >> >> >>>>> >> >> >>>>> On 3/26/14, 1:21 PM, Josh Elser wrote: >> >> >>>>> >> >> >>>>>> Yes, very much experimental at this point. >> >> >>>>>> >> >> >>>>>> What I'm most concerned about is having reasonable hooks up >> front, >> >> not >> >> >>>>>> trying to make an implementation for inclusion 1.6.0. >> >> >>>>>> >> >> >>>>>> Regarding additions, the implementations already contains most >> >> things >> >> >>>>>> I >> >> >>>>>> would want to expose. I haven't come up with anything that would >> be >> >> >>>>>> generally returned through the "API" rather than through this >> >> proposed >> >> >>>>>> implementation (e.g. YARN connection information) >> >> >>>>>> >> >> >>>>>> On 3/26/14, 11:57 AM, Keith Turner wrote: >> >> >>>>>> >> >> >>>>>>> What you are trying to do sounds interesting. It also sounds >> >> >>>>>>> experimental >> >> >>>>>>> and in the early stages. Is there anything specific you think >> >> >>>>>>> should be >> >> >>>>>>> done for 1.6.0 w/ regards to MAC API? >> >> >>>>>>> >> >> >>>>>>> >> >> >>>>>>> On Wed, Mar 26, 2014 at 2:26 PM, Josh Elser < >> [email protected]> >> >> >>>>>>> wrote: >> >> >>>>>>> >> >> >>>>>>> On 3/26/14, 11:13 AM, Keith Turner wrote: >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> On Wed, Mar 26, 2014 at 2:05 PM, Josh Elser < >> >> [email protected]> >> >> >>>>>>>>> >> >> >>>>>>>>> wrote: >> >> >>>>>>>>> >> >> >>>>>>>>> On 3/26/14, 10:57 AM, Keith Turner wrote: >> >> >>>>>>>>> >> >> >>>>>>>>>> >> >> >>>>>>>>>> Can you give an example of what you are thinking of? I >> don't >> >> >>>>>>>>>> understand >> >> >>>>>>>>>> >> >> >>>>>>>>>>> you >> >> >>>>>>>>>>> viewpoint either >> >> >>>>>>>>>>> >> >> >>>>>>>>>>> >> >> >>>>>>>>>>> Sure. One limitation of MAC, in general as a testing >> harness, >> >> >>>>>>>>>>> is >> >> >>>>>>>>>> >> >> >>>>>>>>>> that it >> >> >>>>>>>>>> doesn't adequately exercise multi-node implementations. You >> can >> >> >>>>>>>>>> run >> >> >>>>>>>>>> multiple tservers, but they are all on the same host which >> >> limits >> >> >>> >> >> >>> the >> >> >>>>>>>>>> >> >> >>>>>>>>>> validity of a "robust" test. This is my immediate goal. >> >> >>>>>>>>>> >> >> >>>>>>>>>> Multi-node deployments are capable using something like >> Mesos or >> >> >>>>>>>>>> Yarn. >> >> >>>>>>>>>> Given that there is already functioning support to deploy >> >> Accumulo >> >> >>> >> >> >>> on >> >> >>>>>>>>>> >> >> >>>>>>>>>> Yarn, >> >> >>>>>>>>>> this was my goal. >> >> >>>>>>>>>> >> >> >>>>>>>>>> My goal is to be able to have the ability to run all of our >> >> >>>>>>>>>> AbstractMacIT >> >> >>>>>>>>>> implementations against "real" hardware without changing a >> >> single >> >> >>>>>>>>>> line of >> >> >>>>>>>>>> test code (ok - maybe a line or two to do injection of the >> MAC >> >> >>>>>>>>>> implementation). The point is, I believe there could be a >> huge >> >> >>>>>>>>>> testing >> >> >>>>>>>>>> gain >> >> >>>>>>>>>> from being able to write tests which leverage yarn, have the >> >> same >> >> >>>>>>>>>> programmatic configuration API from MAC, and provide near >> "real" >> >> >>>>>>>>>> Accumulo >> >> >>>>>>>>>> semantics. >> >> >>>>>>>>>> >> >> >>>>>>>>>> >> >> >>>>>>>>>> Ok so you want to MAC to be an interface so that you can >> >> provide >> >> >>>>>>>>>> a >> >> >>>>>>>>> >> >> >>>>>>>>> completely different implementation? >> >> >>>>>>>>> >> >> >>>>>>>>> >> >> >>>>>>>>> Correct. Some things would serve well in a common abstract >> base >> >> >>> >> >> >>> (e.g. >> >> >>>>>>>> >> >> >>>>>>>> numTservers, siteXml configuration), but all the nonsense about >> >> >>>>>>>> creating >> >> >>>>>>>> directory structures and managing Processes is implementation >> >> >>> >> >> >>> specific. >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> Perhaps I could create a new interface that the current >> >> >>> >> >> >>> implementation >> >> >>>>>>>> >> >> >>>>>>>> implements which still provides the same semantics from 1.4 and >> >> 1.5. >> >> >>>>>>>> Let me >> >> >>>>>>>> see if I can mock up what I'm thinking -- that will probably be >> >> >>>>>>>> easier than >> >> >>>>>>>> me trying to write it out. >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>> >> >> >>> >> >> >> >> >> >> >> >> >> >> >> > >> >> >>
