Thanks everyone for the clarification. Just a bit off topic question to end this thread. When a cluster is configured with Ranger and a GRANT/REVOKE is executed, will this RAISE an ERROR, WARN, INFO, other, or none?
Jon Roberts On Thu, Mar 2, 2017 at 9:04 PM, Vineet Goel <[email protected]> wrote: > Ranger community advises against sync between Ranger policies and native > authorization metastores. It complicates design and users haven't shown a > real need to switch between auth models back and forth. > > -Vineet > > > On Thu, Mar 2, 2017 at 6:33 PM Alexander Denissov <[email protected]> > wrote: > > > Not quite. > > > > For the first phase, there will be no integration between GRANT/REVOKE > and > > Ranger policies -- once Ranger is turned on, GRANT/REVOKE from psql will > no > > longer work. > > > > In a later stage, we will likely provide integration HAWQ --> Ranger, > such > > that when GRANT/REVOKE is issued from psql, an appropriate policy is > > created / deleted from Ranger. > > > > I don't think Ranger --> HAWQ integration is planned or is even possible, > > as admin action in Ranger UI would need to be somehow detected and > > translated to HAWQ grants which are not going to be used anyways as > > authorization will be performed in Ranger only. > > > > -- > > Thanks, > > Alex > > > > On Thu, Mar 2, 2017 at 6:20 PM, Jon Roberts <[email protected]> wrote: > > > > > Have I misunderstood the Ranger integration work? When a GRANT/REVOKE > is > > > executed in HAWQ it will be replicated to Ranger and when a > GRANT/REVOKE > > is > > > executed in Ranger, it will be replicated to HAWQ. Right? If yes, > then > > > this is the model that I'm suggesting for Ambari. > > > > > > Jon Roberts > > > Principal Engineer | [email protected] | 615-426-8661 > > <(615)%20426-8661> > > > > > > On Thu, Mar 2, 2017 at 7:27 PM, Vineet Goel <[email protected]> > wrote: > > > > > > > Having worked with Ambari and knowing the complexities of this > desired > > > > integration, I agree with Alex on his comments below. Users need some > > > time > > > > to get used to exclusive HAWQ CLI or Ambari usage model, and avoid > mix > > > and > > > > match despite the fact that HAWQ CLI and Ambari have their > > > > advantages/disadvantages. > > > > > > > > -Vineet > > > > > > > > > > > > On Thu, Mar 2, 2017 at 5:17 PM Alexander Denissov < > > [email protected]> > > > > wrote: > > > > > > > > > Ambari is designed as a tool to sit on top of individual service > CLI > > > > tools > > > > > and provide consistent user experience. Ambari can also have > wizards > > > and > > > > > custom actions that go beyond service lifecycle management. It is > > > assumed > > > > > by Ambari design that once Ambari is used to manage configurations, > > CLI > > > > > tools should not be used, otherwise changes will be out-of-sync or > > will > > > > be > > > > > overwritten by Ambari upon service restart. > > > > > > > > > > Having HAWQ CLI tools communicate back to Ambari is not desirable > > > > because: > > > > > - knowledge of Ambari REST APIs needs to be built into the tools > > > > > - Ambari APIs change outside of service release lifecycle > > > > > - complexity in error handling (what do you do if Ambari call > failed > > ?) > > > > > - introduction of feedback loop and potential infinite cycle > (Ambari > > > > calls > > > > > the tool, the tool calls Ambari back, etc) > > > > > > > > > > If customers want to do some extra automation they can decide how > to > > > tie > > > > > tools and Ambari, if they like, but I will say we should not have > > HAWQ > > > > CLI > > > > > tools call Ambari. > > > > > > > > > > -- > > > > > Thanks, > > > > > Alex Denissov > > > > > > > > > > On Thu, Mar 2, 2017 at 4:44 PM, Alex (Oleksandr) Diachenko < > > > > > [email protected]> wrote: > > > > > > > > > > > Sure, > > > > > > > > > > > > Users can automate configuration changes via Ambari API, which > > > provides > > > > > > unified access > > > > > > to all services in a cluster. It's very likely they need to > > configure > > > > > HAWQ, > > > > > > HDFS and YARN > > > > > > when working with configuration changes, so with Ambari API it > > would > > > be > > > > > > easier to call one API > > > > > > rather than using all different CLI's. > > > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:39 PM, Lei Chang <[email protected]> > > > > wrote: > > > > > > > > > > > > > I think there are some use cases we need What Jon proposed. > > > > > > > > > > > > > > For example, users installed hawq via Ambari, but they want to > > > > automate > > > > > > the > > > > > > > configuration changes, it would be convenient to have hawq CLI > > > change > > > > > the > > > > > > > configurations. > > > > > > > > > > > > > > Cheers > > > > > > > Lei > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Mar 3, 2017 at 8:36 AM, Alex (Oleksandr) Diachenko < > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > I see, that makes sense. > > > > > > > > But is there any action users cannot do via Ambari? > > > > > > > > > > > > > > > > Ranger is also a good example, there we are making > assumption, > > > > > > > > users either use Ranger or HAWQ's authorization engine. > > > > > > > > > > > > > > > > The same logic might be extrapolated to HAWQ/Ambari - users > > might > > > > use > > > > > > > > either Ambari or HAWQ CLI, but not both at the same time. > > > > > > > > In that way, we can keep things simple. > > > > > > > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:28 PM, Jon Roberts < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Right. Just like HAWQ will be operational without Ranger. > > > > > > > > > > > > > > > > > > We have the hawq CLI and will obviously continue to have > it. > > > > Some > > > > > > > people > > > > > > > > > use Ambari while others don't. So just like with Ranger > > > support, > > > > > > > > integrate > > > > > > > > > when possible but don't require it. > > > > > > > > > > > > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 6:26 PM, Alex (Oleksandr) Diachenko > < > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > Not really, because HAWQ should be operational even > without > > > > > > Ambari(if > > > > > > > > > > that's the case). > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:21 PM, Jon Roberts < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > If that is the case, should we remove the "hawq" CLI? > > > > > > > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 6:12 PM, Alex (Oleksandr) > > Diachenko > > > < > > > > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Jon, > > > > > > > > > > > > > > > > > > > > > > > > I think it was designed that Ambari is supposed to be > > > only > > > > > one > > > > > > > > source > > > > > > > > > > of > > > > > > > > > > > > true. > > > > > > > > > > > > The whole purpose of integration id to provide a > > > > > user-friendly > > > > > > > > > > interface > > > > > > > > > > > > and avoid manually editing/distributing config files > > > > > > > > > > > > or running CLI commands. > > > > > > > > > > > > The idea of coupling HAWQ master with Ambari doesn't > > seem > > > > to > > > > > be > > > > > > > > > clean. > > > > > > > > > > > > > > > > > > > > > > > > Regards, Alex. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 2, 2017 at 4:05 PM, Jon Roberts < > > > > > > [email protected] > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > It would be handy if the "hawq config" also updated > > > > > Ambari's > > > > > > > > > database > > > > > > > > > > > so > > > > > > > > > > > > > that changes could be made in either place are > > retained > > > > > when > > > > > > > > > changes > > > > > > > > > > > are > > > > > > > > > > > > > made in either place. > > > > > > > > > > > > > > > > > > > > > > > > > > Register Ambari: > > > > > > > > > > > > > hawq ambari -u admin -w admin -h myhost -p 8080 > > > > > > > > > > > > > > > > > > > > > > > > > > "hawq config" could then raise INFO/WARN messages > > about > > > > > > > updating > > > > > > > > > > > Ambari. > > > > > > > > > > > > > > > > > > > > > > > > > > Example: > > > > > > > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb > > > > > > > > > > > > > INFO: Updated Ambari with > > hawq_rm_stmt_vseg_memory=16gb > > > > > > > > > > > > > or > > > > > > > > > > > > > hawq config -c hawq_rm_stmt_vseg_memory -v 16gb > > > > > > > > > > > > > WARN: Failed to update Ambari with > > > > > > > hawq_rm_stmt_vseg_memory=16gb. > > > > > > > > > > > Please > > > > > > > > > > > > > update Ambari credentials manually to retain this > > > > > > configuration > > > > > > > > > > change > > > > > > > > > > > > > after a restart. > > > > > > > > > > > > > > > > > > > > > > > > > > The implementation would require interacting with > the > > > > > Ambari > > > > > > > APIs > > > > > > > > > and > > > > > > > > > > > > also > > > > > > > > > > > > > storing the credentials in an encrypted file on the > > > HAWQ > > > > > > > Master. > > > > > > > > > > > > > > > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Jon Roberts > > > > > > > > > > > > > Principal Engineer | [email protected] | > > > 615-426-8661 <(615)%20426-8661> > > > > > <(615)%20426-8661> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
