> > As a result, I'm wondering if Knox should support provider configuration > references in topology.xml, rather than having to duplicate it across > descriptors.
+1 to Phil's remark above. Adding to that as Knox provides multi-cluster support, can we have external provider config cluster-wise? Or we can define multiple external provider config in single "global" provider config file and refer them with some key? Also, when services are deployed in HA, we specify HaProvider and multiple Service URLs in the topology. Will it make sense to use Load balancer IP address / External URL for those services in topology instead of multiple service URLs and using HaProvider? Thanks, Krishna Pandey On Wed, Aug 30, 2017 at 10:05 PM, larry mccay <lmc...@apache.org> wrote: > Terrific, Phil! > > > On Wed, Aug 30, 2017 at 11:03 AM, Philip Zampino <pzamp...@gmail.com> > wrote: > > > After giving this KIP some thought, I would like to work on it. I've > added > > more detail to the wiki, and I'll create a JIRA for it. > > > > On Fri, Aug 25, 2017 at 10:03 AM, larry mccay <lmc...@apache.org> wrote: > > > > > Wonderful details, Phil! > > > > > > > > > On Fri, Aug 25, 2017 at 9:40 AM, Philip Zampino <pzamp...@gmail.com> > > > wrote: > > > > > > > Thanks! I've added the Ambari API details to the wiki. > > > > > > > > On Fri, Aug 25, 2017 at 8:27 AM, larry mccay <lmc...@apache.org> > > wrote: > > > > > > > > > HI Phil - > > > > > > > > > > Thank you for digging into this topic! > > > > > > > > > > I've added you as a contributor to the wiki and you should be able > to > > > > edit > > > > > the KIP now. > > > > > > > > > > thanks, > > > > > > > > > > --larry > > > > > > > > > > On Thu, Aug 24, 2017 at 3:45 PM, Philip Zampino < > pzamp...@gmail.com> > > > > > wrote: > > > > > > > > > > > I've put together a quick python POC targeting Ambari as the > > > discovery > > > > > > source, just to prove that we can indeed get the necessary > details > > > via > > > > > > Ambari's REST API. > > > > > > It generates a proper topology.xml descriptor based on a simple > > > > > descriptor > > > > > > (I chose YAML for this POC), which has a reference to what I've > > > called > > > > a > > > > > > policy descriptor (the <gateway/> portion of the topology > > > descriptor). > > > > > > > > > > > > I am currently unable to update the KIP, but I can share the REST > > > APIs > > > > > I've > > > > > > employed if there is interest. > > > > > > > > > > > > As a result, I'm wondering if Knox should support provider > > > > configuration > > > > > > references in topology.xml, rather than having to duplicate it > > across > > > > > > descriptors. > > > > > > So, instead of <gateway><provider>...</gateway> in each > > > topology.xml, > > > > > have > > > > > > a single element that points to an external provider config > (e.g., > > > > > > <policy-ref>$GATEWAY_HOME/conf/policy/my-named-provider- > > > > > > config.xml</policy-ref>). > > > > > > I've already externalized it for input to the POC, but I'm still > > > > copying > > > > > > the contents into the resulting topology descriptor; I think it > > would > > > > be > > > > > > better to copy only the reference. > > > > > > > > > > > > Thoughts? > > > > > > > > > > > > - Phil > > > > > > > > > > > > > > > > > > On Fri, Aug 18, 2017 at 1:55 PM, larry mccay <lmc...@apache.org> > > > > wrote: > > > > > > > > > > > > > Good to hear, Phil. > > > > > > > > > > > > > > Yes, I was looking to go back and add some of the API specifics > > and > > > > > > > investigation details for at least Ambari and ZK. > > > > > > > If others make sense to add such as Consul, etcd, etc that > would > > be > > > > > good > > > > > > as > > > > > > > well and it would help to tease out some of what may be needed > > for > > > > the > > > > > > > abstraction API and config. > > > > > > > > > > > > > > > > > > > > > On Fri, Aug 18, 2017 at 1:52 PM, Philip Zampino < > > > pzamp...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > This sounds like a good improvement from a usability > > perspective. > > > > > > > > > > > > > > > > I agree that Knox must continue its support for direct > topology > > > > (XML) > > > > > > > > definitions, but for those deployments where there is a > service > > > > > > registry > > > > > > > > (e.g., Ambari, Zookeeper, Consul, etc...) with knowledge of > the > > > > > > topology, > > > > > > > > the simplified config coupled with the service discovery > could > > > > reduce > > > > > > > > topology (especially ServiceConnectivity) errors. > > > > > > > > > > > > > > > > I also agree that consolidating provider configurations into > > > named > > > > > > sets, > > > > > > > > which can be referenced from topology configurations, will > be a > > > > good > > > > > > > > change, especially if there are commonly grouped providers > > > employed > > > > > by > > > > > > > > multiple topologies. > > > > > > > > > > > > > > > > Concerning the TODO sections, is your intention is that they > > > > contain > > > > > > the > > > > > > > > respective API facilities that will provide the information > > > needed > > > > to > > > > > > > > populate a topology definition? > > > > > > > > > > > > > > > > - Phil > > > > > > > > > > > > > > > > On Fri, Aug 18, 2017 at 12:39 PM, larry mccay < > > lmc...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > All - > > > > > > > > > > > > > > > > > > I have published an initial stab at a proposal for > > simplifying > > > > > > topology > > > > > > > > > management [1] - as I briefly mentioned on the 0.14.0 > > planning > > > > > > thread. > > > > > > > > > > > > > > > > > > There are a couple TODO sections that need some > investigation > > > for > > > > > > > Ambari > > > > > > > > > API and Zookeeper Registry API as discovery plugins. > > > > > > > > > > > > > > > > > > Please give the KIP a read and let me know if it makes > sense > > or > > > > if > > > > > > you > > > > > > > > > would like to take on specific pieces of this work. If you > > > would > > > > > like > > > > > > > to > > > > > > > > > add other options or ideas, etc. > > > > > > > > > > > > > > > > > > I think that this will have a huge impact as usability > > > > improvements > > > > > > for > > > > > > > > > management of the gateway! > > > > > > > > > > > > > > > > > > thanks, > > > > > > > > > > > > > > > > > > --larry > > > > > > > > > > > > > > > > > > 1. > > > > > > > > > https://cwiki.apache.org/confluence/display/KNOX/KIP-8+ > > > > > > > > > Service+Discovery+and+Topology+Generation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
