Hi Phil - Thanks for bringing this up for discussion.
I do agree with the descriptor author's intent but at the same time, they also intend for the others to be available. There isn't much difference between a topology with a service elements that can't be reached and one without the service elements in it. More than likely, when you deploy a topology and can't access a service - like HIVE - you will go to ambari to check the status on the service. In this case you will notice that it isn't deployed or configured correctly - like in http mode. You take the actions in Ambari and the the service is now accessible. Having to go and add it to the topology after that shouldn't be necessary. I think that we could consider how the monitoring of the discovery service is going to be driven. If it is drive by the simple descriptor - which makes sense - then I think that it could result in a topology with only those services that can be discovered. As long as the others are still in the descriptor they can be discovered later and the topology automagically get updated with the additions. This gives us a situation where only "correct" topologies are deployed and they will be autocorrecting as others come online. Even the HIVE situation would fix itself just by putting it in the right mode. My suggestion would to skip those that can't be fully discovered and log each one as WARNING. Monitoring of discovery service based on descriptors rather than topologies would be able to correct as appropriate. What do you think? thanks, --larry On Tue, Sep 26, 2017 at 10:42 AM, Philip Zampino <[email protected]> wrote: > I’ve been thinking about the behavior wrt topology generation when the > URL(s) for a service declared in a simple descriptor cannot be > correctly/completely determined. > > The options available include: > > > 1. Abort the topology generation because we can’t produce what has been > requested. > > 2. Complete the topology generation without those services whose URL(s) > could not be determined. > The unresolved services could be omitted or commented-out in the resulting > topology file. > > 3. Complete the topology generation, allowing the descriptor deployer > the opportunity to “fill in the blanks” > This will result in Knox deploying a topology it knows to be incorrect. > API deployments may not afford the deployer the opportunity to “fill in > the blanks” (e.g., Ambari-driven deployments). > > > My initial feeling on this is that we should not produce anything less > than what the descriptor declares (i.e., #1). After all, the declared > services are in the descriptor precisely because someone wants to access > them through Knox. > > I could possibly be persuaded that producing a partial topology (i.e., #2) > may be acceptable, but it’s still not what the descriptor author > intends/requires. > > I don’t believe Knox should ever produce or deploy a topology it knows to > be incorrect (i.e., #3). > > One example, which came up during the review of KNOX-1014, is HIVE; If the > hiveserver2 component is not configured for HTTP transport, then there is > no valid URL for that service, as far as Knox is concerned. In this case, I > think we must abort the topology generation or omit the HIVE service from > the generated topology. > > Interested in your thoughts… > > -- > Phil > >
