> I discovered many months ago that just preemptive basic-authentication and SOLRJ required me to write custom code. The standard SOLRJ client will not send the authentication header when performing a PUT operation. As such update operations via Knox would fail.
Rick - Is there a Solr JIRA open for the above issue? Is this with a certain version of Solr? Kevin Risden On Fri, Dec 1, 2017 at 12:57 PM, Rick Kellogg <[email protected]> wrote: > Correct no negative comments. The changes sound very useful. I do wish > the Ambari support was stronger but documenting usage in Wiki is a suitable > compromise. I will try to coordinate some testing with my Ops team time > permitting. > > The third party clients I am referring to are the standard Java clients > (HBase HRemoteTable, SOLR SOLRJ/SolrClient) used by outside clients like > Tomcat. Had not even thought of the Knox Shell. Our job is not complete > if we abandon the Client API developers are accustomed to using. > > In reference to the Lite HBase REST Client I am creating, I fully support > the low level Google Protocol buffers. One item we as a community have not > addressed is the traditional consumer of a service via Knox. I discovered > many months ago that just preemptive basic-authentication and SOLRJ > required me to write custom code. The standard SOLRJ client will not send > the authentication header when performing a PUT operation. As such update > operations via Knox would fail. > > When I started the HBase work, I used the HRemoteTable class. After some > discussion with their development team, they stated it was really only > there for testing purposes. Hence the creation of my new project. My main > goals were: minimize third party dependencies, maintain yet simplify > existing API compatibility plus Kerberos support. At some point, we might > do a complete redesign of the API. > > Hope that clarifies things. Take care, > Rick > > -----Original Message----- > From: larry mccay [mailto:[email protected]] Sent: Friday, December 1, > 2017 9:23 AM > To: [email protected] > Subject: Re: Ambari Enhancements / Knox Clients > > Overall, it seems that you don't have any outstanding concerns - right? > > inline... > > On Fri, Dec 1, 2017 at 9:00 AM, Rick Kellogg <[email protected]> > wrote: > > > Greetings Larry, > > > > As always thanks for the community's good work and collaborative support. > > > > I have been following the traffic on Knox improvements for sometime > > but have not built and used the interim builds. All the deployment > > topology improvements sound very useful in production. To fully close > > the loop for our operations team, the integration with Ambari is going > > to prove critical. Many tools we use (Confluent Kafka REST, Livy, > > etc) require manual installation and then complex configuration. With > > these recent addition to Knox and future improvements to Ambari, it > > would be great to declaratively state which services to proxy and have > > automatic host detection with high availability/failover. Essentially > > a complete end-to-end solution with a low rate of failure opportunity > > for our operations teams. > > > > > I think you are saying that the changes look good. :) I agree that the > Ambari based service discovery is going to be a huge win from an Ops > perspective. > > > > So in the interim, I would ask that changes necessary to support > > automatic deployment/registration of Ambari managed services be > > sufficiently documented in the Knox. That way existing topologies > > could be manually setup or a configuration is set which uses Ambari as > > a registry. I just don't want all the good work done to be > > overshadowed by inadequate documentation. > > > > > I would like to get some wiki pages together for testing this end-to-end > for the upcoming RC - it would be amazing if someone from your ops team > could try it out. > Such folks are so important to our community. > > > > Personally, I have been focused on the Knox to client side of things. > > This includes creating a thin HBase REST client out of their baseline. > > In the near future, we should have Kerberos support from the HBASE > > REST Client API as well. This work is being tracked in HBASE-19207 ( > > https://issues.apache.org/jira/browse/HBASE-19207). The project can > > be found on Github: https://github.com/rmkellogg/hbase-lite-rest-client > . > > After which I will be looking at tackling Kerberos support from the > > SOLRJ API as well. > > > > > Hmmm... what do you mean by "Knox to client side of things"? > That client looks nice - is it at all related to the KnoxShell hbase stuff > - doesn't seem like it but it looks similar? > If not, perhaps we can look at some consolidation where our overall > framework can leverage your API? > Are you handing the base64 decoding bit for it - that is so awkward to > deal with for REST? > > Take care, > > Rick > > > > -----Original Message----- > > From: larry mccay [mailto:[email protected]] > > Sent: Thursday, November 30, 2017 4:52 PM > > To: [email protected] > > Subject: Re: Ambari Enhancements > > > > Hi Rick - > > > > Good to hear from you! > > > > I don't think that there is any danger of getting out of sync with > > Ambari expectations here but I may be missing something. > > > > Let me try and summarize the work being done which is based on KIP-8 > > [1] and you can tell me what your concerns are... > > > > 1. We will continue to support the native topology file which is what > > Ambari is doing 2. We are introducing a simplified descriptor and > > shared provider config mechanism that results in generated topologies > > that Ambari will be unaware of in the near term 3. The topologies are > > generated by using the Ambari API to discover the service details > > which obviously is compliant with the current Ambari REST API but > > Ambari itself is unaware of it 4. The descriptors and provider config > > will be able to be monitored in Zookeeper as well as local file system > > Ambari will remain inline with local default.xml, knoxsso.xml and > > admin.xml topologies and unaware of ZK based configs for now 5. Ambari > > improvements are planned to be able to manage the descriptors and > > provider config and to push them to ZK when available otherwise to > > local disk 6. Editing topologies that are generated from descriptors > > would be problematic and lead to confusing when they get overwritten > > upon changes discovered in the descriptors or service details via > > Ambari REST API. So Sandeep is making the generated topologies > > readonly in the UI with an indication as to why. Ambari is completely > unaware of the Knox Admin-UI currently and will likely remain so. > > > > Does the above address your concerns and/or can you share what your > > concerns are? > > > > thanks! > > > > --larry > > > > 1. > > https://cwiki.apache.org/confluence/display/KNOX/KIP-8+ > > Service+Discovery+and+Topology+Generation > > > > > > On Thu, Nov 30, 2017 at 3:34 PM, Rick Kellogg <[email protected]> > > wrote: > > > > > Greetings, > > > > > > > > > > > > Has there been any enhancements made or even suggested to the Ambari > > team? > > > > > > > > > > > > I see a lot of work being done on the Knox internally and to the > > > Knox Admin UI and have some concerns that Ambari v2.6.0+ might not > > > play well with all the new enhancements. > > > > > > > > > > > > Just my two cents, > > > > > > Rick > > > > > > > > > > > > > > > > > >
