Kevin, The only ticket is one I filed against the Knox project (https://issues.apache.org/jira/browse/KNOX-1066). Within the standard SOLRJ code it explicitly does not allow retry operations to prevent errors with duplicate data. This impact SOLR 6.x for certain. Have not tested with others. A patch is provided.
Good luck! Rick -----Original Message----- From: Kevin Risden [mailto:[email protected]] Sent: Friday, December 1, 2017 2:29 PM To: [email protected] Subject: Re: Ambari Enhancements / Knox Clients > I discovered many months ago that just preemptive basic-authentication and SOLRJ required me to write custom code. The standard SOLRJ client will not send the authentication header when performing a PUT operation. As such update operations via Knox would fail. Rick - Is there a Solr JIRA open for the above issue? Is this with a certain version of Solr? Kevin Risden On Fri, Dec 1, 2017 at 12:57 PM, Rick Kellogg <[email protected]> wrote: > Correct no negative comments. The changes sound very useful. I do > wish the Ambari support was stronger but documenting usage in Wiki is > a suitable compromise. I will try to coordinate some testing with my > Ops team time permitting. > > The third party clients I am referring to are the standard Java > clients (HBase HRemoteTable, SOLR SOLRJ/SolrClient) used by outside clients > like > Tomcat. Had not even thought of the Knox Shell. Our job is not complete > if we abandon the Client API developers are accustomed to using. > > In reference to the Lite HBase REST Client I am creating, I fully > support the low level Google Protocol buffers. One item we as a > community have not addressed is the traditional consumer of a service > via Knox. I discovered many months ago that just preemptive > basic-authentication and SOLRJ required me to write custom code. The > standard SOLRJ client will not send the authentication header when > performing a PUT operation. As such update operations via Knox would fail. > > When I started the HBase work, I used the HRemoteTable class. After > some discussion with their development team, they stated it was really > only there for testing purposes. Hence the creation of my new > project. My main goals were: minimize third party dependencies, > maintain yet simplify existing API compatibility plus Kerberos > support. At some point, we might do a complete redesign of the API. > > Hope that clarifies things. Take care, Rick > > -----Original Message----- > From: larry mccay [mailto:[email protected]] Sent: Friday, December 1, > 2017 9:23 AM > To: [email protected] > Subject: Re: Ambari Enhancements / Knox Clients > > Overall, it seems that you don't have any outstanding concerns - right? > > inline... > > On Fri, Dec 1, 2017 at 9:00 AM, Rick Kellogg <[email protected]> > wrote: > > > Greetings Larry, > > > > As always thanks for the community's good work and collaborative support. > > > > I have been following the traffic on Knox improvements for sometime > > but have not built and used the interim builds. All the deployment > > topology improvements sound very useful in production. To fully > > close the loop for our operations team, the integration with Ambari > > is going to prove critical. Many tools we use (Confluent Kafka > > REST, Livy, > > etc) require manual installation and then complex configuration. > > With these recent addition to Knox and future improvements to > > Ambari, it would be great to declaratively state which services to > > proxy and have automatic host detection with high > > availability/failover. Essentially a complete end-to-end solution > > with a low rate of failure opportunity for our operations teams. > > > > > I think you are saying that the changes look good. :) I agree that the > Ambari based service discovery is going to be a huge win from an Ops > perspective. > > > > So in the interim, I would ask that changes necessary to support > > automatic deployment/registration of Ambari managed services be > > sufficiently documented in the Knox. That way existing topologies > > could be manually setup or a configuration is set which uses Ambari > > as a registry. I just don't want all the good work done to be > > overshadowed by inadequate documentation. > > > > > I would like to get some wiki pages together for testing this > end-to-end for the upcoming RC - it would be amazing if someone from > your ops team could try it out. > Such folks are so important to our community. > > > > Personally, I have been focused on the Knox to client side of things. > > This includes creating a thin HBase REST client out of their baseline. > > In the near future, we should have Kerberos support from the HBASE > > REST Client API as well. This work is being tracked in HBASE-19207 > > ( https://issues.apache.org/jira/browse/HBASE-19207). The project > > can be found on Github: > > https://github.com/rmkellogg/hbase-lite-rest-client > . > > After which I will be looking at tackling Kerberos support from the > > SOLRJ API as well. > > > > > Hmmm... what do you mean by "Knox to client side of things"? > That client looks nice - is it at all related to the KnoxShell hbase > stuff > - doesn't seem like it but it looks similar? > If not, perhaps we can look at some consolidation where our overall > framework can leverage your API? > Are you handing the base64 decoding bit for it - that is so awkward to > deal with for REST? > > Take care, > > Rick > > > > -----Original Message----- > > From: larry mccay [mailto:[email protected]] > > Sent: Thursday, November 30, 2017 4:52 PM > > To: [email protected] > > Subject: Re: Ambari Enhancements > > > > Hi Rick - > > > > Good to hear from you! > > > > I don't think that there is any danger of getting out of sync with > > Ambari expectations here but I may be missing something. > > > > Let me try and summarize the work being done which is based on KIP-8 > > [1] and you can tell me what your concerns are... > > > > 1. We will continue to support the native topology file which is > > what Ambari is doing 2. We are introducing a simplified descriptor > > and shared provider config mechanism that results in generated > > topologies that Ambari will be unaware of in the near term 3. The > > topologies are generated by using the Ambari API to discover the > > service details which obviously is compliant with the current Ambari > > REST API but Ambari itself is unaware of it 4. The descriptors and > > provider config will be able to be monitored in Zookeeper as well as > > local file system Ambari will remain inline with local default.xml, > > knoxsso.xml and admin.xml topologies and unaware of ZK based configs > > for now 5. Ambari improvements are planned to be able to manage the > > descriptors and provider config and to push them to ZK when > > available otherwise to local disk 6. Editing topologies that are > > generated from descriptors would be problematic and lead to > > confusing when they get overwritten upon changes discovered in the > > descriptors or service details via Ambari REST API. So Sandeep is > > making the generated topologies readonly in the UI with an > > indication as to why. Ambari is completely > unaware of the Knox Admin-UI currently and will likely remain so. > > > > Does the above address your concerns and/or can you share what your > > concerns are? > > > > thanks! > > > > --larry > > > > 1. > > https://cwiki.apache.org/confluence/display/KNOX/KIP-8+ > > Service+Discovery+and+Topology+Generation > > > > > > On Thu, Nov 30, 2017 at 3:34 PM, Rick Kellogg > > <[email protected]> > > wrote: > > > > > Greetings, > > > > > > > > > > > > Has there been any enhancements made or even suggested to the > > > Ambari > > team? > > > > > > > > > > > > I see a lot of work being done on the Knox internally and to the > > > Knox Admin UI and have some concerns that Ambari v2.6.0+ might not > > > play well with all the new enhancements. > > > > > > > > > > > > Just my two cents, > > > > > > Rick > > > > > > > > > > > > > > > > > >
