> Assuming an API existed, would that then be a good way? Yep - if there was an API, I think that'd be sufficient (for this use case at least). No reason a user would _want_ to read them out of ZK directly, afaik.
On Mon, Feb 12, 2024 at 9:19 AM Eric Pugh <ep...@opensourceconnections.com> wrote: > Assuming an API existed, would that then be a good way? Or are there > times when an API wouldn’t do what direct ZK does? > > > On Feb 12, 2024, at 9:13 AM, Jason Gerlowski <gerlowsk...@gmail.com> > wrote: > > > >> What other use cases are there for us interacting directly with ZK? > > > > Reading cluster/replica/collection properties for sure. There are some > > Solr APIs to modify these things, but afaik users that want to check the > > value of a particular (e.g.) cluster-property don't have a way to do that > > yet without reading it out of ZK directly. > > > > On Sun, Feb 11, 2024 at 1:58 PM Eric Pugh < > ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com>> > > wrote: > > > >> I like your suggestion David on the CLI supporting configset directly. > >> I’m sort of waiting till the V2 api for putting configsets is done. > >> According to the Solr v2 API Proposed Changes spreadsheet, while there > is a > >> V2 api, it hasn’t been given the “restful/jax-rs/exposed in solrj” > >> treatment yet…. > >> > >> I also agree that figuring out how to make ZK an internal detail is > >> somewhat underlying my question…. > >> > >> What other use cases are there for us interacting directly with ZK? > >> > >>> On Feb 11, 2024, at 12:57 PM, David Smiley <dsmi...@apache.org> wrote: > >>> > >>> I suppose direct ZK has an advantage of not requiring that Solr is > >>> running yet. But maybe that's moot. I do think it would make sense > >>> to have CLI options for configset upload that use Solr's API and don't > >>> refer to ZK. Broadly, this is the direction we're going in; ZK is > >>> more of an internal detail, even though it's obviously a critical > >>> thing. > >>> > >>> On Sun, Feb 11, 2024 at 12:26 PM Eric Pugh > >>> <ep...@opensourceconnections.com <mailto: > ep...@opensourceconnections.com> <mailto:ep...@opensourceconnections.com>> > >> wrote: > >>>> > >>>> Ah.. yeah, I can’t speak to Solr 6.x! In 9x at least you could use > >> the configset API to deploy configs and avoid the direct ZK interaction. > >>>> > >>>> It would be interesting to explore if the process of deploying a > >> configset is risky, has a high chance of things failing, then how do we > >> account for that as part of the process? So you don’t have to do > things > >> like upload the previous config ;-). > >>>> > >>>> And other common reasons to use ZK directly? > >>>> > >>>>> On Feb 11, 2024, at 12:14 PM, Walter Underwood < > wun...@wunderwood.org <mailto:wun...@wunderwood.org>> > >> wrote: > >>>>> > >>>>> The was deploying configs with Jenkins on Solr 6.x. Maybe the APIs > >> were there, but I didn't know about them. > >>>>> > >>>>> Rebuilding the suggester did need external help, since that needs to > >> be done separately on each node. > >>>>> > >>>>> I think working directly with Zookeeper is less risky. If there is > any > >> issue with the upload, then don’t reload the collections. You can back > out > >> the changes by uploading the previous config to Zookeeper. > >>>>> > >>>>> wunder > >>>>> Walter Underwood > >>>>> wun...@wunderwood.org <mailto:wun...@wunderwood.org> <mailto: > wun...@wunderwood.org> <mailto: > >> wun...@wunderwood.org <mailto:wun...@wunderwood.org>> > >>>>> http://observer.wunderwood.org/ (my blog) > >>>>> > >>>>>> On Feb 11, 2024, at 11:07 AM, Eric Pugh < > >> ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com> > <mailto:ep...@opensourceconnections.com> > >> <mailto:ep...@opensourceconnections.com>> wrote: > >>>>>> > >>>>>> Could you share more about “update Solr remotely” that you were > >> doing? Are we missing some APIs that would have made whatever you had > to > >> do require ZK direct access? > >>>>>> > >>>>>> While it’s cool that we can impact Solr via hacking around in ZK, it > >> also seems like an approach fraught with risk! > >>>>>> > >>>>>>> On Feb 11, 2024, at 11:32 AM, Walter Underwood < > >> wun...@wunderwood.org <mailto:wun...@wunderwood.org> <mailto: > wun...@wunderwood.org>> wrote: > >>>>>>> > >>>>>>> I wanted something that didn’t require installing Solr locally in > >> order to update Solr remotely, so I didn’t use the provided zk > commands. I > >> wrote some Python to dig the Zookeeper addresses out of clusterstatus (I > >> think) then uploaded directly to Zookeeper with the Python kazoo > package. > >>>>>>> > >>>>>>> The tool had a bunch of other things, like async reload checking > for > >> results, and rebuilding suggestion dictionaries on each node. > >>>>>>> > >>>>>>> wunder > >>>>>>> Walter Underwood > >>>>>>> wun...@wunderwood.org <mailto:wun...@wunderwood.org> <mailto: > wun...@wunderwood.org> > >>>>>>> http://observer.wunderwood.org/ (my blog) > >>>>>>> > >>>>>>>> On Feb 11, 2024, at 9:04 AM, Gus Heck <gus.h...@gmail.com > <mailto:gus.h...@gmail.com> <mailto: > >> gus.h...@gmail.com <mailto:gus.h...@gmail.com>>> wrote: > >>>>>>>> > >>>>>>>> I pretty much always use zk upconfig, which also works for > >> overwriting > >>>>>>>> existing. I certainly tell my clients to use apis from the ref > >> guide for > >>>>>>>> such operations, but zk upconfig certainly counts as one. Mostly I > >> tell > >>>>>>>> them that they should only break out things like > >>>>>>>> https://github.com/rgs1/zk_shell as a last resort (which is what > I > >> think of > >>>>>>>> as direct modification), and if they are unsure, call me *before* > >> doing > >>>>>>>> anything in zk directly. > >>>>>>>> > >>>>>>>> By the way, I don't know if this has come up in a dev/build > setting > >> or not, > >>>>>>>> but are you aware of https://plugins.gradle.org/search?term=solr > ? > >> It is > >>>>>>>> presently only really suitable for local dev, with a single config > >> set, but > >>>>>>>> could easily grow patches and suggestions welcome of course. > >>>>>>>> > >>>>>>>> On Sun, Feb 11, 2024, 9:10 AM Eric Pugh < > >> ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com> > <mailto:ep...@opensourceconnections.com>> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi all.. I was playing around with a cluster and wanted to > >> upload a > >>>>>>>>> configset into Solr…. > >>>>>>>>> > >>>>>>>>> I ran bin/solr and noticed a bin/solr config -h command, but it > >> just lets > >>>>>>>>> me tweak a config. Then I ran bin/solr create -h and it appears > >> to let me > >>>>>>>>> upload a configset, but I have to create the collection as well, > >> and I’m > >>>>>>>>> not ready to do that. > >>>>>>>>> > >>>>>>>>> Then I poked around and discovered hidden under bin/solr zk a > >> command > >>>>>>>>> upconfig…. So bin/solr zk upconfig will let me get my configset > >> into Solr, > >>>>>>>>> but does require me to remember what my magic ZK string is ;-). > >>>>>>>>> > >>>>>>>>> I went and checked the ref guide, and yes, it states that there > >> are two > >>>>>>>>> ways: > >>>>>>>>> > >>>>>>>>> A configset can be uploaded to ZooKeeper either via the > Configsets > >> API < > >>>>>>>>> > >> > https://solr.apache.org/guide/solr/latest/configuration-guide/configsets-api.html > >>> > >>>>>>>>> or more directly via bin/solr zk upconfig < > >>>>>>>>> > >> > https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html#upload-a-configuration-set > >>> . > >>>>>>>>> The Configsets API has some other operations as well, and > >> likewise, so does > >>>>>>>>> the CLI. > >>>>>>>>> > >>>>>>>>> Are there use cases where interacting directly with ZooKeeper is > >> preferred > >>>>>>>>> over making changes via the APIs? Of is the use of bin/solr zk > >> upconfig > >>>>>>>>> more of a evolutionary byproduct of how we built SolrCloud? > >>>>>>>>> > >>>>>>>>> Eric > >>>>>>>>> > >>>>>>>>> _______________________ > >>>>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > >> 434.466.1467 | > >>>>>>>>> http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> < > >>>>>>>>> http://www.opensourceconnections.com/> | My Free/Busy < > >>>>>>>>> http://tinyurl.com/eric-cal> > >>>>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > >>>>>>>>> > >> > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > >>> > >>>>>>>>> > >>>>>>>>> This e-mail and all contents, including attachments, is > considered > >> to be > >>>>>>>>> Company Confidential unless explicitly stated otherwise, > >> regardless of > >>>>>>>>> whether attachments are marked as such. > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>>> > >>>>>> _______________________ > >>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > >> 434.466.1467 | http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> | My Free/Busy < > >> http://tinyurl.com/eric-cal> > >>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > >> > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > >>> > >>>>>> This e-mail and all contents, including attachments, is considered > to > >> be Company Confidential unless explicitly stated otherwise, regardless > of > >> whether attachments are marked as such. > >>>> > >>>> _______________________ > >>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 > >> | http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> | My Free/Busy < > >> http://tinyurl.com/eric-cal> > >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > >> > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > >>> > >>>> This e-mail and all contents, including attachments, is considered to > >> be Company Confidential unless explicitly stated otherwise, regardless > of > >> whether attachments are marked as such. > >>>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org <mailto: > dev-unsubscr...@solr.apache.org> <mailto: > >> dev-unsubscr...@solr.apache.org <mailto:dev-unsubscr...@solr.apache.org > >> > >>> For additional commands, e-mail: dev-h...@solr.apache.org <mailto: > dev-h...@solr.apache.org> <mailto: > >> dev-h...@solr.apache.org <mailto:dev-h...@solr.apache.org>> > >> _______________________ > >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > >> http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > >> http://www.opensourceconnections.com/> | My Free/Busy < > >> http://tinyurl.com/eric-cal> > >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > >> > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > >> > >> This e-mail and all contents, including attachments, is considered to be > >> Company Confidential unless explicitly stated otherwise, regardless of > >> whether attachments are marked as such. > > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> | My Free/Busy < > http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > >
