> What other use cases are there for us interacting directly with ZK? Reading cluster/replica/collection properties for sure. There are some Solr APIs to modify these things, but afaik users that want to check the value of a particular (e.g.) cluster-property don't have a way to do that yet without reading it out of ZK directly.
On Sun, Feb 11, 2024 at 1:58 PM Eric Pugh <ep...@opensourceconnections.com> wrote: > I like your suggestion David on the CLI supporting configset directly. > I’m sort of waiting till the V2 api for putting configsets is done. > According to the Solr v2 API Proposed Changes spreadsheet, while there is a > V2 api, it hasn’t been given the “restful/jax-rs/exposed in solrj” > treatment yet…. > > I also agree that figuring out how to make ZK an internal detail is > somewhat underlying my question…. > > What other use cases are there for us interacting directly with ZK? > > > On Feb 11, 2024, at 12:57 PM, David Smiley <dsmi...@apache.org> wrote: > > > > I suppose direct ZK has an advantage of not requiring that Solr is > > running yet. But maybe that's moot. I do think it would make sense > > to have CLI options for configset upload that use Solr's API and don't > > refer to ZK. Broadly, this is the direction we're going in; ZK is > > more of an internal detail, even though it's obviously a critical > > thing. > > > > On Sun, Feb 11, 2024 at 12:26 PM Eric Pugh > > <ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com>> > wrote: > >> > >> Ah.. yeah, I can’t speak to Solr 6.x! In 9x at least you could use > the configset API to deploy configs and avoid the direct ZK interaction. > >> > >> It would be interesting to explore if the process of deploying a > configset is risky, has a high chance of things failing, then how do we > account for that as part of the process? So you don’t have to do things > like upload the previous config ;-). > >> > >> And other common reasons to use ZK directly? > >> > >>> On Feb 11, 2024, at 12:14 PM, Walter Underwood <wun...@wunderwood.org> > wrote: > >>> > >>> The was deploying configs with Jenkins on Solr 6.x. Maybe the APIs > were there, but I didn't know about them. > >>> > >>> Rebuilding the suggester did need external help, since that needs to > be done separately on each node. > >>> > >>> I think working directly with Zookeeper is less risky. If there is any > issue with the upload, then don’t reload the collections. You can back out > the changes by uploading the previous config to Zookeeper. > >>> > >>> wunder > >>> Walter Underwood > >>> wun...@wunderwood.org <mailto:wun...@wunderwood.org> <mailto: > wun...@wunderwood.org> > >>> http://observer.wunderwood.org/ (my blog) > >>> > >>>> On Feb 11, 2024, at 11:07 AM, Eric Pugh < > ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com> > <mailto:ep...@opensourceconnections.com>> wrote: > >>>> > >>>> Could you share more about “update Solr remotely” that you were > doing? Are we missing some APIs that would have made whatever you had to > do require ZK direct access? > >>>> > >>>> While it’s cool that we can impact Solr via hacking around in ZK, it > also seems like an approach fraught with risk! > >>>> > >>>>> On Feb 11, 2024, at 11:32 AM, Walter Underwood < > wun...@wunderwood.org <mailto:wun...@wunderwood.org>> wrote: > >>>>> > >>>>> I wanted something that didn’t require installing Solr locally in > order to update Solr remotely, so I didn’t use the provided zk commands. I > wrote some Python to dig the Zookeeper addresses out of clusterstatus (I > think) then uploaded directly to Zookeeper with the Python kazoo package. > >>>>> > >>>>> The tool had a bunch of other things, like async reload checking for > results, and rebuilding suggestion dictionaries on each node. > >>>>> > >>>>> wunder > >>>>> Walter Underwood > >>>>> wun...@wunderwood.org <mailto:wun...@wunderwood.org> > >>>>> http://observer.wunderwood.org/ (my blog) > >>>>> > >>>>>> On Feb 11, 2024, at 9:04 AM, Gus Heck <gus.h...@gmail.com <mailto: > gus.h...@gmail.com>> wrote: > >>>>>> > >>>>>> I pretty much always use zk upconfig, which also works for > overwriting > >>>>>> existing. I certainly tell my clients to use apis from the ref > guide for > >>>>>> such operations, but zk upconfig certainly counts as one. Mostly I > tell > >>>>>> them that they should only break out things like > >>>>>> https://github.com/rgs1/zk_shell as a last resort (which is what I > think of > >>>>>> as direct modification), and if they are unsure, call me *before* > doing > >>>>>> anything in zk directly. > >>>>>> > >>>>>> By the way, I don't know if this has come up in a dev/build setting > or not, > >>>>>> but are you aware of https://plugins.gradle.org/search?term=solr ? > It is > >>>>>> presently only really suitable for local dev, with a single config > set, but > >>>>>> could easily grow patches and suggestions welcome of course. > >>>>>> > >>>>>> On Sun, Feb 11, 2024, 9:10 AM Eric Pugh < > ep...@opensourceconnections.com <mailto:ep...@opensourceconnections.com>> > >>>>>> wrote: > >>>>>> > >>>>>>> Hi all.. I was playing around with a cluster and wanted to > upload a > >>>>>>> configset into Solr…. > >>>>>>> > >>>>>>> I ran bin/solr and noticed a bin/solr config -h command, but it > just lets > >>>>>>> me tweak a config. Then I ran bin/solr create -h and it appears > to let me > >>>>>>> upload a configset, but I have to create the collection as well, > and I’m > >>>>>>> not ready to do that. > >>>>>>> > >>>>>>> Then I poked around and discovered hidden under bin/solr zk a > command > >>>>>>> upconfig…. So bin/solr zk upconfig will let me get my configset > into Solr, > >>>>>>> but does require me to remember what my magic ZK string is ;-). > >>>>>>> > >>>>>>> I went and checked the ref guide, and yes, it states that there > are two > >>>>>>> ways: > >>>>>>> > >>>>>>> A configset can be uploaded to ZooKeeper either via the Configsets > API < > >>>>>>> > https://solr.apache.org/guide/solr/latest/configuration-guide/configsets-api.html > > > >>>>>>> or more directly via bin/solr zk upconfig < > >>>>>>> > https://solr.apache.org/guide/solr/latest/deployment-guide/solr-control-script-reference.html#upload-a-configuration-set > >. > >>>>>>> The Configsets API has some other operations as well, and > likewise, so does > >>>>>>> the CLI. > >>>>>>> > >>>>>>> Are there use cases where interacting directly with ZooKeeper is > preferred > >>>>>>> over making changes via the APIs? Of is the use of bin/solr zk > upconfig > >>>>>>> more of a evolutionary byproduct of how we built SolrCloud? > >>>>>>> > >>>>>>> Eric > >>>>>>> > >>>>>>> _______________________ > >>>>>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > 434.466.1467 | > >>>>>>> http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > >>>>>>> http://www.opensourceconnections.com/> | My Free/Busy < > >>>>>>> http://tinyurl.com/eric-cal> > >>>>>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > >>>>>>> > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > >>>>>>> > >>>>>>> This e-mail and all contents, including attachments, is considered > to be > >>>>>>> Company Confidential unless explicitly stated otherwise, > regardless of > >>>>>>> whether attachments are marked as such. > >>>>>>> > >>>>>>> > >>>>> > >>>> > >>>> _______________________ > >>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | > 434.466.1467 | http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > http://www.opensourceconnections.com/> < > http://www.opensourceconnections.com/> | My Free/Busy < > http://tinyurl.com/eric-cal> > >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > >>>> This e-mail and all contents, including attachments, is considered to > be Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > >> > >> _______________________ > >> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 > | http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> < > http://www.opensourceconnections.com/> | My Free/Busy < > http://tinyurl.com/eric-cal> > >> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw > > > >> This e-mail and all contents, including attachments, is considered to > be Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org <mailto: > dev-unsubscr...@solr.apache.org> > > For additional commands, e-mail: dev-h...@solr.apache.org <mailto: > dev-h...@solr.apache.org> > _______________________ > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | > http://www.opensourceconnections.com < > http://www.opensourceconnections.com/> | My Free/Busy < > http://tinyurl.com/eric-cal> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed < > https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw> > > This e-mail and all contents, including attachments, is considered to be > Company Confidential unless explicitly stated otherwise, regardless of > whether attachments are marked as such. > >
