Shuffling Tuples - Parallel SQL
Hi, I am reading on Solr’s Parallel SQL. How each replica partition the results by using HashQParser plugin and shuffle the tuples with same values in the partition key field to same worker node ? How do we know the partition key field values? Thanks, Swathi.
How to check if solr cores have any issues and the node needs to be restarted
Hello Team, I am trying to restart a node that has issues with the cores. I was hoping to use CORE STATUS api and check whether it has any cores listed in the initFailures or not. Since all such cores do not respond to ping and receive traffic, I want to put effort into finding out the reason from the stack trace of those cores and verify whether the node needs to be restarted. What kind of errors of the core would be shown by using the CORE status API ? Thanks.
Re: Solr Collection Create API queries
Thank you Anshum. I would try the approach of managing it from outside first and see how it works. On Fri, Sep 9, 2016 at 1:51 PM, Anshum Gupta <ans...@anshumgupta.net> wrote: > If you want to build a monitoring tool that maintains a replication factor, > I would suggest you use the Collections APIs (ClusterStatus, AddReplica, > DeleteReplica, etc.) and manage this from outside of Solr. I don't want to > pull you back from trying to build something but I think you'd be biting a > lot for the first bite if you take this up as the first thing to implement > within Solr. > > > On Fri, Sep 9, 2016 at 1:41 PM Swathi Singamsetty < > swathisingamsett...@gmail.com> wrote: > > > I am experimenting on this functionality and see how the overseer > monitors > > and keeps the minimum no of replicas up and running. > > > > > > In heavy indexing/search flow , if any replica goes down we need to keep > > the minimum no. of replicas up and running to serve the traffic and > > mainitain the availability of the cluster. > > > > > > Please let me know if you need more information. > > > > Can you point me to the git repo branch where I can dig deeper and see > this > > functionality ? > > > > > > > > Thanks, > > Swathi. > > > > > > > > > > > > On Fri, Sep 9, 2016 at 1:10 PM, Anshum Gupta <ans...@anshumgupta.net> > > wrote: > > > > > Just to clarify here, I said that I really think it's an XY problem > > here. I > > > still don't know what is being attempted/built. > > > > > > From the last email, sounds like you want to build/support > auto-addition > > of > > > replica but I would wait until you clarify the use case to suggest > > > anything. > > > > > > -Anshum > > > > > > On Fri, Sep 9, 2016 at 8:20 AM Erick Erickson <erickerick...@gmail.com > > > > > wrote: > > > > > > > I think you're missing my point. The _feature_ may be there, > > > > you'll have to investigate. But it is not named "smartCloud" or > > > > "autoManageCluster". Those terms > > > > 1> do not appear in the final patch. > > > > 2> do not appear in any file in Solr 6x. > > > > > > > > They were suggested names, what the final implementation > > > > used should be in the ref guide, although I admit this latter > > > > sometimes lags. > > > > > > > > Best, > > > > Erick > > > > > > > > On Fri, Sep 9, 2016 at 7:51 AM, Swathi Singamsetty > > > > <swathisingamsett...@gmail.com> wrote: > > > > > I am working on solr 6.0.0 to implement this feature. > > > > > I had a chat with Anshum and confirmed that this feature is > available > > > in > > > > > 6.0.0 version. > > > > > > > > > > > > > > > The functionality is that to allow the overseer to bring up > > > > > the minimum no. of replicas for each shard as per the > > > replicationFactor > > > > > set. > > > > > > > > > > I will look into the ref guide as well. > > > > > > > > > > Thanks, > > > > > Swathi. > > > > > > > > > > On Friday, September 9, 2016, Erick Erickson < > > erickerick...@gmail.com> > > > > > wrote: > > > > > > > > > >> You cannot just pick arbitrary parts of a JIRA discussion > > > > >> and expect them to work. JIRAs are places where > > > > >> discussion of alternatives takes place and the discussion > > > > >> often suggests ideas that are not incorporated > > > > >> in the final patch. The patch for the JIRA you mentioned, > > > > >> for instance, does not mention either of those parameters, > > > > >> which implies that they were simply part of the discussion > > > > >> and were never implemented. > > > > >> > > > > >> So this sounds like an "XY" problem. You're asking why > > > > >> properties aren't persisted when you really want to take > > > > >> advantage of some functionality. What is that functionality? > > > > >> > > > > >> BTW, I'd go by the ref guide rather than JIRAs unless you > > > > >> examine the patch and see that the discussion was > > > > >> implemented in the patch. > > > > >> > > > > >> Best, > > > > >> Erick > > > > >> > > > > >> On Thu, Sep 8, 2016 at 9:33 PM, Swathi Singamsetty > > > > >> <swathisingamsett...@gmail.com <javascript:;>> wrote: > > > > >> > Hi Team, > > > > >> > > > > > >> > To implement the feature "Persist and use the > > > > >> > replicationFactor,maxShardsPerNode at Collection level" > am > > > > >> following > > > > >> > the steps mentioned in the jira ticket > > > > >> > https://issues.apache.org/jira/browse/SOLR-4808. > > > > >> > > > > > >> > I used the "smartCloud" and "autoManageCluster" properties to > > > create a > > > > >> > collection in the create collection API to allow the overseer to > > > > bring up > > > > >> > the minimum no. of replicas for each shard as per the > > > > replicationFactor > > > > >> set > > > > >> > . But these 2 properties did not persist in the cluster state. > > Could > > > > >> > someone let me know how to use these properties in this feature? > > > > >> > > > > > >> > > > > > >> > > > > > >> > Thanks & Regards, > > > > >> > Swathi. > > > > >> > > > > > > > > > >
Re: Solr Collection Create API queries
I am experimenting on this functionality and see how the overseer monitors and keeps the minimum no of replicas up and running. In heavy indexing/search flow , if any replica goes down we need to keep the minimum no. of replicas up and running to serve the traffic and mainitain the availability of the cluster. Please let me know if you need more information. Can you point me to the git repo branch where I can dig deeper and see this functionality ? Thanks, Swathi. On Fri, Sep 9, 2016 at 1:10 PM, Anshum Gupta <ans...@anshumgupta.net> wrote: > Just to clarify here, I said that I really think it's an XY problem here. I > still don't know what is being attempted/built. > > From the last email, sounds like you want to build/support auto-addition of > replica but I would wait until you clarify the use case to suggest > anything. > > -Anshum > > On Fri, Sep 9, 2016 at 8:20 AM Erick Erickson <erickerick...@gmail.com> > wrote: > > > I think you're missing my point. The _feature_ may be there, > > you'll have to investigate. But it is not named "smartCloud" or > > "autoManageCluster". Those terms > > 1> do not appear in the final patch. > > 2> do not appear in any file in Solr 6x. > > > > They were suggested names, what the final implementation > > used should be in the ref guide, although I admit this latter > > sometimes lags. > > > > Best, > > Erick > > > > On Fri, Sep 9, 2016 at 7:51 AM, Swathi Singamsetty > > <swathisingamsett...@gmail.com> wrote: > > > I am working on solr 6.0.0 to implement this feature. > > > I had a chat with Anshum and confirmed that this feature is available > in > > > 6.0.0 version. > > > > > > > > > The functionality is that to allow the overseer to bring up > > > the minimum no. of replicas for each shard as per the > replicationFactor > > > set. > > > > > > I will look into the ref guide as well. > > > > > > Thanks, > > > Swathi. > > > > > > On Friday, September 9, 2016, Erick Erickson <erickerick...@gmail.com> > > > wrote: > > > > > >> You cannot just pick arbitrary parts of a JIRA discussion > > >> and expect them to work. JIRAs are places where > > >> discussion of alternatives takes place and the discussion > > >> often suggests ideas that are not incorporated > > >> in the final patch. The patch for the JIRA you mentioned, > > >> for instance, does not mention either of those parameters, > > >> which implies that they were simply part of the discussion > > >> and were never implemented. > > >> > > >> So this sounds like an "XY" problem. You're asking why > > >> properties aren't persisted when you really want to take > > >> advantage of some functionality. What is that functionality? > > >> > > >> BTW, I'd go by the ref guide rather than JIRAs unless you > > >> examine the patch and see that the discussion was > > >> implemented in the patch. > > >> > > >> Best, > > >> Erick > > >> > > >> On Thu, Sep 8, 2016 at 9:33 PM, Swathi Singamsetty > > >> <swathisingamsett...@gmail.com <javascript:;>> wrote: > > >> > Hi Team, > > >> > > > >> > To implement the feature "Persist and use the > > >> > replicationFactor,maxShardsPerNode at Collection level" am > > >> following > > >> > the steps mentioned in the jira ticket > > >> > https://issues.apache.org/jira/browse/SOLR-4808. > > >> > > > >> > I used the "smartCloud" and "autoManageCluster" properties to > create a > > >> > collection in the create collection API to allow the overseer to > > bring up > > >> > the minimum no. of replicas for each shard as per the > > replicationFactor > > >> set > > >> > . But these 2 properties did not persist in the cluster state. Could > > >> > someone let me know how to use these properties in this feature? > > >> > > > >> > > > >> > > > >> > Thanks & Regards, > > >> > Swathi. > > >> > > >
Re: Solr Collection Create API queries
I am working on solr 6.0.0 to implement this feature. I had a chat with Anshum and confirmed that this feature is available in 6.0.0 version. The functionality is that to allow the overseer to bring up the minimum no. of replicas for each shard as per the replicationFactor set. I will look into the ref guide as well. Thanks, Swathi. On Friday, September 9, 2016, Erick Erickson <erickerick...@gmail.com> wrote: > You cannot just pick arbitrary parts of a JIRA discussion > and expect them to work. JIRAs are places where > discussion of alternatives takes place and the discussion > often suggests ideas that are not incorporated > in the final patch. The patch for the JIRA you mentioned, > for instance, does not mention either of those parameters, > which implies that they were simply part of the discussion > and were never implemented. > > So this sounds like an "XY" problem. You're asking why > properties aren't persisted when you really want to take > advantage of some functionality. What is that functionality? > > BTW, I'd go by the ref guide rather than JIRAs unless you > examine the patch and see that the discussion was > implemented in the patch. > > Best, > Erick > > On Thu, Sep 8, 2016 at 9:33 PM, Swathi Singamsetty > <swathisingamsett...@gmail.com <javascript:;>> wrote: > > Hi Team, > > > > To implement the feature "Persist and use the > > replicationFactor,maxShardsPerNode at Collection level" am > following > > the steps mentioned in the jira ticket > > https://issues.apache.org/jira/browse/SOLR-4808. > > > > I used the "smartCloud" and "autoManageCluster" properties to create a > > collection in the create collection API to allow the overseer to bring up > > the minimum no. of replicas for each shard as per the replicationFactor > set > > . But these 2 properties did not persist in the cluster state. Could > > someone let me know how to use these properties in this feature? > > > > > > > > Thanks & Regards, > > Swathi. >
Solr Collection Create API queries
Hi Team, To implement the feature "Persist and use the replicationFactor,maxShardsPerNode at Collection level" am following the steps mentioned in the jira ticket https://issues.apache.org/jira/browse/SOLR-4808. I used the "smartCloud" and "autoManageCluster" properties to create a collection in the create collection API to allow the overseer to bring up the minimum no. of replicas for each shard as per the replicationFactor set . But these 2 properties did not persist in the cluster state. Could someone let me know how to use these properties in this feature? Thanks & Regards, Swathi.
