Re: Accessing Solr collections at different ports

[Erick Erickson]

This is not true. You can run as many separate JVMs on a single physical 
machine as you have available ports. There’s no capability to address a Solr 
_collection_ in the _same_ JVM by a different port though.

But what you didn’t mention is having separate collections per client. A single 
Solr instance (defined by a running JVM) can host a number of different 
collections. So you have URLS like
http://some_sever:port/solr/collection1/query
http://some_sever:port/solr/collection2/query

Now you restrict the URLs available to client1 to collection1, client2 to 
collection2 etc.

Best,
Erick

> On May 3, 2019, at 1:47 AM, Salmaan Rashid Syed  
> wrote:
> 
> Solr in cloud mode with 4 nodes (max as limited by Solr) at 4 different ports

Re: Accessing Solr collections at different ports - Need help

[Shawn Heisey]

On 5/3/2019 12:52 AM, Salmaan Rashid Syed wrote:

I say that the nodes are limited to 4 because when I launch Solr in cloud
mode, the first prompt that I get is to choose number of nodes [1-4]. When
I tried to enter 7, it says that they are more than 4 and choose a smaller
number.


That's the cloud *EXAMPLE*.  It sets everything up on one server that 
would normally be on separate servers, and runs an embedded zookeeper in 
the first node.


Example setups are not meant for production.

Thanks,
Shawn

Re: Accessing Solr collections at different ports

[Shawn Heisey]

On 5/2/2019 11:47 PM, Salmaan Rashid Syed wrote:

I am using Solr 7.6 in cloud mode with external zookeeper installed at ports 
2181, 2182, 2183. Currently we have only one server allocated for Solr. We are 
planning to move to multiple servers for better sharing, replication etc in 
near future.

Now the issue is that, our organisation has data indexed for different clients 
as separate collections. We want to uniquely access, update and index each 
collection separately so that each individual client has access to their 
respective collections at their respective ports. Eg:— Collection1 at port 
8983, Collection2 at port 8984, Collection3 at port 8985 etc.


This is not something you can do with a single instance of Solr.


I have two options I guess, one is to run Solr in cloud mode with 4 nodes (max 
as limited by Solr) at 4 different ports. I don’t know how to go beyond 4 
nodes/ports in this case.


There are no limits to the number of nodes.  I know people are running 
SolrCloud clusters with hundreds of nodes.  And there might be some out 
there with thousands ... although if those exist, they're really pushing 
the limits.



The other option is to run Solr as service and create multiple copies of Solr folder 
within the Server folder and access each Solr at different port with its own 
collection as shown by https://www.youtube.com/watch?v=wmQFwK2sujE 



If you have multiple Solr nodes in a single cluster, you can access any 
collection from any node.  This will probably present a security problem 
for you.  Even if the receiving node doesn't have any of the 
collection's data, SolrCloud will proxy the connection over to the nodes 
that DO have that data.


If you give each Solr node (instance) a different chroot on its zkHost 
string, then each one would be a completely separate cluster from all 
the others, and you can run them all in one zookeeper ensemble.  Each 
one would have zkHost strings that look something like this:


zk1.example.com:2181,zk2.example.com:2181,zk3.example.com/solr1
zk1.example.com:2181,zk2.example.com:2181,zk3.example.com/solr2
zk1.example.com:2181,zk2.example.com:2181,zk3.example.com/solr3
zk1.example.com:2181,zk2.example.com:2181,zk3.example.com/solr4

You'll need to find the section in the documentation that talks about 
creating a chroot in ZK.


I think that would give you what you're after.

It's probably easiest to set up each Solr instance in its own directory, 
not try to run multiple services out of one installation directory.  The 
entire extracted archive is less than 200MB ... tiny by modern standards.


Thanks,
Shawn

Re: Accessing Solr collections at different ports - Need help

[Jörn Franke]

This is just the setup for an experimental cluster (generally it does also not 
make sense to have many instances on the same server). Once you have got more 
experience take a look at 
https://lucene.apache.org/solr/guide/7_7/taking-solr-to-production.html

To see how to set up clusters.

> Am 03.05.2019 um 08:52 schrieb Salmaan Rashid Syed 
> :
> 
> Thanks Jorn for your reply.
> 
> I say that the nodes are limited to 4 because when I launch Solr in cloud
> mode, the first prompt that I get is to choose number of nodes [1-4]. When
> I tried to enter 7, it says that they are more than 4 and choose a smaller
> number.
> 
> 
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
> 
> 
> 
>> On Fri, May 3, 2019 at 12:05 PM Jörn Franke  wrote:
>> 
>> BTW why do you think that SolrCloud is limited to 4 nodes? More are for
>> sure possible.
>> 
>>> Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed <
>> salmaan.ras...@mroads.com>:
>>> 
>>> Hi Solr Users,
>>> 
>>> I am using Solr 7.6 in cloud mode with external zookeeper installed at
>>> ports 2181, 2182, 2183. Currently we have only one server allocated for
>>> Solr. We are planning to move to multiple servers for better sharing,
>>> replication etc in near future.
>>> 
>>> Now the issue is that, our organisation has data indexed for different
>>> clients as separate collections. We want to uniquely access, update and
>>> index each collection separately so that each individual client has
>> access
>>> to their respective collections at their respective ports. Eg:—
>> Collection1
>>> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
>>> 
>>> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
>>> (max as limited by Solr) at 4 different ports. I don’t know how to go
>>> beyond 4 nodes/ports in this case.
>>> 
>>> The other option is to run Solr as service and create multiple copies of
>>> Solr folder within the Server folder and access each Solr at different
>> port
>>> with its own collection as shown by
>>> https://www.youtube.com/watch?v=wmQFwK2sujE
>>> 
>>> I am really confused as to which is the better path to choose. Please
>> help
>>> me out.
>>> 
>>> Thanks.
>>> 
>>> Regards,
>>> Salmaan
>>> 
>>> 
>>> *Thanks and Regards,*
>>> Salmaan Rashid Syed
>>> +91 8978353445 | www.panna.ai |
>>> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
>>> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>> 

Re: Accessing Solr collections at different ports - Need help

[Salmaan Rashid Syed]

Thanks Jorn for your reply.

I say that the nodes are limited to 4 because when I launch Solr in cloud
mode, the first prompt that I get is to choose number of nodes [1-4]. When
I tried to enter 7, it says that they are more than 4 and choose a smaller
number.


*Thanks and Regards,*
Salmaan Rashid Syed
+91 8978353445 | www.panna.ai |
5550 Granite Pkwy, Suite #225, Plano TX-75024.
Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.



On Fri, May 3, 2019 at 12:05 PM Jörn Franke  wrote:

> BTW why do you think that SolrCloud is limited to 4 nodes? More are for
> sure possible.
>
> > Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed <
> salmaan.ras...@mroads.com>:
> >
> > Hi Solr Users,
> >
> > I am using Solr 7.6 in cloud mode with external zookeeper installed at
> > ports 2181, 2182, 2183. Currently we have only one server allocated for
> > Solr. We are planning to move to multiple servers for better sharing,
> > replication etc in near future.
> >
> > Now the issue is that, our organisation has data indexed for different
> > clients as separate collections. We want to uniquely access, update and
> > index each collection separately so that each individual client has
> access
> > to their respective collections at their respective ports. Eg:—
> Collection1
> > at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> >
> > I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> > (max as limited by Solr) at 4 different ports. I don’t know how to go
> > beyond 4 nodes/ports in this case.
> >
> > The other option is to run Solr as service and create multiple copies of
> > Solr folder within the Server folder and access each Solr at different
> port
> > with its own collection as shown by
> > https://www.youtube.com/watch?v=wmQFwK2sujE
> >
> > I am really confused as to which is the better path to choose. Please
> help
> > me out.
> >
> > Thanks.
> >
> > Regards,
> > Salmaan
> >
> >
> > *Thanks and Regards,*
> > Salmaan Rashid Syed
> > +91 8978353445 | www.panna.ai |
> > 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> > Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>

Re: Accessing Solr collections at different ports - Need help

[Salmaan Rashid Syed]

Thanks Walter,

Since I am new to Solr and by looking at your suggestion, it looks like I
am trying to do something very complicated and out-of-box capabilities of
Solr. I really don't want to do that.

I am not from Computer Science background and my specialisation is in
Analytics and AI.

Let me put my case scenario briefly.

We have developed a customised Solr-search engine that can search for data
(prepared, cleaned and preprocessed by us) in each individual Solr
collection.

Every client of ours is from a different vertical (like health,
engineering, public services, finance, casual works etc). They search for
data in their respective Solr collection. They also add, update and
re-index their respective data periodically.

As suggested by you, if I port-out all the collections from a single port,
will not the latency increase, wil not the burden on a single server
increase, will not the computational speed slows down as all the clients
are trying to speak to the same port simultaneously.

Or do you think that Solr-as-service is better option, where I can create
multiple Solr instances at different ports with collections of individual
clients in each solr instance.

To be honest, I really don't know what Solr-as-service is really trying to
accomplish.

Apologies for lengthy question and Thanks in advance.


*Thanks and Regards,*
Salmaan Rashid Syed
+91 8978353445 | www.panna.ai |
5550 Granite Pkwy, Suite #225, Plano TX-75024.
Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.



On Fri, May 3, 2019 at 11:59 AM Walter Underwood 
wrote:

> The best option is to run all the collections at the same port.
> Intra-cluster communication cannot be split over multiple ports, so this
> would require big internal changes to Solr. And what about communication
> that does not belong to a collection, like electing an overseer node?
>
> Why do you want the very non-standard configuration?
>
> If you must have it, run a webserver like nginx on each node, configure it
> to do this crazy multiple port thing for external traffic and to forward
> all traffic to Solr’s single port.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
> > On May 3, 2019, at 7:54 AM, Salmaan Rashid Syed <
> salmaan.ras...@mroads.com> wrote:
> >
> > Hi Solr Users,
> >
> > I am using Solr 7.6 in cloud mode with external zookeeper installed at
> > ports 2181, 2182, 2183. Currently we have only one server allocated for
> > Solr. We are planning to move to multiple servers for better sharing,
> > replication etc in near future.
> >
> > Now the issue is that, our organisation has data indexed for different
> > clients as separate collections. We want to uniquely access, update and
> > index each collection separately so that each individual client has
> access
> > to their respective collections at their respective ports. Eg:—
> Collection1
> > at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> >
> > I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> > (max as limited by Solr) at 4 different ports. I don’t know how to go
> > beyond 4 nodes/ports in this case.
> >
> > The other option is to run Solr as service and create multiple copies of
> > Solr folder within the Server folder and access each Solr at different
> port
> > with its own collection as shown by
> > https://www.youtube.com/watch?v=wmQFwK2sujE
> >
> > I am really confused as to which is the better path to choose. Please
> help
> > me out.
> >
> > Thanks.
> >
> > Regards,
> > Salmaan
> >
> >
> > *Thanks and Regards,*
> > Salmaan Rashid Syed
> > +91 8978353445 | www.panna.ai |
> > 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> > Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.
>
>

Re: Accessing Solr collections at different ports - Need help

[Jörn Franke]

BTW why do you think that SolrCloud is limited to 4 nodes? More are for sure 
possible.

> Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed 
> :
> 
> Hi Solr Users,
> 
> I am using Solr 7.6 in cloud mode with external zookeeper installed at
> ports 2181, 2182, 2183. Currently we have only one server allocated for
> Solr. We are planning to move to multiple servers for better sharing,
> replication etc in near future.
> 
> Now the issue is that, our organisation has data indexed for different
> clients as separate collections. We want to uniquely access, update and
> index each collection separately so that each individual client has access
> to their respective collections at their respective ports. Eg:— Collection1
> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> 
> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> (max as limited by Solr) at 4 different ports. I don’t know how to go
> beyond 4 nodes/ports in this case.
> 
> The other option is to run Solr as service and create multiple copies of
> Solr folder within the Server folder and access each Solr at different port
> with its own collection as shown by
> https://www.youtube.com/watch?v=wmQFwK2sujE
> 
> I am really confused as to which is the better path to choose. Please help
> me out.
> 
> Thanks.
> 
> Regards,
> Salmaan
> 
> 
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.

Re: Accessing Solr collections at different ports - Need help

[Jörn Franke]

You can have dedicarse clusters per Client and/or you can protect it via 
Kerberos or Basic Auth or write your own authorization plugin based on OAuth.

I am not sure why you want to offer this on different ports to different 
clients.

> Am 03.05.2019 um 07:54 schrieb Salmaan Rashid Syed 
> :
> 
> Hi Solr Users,
> 
> I am using Solr 7.6 in cloud mode with external zookeeper installed at
> ports 2181, 2182, 2183. Currently we have only one server allocated for
> Solr. We are planning to move to multiple servers for better sharing,
> replication etc in near future.
> 
> Now the issue is that, our organisation has data indexed for different
> clients as separate collections. We want to uniquely access, update and
> index each collection separately so that each individual client has access
> to their respective collections at their respective ports. Eg:— Collection1
> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> 
> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> (max as limited by Solr) at 4 different ports. I don’t know how to go
> beyond 4 nodes/ports in this case.
> 
> The other option is to run Solr as service and create multiple copies of
> Solr folder within the Server folder and access each Solr at different port
> with its own collection as shown by
> https://www.youtube.com/watch?v=wmQFwK2sujE
> 
> I am really confused as to which is the better path to choose. Please help
> me out.
> 
> Thanks.
> 
> Regards,
> Salmaan
> 
> 
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.

Re: Accessing Solr collections at different ports - Need help

[Walter Underwood]

The best option is to run all the collections at the same port. Intra-cluster 
communication cannot be split over multiple ports, so this would require big 
internal changes to Solr. And what about communication that does not belong to 
a collection, like electing an overseer node?

Why do you want the very non-standard configuration?

If you must have it, run a webserver like nginx on each node, configure it to 
do this crazy multiple port thing for external traffic and to forward all 
traffic to Solr’s single port.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On May 3, 2019, at 7:54 AM, Salmaan Rashid Syed  
> wrote:
> 
> Hi Solr Users,
> 
> I am using Solr 7.6 in cloud mode with external zookeeper installed at
> ports 2181, 2182, 2183. Currently we have only one server allocated for
> Solr. We are planning to move to multiple servers for better sharing,
> replication etc in near future.
> 
> Now the issue is that, our organisation has data indexed for different
> clients as separate collections. We want to uniquely access, update and
> index each collection separately so that each individual client has access
> to their respective collections at their respective ports. Eg:— Collection1
> at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.
> 
> I have two options I guess, one is to run Solr in cloud mode with 4 nodes
> (max as limited by Solr) at 4 different ports. I don’t know how to go
> beyond 4 nodes/ports in this case.
> 
> The other option is to run Solr as service and create multiple copies of
> Solr folder within the Server folder and access each Solr at different port
> with its own collection as shown by
> https://www.youtube.com/watch?v=wmQFwK2sujE
> 
> I am really confused as to which is the better path to choose. Please help
> me out.
> 
> Thanks.
> 
> Regards,
> Salmaan
> 
> 
> *Thanks and Regards,*
> Salmaan Rashid Syed
> +91 8978353445 | www.panna.ai |
> 5550 Granite Pkwy, Suite #225, Plano TX-75024.
> Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.

Accessing Solr collections at different ports

[Salmaan Rashid Syed]

Hi Solr Users,

I am using Solr 7.6 in cloud mode with external zookeeper installed at ports 
2181, 2182, 2183. Currently we have only one server allocated for Solr. We are 
planning to move to multiple servers for better sharing, replication etc in 
near future.

Now the issue is that, our organisation has data indexed for different clients 
as separate collections. We want to uniquely access, update and index each 
collection separately so that each individual client has access to their 
respective collections at their respective ports. Eg:— Collection1 at port 
8983, Collection2 at port 8984, Collection3 at port 8985 etc.

I have two options I guess, one is to run Solr in cloud mode with 4 nodes (max 
as limited by Solr) at 4 different ports. I don’t know how to go beyond 4 
nodes/ports in this case.

The other option is to run Solr as service and create multiple copies of Solr 
folder within the Server folder and access each Solr at different port with its 
own collection as shown by https://www.youtube.com/watch?v=wmQFwK2sujE 
 

I am really confused as to which is the better path to choose. Please help me 
out.

Thanks.

Regards,
Salmaan

Accessing Solr collections at different ports - Need help

[Salmaan Rashid Syed]

Hi Solr Users,

I am using Solr 7.6 in cloud mode with external zookeeper installed at
ports 2181, 2182, 2183. Currently we have only one server allocated for
Solr. We are planning to move to multiple servers for better sharing,
replication etc in near future.

Now the issue is that, our organisation has data indexed for different
clients as separate collections. We want to uniquely access, update and
index each collection separately so that each individual client has access
to their respective collections at their respective ports. Eg:— Collection1
at port 8983, Collection2 at port 8984, Collection3 at port 8985 etc.

I have two options I guess, one is to run Solr in cloud mode with 4 nodes
(max as limited by Solr) at 4 different ports. I don’t know how to go
beyond 4 nodes/ports in this case.

The other option is to run Solr as service and create multiple copies of
Solr folder within the Server folder and access each Solr at different port
with its own collection as shown by
https://www.youtube.com/watch?v=wmQFwK2sujE

I am really confused as to which is the better path to choose. Please help
me out.

Thanks.

Regards,
Salmaan


*Thanks and Regards,*
Salmaan Rashid Syed
+91 8978353445 | www.panna.ai |
5550 Granite Pkwy, Suite #225, Plano TX-75024.
Cyber Gateways, Hi-tech City, Hyderabad, Telangana, India.