RE: Solr hardware memory question

2013-12-12 Thread Hoggarth, Gil
r@lucene.apache.org Subject: Re: Solr hardware memory question On Tue, 2013-12-10 at 17:51 +0100, Hoggarth, Gil wrote: > We're probably going to be building a Solr service to handle a dataset > of ~60TB, which for our data and schema typically gives a Solr index > size of 1/10th - i.e

RE: Solr hardware memory question

2013-12-10 Thread Hoggarth, Gil
rag.org] Sent: 10 December 2013 17:37 To: solr-user@lucene.apache.org Subject: Re: Solr hardware memory question On 12/10/2013 9:51 AM, Hoggarth, Gil wrote: > We're probably going to be building a Solr service to handle a dataset > of ~60TB, which for our data and schema typically gives

Solr hardware memory question

2013-12-10 Thread Hoggarth, Gil
We're probably going to be building a Solr service to handle a dataset of ~60TB, which for our data and schema typically gives a Solr index size of 1/10th - i.e., 6TB. Given there's a general rule about the amount of hardware memory required should exceed the size of the Solr index (exceed to also

RE: How to work with remote solr savely?

2013-11-22 Thread Hoggarth, Gil
You could also use one of the proxy scripts, such as http://code.google.com/p/solr-php-client/, which is coincidentally linked (eventually) from Michael's suggested SolrSecurity URL. -Original Message- From: michael.boom [mailto:my_sky...@yahoo.com] Sent: 22 November 2013 14:53 To: solr-u

RE: How to work with remote solr savely?

2013-11-22 Thread Hoggarth, Gil
We solved this issue outside of Solr. As you've done, restrict the server to localhost access to Solr, add firewall rules to allow your developers on port 80, and proxypass allowed port 80 transfer to Solr. Remember to include the proxypassreverse too. (This runs on linux and apache httpd btw.) --

RE: Why do people want to deploy to Tomcat?

2013-11-12 Thread Hoggarth, Gil
For me, a side-affect of 'example' is that it's just that, not appropriate for production. But also, there's the organisation factor beyond Solr that is about staff expertise - we don't have any systems that utilise jetty so we're unfamiliar with its configuration, issues, or oddities. Tomcat is

RE: How to cancel a collection 'optimize'?

2013-11-11 Thread Hoggarth, Gil
his might do to the index, but maybe somebody else can jump in and comment on this approach or suggest a better one. Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Mon, Nov 11, 2013 at 10:44 AM, Hoggarth, Gil wrote: >

How to cancel a collection 'optimize'?

2013-11-11 Thread Hoggarth, Gil
We have an internal Solr collection with ~1 billion documents. It's split across 24 shards and uses ~3.2TB of disk space. Unfortunately we've triggered an 'optimize' on the collection (via a restarted browser tab), which has raised the disk usage to 4.6TB, with 130GB left on the disk volume. As

RE: New shard leaders or existing shard replicas depends on zookeeper?

2013-10-24 Thread Hoggarth, Gil
lose the other Solr collection information.) Thanks in advance, Gil -----Original Message- From: Hoggarth, Gil [mailto:gil.hogga...@bl.uk] Sent: 24 October 2013 10:13 To: solr-user@lucene.apache.org Subject: RE: New shard leaders or existing shard replicas depends on zookeeper? Absolutely, the sc

RE: New shard leaders or existing shard replicas depends on zookeeper?

2013-10-24 Thread Hoggarth, Gil
uess is that this ZK ensemble has the ldwa01 collection defined as having only one shard I admit I pretty much skimmed your post though... Best, Erick On Wed, Oct 23, 2013 at 12:54 PM, Hoggarth, Gil wrote: > Hi solr-users, > > > > I'm seeing some confusing behaviour in Solr/

New shard leaders or existing shard replicas depends on zookeeper?

2013-10-23 Thread Hoggarth, Gil
Hi solr-users, I'm seeing some confusing behaviour in Solr/zookeeper and hope you can shed some light on what's happening/how I can correct it. We have two physical servers running automated builds of RedHat 6.4 and Solr 4.4.0 that host two separate Solr services. The first server (called l

RE: Solr 4.3.0: Shard instances using incorrect data directory on machine boot

2013-05-16 Thread Hoggarth, Gil
Thanks for your response Shawn, very much appreciated. Gil -Original Message- From: Shawn Heisey [mailto:s...@elyograg.org] Sent: 16 May 2013 15:59 To: solr-user@lucene.apache.org Subject: RE: Solr 4.3.0: Shard instances using incorrect data directory on machine boot > The dataDir is set

RE: Solr 4.3.0: Shard instances using incorrect data directory on machine boot

2013-05-16 Thread Hoggarth, Gil
understand it, datatDir is set from the solrconfig.xml file, so either your instances are picking up the "wrong" file, or you have some override which is incorrect? Where do you set solr.data.dir, at the environment when you start Solr or in solrconfig? On 16 May 2013 12:23, Hoggarth, Gil

Solr 4.3.0: Shard instances using incorrect data directory on machine boot

2013-05-16 Thread Hoggarth, Gil
Hi all, I hope you can advise a solution to our incorrect data directory issue. We have 2 physical servers using Solr 4.3.0, each with 24 separate tomcat instances (RedHat 6.4, java 1.7.0_10-b18, tomcat 7.0.34) with a solr shard in each. This configuration means that each shard has its own data