Re: Replicating files containing external file fields
Jack and Erick,
Thanks for your replies. I am able to replicate ext file fields by
specifying the relative paths for each individual file. confFiles in
solrconfig.xml is really long now with lot of ../ and I got 5 ext file
field files. Would be really nice if wild-cards were supported here :-).
About the reloadCache on slave: following
http://docs.lucidworks.com/display/solr/Working+with+External+Files+and+Processes
I
set up listeners to reload the ext file fields after commits. Since the
slave replicationHandler issues a commit after it replicates the files (as
mentioned in
https://wiki.apache.org/solr/SolrReplication#How_does_the_slave_replicate.3F),
I believe the ext file fields get reloaded to the slave cache after
replication. This is exactly what I was looking for.
On Fri, Jun 28, 2013 at 5:08 PM, Jack Krupansky j...@basetechnology.comwrote:
Yes, you need to list that EFF file in the confFiles list - only those
listed files will be replicated.
str
name=confFilessolrconfig.**xml,data-config.xml,schema.**
xml,stopwords.txt,synonyms.**txt,elevate.xml,
/var/solr-data/List/external_***/str
Oops... sorry, no wildcards... you must list the individual files.
Technically, the path is supposed to be relative to the Solr collection
conf directory, so you MAY have you may have to put lots of ../ in the
path to get to the files, like:
../../../../solr-data/List/**external_1
Tor each file.
(This is what Erick was referring to.)
Sorry, I don't have the answer to the reload question at the tip of my
tongue.
-- Jack Krupansky
-Original Message- From: Arun Rangarajan
Sent: Friday, June 28, 2013 7:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Replicating files containing external file fields
Jack,
Here is the ReplicationHandler definition from solrconfig.xml:
requestHandler name=/replication class=solr.**ReplicationHandler
lst name=master
str name=enable${enable.master:**false}/str
str name=replicateAfterstartup**/str
str name=replicateAftercommit/**str
str name=replicateAfter**optimize/str
str
name=confFilessolrconfig.**xml,data-config.xml,schema.**
xml,stopwords.txt,synonyms.**txt,elevate.xml/str
/lst
lst name=slave
str name=enable${enable.slave:**false}/str
str name=masterUrlhttp://${**master.ip}:${master.port}/**solr/${
solr.core.name}/replication/**str
str name=pollInterval00:01:00/**str
/lst
/requestHandler
The confFiles are under the dir:
/var/solr/application-cores/**List/conf
and the external file fields are like:
/var/solr-data/List/external_*
Should I add
/var/solr-data/List/external_*
to confFiles like this?
str
name=confFilessolrconfig.**xml,data-config.xml,schema.**
xml,stopwords.txt,synonyms.**txt,elevate.xml,
/var/solr-data/List/external_***/str
Also, can you tell me when (or whether) I need to do reloadCache on the
slave after the ext file fields are replicated?
Thx.
On Fri, Jun 28, 2013 at 10:13 AM, Jack Krupansky j...@basetechnology.com
**wrote:
Show us your confFiles directive. Maybe there is some subtle error in
the file name.
-- Jack Krupansky
-Original Message- From: Arun Rangarajan
Sent: Friday, June 28, 2013 1:06 PM
To: solr-user@lucene.apache.org
Subject: Re: Replicating files containing external file fields
Erick,
Thx for your reply. The external file field fields are already under
dataDir specified in solrconfig.xml. They are not getting replicated.
(Solr version 4.2.1.)
On Thu, Jun 27, 2013 at 10:50 AM, Erick Erickson erickerick...@gmail.com
**wrote:
Haven't tried this, but I _think_ you can use the
confFiles trick with relative paths, see:
http://wiki.apache.org/solr/SolrReplicationhttp://wiki.apache.org/solr/**SolrReplication
http://wiki.**apache.org/solr/**SolrReplicationhttp://wiki.apache.org/solr/SolrReplication
Or just put your EFF files in the data dir?
Best
Erick
On Wed, Jun 26, 2013 at 9:01 PM, Arun Rangarajan
arunrangara...@gmail.comwrote:
From
https://wiki.apache.org/solr/SolrReplicationhttps://wiki.apache.org/solr/**SolrReplication
https://wiki.**apache.org/solr/**SolrReplicationhttps://wiki.apache.org/solr/SolrReplicationI
understand that
index
dir and any files under the conf dir can be replicated to slaves. I
want
to
know if there is any way the files under the data dir containing
external
file fields can be replicated. These are not replicated by default.
Currently we are running the ext file field reload script on both the
master and the slave and then running reloadCache on each server once
they
are loaded.
Re: Replicating files containing external file fields
Erick, Thx for your reply. The external file field fields are already under dataDir specified in solrconfig.xml. They are not getting replicated. (Solr version 4.2.1.) On Thu, Jun 27, 2013 at 10:50 AM, Erick Erickson erickerick...@gmail.comwrote: Haven't tried this, but I _think_ you can use the confFiles trick with relative paths, see: http://wiki.apache.org/solr/SolrReplication Or just put your EFF files in the data dir? Best Erick On Wed, Jun 26, 2013 at 9:01 PM, Arun Rangarajan arunrangara...@gmail.comwrote: From https://wiki.apache.org/solr/SolrReplication I understand that index dir and any files under the conf dir can be replicated to slaves. I want to know if there is any way the files under the data dir containing external file fields can be replicated. These are not replicated by default. Currently we are running the ext file field reload script on both the master and the slave and then running reloadCache on each server once they are loaded.
Re: Replicating files containing external file fields
Show us your confFiles directive. Maybe there is some subtle error in the file name. -- Jack Krupansky -Original Message- From: Arun Rangarajan Sent: Friday, June 28, 2013 1:06 PM To: solr-user@lucene.apache.org Subject: Re: Replicating files containing external file fields Erick, Thx for your reply. The external file field fields are already under dataDir specified in solrconfig.xml. They are not getting replicated. (Solr version 4.2.1.) On Thu, Jun 27, 2013 at 10:50 AM, Erick Erickson erickerick...@gmail.comwrote: Haven't tried this, but I _think_ you can use the confFiles trick with relative paths, see: http://wiki.apache.org/solr/SolrReplication Or just put your EFF files in the data dir? Best Erick On Wed, Jun 26, 2013 at 9:01 PM, Arun Rangarajan arunrangara...@gmail.comwrote: From https://wiki.apache.org/solr/SolrReplication I understand that index dir and any files under the conf dir can be replicated to slaves. I want to know if there is any way the files under the data dir containing external file fields can be replicated. These are not replicated by default. Currently we are running the ext file field reload script on both the master and the slave and then running reloadCache on each server once they are loaded.
Re: Replicating files containing external file fields
Jack,
Here is the ReplicationHandler definition from solrconfig.xml:
requestHandler name=/replication class=solr.ReplicationHandler
lst name=master
str name=enable${enable.master:false}/str
str name=replicateAfterstartup/str
str name=replicateAftercommit/str
str name=replicateAfteroptimize/str
str
name=confFilessolrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml/str
/lst
lst name=slave
str name=enable${enable.slave:false}/str
str name=masterUrlhttp://${master.ip}:${master.port}/solr/${
solr.core.name}/replication/str
str name=pollInterval00:01:00/str
/lst
/requestHandler
The confFiles are under the dir:
/var/solr/application-cores/List/conf
and the external file fields are like:
/var/solr-data/List/external_*
Should I add
/var/solr-data/List/external_*
to confFiles like this?
str
name=confFilessolrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml,
/var/solr-data/List/external_*/str
Also, can you tell me when (or whether) I need to do reloadCache on the
slave after the ext file fields are replicated?
Thx.
On Fri, Jun 28, 2013 at 10:13 AM, Jack Krupansky j...@basetechnology.comwrote:
Show us your confFiles directive. Maybe there is some subtle error in
the file name.
-- Jack Krupansky
-Original Message- From: Arun Rangarajan
Sent: Friday, June 28, 2013 1:06 PM
To: solr-user@lucene.apache.org
Subject: Re: Replicating files containing external file fields
Erick,
Thx for your reply. The external file field fields are already under
dataDir specified in solrconfig.xml. They are not getting replicated.
(Solr version 4.2.1.)
On Thu, Jun 27, 2013 at 10:50 AM, Erick Erickson erickerick...@gmail.com
**wrote:
Haven't tried this, but I _think_ you can use the
confFiles trick with relative paths, see:
http://wiki.apache.org/solr/**SolrReplicationhttp://wiki.apache.org/solr/SolrReplication
Or just put your EFF files in the data dir?
Best
Erick
On Wed, Jun 26, 2013 at 9:01 PM, Arun Rangarajan
arunrangara...@gmail.com**wrote:
From
https://wiki.apache.org/solr/**SolrReplicationhttps://wiki.apache.org/solr/SolrReplicationI
understand that
index
dir and any files under the conf dir can be replicated to slaves. I want
to
know if there is any way the files under the data dir containing
external
file fields can be replicated. These are not replicated by default.
Currently we are running the ext file field reload script on both the
master and the slave and then running reloadCache on each server once
they
are loaded.
Re: Replicating files containing external file fields
Yes, you need to list that EFF file in the confFiles list - only those
listed files will be replicated.
str
name=confFilessolrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml,
/var/solr-data/List/external_*/str
Oops... sorry, no wildcards... you must list the individual files.
Technically, the path is supposed to be relative to the Solr collection
conf directory, so you MAY have you may have to put lots of ../ in the
path to get to the files, like:
../../../../solr-data/List/external_1
Tor each file.
(This is what Erick was referring to.)
Sorry, I don't have the answer to the reload question at the tip of my
tongue.
-- Jack Krupansky
-Original Message-
From: Arun Rangarajan
Sent: Friday, June 28, 2013 7:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Replicating files containing external file fields
Jack,
Here is the ReplicationHandler definition from solrconfig.xml:
requestHandler name=/replication class=solr.ReplicationHandler
lst name=master
str name=enable${enable.master:false}/str
str name=replicateAfterstartup/str
str name=replicateAftercommit/str
str name=replicateAfteroptimize/str
str
name=confFilessolrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml/str
/lst
lst name=slave
str name=enable${enable.slave:false}/str
str name=masterUrlhttp://${master.ip}:${master.port}/solr/${
solr.core.name}/replication/str
str name=pollInterval00:01:00/str
/lst
/requestHandler
The confFiles are under the dir:
/var/solr/application-cores/List/conf
and the external file fields are like:
/var/solr-data/List/external_*
Should I add
/var/solr-data/List/external_*
to confFiles like this?
str
name=confFilessolrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml,
/var/solr-data/List/external_*/str
Also, can you tell me when (or whether) I need to do reloadCache on the
slave after the ext file fields are replicated?
Thx.
On Fri, Jun 28, 2013 at 10:13 AM, Jack Krupansky
j...@basetechnology.comwrote:
Show us your confFiles directive. Maybe there is some subtle error in
the file name.
-- Jack Krupansky
-Original Message- From: Arun Rangarajan
Sent: Friday, June 28, 2013 1:06 PM
To: solr-user@lucene.apache.org
Subject: Re: Replicating files containing external file fields
Erick,
Thx for your reply. The external file field fields are already under
dataDir specified in solrconfig.xml. They are not getting replicated.
(Solr version 4.2.1.)
On Thu, Jun 27, 2013 at 10:50 AM, Erick Erickson erickerick...@gmail.com
**wrote:
Haven't tried this, but I _think_ you can use the
confFiles trick with relative paths, see:
http://wiki.apache.org/solr/**SolrReplicationhttp://wiki.apache.org/solr/SolrReplication
Or just put your EFF files in the data dir?
Best
Erick
On Wed, Jun 26, 2013 at 9:01 PM, Arun Rangarajan
arunrangara...@gmail.com**wrote:
From
https://wiki.apache.org/solr/**SolrReplicationhttps://wiki.apache.org/solr/SolrReplicationI
understand that
index
dir and any files under the conf dir can be replicated to slaves. I
want
to
know if there is any way the files under the data dir containing
external
file fields can be replicated. These are not replicated by default.
Currently we are running the ext file field reload script on both the
master and the slave and then running reloadCache on each server once
they
are loaded.
Re: Replicating files containing external file fields
Haven't tried this, but I _think_ you can use the confFiles trick with relative paths, see: http://wiki.apache.org/solr/SolrReplication Or just put your EFF files in the data dir? Best Erick On Wed, Jun 26, 2013 at 9:01 PM, Arun Rangarajan arunrangara...@gmail.comwrote: From https://wiki.apache.org/solr/SolrReplication I understand that index dir and any files under the conf dir can be replicated to slaves. I want to know if there is any way the files under the data dir containing external file fields can be replicated. These are not replicated by default. Currently we are running the ext file field reload script on both the master and the slave and then running reloadCache on each server once they are loaded.
Replicating files containing external file fields
From https://wiki.apache.org/solr/SolrReplication I understand that index dir and any files under the conf dir can be replicated to slaves. I want to know if there is any way the files under the data dir containing external file fields can be replicated. These are not replicated by default. Currently we are running the ext file field reload script on both the master and the slave and then running reloadCache on each server once they are loaded.
