date:20091023

[jira] Updated: (SOLR-551) Solr replication should include the schema also

2009-10-23 Thread Koji Sekiguchi (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Sekiguchi updated SOLR-551:


Component/s: (was: replication (scripts))
 replication (java)

change component from scripts to java

> Solr replication should include the schema also
> ---
>
> Key: SOLR-551
> URL: https://issues.apache.org/jira/browse/SOLR-551
> Project: Solr
>  Issue Type: Improvement
>  Components: replication (java)
>Affects Versions: 1.4
>Reporter: Noble Paul
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
>
> The current Solr replication just copy the data directory . So if the
> schema changes and I do a re-index it will blissfully copy the index
> and the slaves will fail because of incompatible schema.
> So the steps we follow are
>  * Stop rsync on slaves
>  * Update the master with new schema
>  * re-index data
>  * forEach slave
>  ** Kill the slave
>  ** clean the data directory
>  ** install the new schema
>  ** restart
>  ** do a manual snappull
> The amount of work the admin needs to do is quite significant
> (depending on the no:of slaves). These are manual steps and very error
> prone
> The solution :
> Make the replication mechanism handle the schema replication also. So
> all I need to do is to just change the master and the slaves synch
> automatically
> What is a good way to implement this?
> We have an idea along the following lines
> This should involve changes to the snapshooter and snappuller scripts
> and the snapinstaller components
> Everytime the snapshooter takes a snapshot it must keep the timestamps
> of schema.xml and elevate.xml (all the files which might affect the
> runtime behavior in slaves)
> For subsequent snapshots if the timestamps of any of them is changed
> it must copy the all of them also for replication.
> The snappuller copies the new directory as usual
> The snapinstaller checks if these config files are present ,
> if yes,
>  * It can create a temporary core
>  * install the changed index and configuration
>  * load it completely and swap it out with the original core

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (SOLR-561) Solr replication by Solr (for windows also)

2009-10-23 Thread Koji Sekiguchi (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Koji Sekiguchi updated SOLR-561:


Component/s: (was: replication (scripts))
 replication (java)

change component from scripts to java

> Solr replication by Solr (for windows also)
> ---
>
> Key: SOLR-561
> URL: https://issues.apache.org/jira/browse/SOLR-561
> Project: Solr
>  Issue Type: New Feature
>  Components: replication (java)
>Affects Versions: 1.4
> Environment: All
>Reporter: Noble Paul
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: deletion_policy.patch, SOLR-561-core.patch, 
> SOLR-561-fixes.patch, SOLR-561-fixes.patch, SOLR-561-fixes.patch, 
> SOLR-561-full.patch, SOLR-561-full.patch, SOLR-561-full.patch, 
> SOLR-561-full.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, 
> SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, 
> SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, 
> SOLR-561.patch, SOLR-561.patch, SOLR-561.patch
>
>
> The current replication strategy in solr involves shell scripts . The 
> following are the drawbacks with the approach
> *  It does not work with windows
> * Replication works as a separate piece not integrated with solr.
> * Cannot control replication from solr admin/JMX
> * Each operation requires manual telnet to the host
> Doing the replication in java has the following advantages
> * Platform independence
> * Manual steps can be completely eliminated. Everything can be driven from 
> solrconfig.xml .
> ** Adding the url of the master in the slaves should be good enough to enable 
> replication. Other things like frequency of
> snapshoot/snappull can also be configured . All other information can be 
> automatically obtained.
> * Start/stop can be triggered from solr/admin or JMX
> * Can get the status/progress while replication is going on. It can also 
> abort an ongoing replication
> * No need to have a login into the machine 
> * From a development perspective, we can unit test it
> This issue can track the implementation of solr replication in java

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Re: Setting solr.home in web.xml

2009-10-23 Thread Mark Miller

Mark Miller wrote:
> Anyone able to get this to work? I can't.
>
>   
>
> 
>solr/home
>/put/your/solr/home/here
>java.lang.String
> 
>
> For some reason, I see an error in web.xml when uncommenting that -
> though it looks right from the doc info I can find on it.
>
>   
Whew - never mind - its actually easy as this comment would imply.

You just have to:

1. If you want to lose the web.xml error on validation, change web.xml
from 2.3 to 2.4 (done in a different way than 2.3 is specified). Not
required though.

2. Add the jetty-plus jar and jetty-naming jar to example/lib

3. Add
to jetty.xml in the webappdeployer section.

4. Make sure you specify an initial naming factory: 

Then you can actually use this method to specify different solr.homes
for multiple Solr installs on a box.

-- 
- Mark

http://www.lucidimagination.com

Re: deploy solr in Eclipse IDE

2009-10-23 Thread Amit Nithian

Apparently this is a more popular topic than I thought!
I have received several requests to send the files personally so now I have
made them available for download at
http://filebox.vt.edu/users/anithian/eclipse_settings.zip

Thanks
Amit

On Tue, Oct 20, 2009 at 9:58 PM, Amit Nithian  wrote:

> Pradeep,
> Attached are the files. You may have to open them in a text editor and
> rename the project to match yours but should be pretty straightforward. I
> used this with 1.3 trunk at the time so things may have changed but it's
> easy enough to modify in eclipse.
>
> - Amit
>
>
> On Mon, Oct 19, 2009 at 4:16 PM, Pradeep Pujari 
> wrote:
>
>> This is ulr is helpful. If I checkout in Eclipse using SVN(Subclipse), the
>> source files are not as per package structure. Can you please send me your
>> ..project and .classpth files? Thank you in advance.
>>
>> Pradeep
>>
>> --- On Sun, 10/18/09, Amit Nithian  wrote:
>>
>> > From: Amit Nithian 
>> > Subject: Re: deploy solr in Eclipse IDE
>> > To: solr-dev@lucene.apache.org
>> > Date: Sunday, October 18, 2009, 11:06 PM
>> > Hey Pradeep,
>> > Check out
>> >
>> http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29
>> >
>> > <
>> http://lucene.apache.org/solr/version_control.html#Anonymous+Access+%28read-only%29
>> >If
>> > you need more help with setting up Eclipse and Solr trunk
>> > send me an email.
>> > I can send you my .project and .classpath files as I have
>> > it for my setup.
>> >
>> > Take care
>> > Amit
>> >
>> > On Sun, Oct 18, 2009 at 11:34 AM, Pradeep Pujari <
>> prade...@rocketmail.com>wrote:
>> >
>> > > Hi Amit,
>> > > This is what I am looking for. Do you know the URL for
>> > trunk?
>> > >
>> > > Thanks,
>> > > Pradeep.
>> > >
>> > > --- On Sun, 10/18/09, Amit Nithian 
>> > wrote:
>> > >
>> > > > From: Amit Nithian 
>> > > > Subject: Re: deploy solr in Eclipse IDE
>> > > > To: solr-dev@lucene.apache.org
>> > > > Date: Sunday, October 18, 2009, 12:55 AM
>> > > > I think you may have better luck
>> > > > setting up Eclipse, Subclipse etc and hook
>> > > > off of trunk rather than having to re-create the
>> > eclipse
>> > > > project every time
>> > > > a nightly build comes out.
>> > > > I simply have an eclipse project tied to trunk
>> > and every so
>> > > > often i'll do an
>> > > > SVN update when I want/need the latest code.
>> > > >
>> > > > hope that helps some!
>> > > > Amit
>> > > >
>> > > > On Thu, Oct 15, 2009 at 2:31 AM, Brian Carmalt
>> > 
>> > > > wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > I Start Solr with Jetty using the following
>> > code. If
>> > > > the classpath and
>> > > > > src paths are set correctly in Eclipse and
>> > you pass
>> > > > the solr.home to the
>> > > > > VM on startup, you just have to start this
>> > class and
>> > > > you can debug Solr
>> > > > > in Eclipse.
>> > > > >
>> > > > > 
>> > > > > import org.mortbay.jetty.Connector;
>> > > > > import org.mortbay.jetty.Server;
>> > > > > import
>> > org.mortbay.jetty.webapp.WebAppContext;
>> > > > >
>> > > > > public class JettyStarter {
>> > > > >
>> > > > >/**
>> > > > > *
>> > @param args
>> > > > > */
>> > > > >public static
>> > void
>> > > > main(String[] args) {
>> > > > >
>> > > > >
>> > > > try {
>> > > > >
>> > > > >
>> > > > Server
>> > server = new
>> > > > Server();
>> > > > >
>> > > > >
>> > > >
>> >WebAppContext solr = new
>> > > > WebAppContext();
>> > > > >
>> > > >
>> >solr.setContextPath("/solr");
>> > > > > solr.setWar("Path to solr directory or
>> > war");
>> > > > >
>> > > >
>> >server.addHandler(solr);
>> > > > >
>> > > >
>> >server.setStopAtShutdown(true);
>> > > > >
>> > > >
>> >server.start();
>> > > > >
>> > > > } catch (Exception e) {
>> > > > >
>> > > > // TODO
>> > Auto-generated catch
>> > > > block
>> > > > >
>> > > >
>> >e.printStackTrace();
>> > > > >
>> > > > }
>> > > > >}
>> > > > >
>> > > > > }
>> > > > >
>> > > > > 
>> > > > >
>> > > > >
>> > > > > Am Dienstag, den 13.10.2009, 16:43 -0700
>> > schrieb
>> > > > Pradeep Pujari:
>> > > > > > Hi All,
>> > > > > >
>> > > > > > I am trying to install solr nightly
>> > build into
>> > > > Eclipse IDE and facing lot
>> > > > > of issues while importing the zip file. The
>> > build
>> > > > path, libs and various
>> > > > > source files are scattered. It took me lot
>> > of tine to
>> > > > configure and make it
>> > > > > run.
>> > > > > >
>> > > > > > What development environment are being
>> > used and
>> > > > is there a smooth way of
>> > > > > importing daily-nightly build into eclipse?
>> > > > > >
>> > > > > > Please help.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Pradeep.
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> >
>>
>>
>

[jira] Commented: (SOLR-1521) eliminate contrib/clustering/example by moving to main example disabled by default

2009-10-23 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12769301#action_12769301
 ] 

Yonik Seeley commented on SOLR-1521:


I'm +1 for committing...

> eliminate contrib/clustering/example by moving to main example disabled by 
> default
> --
>
> Key: SOLR-1521
> URL: https://issues.apache.org/jira/browse/SOLR-1521
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
> Fix For: 1.4
>
> Attachments: SOLR-1521.patch
>
>
> As noted in email, there is a tenuous connection between the main example and 
> the clustering example in that they both use the same dataDir, so their 
> schema files have to be kept in sync or the example will break...
> http://www.nabble.com/clustering-schema-td25982682.html

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Setting solr.home in web.xml

2009-10-23 Thread Mark Miller

Anyone able to get this to work? I can't.

  


   solr/home
   /put/your/solr/home/here
   java.lang.String


For some reason, I see an error in web.xml when uncommenting that -
though it looks right from the doc info I can find on it.

-- 
- Mark

http://www.lucidimagination.com

Re: patch for ExtractingDocumentLoader

2009-10-23 Thread Grant Ingersoll


Applied.  Thanks John!

On Oct 23, 2009, at 10:37 AM, John Thorhauer wrote:


Hi,

I would like to extend the ExtractingRequestHandler and in doing so I
will need to instantiate the ExtractingDocumentLoader in my own
package.  So could someone please apply this simple patch that would
allow me to create an instance of the ExtractingDocumentLoader?

Thanks for your help,
John Thorhauer

Re: patch for ExtractingDocumentLoader

2009-10-23 Thread Noble Paul നോബിള്‍ नोब्ळ्

Grant , is there any reason why the constructor can't be public?

On Fri, Oct 23, 2009 at 8:07 PM, John Thorhauer  wrote:
> Hi,
>
> I would like to extend the ExtractingRequestHandler and in doing so I
> will need to instantiate the ExtractingDocumentLoader in my own
> package.  So could someone please apply this simple patch that would
> allow me to create an instance of the ExtractingDocumentLoader?
>
> Thanks for your help,
> John Thorhauer
>



-- 
-
Noble Paul | Principal Engineer| AOL | http://aol.com

patch for ExtractingDocumentLoader

2009-10-23 Thread John Thorhauer

Hi,

I would like to extend the ExtractingRequestHandler and in doing so I
will need to instantiate the ExtractingDocumentLoader in my own
package.  So could someone please apply this simple patch that would
allow me to create an instance of the ExtractingDocumentLoader?

Thanks for your help,
John Thorhauer
Index: contrib/extraction/src/main/java/org/apache/solr/handler/extraction/ExtractingDocumentLoader.java
===
--- contrib/extraction/src/main/java/org/apache/solr/handler/extraction/ExtractingDocumentLoader.java	(revision 829075)
+++ contrib/extraction/src/main/java/org/apache/solr/handler/extraction/ExtractingDocumentLoader.java	(working copy)
@@ -79,7 +79,7 @@
   protected SolrContentHandlerFactory factory;
   //protected Collection dateFormats = DateUtil.DEFAULT_DATE_FORMATS;
 
-  ExtractingDocumentLoader(SolrQueryRequest req, UpdateRequestProcessor processor,
+  public ExtractingDocumentLoader(SolrQueryRequest req, UpdateRequestProcessor processor,
TikaConfig config, SolrContentHandlerFactory factory) {
 this.params = req.getParams();
 schema = req.getSchema();

RE: Dinamic field name with Data import handler

2009-10-23 Thread Renata Mota

Yes, this was the problem.

Thanks,


-Original Message-
From: noble.p...@gmail.com [mailto:noble.p...@gmail.com] On Behalf Of Noble
Paul ??? ??
Sent: Friday, October 23, 2009 10:53 AM
To: solr-dev@lucene.apache.org
Subject: Re: Dinamic field name with Data import handler

try putting the  tag directly under the  tag . it
does not read it if it is under the  tag



On Fri, Oct 23, 2009 at 6:18 PM, Renata Mota
 wrote:
>
> I did, using the example:
>
>
> 
>     driver="oracle.jdbc.driver.OracleDriver"
> url="jdbc:oracle:thin:@127.0.0.1:1521:XY" user="root"/>
>    
>