Hi
I hacked luence index dynamicly update one year ago(I used quartz as
scheduler). And hope the idea helps.
In indexing phase:
1. write index into directory
2. write one flag, say "done" when indexing is completed
3. try to delete the index some days ago(say two days ago).
In search phase:
1. scan the search.dir and try to find the lastest indices with "done" flag
2. search it.
I remembered Doug posted some great idea in 2005 year, pls try to
search the maillist.
Good luck!
/Jack
On 3/15/06, TDLN <[EMAIL PROTECTED]> wrote:
> That's about where I got stuck as well. I didn't have time to look into this
> problem any further then. I think it's best to either check the trunk and
> see how it is solved there or just to move to 0.8 altogether.
>
> Rgrds, Thomas
>
>
> On 3/14/06, Laurent Michenaud <[EMAIL PROTECTED]> wrote:
> >
> > Here where i am :
> >
> > *** In the NutchBean
> >
> > I have made the segments and localFileSystem
> > Variables as instance variables.
> >
> > private FetchedSegments segments ;
> > private LocalFileSystem localFileSystem ;
> >
> > In the init method :
> > localFileSystem = new LocalFileSystem();
> > segments = new FetchedSegments(localFileSystem,segmentsDir.toString());
> >
> > And i've created the method closeSegment :
> >
> > public void closeSegments() throws IOException {
> > segments.closeSegments();
> > localFileSystem.close();
> > }
> >
> > ** In the class FetchSegments
> >
> > I have added the method :
> > public void closeSegments() throws IOException {
> > Iterator iterSegments = segments.values().iterator();
> > while( iterSegments.hasNext()) {
> > Segment oSegment = (Segment) iterSegments.next();
> > oSegment.close();
> > }
> > }
> >
> > ** In the subclass FetchSegments.Segment
> >
> > I have added the method :
> > public void close() throws IOException {
> > if ( fetcher != null )
> > fetcher.close();
> > if ( content != null )
> > content.close();
> > if ( text != null )
> > text.close();
> > if ( parsedata != null )
> > parsedata.close();
> > }
> >
> >
> >
> >
> > My test program :
> > I instanciate the NutchBean.
> > I make a search.
> > I call oNutchBean.closeSegments()
> > Finally, I try to delete the index dir and i get :
> >
> > java.io.IOException: Unable to delete file:
> > D:\Projets\application-data\recherche\indexations\indexes-
> > 20060314-17h33.54\segments\20060314173333\parse_text\data
> > at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:951)
> > at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java
> > :713)
> > at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java
> > :680)
> > at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:943)
> > at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java
> > :713)
> > at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java
> > :680)
> > at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:943)
> > at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java
> > :713)
> > at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java
> > :680)
> > at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:943)
> > at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java
> > :713)
> > at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java
> > :680)
> >
> > So, it is not ok yet.
> >
> >
> >
> >
> > -----Message d'origine-----
> > De : TDLN [mailto:[EMAIL PROTECTED]
> > Envoyé : mardi 14 mars 2006 12:39
> > À : [email protected]
> > Objet : Re: Problems
> >
> > I think so too, maybe as part of the upcoming 0.7.2 release.
> >
> > In the meantime, you can try to implement the fix suggested in this post
> >
> > http://www.mail-archive.com/[email protected]/msg03011.html
> >
> > If it works, for you, please let me know, I need the fix as well :)
> >
> > Rgrds, Thomas
> >
> >
> >
> > On 3/14/06, Laurent Michenaud <[EMAIL PROTECTED]> wrote:
> > >
> > > It would be interesting to have a fix for 0.7
> > >
> > > -----Message d'origine-----
> > > De : TDLN [mailto:[EMAIL PROTECTED] Envoyé : mardi 14 mars 2006
> > > 12:32 À : [email protected] Objet : Re: Problems
> > >
> > > Unfortunately, in the 0.7 release, the NutchBean does not clean up
> > > properly after itself, so some SegementReaders and IndexReaders remain
> > open.
> > > I think this is fixed in the current code line. I had similar problems
> > > in my app based on 0.7 - all that helped was killing the processes
> > > blocking the index.
> > >
> > > Rgrds, Thomas
> > >
> > >
> > > On 3/13/06, Laurent Michenaud <[EMAIL PROTECTED]> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I have integrated nutch in my application.
> > > >
> > > > In the admin part, i can launch the indexation.
> > > > In the client part, i can launch searches.
> > > >
> > > > After a search, i can't do an indexation again because the index
> > > > files are used by the NutchBean.
> > > >
> > > > How can i do that please ?
> > > >
> > > > thx
> > > >
> > > >
> > >
> >
>
>
--
Keep Discovering ... ...
http://www.jroller.com/page/jmars
-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
