Hi Patrick, If you intend to be dynamically updating the index (i.e. while the searcher is still alive) you can't avoid synchronization because the IndexSearcher is unaware of new docs until it is refreshed/reinstantiated.
Defining an interface is very important for compartmentalizing the search engine. The clients (like your ASP.NET website) of your search engine should be hidden from the IndexSearcher, Modifier, Writer and Readers to protect them from headache synching concerns. Also, I found creating wrapper objects and extending lucene classes allowed me to further exclude Lucene.Net.* classes from my search engine interface. Decoupling Lucene.Net and it's wrapping consumption classes (YourIndexBuilder, YourIndexUpdater,etc) is a good start for scaling your search engine too. During the last Lucene.Net project I was involved with, we put Lucene.Net inside a Windows service, and exposed it with a static singleton remoting interface. Vijay Santhanam B.Eng.(Soft.) Spectrum Wired - Software Engineer T: +61 2 4925 3266 F: +61 2 4925 3255 M: +61 407 525 087 W: www.spectrumwired.com Disclaimer: This email and any attached files are intended solely for the named addressee, are confidential and may contain legally privileged information. The copying or distribution of them or any information they contain, by anyone other than the addressee, is prohibited. If you have received this email in error, please let us know by telephone or return the email to the sender and destroy all copies. Thank you. -----Original Message----- From: Patrick Burrows [mailto:[EMAIL PROTECTED] Sent: Monday, 25 June 2007 11:27 PM To: [email protected] Subject: Re: FileNotFound Exception Ooh... is that right? Cause I access it via a website without any sort of sync locking. The site isn't live. But, by the very nature of a website, it is multithreaded. I also have separate processes which are constantly updating the index. And yet another process that validates the index once a week (makes sure there are no dupes or missed records). Access to the index through all these things must be synchronized? That seems... cumbersome. At best. On 6/25/07, Torsten Rendelmann <[EMAIL PROTECTED]> wrote: > > These kind of errors we also got - the reason was: > We accessed the index by multiple threads. Think, the same > happens if you access the index by two processes as > it seems examining the callstack (guess). > > > TorstenR > > > -----Original Message----- > > From: Patrick Burrows [mailto:[EMAIL PROTECTED] > > Sent: Sunday, June 24, 2007 7:21 PM > > To: [email protected] > > Subject: Re: FileNotFound Exception > > > > I deleted and recreated my index and things seem to be > > indexing now just > > fine. I went ahead and deleted it because everything google > > said was "wow, > > that seems bad" whenever someone else got this error. > > > > On 6/24/07, Patrick Burrows <[EMAIL PROTECTED]> wrote: > > > > > > If I call .Optimize() I get the same error... > > > > > > > > > > > > On 6/24/07, Patrick Burrows <[EMAIL PROTECTED]> wrote: > > > > > > > > I am in a tight loop adding items into my index. After > > running for a > > > > couple minutes in the loop just fine, I get the error > > posted below. If I > > > > then step through (don't stop the debugger, just hit F10 > > to keep stepping), > > > > it adds just fine. If I let it run, it will get the error > > again immediately. > > > > If I keep stepping through, though, I get no error. Only > > when it is running > > > > continuously. > > > > > > > > I added a sleep statement in my attempt to "program by > > coincidence" but > > > > it had no effect. Here is the code I am executing. The > > error is below that. > > > > The error occurs on the iw.AddDocument line: > > > > > > > > > > > > public > > > > static void AddPostsToIndex( List<Post> posts) > > > > > > > > { > > > > > > > > IndexWriter iw = GetIndexWriter(); > > > > > > > > foreach (Post post in posts) > > > > > > > > { > > > > > > > > DateTime loopItemStart = DateTime.Now; > > > > > > > > iw.AddDocument(post.ToDocument()); > > > > > > > > System.Threading. > > > > Thread.Sleep(10); > > > > > > > > log.DebugFormat( > > > > "Added post for feedItem {0} in {1}", post.FeedItemId, > > > > > > > > DateTime.Now.Subtract(loopItemStart)); > > > > > > > > } > > > > > > > > iw.Close(); > > > > > > > > } > > > > > > > > System.IO.FileNotFoundException was unhandled > > > > Message="Could not find file > > > > 'C:\\FeedReader\\FullTextSearch\\_oy.fnm'." > > > > Source="mscorlib" > > > > FileName="C:\\FeedReader\\FullTextSearch\\_oy.fnm" > > > > StackTrace: > > > > at System.IO.__Error.WinIOError(Int32 errorCode, String > > > > maybeFullPath) > > > > at System.IO.FileStream.Init(String path, FileMode mode, > > > > FileAccess access, Int32 rights, Boolean useRights, > > FileShare share, Int32 > > > > bufferSize, FileOptions options, SECURITY_ATTRIBUTES > > secAttrs, String > > > > msgPath, Boolean bFromProxy) > > > > at System.IO.FileStream..ctor(String path, FileMode mode, > > > > FileAccess access, FileShare share) > > > > at > > Lucene.Net.Store.FSIndexInput.Descriptor..ctor(FSIndexInput > > > > enclosingInstance, FileInfo file, FileAccess mode) > > > > at Lucene.Net.Store.FSIndexInput..ctor(FileInfo path) > > > > at Lucene.Net.Store.FSDirectory.OpenInput(String name) > > > > at Lucene.Net.Index.FieldInfos..ctor(Directory d, > > String name) > > > > at Lucene.Net.Index.SegmentReader.Initialize > > (SegmentInfo si) > > > > at Lucene.Net.Index.SegmentReader.Get(Directory > > dir, SegmentInfo > > > > si, SegmentInfos sis, Boolean closeDir, Boolean ownDir) > > > > at Lucene.Net.Index.SegmentReader.Get(SegmentInfo si) > > > > at > > Lucene.Net.Index.IndexWriter.MergeSegments(Int32 minSegment, > > > > Int32 end) > > > > at > > Lucene.Net.Index.IndexWriter.MergeSegments(Int32 minSegment) > > > > at Lucene.Net.Index.IndexWriter.MaybeMergeSegments() > > > > at Lucene.Net.Index.IndexWriter.AddDocument(Document doc, > > > > Analyzer analyzer) > > > > at Lucene.Net.Index.IndexWriter.AddDocument(Document doc) > > > > at FullTextSearch.Tasks.IndexManager.AddPostsToIndex(List`1 > > > > posts) > > > > at FullTextSearch.Tasks.IndexManager.ValidateIndex() > > > > at Indox.Program.RefreshDocsInIndex() in > > > > > > C:\Dev\WebSites\FeedReader\FullTextSearch\System\Indox\Program > .cs:line 61 > > > > at Indox.Program.HandleArguments (String[] args) in > > > > > > C:\Dev\WebSites\FeedReader\FullTextSearch\System\Indox\Program > .cs:line 40 > > > > at Indox.Program.Main(String[] args) in > > > > > > C:\Dev\WebSites\FeedReader\FullTextSearch\System\Indox\Program > .cs:line 23 > > > > at System.AppDomain.nExecuteAssembly(Assembly > > assembly, String[] > > > > args) > > > > at System.AppDomain.ExecuteAssembly(String > > assemblyFile, Evidence > > > > assemblySecurity, String[] args) > > > > at > > > > Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly () > > > > at System.Threading.ThreadHelper.ThreadStart_Context(Object > > > > state) > > > > at System.Threading.ExecutionContext.Run(ExecutionContext > > > > executionContext, ContextCallback callback, Object state) > > > > at System.Threading.ThreadHelper.ThreadStart () > > > > > > > > > > > > -- > > > > - > > > > P > > > > > > > > > > > > > > > > -- > > > - > > > P > > > > > > > > > > -- > > - > > P > > > > -- - P __________ NOD32 2220 (20070426) Information __________ This message was checked by NOD32 antivirus system. http://www.eset.com
