OK, here is the code that can be compiled against .NET 2.0 http://pastebin.com/k2f7JfPd
DIGY -----Original Message----- From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com] Sent: Wednesday, November 30, 2011 9:26 PM To: lucene-net-dev@lucene.apache.org Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2 DIGY, Thanks for the tip, but could you be a little more specific? Where and how are "extension-methods calls" replaced? For example, how would I change the CloseableThreadLocalExtensions method public static void Set<T>(this ThreadLocal<T> t, T val) { t.Value = val; } to eliminate the compile error Error 2 Cannot define a new extension method because the compiler required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be found. Are you missing a reference to System.Core.dll? due to the Set<T> parameter "this ThreadLocal<t> t" ? - Neal -----Original Message----- From: Digy [mailto:digyd...@gmail.com] Sent: Wednesday, November 30, 2011 12:27 PM To: lucene-net-dev@lucene.apache.org Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2 FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in CloseableThreadLocal (like uncommenting ThreadLocal<T> class and replacing extension-methods calls with static calls to CloseableThreadLocalExtensions) DIGY -----Original Message----- From: Christopher Currens [mailto:currens.ch...@gmail.com] Sent: Wednesday, November 30, 2011 7:26 PM To: lucene-net-dev@lucene.apache.org Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2 Trevor, I'm not sure if you can use 2.9.4, though, it looks like you're using VS2005 and .NET 2.0. 2.9.4 targets .NET 4.0, and I'm fairly certain we use classes only available in 4.0 (or 3.5?). However, if you can, I would suggest updating, as 2.9.4 should be a fairly stable release. The leak I'm talking about is addressed here: https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4. It may or may not be what your issue is. You say that it was at one time working fine, I assume you mean no memory leak. I would take some time to see what else in your code has changed. Make sure you're calling Close on whatever needs to be closed (IndexWriter/IndexReader/Analyzers, etc). Unfortunately for us, memory leaks are hard to debug over email, and it's difficult for us to tell if it's any change to your code or an issue with Lucene .NET. As far as I can tell, this is the only memory leak I can find that affects 2.9.2. Thanks, Christopher On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser <geobmx...@hotmail.com>wrote: > We just released 2.9.4 - the website didn't update last night, so ill have > to try and update it later today. But if you follow the link to download > 2.9.2 dist you'll see folders for 2.9.4. > > I'll send an email to the user and dev lists once i get the website to > update > ________________________________ > From: Trevor Watson > Sent: 11/30/2011 8:14 AM > To: lucene-net-dev@lucene.apache.org > Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2 > > You said "pre 2.9.3" I checked the apache lucene.net page to try to see > if > I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2 and > 2.9.4(g). Was this an un-released version? Or am I looking in the wrong > spot for updates to lucene.net? > > Thanks for all your help > > On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson < > powersearchsoftw...@gmail.com> wrote: > > > I can send you the dll that I am using if you would like. The documents > > are _mostly_ small documents. Emails and office docs size of plain text > > > > > > On Tuesday, November 29, 2011, Christopher Currens < > > currens.ch...@gmail.com> wrote: > > > Do you know how big the documents are that you are trying to > > delete/update? > > > I'm trying to find a copy of 2.9.2 to see if I can reproduce it. > > > > > > > > > Thanks, > > > Christopher > > > > > > On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson < > > > powersearchsoftw...@gmail.com> wrote: > > > > > >> Sorry for the duplicate post. I was on the road and posted both via my > > web > > >> mail and office mail by mistake > > >> > > >> The increase is a very gradual, the program starts at about 160,000k > > >> according to task manager (I know that's not entirely accurate, but it > > was > > >> the best I had at the time) and would, after adding 25,000-40,000 > > result in > > >> an out of memory exception (800,000k according to taskmanager). I > tried > > >> building a copy of 2.9.4 to test, but could not find one that worked > in > > >> visual studio 2005 > > >> > > >> I did notice using Ants memory profiler that there were a number of > > >> byte[32789] arrays that I didn't know where they came from in memory. > > >> > > >> On Monday, November 28, 2011, Christopher Currens < > > currens.ch...@gmail.com > > >> > > > >> wrote: > > >> > Hi Trevor, > > >> > > > >> > What kind of memory increase are we talking about? Also, how big > are > > the > > >> > documents that you are indexing, the ones returned from > > getFileInfoDoc()? > > >> > Is it putting an entire file into the index? Pre 2.9.3 versions > had > > >> > issues with holding onto allocated byte arrays far beyond when they > > were > > >> > used. The memory could only be freed via closing the IndexWriter. > > >> > > > >> > I'm a little unclear on exactly what's happening. Are you noticing > > >> memory > > >> > spike and stay constant at that level or is it a gradual increase? > > Is it > > >> > causing your application to error, (ie OutOfMemory exception, etc)? > > >> > > > >> > > > >> > Thanks, > > >> > Christopher > > >> > > > >> > On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson < > > >> > powersearchsoftw...@gmail.com> wrote: > > >> > > > >> >> I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005 > > (.NET > > >> >> 2.0) environment. We had a piece of software that WAS working. > I'm > > not > > >> >> sure what has changed however, the following code results in a > memory > > >> leak > > >> >> in the Lucene.Net component (or a failure to clean up used memory). > > >> >> > > >> >> The code in issue is here: > > >> >> > > >> >> private void SaveFileToFileInfo(Lucene.Net.Index.IndexWriter iw, > > bool > > >> >> delayCommit, string sDataPath) > > >> >> { > > >> >> Document doc = getFileInfoDoc(sDataPath); > > >> >> Analyzer analyzer = clsLuceneFunctions.getAnalyzer(); > > >> >> if (this.FileID == 0) > > >> >> { > > >> >> string s = ""; > > >> >> } > > >> >> iw.UpdateDocument(new Lucene.Net.Index.Term("FileId", > > >> >> this.fileID.ToString("000000000")), doc, analyzer); > > >> >> > > >> >> analyzer = null; > > >> >> doc = null; > > >> >> if (!delayCommit) > > >> >> iw.Commit(); > > >> >> } > > >> >> > > >> >> Commenting out the line iw.UpdateDocument resulted in no memory > > >> increase. > > >> >> I also tried replacing it with a deleteDocument and AddDocument > and > > the > > >> >> memory increased the same as using the UpdateDocument function > > >> >> > > >> >> The getAnalyzer() function returns a ExtendedStandardAnalyzer, but > > it's > > >> the > > >> >> UpdateDocument line specifically that gives me the issue. > > >> >> > > >> >> Any assistance would be greatly appreciated. > > >> >> > > >> >> Trevor Watson > > >> >> > > >> > > > >> > > > > > > ----- Checked by AVG - www.avg.com Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11 ----- Checked by AVG - www.avg.com Version: 2012.0.1873 / Virus Database: 2102/4648 - Release Date: 11/30/11
