I would like to add async/await, but I think adding a MMapDirectory, first, is more important, and also something we could do now for the 3.x versions of lucene. I know we've looked at it before (LUCENE-425, I think?), and it was turned down because it had poor performance. I've tested a quick port that I wrote and it performs the same under light load (8 threads) as it does under heavy load (250 threads), able to complete 6000 queries in 74 seconds. It performs the same as SimpleFSDirectory when it comes to light load, but the SimpleFSDirectory performs horribly under heavy load, where the 250 threads querying 6000 times total took about 160 seconds.
Another problem with supporting both .NET 3.5 and .NET 4.0, is that it makes it more difficult in some respects to add things like methods that return Task<T>, since it's not present in 3.5. I think we need to maintain two separate ports, a 3.x branch and a 4.x branch, although we don't have a committer base that is active enough to do that. The 3.x branch could support .NET 3.5 and 4.0, where the 4.0 branch would only support .NET 4.0. If you look at the differences between 3.0.3 and 3.1, there are huge performance optimizations that have been done, and to not give those to .NET 3.5 users seems to be a wasted opportunity, in my opinion. A lot of the difficulties that come with porting Lucene 4.0 into .NET 3.5 come in the later versions of the 3.x branch, probably around 3.5 or 3.6. On Tue, Jan 1, 2013 at 3:43 PM, Nicholas Paldino [.NET/C# MVP] < [email protected]> wrote: > As a somewhat related issue with .NET 4.0, are there plans to move to > asynchronous calls at any point for what are now blocking I/O operations? > > Task<T> in 4.0 has made removing blocking I/O operations in C#/.NET 4.0 > fairly simple (although not as simple as C# 4.5 with async/await and core > framework changes to support Task<T>). > > I ask because I'm thinking the throughput and/or performance of of > Lucene.NET would be increased dramatically (by reducing waits on file-based > I/O operations) but it would be a large architectural change. > > Perhaps it's something to keep in mind for the future. > > - Nicholas Paldino > > On Jan 1, 2013, at 5:38 PM, "Christopher Currens" <[email protected]> > wrote: > > > There are issues that need to be discussed about the 4.0 port and being > > able to continue support of .NET 3.5. I can only think of one example > > right now, but I've looked through most of the code for lucene 4.0, and > > there's a *very* heavy use of variance that would be difficult to > maintain > > outside of .NET 4.x. If you want a good example of this, check out the > > lucene.util.ast package and its usages. This isn't the only area that > uses > > both contravariance and covariance, but it's the only one I could think > of > > off of the top of my head. > > > > I started porting parts of it, just to see what could be done (nothing > > significant). It's been a month or so since I worked on it, so my memory > > of what I found is foggy. The variance was the biggest thing I saw that > > was an issue, but I think there were a few other things. I'd have to > look > > at it again to see. > > > > > > Thanks, > > Christopher > > > > > > On Tue, Jan 1, 2013 at 12:50 AM, Itamar Syn-Hershko <[email protected] > >wrote: > > > >> The general direction should be to put most of the efforts on a v4 port > >> (4.1 probably...) and to start finding pieces in the codebase we can > easily > >> isolate and .NET-ify. Mostly readers, writers, structures and > conversions. > >> > >> Re git on apache - I believe you should ask infra. > >> > >> > >> On Tue, Dec 18, 2012 at 6:54 PM, Christopher Currens < > >> [email protected]> wrote: > >> > >>> Don't think I've forgotten about this. While I have no idea how git > >> under > >>> Apache looks like, I have a lot of comments on Lucene 3.6 (and 4.0) > that > >> I > >>> need to discuss on this mailing list directly relating to the porting > >> work > >>> and the future direction of lucene.net. I've had an email in my > drafts > >>> folder for about 3 weeks now but this time of year has been so busy I > >>> haven't had a chance to finish it. Soon, though, I hope. > >>> > >>> > >>> Thanks, > >>> Christopher > >>> > >>> > >>> On Sat, Dec 15, 2012 at 1:09 PM, Prescott Nasser < > [email protected] > >>>> wrote: > >>> > >>>> Hey Guys - I've been quietly working in the background on > >> administrative > >>>> stuff for a while. I really only have two things on my to-do list - - > >>> What > >>>> does git under apache look like? I can't remember who asked this, but > I > >>>> know I owe digging up the answer- Lucene 3.6 - planning, moving > >> forward ( > >>>> https://github.com/apache/lucene-solr/compare/5261b571...e4402c22c). > >> Do > >>>> we just want to start picking things off and committing them? Are > there > >>>> other refactoring issues we want to tackle with 3.6? We should make > >> jira > >>>> tickets and start tracking. Happy holidays everyone! ~Prescott > >> > >
