if you need to do frequent batch indexing I would suggest to get the data directly with a sql datareader. You could load your objects from it and use the nhibernate index methods. You can also use the createdocument function of nhibernate.search to get the document (the basic element that lucene index) and feed an indexwriter directly, so you could get more control of the indexing process.
The lack of an scrollable resultset is a pain when working with large information volumes. To compare with your results, with this approach, i'm indexing 3 million records (name, description and address) in about 30 minutes. Howewer, i'm using lucene.net 2.3.2 candidate, that provides a big performance improvements in the indexing speeds. (with lucene 2.1 it tooks about 1:20 minutes.) This lucene version it's only in svn, but you could grab from it, compile, and substitute it from sharedlibs because it doesn't have any api changes. On 22 nov, 01:47, John Wheeler <[EMAIL PROTECTED]> wrote: > I got trunk, as of yesterday, of NHibernate Search from nhcontrib's > svn repo. I tried following the batch indexing advice located > here:http://www.hibernate.org/hib_docs/search/reference/en/html_single/#se..., > which basically uses a JDBC Scrollable ResultSet, which does not exist > in ADO. Also, I looked through the code and couldn't find the > hibernate.search.worker.batch_size being used anywhere. I saw the > constant that declares the parameter but nothing referencing it. Right > now, I am indexing 200K Address POCOs at around 3.5 hours with 2.8 > Xeon and 8 gigs. I manually commit them in batches of 500. I looked at > the log4net logs and noticed that the actual index writing is slow > taking about 25 seconds for each batch of 500. > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "NHibernate Contrib - Development Group" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com.ar/group/nhcdevs?hl=en -~----------~----~----~----~------~----~------~--~---
