Thanks.  I wound up writing my own performance test tool, since, among other 
things, I want to be sure that the index is big enough to not fit in memory 
(either in a Lucene cache or the OS disk cache).  I will take your 
recommendation about nightly tests, though.  It's hard to have too many unit 
tests!

Scott


> -----Original Message-----
> From: Uwe Schindler [mailto:u...@thetaphi.de]
> Sent: Friday, January 24, 2014 3:03 AM
> To: java-user@lucene.apache.org
> Subject: RE: Performance testing Lucene
> 
> Hi Scott,
> 
> the unit tests are also a good performance test. But to compare your
> directory with another one, be sure to:
> - use a defined directory instance to compare. The most performant
> Lucene one is: -Dtests.directory=MMapDirectory - so compare you results
> with that one. If you don't define a diferectly, it uses RAMDirectory
> in most cases.
> - use a defined random seed when comparing results. Lucene tests
> randomize a lot. Randomization can be prevented by explicitely stating
> a given random seed (one example is given on startup). Also run "ant
> test-help" to get more usage help.
> - to do more stress testing - this will create larger indexes: -
> Dtests.nightly=true
> - use a single JVM: -Dtests.jvms=1
> 
> Uwe
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: u...@thetaphi.de
> 
> 
> > -----Original Message-----
> > From: Scott Schneider [mailto:scott_schnei...@symantec.com]
> > Sent: Friday, January 24, 2014 2:41 AM
> > To: java-user@lucene.apache.org
> > Subject: RE: Performance testing Lucene
> >
> > Thanks!  I ran this Directory subclass through the Lucene unit tests
> (and
> > found 3 race conditions).  Unit tests are wonderful.
> >
> > Scott
> >
> >
> > > -----Original Message-----
> > > From: Michael McCandless [mailto:luc...@mikemccandless.com]
> > > Sent: Wednesday, January 22, 2014 7:05 AM
> > > To: Lucene Users
> > > Subject: Re: Performance testing Lucene
> > >
> > > All the source code for the nightly Lucene perf tests I run (
> > > http://people.apache.org/~mikemccand/lucenebench/ ) are here:
> > > https://code.google.com/a/apache-extras.org/p/luceneutil/
> > >
> > > These are also the scripts I use for A/B performance tests for a
> new
> > > patch.
> > >
> > > It's somewhat tricky getting those Python scripts set up to run ...
> > > but I think it'd be a good way to smoke test your new Directory.
> > >
> > > The queries are "synthetic"; it's a real problem, not having a real
> > > world, biggish corpus plus real queries, for better performance
> > > testing...
> > >
> > > Mike McCandless
> > >
> > > http://blog.mikemccandless.com
> > >
> > >
> > > On Mon, Jan 20, 2014 at 11:22 PM, Scott Schneider
> > > <scott_schnei...@symantec.com> wrote:
> > > > Hello,
> > > >
> > > > Would you folks mind giving me a few tips on performance testing
> > > Lucene?  I want to test the performance impact of a Directory
> subclass.
> > > >
> > > > What is a good testing tool to use?  I don't see a great way to
> get
> > > SolrMeter to run the max # updates/minute and measure throughput
> that
> > > way.  When I set the # updates/minute to a large #, SolrMeter logs
> > > NullPointerExceptions.  (I assume these are within SolrMeter, as I
> > > don't see errors in Solr.)  Mike McCandless's nightly Lucene
> > > performance tests look good, though I've only just started looking
> at
> > > it.
> > > >
> > > > Are there any particularly standard or good test sets?  I'd like
> to
> > > test 3 scenarios:  indexing only, querying only, and indexing plus
> > > querying.  McCandless's indexing test uses wikipedia, which seems
> > > great, but he has a slew of tests that are each specific to some
> > > querying feature.  I'd like a single, general query test.  It's not
> > > hard to come up with a decent set of queries, but I'd really like
> > > something representative of real world queries.  If there some
> > > standard set of commonly used queries, that would be ideal.
> > > >
> > > > Thanks!
> > > >
> > > > Scott
> > > >
> > >
> > > -------------------------------------------------------------------
> --
> > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: java-user-h...@lucene.apache.org
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: java-user-h...@lucene.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to