I did clean up the indexes and re-started the index process from scratch (new index file). As another test if I use simple numeric counter for unique id the index speed is fast (within 20 sec for commit 50k records). I'm thinking UUID might not be the way to go for unique id - I'll look into using sequence# instead.
Thanks, -vivek On Tue, May 5, 2009 at 11:03 AM, Otis Gospodnetic <otis_gospodne...@yahoo.com> wrote: > > You really had nothing in uniqueKey element in schema.xml at first? I'm not > looking at Solr code right now, but it could be the lack of the cost of that > lookup that made things faster. Now you have a lookup + generation + more > data to pass through analyzer + write out, though I can't imagine how that > would make things 2x slower. You didn't say whether you cleared the old > index after adding UUID key.... did you do that? > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- >> From: vivek sar <vivex...@gmail.com> >> To: solr-user@lucene.apache.org >> Sent: Tuesday, May 5, 2009 1:49:21 PM >> Subject: Using UUID for unique key >> >> Hi, >> >> I've a distributed Solr instances. I'm using Java's UUID >> (UUID.randomUUID()) to generate the unique id for my documents. Before >> adding unique key I was able to commit 50K records in 15sec (pretty >> constant over the growing index), after adding unique key it's taking >> over 35 sec for 50k and the time is increasing as the index size >> grows. Here is my schema setting for unique key, >> >> >> required="true" omitNorms="true" compressed="false"/> >> >> Why is commit taking so long? Should I not be using UUID key for >> unique keys? What are other options - timestamp etc.? >> >> Thanks, >> -vivek > >
