Isn't it possible that the production equipment is simply under much
higher load (given that, since it's in production, your various users
are all actually using it), vs the QA equipment, which is only in use
by the people doing QA?
We've found the same thing at one point - we had a very small index (<
40000 rows), so small it didn't seem worth the effort to do delta
updates. So we would just refresh the whole thing every time - or so
we planned. In the test environment it updated within a minute. In
production, it would take as long as 15 minutes. What we finally
realized was, because the DB was under much higher load in production
than in the test environment, especially considering the amount of
joins that needed to take place to pull out the data properly, various
writes from the users to the affected tables would slow down the data
selection process dramatically as the indexer would have to wait for
locks to clear. Now of course we do delta updates and everything's
fine (and blazingly fast in both environments).
Try simulating higher load (involving a "normal" amount of writes to
the DB) against your QA equipment and then building the index. See if
the QA equipment still runs so quickly.
--
Steve
On Jun 20, 2009, at 11:29 PM, Otis Gospodnetic wrote:
Hi Francis,
I can't tell what the problem is from the information you've
provided so far. My gut instinct is that this is due to some
difference in QA vs. PROD environments that isn't Solr-specific.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
From: Francis Yakin <fya...@liquid.com>
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Sent: Saturday, June 20, 2009 2:18:07 AM
Subject: RE: Slowness during submit the index
The amount of data in Prod is about 20% more than QA.
We tested the network speed is fine. The hardware in Prod is larger
and more
powerful than QA.
But QA is faster during reload. It takes QA only one hour than 6
hours in Prod.
That's why we don't understand what's the reason, the amount of
data is only 20%
more but it will not take 5 times slower because the data only 20%
more.
So, we looked into the config file for solr, but it's not much
different, except
Prod has master/slave environment which QA only master.
Thanks for the response.
Francis
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com]
Sent: Friday, June 19, 2009 8:58 PM
To: solr-user@lucene.apache.org
Subject: Re: Slowness during submit the index
Francis,
So it could easily be that your QA and PROD DBs are really just
simply different
(different amount of data, different network speed, different
hardware...)
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
From: Francis Yakin
To: "solr-user@lucene.apache.org"
Sent: Friday, June 19, 2009 10:39:48 PM
Subject: RE: Slowness during submit the index
* is the java version the same on both machines (QA vs. PROD) - YES
* are the same java parameters being used on both machines -
YES
* is the connection to the DB the same on both machines -
Not sure,
need
to ask the network guy
* are both the PROD and QA DB servers the same and are both DB
instances the
same - they are not from the same DB
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com]
Sent: Friday, June 19, 2009 6:23 PM
To: solr-user@lucene.apache.org
Subject: Re: Slowness during submit the index
Francis,
I'm not sure if I understood your email correctly, but I think you
are saying
you are indexing your DB content into a Solr index. If this is
correct, here
are things to look at:
* is the java version the same on both machines (QA vs. PROD)
* are the same java parameters being used on both machines
* is the connection to the DB the same on both machines
* are both the PROD and QA DB servers the same and are both DB
instances the
same
...
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
From: Francis Yakin
To: "solr-user@lucene.apache.org"
Sent: Friday, June 19, 2009 5:27:59 PM
Subject: Slowness during submit the index
We are experiencing slowness during reloading/resubmitting index
from
Database
to the master.
We have two environments:
QA and Prod.
The slowness is happened only in Production but not in QA.
It only takes one hours to reload 2.5Mil indexes compare 5-6
hours to load
the
same size of index in Prod.
I checked both the config files in QA and Prod, they are all
identical,
except:
In QA:
false
In Prod:
true
I believe that we use "http" protocol reload/submit the index
from Database
to
Solr Master.
I did test copying big files thru network from database to the
solr box, I
don't
see any issue.
We are running solr 1.2
Any inputs will be much appreciated.
