Re: What happened to search? (only google custom search with crippled possibilities?)

2011-02-12 Thread Jeff Marshall
HI Christian.

Actually, we've been using Google's search for our front-page site-wide
search for years.  Our internally-developed Lucene search has always been on
the individual lists and still is.  So we've never had Lucene search
parameter controls on that site-wide search.

The only change that took place was Google required us to update the search
box code we use on the front page, and now it has that Google Custom Search
logo, and ads are now showing on the right side of the search results.  The
old site-wide search had Google's ads above the message search results and
we thought getting them off to the side was less intrusive.

It's unfortunate that Google insists on having their logo on our front-page
search box now and I'm trying to reach a Google rep to see what we can do
about that.  As for the ads on the site-wide search results we'd prefer to
turn them off completely since they kind of get in the way and only generate
a few dollars anyway.

The reason we went with Google's search for our site-wide results years ago
is because we couldn't efficiently generate a Lucene index that spans all
messages across all lists.  Perhaps it's time for us to revisit this now
that we aren't particularly happy with the Google logo or the change in ad
display requirements.

Jeff


On Sat, Feb 12, 2011 at 11:49 AM, Christian Lohmaier 
lohma...@googlemail.com wrote:

 Hi *,

 what I always liked about mail-archive.com was its powerful search,
 but it seems it has been replaced by a google search, that doesn't
 support the lucene parameters anymore.

 The first change to the worse since I've been using mail-archive.com

 So if this is a permanent change (the message back in November last year
 http://www.mail-archive.com/gossip@jab.org/msg01352.html
 sounded that while there was a problem, work is towards reenabling the
 feature), please update the FAQ wrt. search syntax accordingly.

 Thankfully perl-list search is still using lucene search, but
 searching multiple/all lists is where the more powerful search syntax
 is most helpful...

 ciao
 Christian


 --
 To unsubscribe, send mail to gossip-unsubscr...@jab.org.



Re: What happened to search? (only google custom search with crippled possibilities?)

2011-02-12 Thread Jeff Breidenbach
Perhaps it's time for us to revisit [Lucene for site-wide search]

A few weeks ago I tested Lucene's ability to search across multiple
indexes (MultiSearcher) and it is hopelessly slow; queries take 5
seconds across just a few hundred indexes.

Right now I'm trying index merging (addIndexesNoOptimize). With an HDD
destination there is a load spike (around 20 to 30) and merge at a
rate of about 2GB per minute. That's approximately 3 hours. With an
SSD destination the load spike is gone and merge rate jumps 3GB
minute. It remains to be seen what the search performance will be once
indexing is complete.

There is also chatter that Lucene 3 has a slightly different API for
index merging, but if I'm reading it right there is no performance
difference.

-Jeff


-- 
To unsubscribe, send mail to gossip-unsubscr...@jab.org.


Re: What happened to search? (only google custom search with crippled possibilities?)

2011-02-12 Thread Jeff Breidenbach
I was doing some experiments today, and managed to briefly knock over
a server in the process. I looked at searching  multiple indexes
(Lucene's MultiSearcher) and merging
(IndexWriter.addIndexesNoOptimize). The former is unusably slow. The
latter seems to be on track for about 6 hours if the destination is
HDD. About 4 if destination is SDD. When it finishes, we'll see what
query speed looks like.

-Jeff


-- 
To unsubscribe, send mail to gossip-unsubscr...@jab.org.


Re: What happened to search? (only google custom search with crippled possibilities?)

2011-02-12 Thread Jeff Breidenbach
Oops, sorry for the more-or-less duplicate message.  The extra factor
of 2 in time is because the temporary files are turning out twice as
big as I was expecting. Earl, good suggestion, and no we haven't
explored it (yet).


-- 
To unsubscribe, send mail to gossip-unsubscr...@jab.org.