Looking for Solr/Lucene Developers in India(Pune)

[ST ST]

If you are a Solr/Lucene developer in Pune, India and are interested in a
consulting opportunity overseas,
or on Projects local to the area, please get in touch with me.

Thanks

Query Timings increase after system is idle

[ST ST]

Folks,

We have a problem in our environment where after a system is idle the query
time goes up from a few 100ms to 4+ seconds after 9 hours of idle time on
the system.

System Details:
 - Solr 1.4
 - 10 Million Index.
 - Use MMAP for mapping the index files in memory

Test Details:
-  8 hour performance run with ingestion (@ 8 docs/sec) , query rate - 3
Queries per sec.
-  Commit is per hour.

Issue:
- After 9 hours of idle time (ie no queries, no ingestion ) every query
takes 4+ seconds, subsequent queries are fast.

I have a few specific questions:
A. Does Lucene/Solr have internal caches which may be flushed out of memory
when the system is idle ?
B. What operations are done on a per term basis (example: build doc lists )
for first time queries.
C. Any pointers to what else may be an issue here.

Really appreciate any help you can provide.

ST

Solr integration with document management systems

[ST ST]

Folks,

Does Solr 1.4 come with integration with existing document management
systems ?

Are there any other open source projects based on Solr which provide this
capability ?

Thanks

Fwd: Lucene MMAP Usage with Solr

[ST ST]

Folks,

I am trying to get Lucene MMAP to work in solr.

I am assuming that when I configure MMAP the entire index will be loaded
into RAM.
Is that the right assumption ?

I have tried the following ways for using MMAP:

Option 1. Using the solr config below for MMAP configuration

-Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.MMapDirectory

   With this config, when I start solr with a 30G index, I expected that the
RAM usage should go up, but it did not.

Option 2. By Code Change
I made the following code change :

   Changed org.apache.solr.core.StandardDirectoryFactory to use MMAP instead
of FSDirectory.
   Code snippet pasted below.


Could you help me to understand if these are the right way to use MMAP?

Thanks much
/ST.

Code SNippet for Option 2:

package org.apache.solr.core;
/**
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the License); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an AS IS BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

import java.io.File;
import java.io.IOException;

import org.apache.lucene.store.Directory;
import org.apache.lucene.store.MMapDirectory;

/**
 * Directory provider which mimics original Solr FSDirectory based behavior.
 *
 */
public class StandardDirectoryFactory extends DirectoryFactory {

  public Directory open(String path) throws IOException {
return MMapDirectory.open(new File(path));
  }
}