Please send a log covering at least the 2.5 minutes you discuss, but upwards of 5 minutes would be good.

On Oct 15, 2009, at 1:26 PM, nonrenewable wrote:


Did I read that right?  330K docs == 12 GB index.

Ops, missed the dot - 1.2GB, but i don't think that should really make the difference in this case. Even if it was 12 GB it would just have some really
juicy documents, right? :)

Can you share the Solr logs and/or your config?  Is this happening
around a commit or some warming process?  After startup, with no
requests hitting it and no warming/commits/indexing, I don't see why
it would be growing.  Do you have custom code?

There is custom code around the solrj API however it does not explain this behaviour because of the lack of requests coming through it. There are no
indexing, commits or queries sent to the server after it's started up,
except for the initial 2 warming queries (can those be to blame for this even with no caches present??). Here are these in the log (it's on it's default verbosity so i'll refrain from posting the whole start up until necessary) After the initial start up, what you see in the log is GC every
2.5 min and Full GC every 30min. No actual activity is present.

Oct 15, 2009 1:13:36 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null params={start=0&q=fast_warm&rows=10} hits=0
status=0 QTime=16853
Oct 15, 2009 1:13:36 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
Oct 15, 2009 1:13:36 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={q=static+firstSearcher+warming+query+from+solrconfig.xml} hits=0
status=0 QTime=204
Oct 15, 2009 1:13:36 PM org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done


here is the config on it:

<config>

<abortOnConfigurationError>${solr.abortOnConfigurationError:true}</ abortOnConfigurationError>
 <dataDir>/r9/flare1.data/solr/data</dataDir>
 <indexDefaults>
   <useCompoundFile>false</useCompoundFile>
   <mergeFactor>10</mergeFactor>
   <ramBufferSizeMB>32</ramBufferSizeMB>
   <maxMergeDocs>2147483647</maxMergeDocs>
   <maxFieldLength>10000</maxFieldLength>
   <writeLockTimeout>1000</writeLockTimeout>
   <commitLockTimeout>10000</commitLockTimeout>
   <lockType>single</lockType>
 </indexDefaults>

 <mainIndex>
   <useCompoundFile>false</useCompoundFile>
   <ramBufferSizeMB>32</ramBufferSizeMB>
   <mergeFactor>10</mergeFactor>
   <maxMergeDocs>2147483647</maxMergeDocs>
   <maxFieldLength>10000</maxFieldLength>
   <unlockOnStartup>false</unlockOnStartup>
 </mainIndex>
 <jmx />

 <updateHandler class="solr.DirectUpdateHandler2">
 </updateHandler>


 <query>
   <maxBooleanClauses>1024</maxBooleanClauses>
   <queryResultWindowSize>50</queryResultWindowSize>
   <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
   <HashDocSet maxSize="3000" loadFactor="0.75"/>
   <listener event="newSearcher" class="solr.QuerySenderListener">
     <arr name="queries">
       <lst> <str name="q">solr</str> <str name="start">0</str> <str
name="rows">10</str> </lst>
       <lst> <str name="q">rocks</str> <str name="start">0</str> <str
name="rows">10</str> </lst>
       <lst><str name="q">static newSearcher warming query from
solrconfig.xml</str></lst>
     </arr>
   </listener>
   <listener event="firstSearcher" class="solr.QuerySenderListener">
     <arr name="queries">
<lst> <str name="q">fast_warm</str> <str name="start">0</str> <str
name="rows">10</str> </lst>
       <lst><str name="q">static firstSearcher warming query from
solrconfig.xml</str></lst>
     </arr>
   </listener>
   <useColdSearcher>false</useColdSearcher>
   <maxWarmingSearchers>2</maxWarmingSearchers>
 </query>

 <requestDispatcher handleSelect="true" >
   <requestParsers enableRemoteStreaming="false"
multipartUploadLimitInKB="2048" />
 </requestDispatcher>

<requestHandler name="standard" class="solr.SearchHandler" default="true">
    <lst name="defaults">
      <str name="echoParams">explicit</str>
    </lst>
 </requestHandler>

 <requestHandler name="dismax" class="solr.SearchHandler" >
   <lst name="defaults">
    <str name="defType">dismax</str>
    <str name="echoParams">explicit</str>
    <float name="tie">0.01</float>
    <str name="qf">
       text^0.5 address_t^2.0 name^1.5 brand^1.1 airport_name_t^1.0
    </str>
    <str name="pf">
       text^0.2 address_t^1.1 name^1.5 brand^1.4 brand_exact^1.9
airport_name_t^1.0
    </str>
    <str name="fl">
       id,name,price,score
    </str>
    <int name="ps">100</int>
    <str name="q.alt">*:*</str>
    <str name="hl.fl">text features name</str>
    <str name="f.name.hl.fragsize">0</str>
    <str name="f.name.hl.alternateField">name</str>
<str name="f.text.hl.fragmenter">regex</str> <!-- defined below -->
    <str name="spellcheck">true</str>
    <str name="spellcheck.extendedResults">true</str>
    <str name="spellcheck.collate">true</str>
    <str name="spellcheck.count">5</str>
   </lst>
    <arr name="last-components">
     <str>spellcheck</str>
   </arr>
 </requestHandler>
 <requestHandler name="partitioned" class="solr.SearchHandler" >
   <lst name="defaults">
    <str name="defType">dismax</str>
    <str name="echoParams">explicit</str>
    <str name="qf">text^0.5 features^1.0 name^1.2 id^10.0</str>
    <str name="mm">2&lt;-1 5&lt;-2 6&lt;90%</str>
    <str name="bq">incubationdate_dt:[* TO NOW/DAY-1MONTH]^2.2</str>
   </lst>
   <lst name="appends">
     <str name="fq">inStock:true</str>
   </lst>
</requestHandler>

 <updateRequestProcessor>
   <factory name="standard" class="solr.ChainedUpdateProcessorFactory"
default="true">
     <chain
class="com.pjaol.search.solr.update.LocalUpdateProcessorFactory">
       <str name="latField">lat</str>
       <str name="lngField">lng</str>
       <int name="startTier">9</int>
       <int name="endTier">17</int>
     </chain>
     <chain class="solr.LogUpdateProcessorFactory" >
      </chain>
     <chain class="solr.RunUpdateProcessorFactory" />
   </factory>
 </updateRequestProcessor>

 <requestHandler name="geo"
class="com.pjaol.search.solr.LocalSolrRequestHandler">
     <str name="latField">lat</str>
    <str name="lngField">lng</str>
 </requestHandler>

 <searchComponent name="spellcheck" class="solr.SpellCheckComponent">

   <str name="queryAnalyzerFieldType">textSpell</str>

   <lst name="spellchecker">
     <str name="name">default</str>
     <str name="field">spell</str>
     <str name="spellcheckIndexDir">./spellchecker1</str>

   </lst>
   <lst name="spellchecker">
     <str name="name">jarowinkler</str>
     <str name="field">spell</str>
      <str
name = "distanceMeasure ">org.apache.lucene.search.spell.JaroWinklerDistance</str>
     <str name="spellcheckIndexDir">./spellchecker2</str>

   </lst>
</searchComponent>

  <requestHandler name="/spellCheckCompRH" class="solr.SearchHandler">
   <lst name="defaults">
      <str name="spellcheck.onlyMorePopular">false</str>
      <str name="spellcheck.extendedResults">false</str>
      <str name="spellcheck.count">1</str>
   </lst>
   <arr name="last-components">
     <str>spellcheck</str>
   </arr>
 </requestHandler>

<searchComponent name="elevator" class="solr.QueryElevationComponent" >
    <str name="queryFieldType">string</str>
   <str name="config-file">elevate.xml</str>
 </searchComponent>

  <requestHandler name="/elevate" class="solr.SearchHandler"
startup="lazy">
   <lst name="defaults">
     <str name="echoParams">explicit</str>
   </lst>
   <arr name="last-components">
     <str>elevator</str>
   </arr>
 </requestHandler>


<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" /> <requestHandler name="/analysis" class="solr.AnalysisRequestHandler" />


 <requestHandler name="/update/csv" class="solr.CSVRequestHandler"
startup="lazy" />


 <requestHandler name="/admin/"
class="org.apache.solr.handler.admin.AdminHandlers" />

  <requestHandler name="/admin/ping" class="PingRequestHandler">
   <lst name="defaults">
     <str name="qt">standard</str>
     <str name="q">solrpingquery</str>
     <str name="echoParams">all</str>
   </lst>
 </requestHandler>

 <requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
   <lst name="defaults">
    <str name="echoParams">explicit</str>
    <str name="echoHandler">true</str>
   </lst>
 </requestHandler>

 <highlighting>
<fragmenter name="gap" class="org.apache.solr.highlight.GapFragmenter"
default="true">
   <lst name="defaults">
    <int name="hl.fragsize">100</int>
   </lst>
  </fragmenter>

  <fragmenter name="regex"
class="org.apache.solr.highlight.RegexFragmenter">
   <lst name="defaults">
      <int name="hl.fragsize">70</int>
      <float name="hl.regex.slop">0.5</float>
      <str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
   </lst>
  </fragmenter>

<formatter name="html" class="org.apache.solr.highlight.HtmlFormatter"
default="true">
   <lst name="defaults">
    <str name="hl.simple.pre"><![CDATA[<em>]]></str>
    <str name="hl.simple.post"><![CDATA[</em>]]></str>
   </lst>
  </formatter>
 </highlighting>

 <queryResponseWriter name="xslt"
class="org.apache.solr.request.XSLTResponseWriter">
   <int name="xsltCacheLifetimeSeconds">5</int>
 </queryResponseWriter>

 <admin>
   <defaultQuery>solr</defaultQuery>

 </admin>

</config>
--
View this message in context: 
http://www.nabble.com/Solr-Lucene-keeps-eating-up-memory-while-idling-tp25894357p25912727.html
Sent from the Solr - User mailing list archive at Nabble.com.


--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene:
http://www.lucidimagination.com/search

Reply via email to