What is the precise error? What kind of machine? File buffers are a robust part of the OS. Unix has had file buffer caching for decades.
wunder On Aug 26, 2013, at 1:37 AM, Furkan KAMACI wrote: > Hi Walter; > > You are right about performance. However when I index documents on a > machine that has a high percentage of Physical Memory usage I get EOF > errors? > > > 2013/8/26 Walter Underwood <wun...@wunderwood.org> > >> On Aug 25, 2013, at 1:41 PM, Furkan KAMACI wrote: >> >>> Sometimes Physical Memory usage of Solr is over %99 and this may cause >>> problems. Do you run such kind of a command periodically: >>> >>> sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches" >>> >>> to force dropping caches of machine that Solr runs at and avoid problems? >> >> >> This is a terrible idea. The OS automatically manages the file buffers. >> When they are all used, that is a good thing, because it reduced disk IO. >> >> After this, no files will be cached in RAM. Every single read from a file >> will have to go to disk. This will cause very slow performance until the >> files are recached. >> >> Recently, I did exactly the opposite to improve performance in our Solr >> installation. Before starting the Solr process, a script reads every file >> in the index so that it will already be in file buffers. This avoids >> several minutes of high disk IO and slow performance after startup. >> >> wunder >> Search Guy, Chegg.com >> >> >> -- Walter Underwood wun...@wunderwood.org
