Your right, its nasty, but its how Fork works. I would say its something
that should be fixed, its so nasty, but with the new all Java
replication, its probably a moot point.
Forking for a small script on something that can have such a large
memory footprint is just a huge waste of resources. Ideally you might
have a tiny program running, listening on a socket or something, and it
can be alerted and do the actual fork (being small itself). Or some
other such workaround, other than copying a few gig into RAM or swap :)
The new all Java replication looks a little nicer in the face of this
(someone was asking about the differences earlier).
- Mark
Brian Whitman wrote:
Thanks for the pointer. (It seems really weird to alloc 5GB of swap just
because the JVM needs to run a shell script.. but I get hoss's explanation
in the following post)
On Fri, Jan 2, 2009 at 2:37 PM, Bill Au <bill.w...@gmail.com> wrote:
add more swap space:
http://www.nabble.com/Not-enough-space-to11423199.html#a11424938
Bill
On Fri, Jan 2, 2009 at 10:52 AM, Brian Whitman <br...@echonest.com> wrote:
I have an indexing machine on a test server (a mid-level EC2 instance,
8GB
of RAM) and I run jetty like:
java -server -Xms5g -Xmx5g -XX:MaxPermSize=128m
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heap
-Dsolr.solr.home=/vol/solr -Djava.awt.headless=true -jar start.jar
The indexing master is set to snapshoot on commit. Sometimes (not always)
the snapshot fails with
SEVERE: java.io.IOException: Cannot run program
"/vol/solr/bin/snapshooter":
java.io.IOException: error=12, Cannot allocate memory
at java.lang.ProcessBuilder.start(Unknown Source)
Why would snapshooter need more than 2GB ram? /proc/meminfo says (with
solr
running & nothing else)
MemTotal: 7872040 kB
MemFree: 2018404 kB
Buffers: 67704 kB
Cached: 2161880 kB
SwapCached: 0 kB
Active: 3446348 kB
Inactive: 2186964 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 8 kB
Writeback: 0 kB
AnonPages: 3403728 kB
Mapped: 12016 kB
Slab: 37804 kB
SReclaimable: 20048 kB
SUnreclaim: 17756 kB
PageTables: 7476 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 3936020 kB
Committed_AS: 5383624 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 340 kB
VmallocChunk: 34359738027 kB
