I have a largish index with a lot of churn, and inserts that come in
large bursts. My server is a multiprocessor with plenty of memory, so I
can multi-thread and stuff in about 1.6 million records per hour, going
full speed. I use a dozen or so threads to post curl inserts, and
monitor the responses.
Using jetty, there is ~10% failure rate with no server response code
received. Switching to tomcat reduces the error rate to around 2%.
(which makes me like tomcat a lot, even though I'm a dog person...). I
suspect I'm overrunning the capacity of the servlet container. Tweaking
parameters in Jetty improved performance, and I can tune Tomcat. But
then I'll just be overrunning a tuned system, at a slightly faster rate.
My work around is to keep track of which inserts fail, but I suspect
there's a better approach. Any suggestions how I can balance maximum
insert speed with a low error rate? Thanks!
-Jim
- No server response code on insert: how do I avoid this a... Paleo Tek
-