On Fri, Jun 3, 2016 at 6:34 PM, Josh Elser <[email protected]> wrote:
> Great stuff, Keith! Looking forward to picking it apart in more detail: > > bq. In the previous run Fluo worker processes were constantly being killed > by YARN for exceeding memory limits > > I remember stumbling across some (surprising) recommendations which were > to just completely disable the vmem/pmem monitoring to avoid this. Nice > that you were actually able to work around it "properly". I am not quite sure why the workers processes are going over the heap usage by so much, because no native code should be in use. One possibility is thread stacks. The -Xmx jvm option only controls heap usage. If there are lots of threads where the stack has been deep in the past, then those threads may continue to use a good bit of memory. > > > Keith Turner wrote: > >> A 2nd multi-day test of Fluo was completed on EC2 a week or two ago. I >> just got around to finishing writing it up. The test was run using >> Accumulo 1.7.1. >> >> http://fluo.io/blog/2016/05/17/webindex-long-run-2 >> >
