Sergio, Is thus for Gioco Digitale?
Gary O Nakamura Terracotta, Inc. 415-990-9844 Mobile 415-738-4011 Office On Nov 17, 2010, at 11:55 AM, Sergio Bossa <sergio.bo...@gmail.com> wrote: > To recap, there seems to be three distinct problems: > > 1) Within a running cluster, adding new nodes causes a lot of object > faults. > 2) Upon restart of a whole cluster, the active master does a lot of > faulting, eventually going OOM. > 3) Upon restart, the passive also goes OOM while trying to sync up. > > I tried to set the master eviction percentage to 25 and this seems to > help, but there's still lots of faulting activity ... > > Masters have both 1GB of heap, which is a reasonable amount of memory > I think to hold just a few GBs of data. > > Any help, maybe some tuning tips? > > Sergio Bossa > Sent by iPhone > > Il giorno 17/nov/2010, alle ore 18.36, Sergio Bossa <sergio.bo...@gmail.com >> ha scritto: > >> I tried to wait several minutes for the masters to startup and stop >> logging faults, then started the clients and tried working with them. >> But, after a few minutes of usage, masters crash with OOME ... >> >> On Wed, Nov 17, 2010 at 6:01 PM, Sergio Bossa >> <sergio.bo...@gmail.com> wrote: >>> After restarting all masters and clients, the cluster is completely >>> unusable: masters are very busy and clients cannot connect at all. >>> I'm attaching the active master logs, where I see lots of object >>> faulting and strange messages from the ObjectManager. >>> >>> Here are some info about my setup: >>> - Terracotta 3.4.0. >>> - 1.9GB of objectdb (on disk). >>> - 1 active and 1 master, each one with 1GB. >>> - 2 clients, each one with 1.5GB. >>> >>> On Wed, Nov 17, 2010 at 4:36 PM, Sergio Bossa >>> <sergio.bo...@gmail.com> wrote: >>>> Hi guys, >>>> >>>> when I try to connect a new client to a Terracotta master holding a >>>> few millions of objects (I can give you an approximated size if >>>> needed), the client fails to connect with a timeout and the master >>>> gets heavily busy trying to fault objects to send to the new client. >>>> I'm attaching a thread dump of the master: you'll notice several >>>> threads being busy at faulting in objects (see >>>> managed_object_fault_stage). >>>> Two questions: >>>> 1) Why is the master trying to fault so many objects at client >>>> connection? >>>> 2) Why is the master still busy in the fault stage even *after* the >>>> client is completely disconnected? >>>> >>>> -- >>>> Sergio Bossa >>>> http://www.linkedin.com/in/sergiob >>>> >>> >>> >>> >>> -- >>> Sergio Bossa >>> http://www.linkedin.com/in/sergiob >>> >> >> >> >> -- >> Sergio Bossa >> http://www.linkedin.com/in/sergiob > _______________________________________________ > tc-dev mailing list > tc-dev@lists.terracotta.org > http://lists.terracotta.org/mailman/listinfo/tc-dev _______________________________________________ tc-dev mailing list tc-dev@lists.terracotta.org http://lists.terracotta.org/mailman/listinfo/tc-dev
