We are using AMazon EC2 m3.x2large nodes and while the freeze is occurring
free reports
total used free shared buffers cached
Mem: 30623232 8818792 21804440 0 88092 4411832
-/+ buffers/cache: 4318868 26304364
Swap: 0 0 0
The Erlang processes seem to be unkillable because "shutdown -r now" is
also hanging. Right now these nodes are just being used for some testing,
but eventually we will go into production and I really need to have a plan
for how to detect and then deal with these Erlang freezes. Or better yet, a
way to avoid them even if it means detecting some condition in advance and
then rebooting the node.
On Wed, Mar 19, 2014 at 2:07 PM, Matthew Von-Maszewski
<[email protected]>wrote:
>
> Any chance you are overflowing into swap? Or in the case of XEN have you
> exceeded the guaranteed RAM for the VM memory and moved into the disk
> backed portion of "ram"?
>
> What backend do you use within riak? Do you have memory statistics from
> before and after the seizure/freeze?
>
> Matthew
>
>
> On Mar 19, 2014, at 4:56 PM, Michael Dillon <[email protected]>
> wrote:
>
> > I've run into a problem with Riak freezing completely on one node
> running on Ubuntu 12.04 LTS on a XEN VM (EC2). If I ssh into the node and
> run "ps ax" that shell session also freezes. I also tried another ssh
> session with "netstat -lnp" to see if I could find the process ID to kill,
> but that also froze.
> >
> > I must admit that I have seen a similar problem with RabbitMQ running on
> Ubuntu 10 LTS on a an OpenVPS VM a few years ago.
> >
> > I suppose this is an Erlang issue of some sort, but I would really like
> some way to kill the Riak processes without a reboot if possible.
> >
> > --
> > PageFreezer.com
> > #200 - 311 Water Street
> > Vancouver, BC V6B 1B8
> > _______________________________________________
> > riak-users mailing list
> > [email protected]
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
--
PageFreezer.com
#200 - 311 Water Street
Vancouver, BC V6B 1B8
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
