Greetings Phil,
I got it, this is less a failover configuration question and more of a
notification question. It sounds like either Swarm and/or Kubernetes is
juggling the instances just fine, and you just want actionable notice as
to the status of the 5 nodes, so that you can do something (e.g. grow
storage & memory on surviving zLinux instances, start replacement zLinux
instances, drive to package store, call Mom, etc. :^). I like the "call
Mom" option myself, as long as Mom stands for "Massive Object Monitor",
you Jersey Boys do things in a big way. After 167 docker instances on
three zLinuxen you get 250 docker instances on two zLinuxen and finally
the whole plate of 500 instances on one solitary standing virtual, and
this could happen in milliseconds, if Mom is flesh and blood she will not
have the time to react before the whole calliope comes to a complete halt.
Have a great weekend and...
Kindest Regards,
Paul Flint
On Fri, 28 Oct 2016, PHILIP TULLY wrote:
Date: Fri, 28 Oct 2016 13:44:01 -0400
From: PHILIP TULLY <tull...@optonline.net>
Reply-To: Linux on 390 Port <LINUX-390@VM.MARIST.EDU>
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on Z
Paul,
The fall over works.. Our current test has 100 containers running in each of
5 nodes. One node fails the other nodes now have 125 containers. This works
but if we fail another node, the additional 42 nodes might not fit on the
remaining 3 nodes, so I could issue commands to grow the memory of these
nodes, but i am not receiving any notification that the remains 3 nodes must
grow to accommodate the additional workload
On Fri, Oct 28, 2016 at 12:49 PM, Paul Flint wrote:
Greetings Phil,
My thoughts revolved around a Hot Standby type solution to maybe get the
ball rolling in what is clearly the right direction. Clustering could come
later.
Anyway, have a great weekend. It is snowing here in Vermont.
Regards,
Paul
On Fri, 28 Oct 2016, PHILIP TULLY wrote:
Date: Fri, 28 Oct 2016 10:00:32 -0400
From: PHILIP TULLY Reply-To: Linux on 390 Port To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on Z
The tools we use to do this re-swizzling of workload is
(Swarm/Kubernetes)
On Thu, Oct 27, 2016 at 10:12 PM, Phil Tully wrote:
Mike We know how to modify the CPU/memory dynamically. The issue is
how do we get the docker components to signal that they are about to
deploy more workload than the current memory size can handle so we can
grow it. Phil
Sent from my iPhone
On Oct 27, 2016, at 9:52 PM, Mike Friesenegger wrote:
Have you looked at cpuplugd -
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
- which uses a set of rules to dynamically enable or disable CPUs and
also dynamically add or remove memory.
Mike Friesenegger
On 10/27/2016 06:56 PM, PHILIP TULLY wrote:
The issue here is we have multiple docker engines on multiple lpars
(
we still think from an economics and manageability point of view
that
under VM is better than on the metal).
We have been doing the testing to have one engine pick up the
workload
form another that has failed, this works.
We are still trying to make the environment more flexible. These
docker engine VMs are sized at 60G and 8 vcpu but can grow (without
ipl) to 120G and 16 vcpu's. It is the automation piece to exploit
the flexibility of the platform that we need to figure out. Yes, we
can define full and "waste" resources but at these sizes the
resources
are big.
On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:
On Tue, 25 Oct 2016, PHILIP TULLY wrote:
We are looking to implement Docker on Z, as we have begun the
testing
part of the issue is to be able to grow a docker engine and
growing it
dynamically based on it's current needs especially when a node in
the
Docker cluster fails.
So the question is does anyone see a way for the VM system to see
the
memory resource grow, which would allow me to add more
dynamically.
I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down
-- Russ herrold
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Kindest Regards,
☮ Paul Flint
(802) 479-2360 Home
(802) 595-9365 Cell
/************************************
Based upon email reliability concerns,
please send an acknowledgement in response to this note.
Paul Flint
17 Averill Street
Barre, VT
05641
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
Kindest Regards,
☮ Paul Flint
(802) 479-2360 Home
(802) 595-9365 Cell
/************************************
Based upon email reliability concerns,
please send an acknowledgement in response to this note.
Paul Flint
17 Averill Street
Barre, VT
05641
----------------------------------------------------------------------
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
----------------------------------------------------------------------
For more information on Linux on System z, visit
http://wiki.linuxvm.org/
