Re: Docker on Z

[Paul Flint]

Greetings Phil,

I got it, this is less a failover configuration question and more of a 
notification question.  It sounds like either Swarm and/or Kubernetes is 
juggling the instances just fine, and you just want actionable notice as 
to the status of the 5 nodes, so that you can do something (e.g. grow 
storage & memory on surviving zLinux instances, start replacement zLinux 
instances, drive to package store, call Mom, etc. :^).  I like the "call 
Mom" option myself, as long as Mom stands for "Massive Object Monitor", 
you Jersey Boys do things in a big way.  After 167 docker instances on 
three zLinuxen you get 250 docker instances on two zLinuxen and finally 
the whole plate of 500 instances on one solitary standing virtual, and 
this could happen in milliseconds, if Mom is flesh and blood she will not 
have the time to react before the whole calliope comes to a complete halt.


Have a great weekend and...

Kindest Regards,

Paul Flint

On Fri, 28 Oct 2016, PHILIP TULLY wrote:


Date: Fri, 28 Oct 2016 13:44:01 -0400
From: PHILIP TULLY <tull...@optonline.net>
Reply-To: Linux on 390 Port <LINUX-390@VM.MARIST.EDU>
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on Z

Paul,
The fall over works..  Our current test has 100 containers running in each of 
5 nodes.  One node fails the other nodes now have 125 containers.  This works 
but if we fail another node, the additional 42 nodes might not fit on the 
remaining 3 nodes, so I could issue commands to grow the memory of these 
nodes, but i am not receiving any notification that the remains 3 nodes must 
grow to accommodate the additional workload



On Fri, Oct 28, 2016 at 12:49 PM, Paul Flint wrote:


Greetings Phil,

My thoughts revolved around a Hot Standby type solution to maybe get the 
ball rolling in what is clearly the right direction.  Clustering could come 
later.


Anyway, have a great weekend.  It is snowing here in Vermont.

Regards,

Paul

On Fri, 28 Oct 2016, PHILIP TULLY wrote:


Date: Fri, 28 Oct 2016 10:00:32 -0400
From: PHILIP TULLY Reply-To: Linux on 390 Port To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on Z

The tools we use to do this re-swizzling of workload is
(Swarm/Kubernetes)


On Thu, Oct 27, 2016 at 10:12 PM, Phil Tully wrote:


Mike We know how to modify the CPU/memory dynamically. The issue is
how do we get the docker components to signal that they are about to
deploy more workload than the current memory size can handle so we can
grow it. Phil

Sent from my iPhone


On Oct 27, 2016, at 9:52 PM, Mike Friesenegger  wrote:

Have you looked at cpuplugd -


https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
- which uses a set of rules to dynamically enable or disable CPUs and
also dynamically add or remove memory.

Mike Friesenegger


On 10/27/2016 06:56 PM, PHILIP TULLY wrote:
The issue here is we have multiple docker engines on multiple lpars
(
we still think from an economics and manageability point of view
that
under VM is better than on the metal).
We have been doing the testing to have one engine pick up the
workload
form another that has failed, this works.
We are still trying to make the environment more flexible.  These
docker engine VMs are sized at 60G and 8 vcpu but can grow (without
ipl) to 120G and 16 vcpu's.   It is the automation piece to exploit
the flexibility of the platform that we need to figure out.  Yes, we
can define full and "waste" resources but at these sizes the
resources
are big.




On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:


On Tue, 25 Oct 2016, PHILIP TULLY wrote:

We are looking to implement Docker on Z, as we have begun the
testing
part of the issue is to be able to grow a docker engine and
growing it
dynamically based on it's current needs especially when a node in
the
Docker cluster fails.

So the question is does anyone see a way for the VM system to see
the
memory resource grow, which would allow me to add more
dynamically.


I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down

-- Russ herrold



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390


--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/





--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit

Re: Docker on Z

[PHILIP TULLY]

Paul,
The fall over works..  Our current test has 100 containers running in 
each of 5 nodes.  One node fails the other nodes now have 125 
containers.  This works but if we fail another node, the additional 42 
nodes might not fit on the remaining 3 nodes, so I could issue commands 
to grow the memory of these nodes, but i am not receiving any 
notification that the remains 3 nodes must grow to accommodate the 
additional workload



On Fri, Oct 28, 2016 at 12:49 PM, Paul Flint wrote:


Greetings Phil,

My thoughts revolved around a Hot Standby type solution to maybe get 
the ball rolling in what is clearly the right direction.  Clustering 
could come later.


Anyway, have a great weekend.  It is snowing here in Vermont.

Regards,

Paul

On Fri, 28 Oct 2016, PHILIP TULLY wrote:


Date: Fri, 28 Oct 2016 10:00:32 -0400
From: PHILIP TULLY Reply-To: Linux on 390 Port To: 
LINUX-390@VM.MARIST.EDU

Subject: Re: Docker on Z

The tools we use to do this re-swizzling of workload is
(Swarm/Kubernetes)


On Thu, Oct 27, 2016 at 10:12 PM, Phil Tully wrote:


Mike We know how to modify the CPU/memory dynamically. The issue is
how do we get the docker components to signal that they are about to
deploy more workload than the current memory size can handle so we 
can

grow it. Phil

Sent from my iPhone


On Oct 27, 2016, at 9:52 PM, Mike Friesenegger  wrote:

Have you looked at cpuplugd -


https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
- which uses a set of rules to dynamically enable or disable CPUs 
and

also dynamically add or remove memory.

Mike Friesenegger


On 10/27/2016 06:56 PM, PHILIP TULLY wrote:
The issue here is we have multiple docker engines on multiple 
lpars

(
we still think from an economics and manageability point of view
that
under VM is better than on the metal).
We have been doing the testing to have one engine pick up the
workload
form another that has failed, this works.
We are still trying to make the environment more flexible.  These
docker engine VMs are sized at 60G and 8 vcpu but can grow 
(without
ipl) to 120G and 16 vcpu's.   It is the automation piece to 
exploit
the flexibility of the platform that we need to figure out.  Yes, 
we

can define full and "waste" resources but at these sizes the
resources
are big.




On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:


On Tue, 25 Oct 2016, PHILIP TULLY wrote:

We are looking to implement Docker on Z, as we have begun the
testing
part of the issue is to be able to grow a docker engine and
growing it
dynamically based on it's current needs especially when a node 
in

the
Docker cluster fails.

So the question is does anyone see a way for the VM system to 
see

the
memory resource grow, which would allow me to add more
dynamically.


I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down

-- Russ herrold



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390


--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/





--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390


--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/





--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO 
LINUX-390

or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390


--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO 
LINUX-390

or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 
or visit

http://www.marist

Re: Docker on Z

[Paul Flint]

Greetings Phil,

My thoughts revolved around a Hot Standby type solution to maybe get the 
ball rolling in what is clearly the right direction.  Clustering could 
come later.


Anyway, have a great weekend.  It is snowing here in Vermont.

Regards,

Paul

On Fri, 28 Oct 2016, PHILIP TULLY wrote:


Date: Fri, 28 Oct 2016 10:00:32 -0400
From: PHILIP TULLY <tull...@optonline.net>
Reply-To: Linux on 390 Port <LINUX-390@VM.MARIST.EDU>
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on Z

The tools we use to do this re-swizzling of workload is
(Swarm/Kubernetes)


On Thu, Oct 27, 2016 at 10:12 PM, Phil Tully wrote:


Mike We know how to modify the CPU/memory dynamically. The issue is
how do we get the docker components to signal that they are about to
deploy more workload than the current memory size can handle so we can
grow it. Phil

Sent from my iPhone


On Oct 27, 2016, at 9:52 PM, Mike Friesenegger  wrote:

Have you looked at cpuplugd -

https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
- which uses a set of rules to dynamically enable or disable CPUs and
also dynamically add or remove memory.

Mike Friesenegger


On 10/27/2016 06:56 PM, PHILIP TULLY wrote:
The issue here is we have multiple docker engines on multiple lpars
(
we still think from an economics and manageability point of view
that
under VM is better than on the metal).
We have been doing the testing to have one engine pick up the
workload
form another that has failed, this works.
We are still trying to make the environment more flexible.  These
docker engine VMs are sized at 60G and 8 vcpu but can grow (without
ipl) to 120G and 16 vcpu's.   It is the automation piece to exploit
the flexibility of the platform that we need to figure out.  Yes, we
can define full and "waste" resources but at these sizes the
resources
are big.




On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:


On Tue, 25 Oct 2016, PHILIP TULLY wrote:

We are looking to implement Docker on Z, as we have begun the
testing
part of the issue is to be able to grow a docker engine and
growing it
dynamically based on it's current needs especially when a node in
the
Docker cluster fails.

So the question is does anyone see a way for the VM system to see
the
memory resource grow, which would allow me to add more
dynamically.


I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down

-- Russ herrold


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or 
visit

http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



Kindest Regards,



☮ Paul Flint
(802) 479-2360 Home
(802) 595-9365 Cell

/
Based upon email reliability concerns,
please send an acknowledgement in response to this note.

Paul Flint
17 Averill 

Re: Docker on Z

[PHILIP TULLY]

The tools we use to do this re-swizzling of workload is
(Swarm/Kubernetes)


On Thu, Oct 27, 2016 at 10:12 PM, Phil Tully wrote:


Mike We know how to modify the CPU/memory dynamically. The issue is
how do we get the docker components to signal that they are about to
deploy more workload than the current memory size can handle so we can
grow it. Phil

Sent from my iPhone


On Oct 27, 2016, at 9:52 PM, Mike Friesenegger  wrote:

Have you looked at cpuplugd -

https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
- which uses a set of rules to dynamically enable or disable CPUs and
also dynamically add or remove memory.

Mike Friesenegger


On 10/27/2016 06:56 PM, PHILIP TULLY wrote:
The issue here is we have multiple docker engines on multiple lpars
(
we still think from an economics and manageability point of view
that
under VM is better than on the metal).
We have been doing the testing to have one engine pick up the
workload
form another that has failed, this works.
We are still trying to make the environment more flexible.  These
docker engine VMs are sized at 60G and 8 vcpu but can grow (without
ipl) to 120G and 16 vcpu's.   It is the automation piece to exploit
the flexibility of the platform that we need to figure out.  Yes, we
can define full and "waste" resources but at these sizes the
resources
are big.




On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:


On Tue, 25 Oct 2016, PHILIP TULLY wrote:

We are looking to implement Docker on Z, as we have begun the
testing
part of the issue is to be able to grow a docker engine and
growing it
dynamically based on it's current needs especially when a node in
the
Docker cluster fails.

So the question is does anyone see a way for the VM system to see
the
memory resource grow, which would allow me to add more
dynamically.


I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down

-- Russ herrold


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO
LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/




--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390

--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/


--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z

[Phil Tully]

Mike 
We know how to modify the CPU/memory dynamically. The issue is how do we get 
the docker components to signal that they are about to deploy more workload 
than the current memory size can handle so we can grow it. 
Phil

Sent from my iPhone

> On Oct 27, 2016, at 9:52 PM, Mike Friesenegger  wrote:
> 
> Have you looked at cpuplugd -
> https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
> - which uses a set of rules to dynamically enable or disable CPUs and
> also dynamically add or remove memory.
> 
> Mike Friesenegger
> 
>> On 10/27/2016 06:56 PM, PHILIP TULLY wrote:
>> The issue here is we have multiple docker engines on multiple lpars (
>> we still think from an economics and manageability point of view that
>> under VM is better than on the metal).
>> We have been doing the testing to have one engine pick up the workload
>> form another that has failed, this works.
>> We are still trying to make the environment more flexible.  These
>> docker engine VMs are sized at 60G and 8 vcpu but can grow (without
>> ipl) to 120G and 16 vcpu's.   It is the automation piece to exploit
>> the flexibility of the platform that we need to figure out.  Yes, we
>> can define full and "waste" resources but at these sizes the resources
>> are big.
>> 
>> 
>> 
>>> On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:
>>> 
 On Tue, 25 Oct 2016, PHILIP TULLY wrote:
 
 We are looking to implement Docker on Z, as we have begun the testing
 part of the issue is to be able to grow a docker engine and growing it
 dynamically based on it's current needs especially when a node in the
 Docker cluster fails.
 
 So the question is does anyone see a way for the VM system to see the
 memory resource grow, which would allow me to add more dynamically.
>>> 
>>> I thought one point of Docker was to have 'fast to spin up'
>>> instances, ready to spin up, which then pulled in ephemeral
>>> data from a back end persistent store, so that a swarm of them
>>> handled load spikes, and once the spike passes, that the
>>> excess units are shut down
>>> 
>>> -- Russ herrold
>>> 
>>> --
>>> For LINUX-390 subscribe / signoff / archive access instructions,
>>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
>>> or visit
>>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>>> --
>>> For more information on Linux on System z, visit
>>> http://wiki.linuxvm.org/
>>> 
>> 
>> --
>> For LINUX-390 subscribe / signoff / archive access instructions,
>> send email to lists...@vm.marist.edu with the message: INFO LINUX-390
>> or visit
>> http://www.marist.edu/htbin/wlvindex?LINUX-390
>> --
>> For more information on Linux on System z, visit
>> http://wiki.linuxvm.org/
>> 
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z

[Mike Friesenegger]

Have you looked at cpuplugd -
https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_r_cpuplugdcmd.html
- which uses a set of rules to dynamically enable or disable CPUs and
also dynamically add or remove memory.

Mike Friesenegger

On 10/27/2016 06:56 PM, PHILIP TULLY wrote:

The issue here is we have multiple docker engines on multiple lpars (
we still think from an economics and manageability point of view that
under VM is better than on the metal).
We have been doing the testing to have one engine pick up the workload
form another that has failed, this works.
We are still trying to make the environment more flexible.  These
docker engine VMs are sized at 60G and 8 vcpu but can grow (without
ipl) to 120G and 16 vcpu's.   It is the automation piece to exploit
the flexibility of the platform that we need to figure out.  Yes, we
can define full and "waste" resources but at these sizes the resources
are big.



On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:


On Tue, 25 Oct 2016, PHILIP TULLY wrote:


We are looking to implement Docker on Z, as we have begun the testing
part of the issue is to be able to grow a docker engine and growing it
dynamically based on it's current needs especially when a node in the
Docker cluster fails.

So the question is does anyone see a way for the VM system to see the
memory resource grow, which would allow me to add more dynamically.


I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down

-- Russ herrold

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390
or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z

[PHILIP TULLY]

The issue here is we have multiple docker engines on multiple lpars ( we 
still think from an economics and manageability point of view that under 
VM is better than on the metal). 
We have been doing the testing to have one engine pick up the workload 
form another that has failed, this works.
We are still trying to make the environment more flexible.  These docker 
engine VMs are sized at 60G and 8 vcpu but can grow (without ipl) to 
120G and 16 vcpu's.   It is the automation piece to exploit the 
flexibility of the platform that we need to figure out.  Yes, we can 
define full and "waste" resources but at these sizes the resources are 
big.




On Tue, Oct 25, 2016 at 04:57 PM, R P Herrold wrote:


On Tue, 25 Oct 2016, PHILIP TULLY wrote:


We are looking to implement Docker on Z, as we have begun the testing
part of the issue is to be able to grow a docker engine and growing 
it

dynamically based on it's current needs especially when a node in the
Docker cluster fails.

So the question is does anyone see a way for the VM system to see the
memory resource grow, which would allow me to add more dynamically.


I thought one point of Docker was to have 'fast to spin up'
instances, ready to spin up, which then pulled in ephemeral
data from a back end persistent store, so that a swarm of them
handled load spikes, and once the spike passes, that the
excess units are shut down

-- Russ herrold

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 
or visit

http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z - One less lier?

[Alan Altmark]

On Wednesday, 10/26/2016 at 11:05 GMT, Paul Flint  wrote:
> In zVM land you have a kick ass memory manager that essentially lies to
> each Virtual Machine and tells it that the memory limit is in the 
Exabytes
> (gee, I love that word :^).  The guest operating system on the Virtual
> Machine in turn uses this lie to set the limit the docker engine can
> operate a docker instance based upon the lie it got from zVM.

While the *architecture* limits the memory to 16EB, the *machine* may (and 
will) establish a smaller value based on construction.

You can figure it out by setting the MAXSTOR value in your directory entry 
to 16E and then trying to DEFINE STORAGE 16E.  If you exceed the machine 
maximum, you get an error like this

Storage size (16E) exceeds hardware maximum (16T)

That's a hardware statement.  From a software point of view, z/VM supports 
virtual machines up to 1TB.   You can define larger, but they aren't 
supported. 

Alan Altmark

Senior Managing z/VM and Linux Consultant
Lab Services System z Delivery Practice
IBM Systems & Technology Group
ibm.com/systems/services/labservices
office: 607.429.3323
mobile; 607.321.7556
alan_altm...@us.ibm.com
IBM Endicott

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z - One less lier?

[Paul Flint]

Greetings Rick,

Never thought of the implications of using zLinux "heavy", and then lotsa 
dockers...  Very interesting, and saves the cost of a zVM (or ifL) 
license.


Nice!

Regards,

Flint

On Wed, 26 Oct 2016, Rick Troth wrote:


Date: Wed, 26 Oct 2016 09:15:48 -0400
From: Rick Troth <r...@casita.net>
Reply-To: Linux on 390 Port <LINUX-390@VM.MARIST.EDU>
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on Z - One less lier?

On 10/26/2016 07:05 AM, Paul Flint wrote:

In '86 land the OS, say Linux, essentially eats all the memory, and
makes it virtual (as opposed to virtuous :^).   ...


Linux does the same on z: it just parks on all memory it sees.



In zVM land you have a kick ass memory manager that essentially lies
to each Virtual Machine and tells it that the memory limit is in the
Exabytes (gee, I love that word :^).  The guest operating system on
the Virtual Machine in turn uses this lie to set the limit the docker
engine can operate a docker instance based upon the lie it got from zVM.


z/VM is a hypervisor.
All hypervisors "lie" to their guests about how much memory is installed.
You see the same thing with VMware, KVM, VirtualBox, Hyper-V, ... any of
them.



Kinda makes you wonder how hard it would be to make the docker engine
talk directly to zVM and get the lie directly from the lier...


Trivial.
And conceptually the same for any hypervisor because they all "lie"
about memory for the sake of establishing a fake "machine" which the
guest sees. (z/VM provides a direct interface which the bare metal
doesn't have, but the guest could use either that or the bare metal
method to figger it out.)



If I had a zVM account somewhere or an emulator that legally ran the
current zVM I could look into this, right now I just want to know if I
am following the your issue correctly.  I likely need to find out more
about docker clusters, but right now I have a hunch the answer to
better clustering is on the zVM side.


I should probably reply on the other thread.
I /think/ what Phil is trying to do is get some measurements from Docker
on z for the sake of sizing "production" hosting with Docker. So if I am
understanding, Phil needs to incrementally scale up the memory seen by
this one specific z/VM guest. But adding memory on-the-fly to a guest
usually requires a reboot of the guest. So others on the list have
suggested a couple of the usual tricks for tweaking memory w/o rebooting
the guest.

Docker is a good candidate for running Linux "heavy". That is, Docker
could use Linux in its own LPAR and without z/VM. (KVM is another such
candidate.) Even so, LPARs get lied to by PR/SM about how much memory is
present. (Seems these lies abound!) So Phil will want to continue his
measurements to know how big the Docker LPAR(s) should be.

-- R; <><






--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/



Kindest Regards,



☮ Paul Flint
(802) 479-2360 Home
(802) 595-9365 Cell

/
Based upon email reliability concerns,
please send an acknowledgement in response to this note.

Paul Flint
17 Averill Street
Barre, VT
05641

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z - One less lier?

[Rick Troth]

On 10/26/2016 07:05 AM, Paul Flint wrote:
> In '86 land the OS, say Linux, essentially eats all the memory, and
> makes it virtual (as opposed to virtuous :^).   ...

Linux does the same on z: it just parks on all memory it sees.


> In zVM land you have a kick ass memory manager that essentially lies
> to each Virtual Machine and tells it that the memory limit is in the
> Exabytes (gee, I love that word :^).  The guest operating system on
> the Virtual Machine in turn uses this lie to set the limit the docker
> engine can operate a docker instance based upon the lie it got from zVM.

z/VM is a hypervisor.
All hypervisors "lie" to their guests about how much memory is installed.
You see the same thing with VMware, KVM, VirtualBox, Hyper-V, ... any of
them.


> Kinda makes you wonder how hard it would be to make the docker engine
> talk directly to zVM and get the lie directly from the lier...

Trivial.
And conceptually the same for any hypervisor because they all "lie"
about memory for the sake of establishing a fake "machine" which the
guest sees. (z/VM provides a direct interface which the bare metal
doesn't have, but the guest could use either that or the bare metal
method to figger it out.)


> If I had a zVM account somewhere or an emulator that legally ran the
> current zVM I could look into this, right now I just want to know if I
> am following the your issue correctly.  I likely need to find out more
> about docker clusters, but right now I have a hunch the answer to
> better clustering is on the zVM side.

I should probably reply on the other thread.
I /think/ what Phil is trying to do is get some measurements from Docker
on z for the sake of sizing "production" hosting with Docker. So if I am
understanding, Phil needs to incrementally scale up the memory seen by
this one specific z/VM guest. But adding memory on-the-fly to a guest
usually requires a reboot of the guest. So others on the list have
suggested a couple of the usual tricks for tweaking memory w/o rebooting
the guest.

Docker is a good candidate for running Linux "heavy". That is, Docker
could use Linux in its own LPAR and without z/VM. (KVM is another such
candidate.) Even so, LPARs get lied to by PR/SM about how much memory is
present. (Seems these lies abound!) So Phil will want to continue his
measurements to know how big the Docker LPAR(s) should be.

-- R; <><






--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z

[Neale Ferguson]

We (SNA) are customizing the OpenShift stuff and will be offering it for
support and publishing images on dockerhub. We have recently published
some hyperledger images if people are interested in experimenting with
this technology. See https://hub.docker.com/u/sinenomine for all our
images and 
https://www.ibm.com/developerworks/cloud/library/cl-ibm-blockchain-101-quic
k-start-guide-for-developers-bluemix-trs/index.html for using the
hyperledger images.

Neale

On 10/25/16, 5:26 PM, "Linux on 390 Port on behalf of Filipe Miranda"
 wrote:

>Just a thought:
>
>Have you seem Origin Openshift? The community version can be built on
>Linux for z Systems:
>https://github.com/linux-on-ibm-z/docs/wiki/Building-OpenShift-Origin
>
>IBM actually did all the heavy lifting porting the components from x86 to
>s390x.
>Origin Openshift can help you to manage the resources using containers to
>provide a DevOps environment.

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z

[Filipe Miranda]

Just a thought:

Have you seem Origin Openshift? The community version can be built on Linux for 
z Systems:
https://github.com/linux-on-ibm-z/docs/wiki/Building-OpenShift-Origin

IBM actually did all the heavy lifting porting the components from x86 to s390x.
Origin Openshift can help you to manage the resources using containers to 
provide a DevOps environment.

 
--
Filipe

> On Oct 25, 2016, at 2:00 PM, R P Herrold  wrote:
> 
>> On Tue, 25 Oct 2016, PHILIP TULLY wrote:
>> 
>> We are looking to implement Docker on Z, as we have begun the testing
>> part of the issue is to be able to grow a docker engine and growing it
>> dynamically based on it's current needs especially when a node in the
>> Docker cluster fails.
>> 
>> So the question is does anyone see a way for the VM system to see the
>> memory resource grow, which would allow me to add more dynamically.
> 
> I thought one point of Docker was to have 'fast to spin up'
> instances, ready to spin up, which then pulled in ephemeral
> data from a back end persistent store, so that a swarm of them
> handled load spikes, and once the spike passes, that the
> excess units are shut down
> 
> -- Russ herrold
> 
> --
> For LINUX-390 subscribe / signoff / archive access instructions,
> send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
> http://www.marist.edu/htbin/wlvindex?LINUX-390
> --
> For more information on Linux on System z, visit
> http://wiki.linuxvm.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on Z

[Marcy Cortes]

You could define it big and run Rob's cmmflush on a periodic basis - once a 
day?  after each shift?   

I think the writeup is here somewhere
https://zvmperf.wordpress.com/

but i can't verify at the moment since it seems we have it blocked :(

Alternatively, you could write a velocity macro and pull it from esaucd2 and 
then do a send to the console for the chmem -e if you have root logged into the 
console.



-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of PHILIP 
TULLY
Sent: Tuesday, October 25, 2016 1:29 PM
To: LINUX-390@VM.MARIST.EDU
Subject: [LINUX-390] Docker on Z

We are looking to implement Docker on Z, as we have begun the testing part of 
the issue is to be able to grow a docker engine and growing it dynamically 
based on it's current needs especially when a node in the Docker cluster fails.

So the question is does anyone see a way for the VM system to see the memory 
resource grow, which would allow me to add more dynamically.

phil

--
For LINUX-390 subscribe / signoff / archive access instructions, send email to 
lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit http://wiki.linuxvm.org/

Re: Docker on z

[Mark Post]

 On 3/5/2015 at 10:50 AM, Utz Bacher utz.bac...@de.ibm.com wrote: 
 Hi there,
 
 there already has been some talk on Docker and IBM z Systems. Meanwhile, an
 s390x Docker build is now available for RHEL 7 and SLES 12. Please refer to
 http://www.ibm.com/developerworks/linux/linux390/docker.html for details.

Hmm.  I thought IBM employees were not allowed to distributed binaries of open 
source software outside of the company.  Has this changed for the better?


Mark Post

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on z

[Utz Bacher]

Hi Mark,

Mark Post mp...@suse.com wrote on 06.03.2015 20:10:15:
 Hmm.  I thought IBM employees were not allowed to distributed
 binaries of open source software outside of the company.  Has this
 changed for the better?

well, this is what our processes in this specific case ended up with. So I
believe your statement may not be valid as generic as it is phrased...

Thanks,
Utz

:wq
--
IBM Deutschland Research  Development GmbH; Vorsitzende des Aufsichtsrats:
Martina Koederitz; Geschäftsführung: Dirk Wittkopp; Sitz der Gesellschaft:
Böblingen; Registergericht: Amtsgericht Stuttgart, HRB 243294
--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on z

[Rob van der Heij]

On 6 March 2015 at 22:08, Utz Bacher utz.bac...@de.ibm.com wrote:

 Hi Mark,

 Mark Post mp...@suse.com wrote on 06.03.2015 20:10:15:
  Hmm.  I thought IBM employees were not allowed to distributed
  binaries of open source software outside of the company.  Has this
  changed for the better?

 well, this is what our processes in this specific case ended up with. So I
 believe your statement may not be valid as generic as it is phrased...

 Thanks,
 Utz


Must be just fine... ;-) https://www.utzcertified.org/

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on z

[Agblad Tore]

Yes, thank's for the tip, now we don't have to compile it ourselves :-)

BR /Tore

 
Tore Agblad 
zOpen Teamleader
IT Services

Volvo Group Headquarters
Corporate Process  IT
Assar Gabrielssons väg 9
SE-405 08, Gothenburg  Sweden 
E-mail: tore.agb...@volvo.com 
http://www.volvo.com/volvoit/global/en-gb/ 


-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Tito 
Garrido
Sent: den 5 mars 2015 5:20
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on z

That is awesome! Thanks Utz I will take a look :)

On Thu, Mar 5, 2015 at 12:50 PM, Utz Bacher utz.bac...@de.ibm.com wrote:

 Hi there,

 there already has been some talk on Docker and IBM z Systems. Meanwhile, an
 s390x Docker build is now available for RHEL 7 and SLES 12. Please refer to
 http://www.ibm.com/developerworks/linux/linux390/docker.html for details.

 At this time, http://containerz.blogspot.com/ contains an intro and
 tutorial with z in mind, and more technical aspects around Docker on z are
 to come; and http://mainframedebate.com/ is a very good source for hot
 infrastructure topics and more, including Docker.

 Please note this is not a product, but intended to enable everyone to gain
 early experience with Docker on z Systems. As always, we are interested in
 hearing your opinion.

 Thanks,
 Utz

 Linux on System z Lead Architect

 :wq
 --
 IBM Deutschland Research  Development GmbH; Vorsitzende des Aufsichtsrats:
 Martina Koederitz; Geschäftsführung: Dirk Wittkopp; Sitz der Gesellschaft:
 Böblingen; Registergericht: Amtsgericht Stuttgart, HRB 243294
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/




-- 

Linux User #387870
.
 _/_õ|__|
..º[ .-.___.-._| . . . .
.__( o)__( o).:___

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on z

[Marcy Cortes]

At SHARE this week. Apparently you weren't the only ones thinking of doing it 
or starting to have done it already.

Hmm. An opportunity for some better community collaboration??



Marcy


-Original Message-
From: Agblad Tore [tore.agb...@volvo.commailto:tore.agb...@volvo.com]
Sent: Friday, March 06, 2015 02:26 AM Central Standard Time
To: LINUX-390@VM.MARIST.EDU
Subject: Re: [LINUX-390] Docker on z


Yes, thank's for the tip, now we don't have to compile it ourselves :-)

BR /Tore


Tore Agblad
zOpen Teamleader
IT Services

Volvo Group Headquarters
Corporate Process  IT
Assar Gabrielssons väg 9
SE-405 08, Gothenburg  Sweden
E-mail: tore.agb...@volvo.com
http://www.volvo.com/volvoit/global/en-gb/


-Original Message-
From: Linux on 390 Port [mailto:LINUX-390@VM.MARIST.EDU] On Behalf Of Tito 
Garrido
Sent: den 5 mars 2015 5:20
To: LINUX-390@VM.MARIST.EDU
Subject: Re: Docker on z

That is awesome! Thanks Utz I will take a look :)

On Thu, Mar 5, 2015 at 12:50 PM, Utz Bacher utz.bac...@de.ibm.com wrote:

 Hi there,

 there already has been some talk on Docker and IBM z Systems. Meanwhile, an
 s390x Docker build is now available for RHEL 7 and SLES 12. Please refer to
 http://www.ibm.com/developerworks/linux/linux390/docker.html for details.

 At this time, http://containerz.blogspot.com/ contains an intro and
 tutorial with z in mind, and more technical aspects around Docker on z are
 to come; and http://mainframedebate.com/ is a very good source for hot
 infrastructure topics and more, including Docker.

 Please note this is not a product, but intended to enable everyone to gain
 early experience with Docker on z Systems. As always, we are interested in
 hearing your opinion.

 Thanks,
 Utz

 Linux on System z Lead Architect

 :wq
 --
 IBM Deutschland Research  Development GmbH; Vorsitzende des Aufsichtsrats:
 Martina Koederitz; Geschäftsführung: Dirk Wittkopp; Sitz der Gesellschaft:
 Böblingen; Registergericht: Amtsgericht Stuttgart, HRB 243294
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/




--

Linux User #387870
.
 _/_õ|__|
..º[ .-.___.-._| . . . .
.__( o)__( o).:___

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/

Re: Docker on z

[Tito Garrido]

That is awesome! Thanks Utz I will take a look :)

On Thu, Mar 5, 2015 at 12:50 PM, Utz Bacher utz.bac...@de.ibm.com wrote:

 Hi there,

 there already has been some talk on Docker and IBM z Systems. Meanwhile, an
 s390x Docker build is now available for RHEL 7 and SLES 12. Please refer to
 http://www.ibm.com/developerworks/linux/linux390/docker.html for details.

 At this time, http://containerz.blogspot.com/ contains an intro and
 tutorial with z in mind, and more technical aspects around Docker on z are
 to come; and http://mainframedebate.com/ is a very good source for hot
 infrastructure topics and more, including Docker.

 Please note this is not a product, but intended to enable everyone to gain
 early experience with Docker on z Systems. As always, we are interested in
 hearing your opinion.

 Thanks,
 Utz

 Linux on System z Lead Architect

 :wq
 --
 IBM Deutschland Research  Development GmbH; Vorsitzende des Aufsichtsrats:
 Martina Koederitz; Geschäftsführung: Dirk Wittkopp; Sitz der Gesellschaft:
 Böblingen; Registergericht: Amtsgericht Stuttgart, HRB 243294
 --
 For LINUX-390 subscribe / signoff / archive access instructions,
 send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or
 visit
 http://www.marist.edu/htbin/wlvindex?LINUX-390
 --
 For more information on Linux on System z, visit
 http://wiki.linuxvm.org/




-- 

Linux User #387870
.
 _/_õ|__|
..º[ .-.___.-._| . . . .
.__( o)__( o).:___

--
For LINUX-390 subscribe / signoff / archive access instructions,
send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit
http://www.marist.edu/htbin/wlvindex?LINUX-390
--
For more information on Linux on System z, visit
http://wiki.linuxvm.org/