Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

2016-03-15 Thread Steven Dake (stdake) 
Overlayfs is definitely faster, especially at deploy from a registry.

Regards,
-steve

From: Sam Yaple <sam...@yaple.net<mailto:sam...@yaple.net>>
Reply-To: "s...@yaple.net<mailto:s...@yaple.net>" 
<s...@yaple.net<mailto:s...@yaple.net>>, "OpenStack Development Mailing List 
(not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Tuesday, March 15, 2016 at 9:38 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

Steve,

overlayfs does not reduce the disk usage at all

Paul, we can bump the size of the docker mountpoint up to ~20GB if you check 
all the gates for the appropriate space.

Sam Yaple

On Mon, Mar 14, 2016 at 7:20 PM, Steven Dake (stdake) 
<std...@cisco.com<mailto:std...@cisco.com>> wrote:
Vikarm,

/var/lib/docker is a btrfs filesystem.  It would be nice to just use overlayfs 
as it doesn't take nearly as much disk space and is much faster then btrfs.  I 
use overlay daily and it works like a champ unless I want to rebuild, in which 
case I think a bug in the ovl yum plugin prevents rebuilding if an image 
already exists.

https://github.com/openstack/kolla/blob/master/tools/setup_RedHat.sh#L13

Regards,
-steve


From: "Vikram Hosakote (vhosakot)" 
<vhosa...@cisco.com<mailto:vhosa...@cisco.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Monday, March 14, 2016 at 11:51 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

/var/lib/docker/aufs  takes a lot of space if many containers are running and
writing data to their volumes.

If the gate VM's  /var/lib/docker  partition is small, you could move
/var/lib/docker  to a different partition that is big enough (like /home or 
/tmp)
and create a symbolic link to  /var/lib/docker.

mv  /var/lib/docker  /home/new_docker_path_with_more_space
ln -s  /home/new_docker_path_with_more_space  /var/lib/docker

Alternatively, you can use the -g option for the docker daemon to use a 
different
path instead of  /var/lib/docker  if it runs out of disk space.

DOCKER_OPTS="-g /home/new_docker_path_with_more_space"

Restart the docker daemon after making the above change.

https://github.com/docker/docker/issues/3127

Regards,
Vikram Hosakote
IRC: vhosakot

From: Paul Bourke <paul.bou...@oracle.com<mailto:paul.bou...@oracle.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Monday, March 14, 2016 at 11:25 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: [openstack-dev] [kolla] [infra] Size of images built in the gate

Hi all,

I'm currently working to add new gates for the oraclelinux image base
type (https://blueprints.launchpad.net/kolla/+spec/oraclelinux-gate) and
had a question on size available in the gate VMs.

Right now the binary build is running out of space in the gate, as it
exceeds the 10GB we're allocating for /var/lib/docker. For me, a current
local build using binary+oraclelinux is clocking in at 10.01GB, where as
the centos+binary is a little smaller at 8.89GB.

Is it written anywhere exactly what space is available in the gate VMs?
Sam mentioned briefly that different providers used in the gate give a
variety of disk sizes, so do we have an idea what we can reasonably bump
the docker partition to?

The above number indicates centos will likely soon run into the same
problem as we dockerise more of the big tent, so I think it's a good
idea to check this now before more gates start falling over.

Thanks,
-Paul

p.s. if people are interested here's lists sorted by name of the
oraclelinux and centos builds:

oraclelinux: http://paste.fedoraproject.org/339672/96915914
centos: http://paste.fedoraproject.org/339673/57969163

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (no

Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

2016-03-15 Thread Sam Yaple 
Steve,

overlayfs does not reduce the disk usage at all

Paul, we can bump the size of the docker mountpoint up to ~20GB if you
check all the gates for the appropriate space.

Sam Yaple

On Mon, Mar 14, 2016 at 7:20 PM, Steven Dake (stdake) <std...@cisco.com>
wrote:

> Vikarm,
>
> /var/lib/docker is a btrfs filesystem.  It would be nice to just use
> overlayfs as it doesn't take nearly as much disk space and is much faster
> then btrfs.  I use overlay daily and it works like a champ unless I want to
> rebuild, in which case I think a bug in the ovl yum plugin prevents
> rebuilding if an image already exists.
>
> https://github.com/openstack/kolla/blob/master/tools/setup_RedHat.sh#L13
>
> Regards,
> -steve
>
>
> From: "Vikram Hosakote (vhosakot)" <vhosa...@cisco.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Monday, March 14, 2016 at 11:51 AM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [kolla] [infra] Size of images built in the
> gate
>
> /var/lib/docker/aufs  takes a lot of space if many containers are running
> and
> writing data to their volumes.
>
> If the gate VM’s  /var/lib/docker  partition is small, you could move
> /var/lib/docker  to a different partition that is big enough (like /home
> or /tmp)
> and create a symbolic link to  /var/lib/docker.
>
> mv  /var/lib/docker  /home/new_docker_path_with_more_space
> ln -s  /home/new_docker_path_with_more_space  /var/lib/docker
>
> Alternatively, you can use the -g option for the docker daemon to use a
> different
> path instead of  /var/lib/docker  if it runs out of disk space.
>
> DOCKER_OPTS="-g /home/new_docker_path_with_more_space"
>
> Restart the docker daemon after making the above change.
>
> https://github.com/docker/docker/issues/3127
>
> Regards,
> Vikram Hosakote
> IRC: vhosakot
>
> From: Paul Bourke <paul.bou...@oracle.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Monday, March 14, 2016 at 11:25 AM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: [openstack-dev] [kolla] [infra] Size of images built in the gate
>
> Hi all,
>
> I'm currently working to add new gates for the oraclelinux image base
> type (https://blueprints.launchpad.net/kolla/+spec/oraclelinux-gate) and
> had a question on size available in the gate VMs.
>
> Right now the binary build is running out of space in the gate, as it
> exceeds the 10GB we're allocating for /var/lib/docker. For me, a current
> local build using binary+oraclelinux is clocking in at 10.01GB, where as
> the centos+binary is a little smaller at 8.89GB.
>
> Is it written anywhere exactly what space is available in the gate VMs?
> Sam mentioned briefly that different providers used in the gate give a
> variety of disk sizes, so do we have an idea what we can reasonably bump
> the docker partition to?
>
> The above number indicates centos will likely soon run into the same
> problem as we dockerise more of the big tent, so I think it's a good
> idea to check this now before more gates start falling over.
>
> Thanks,
> -Paul
>
> p.s. if people are interested here's lists sorted by name of the
> oraclelinux and centos builds:
>
> oraclelinux: http://paste.fedoraproject.org/339672/96915914
> centos: http://paste.fedoraproject.org/339673/57969163
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

2016-03-14 Thread Steven Dake (stdake) 
Vikarm,

/var/lib/docker is a btrfs filesystem.  It would be nice to just use overlayfs 
as it doesn't take nearly as much disk space and is much faster then btrfs.  I 
use overlay daily and it works like a champ unless I want to rebuild, in which 
case I think a bug in the ovl yum plugin prevents rebuilding if an image 
already exists.

https://github.com/openstack/kolla/blob/master/tools/setup_RedHat.sh#L13

Regards,
-steve


From: "Vikram Hosakote (vhosakot)" 
<vhosa...@cisco.com<mailto:vhosa...@cisco.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Monday, March 14, 2016 at 11:51 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

/var/lib/docker/aufs  takes a lot of space if many containers are running and
writing data to their volumes.

If the gate VM's  /var/lib/docker  partition is small, you could move
/var/lib/docker  to a different partition that is big enough (like /home or 
/tmp)
and create a symbolic link to  /var/lib/docker.

mv  /var/lib/docker  /home/new_docker_path_with_more_space
ln -s  /home/new_docker_path_with_more_space  /var/lib/docker

Alternatively, you can use the -g option for the docker daemon to use a 
different
path instead of  /var/lib/docker  if it runs out of disk space.

DOCKER_OPTS="-g /home/new_docker_path_with_more_space"

Restart the docker daemon after making the above change.

https://github.com/docker/docker/issues/3127

Regards,
Vikram Hosakote
IRC: vhosakot

From: Paul Bourke <paul.bou...@oracle.com<mailto:paul.bou...@oracle.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Monday, March 14, 2016 at 11:25 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: [openstack-dev] [kolla] [infra] Size of images built in the gate

Hi all,

I'm currently working to add new gates for the oraclelinux image base
type (https://blueprints.launchpad.net/kolla/+spec/oraclelinux-gate) and
had a question on size available in the gate VMs.

Right now the binary build is running out of space in the gate, as it
exceeds the 10GB we're allocating for /var/lib/docker. For me, a current
local build using binary+oraclelinux is clocking in at 10.01GB, where as
the centos+binary is a little smaller at 8.89GB.

Is it written anywhere exactly what space is available in the gate VMs?
Sam mentioned briefly that different providers used in the gate give a
variety of disk sizes, so do we have an idea what we can reasonably bump
the docker partition to?

The above number indicates centos will likely soon run into the same
problem as we dockerise more of the big tent, so I think it's a good
idea to check this now before more gates start falling over.

Thanks,
-Paul

p.s. if people are interested here's lists sorted by name of the
oraclelinux and centos builds:

oraclelinux: http://paste.fedoraproject.org/339672/96915914
centos: http://paste.fedoraproject.org/339673/57969163

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

2016-03-14 Thread Vikram Hosakote (vhosakot) 
/var/lib/docker/aufs  takes a lot of space if many containers are running and
writing data to their volumes.

If the gate VM's  /var/lib/docker  partition is small, you could move
/var/lib/docker  to a different partition that is big enough (like /home or 
/tmp)
and create a symbolic link to  /var/lib/docker.

mv  /var/lib/docker  /home/new_docker_path_with_more_space
ln -s  /home/new_docker_path_with_more_space  /var/lib/docker

Alternatively, you can use the -g option for the docker daemon to use a 
different
path instead of  /var/lib/docker  if it runs out of disk space.

DOCKER_OPTS="-g /home/new_docker_path_with_more_space"

Restart the docker daemon after making the above change.

https://github.com/docker/docker/issues/3127

Regards,
Vikram Hosakote
IRC: vhosakot

From: Paul Bourke <paul.bou...@oracle.com<mailto:paul.bou...@oracle.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Monday, March 14, 2016 at 11:25 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: [openstack-dev] [kolla] [infra] Size of images built in the gate

Hi all,

I'm currently working to add new gates for the oraclelinux image base
type (https://blueprints.launchpad.net/kolla/+spec/oraclelinux-gate) and
had a question on size available in the gate VMs.

Right now the binary build is running out of space in the gate, as it
exceeds the 10GB we're allocating for /var/lib/docker. For me, a current
local build using binary+oraclelinux is clocking in at 10.01GB, where as
the centos+binary is a little smaller at 8.89GB.

Is it written anywhere exactly what space is available in the gate VMs?
Sam mentioned briefly that different providers used in the gate give a
variety of disk sizes, so do we have an idea what we can reasonably bump
the docker partition to?

The above number indicates centos will likely soon run into the same
problem as we dockerise more of the big tent, so I think it's a good
idea to check this now before more gates start falling over.

Thanks,
-Paul

p.s. if people are interested here's lists sorted by name of the
oraclelinux and centos builds:

oraclelinux: http://paste.fedoraproject.org/339672/96915914
centos: http://paste.fedoraproject.org/339673/57969163

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org<mailto:openstack-dev-requ...@lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] [infra] Size of images built in the gate

2016-03-14 Thread Jeffrey Zhang 
​right now, there is no extra disk in the test vms. we set up a loop
device by manually. see these two link[0][1]. I think we can increase
the value to a bigger one.

[0] https://github.com/openstack/kolla/blob/master/tools/setup_RedHat.sh#L13
[1] https://github.com/openstack/kolla/blob/master/tools/setup_Debian.sh#L29
​

On Mon, Mar 14, 2016 at 11:25 PM, Paul Bourke 
wrote:

> Hi all,
>
> I'm currently working to add new gates for the oraclelinux image base type
> (https://blueprints.launchpad.net/kolla/+spec/oraclelinux-gate) and had a
> question on size available in the gate VMs.
>
> Right now the binary build is running out of space in the gate, as it
> exceeds the 10GB we're allocating for /var/lib/docker. For me, a current
> local build using binary+oraclelinux is clocking in at 10.01GB, where as
> the centos+binary is a little smaller at 8.89GB.
>
> Is it written anywhere exactly what space is available in the gate VMs?
> Sam mentioned briefly that different providers used in the gate give a
> variety of disk sizes, so do we have an idea what we can reasonably bump
> the docker partition to?
>
> The above number indicates centos will likely soon run into the same
> problem as we dockerise more of the big tent, so I think it's a good idea
> to check this now before more gates start falling over.
>
> Thanks,
> -Paul
>
> p.s. if people are interested here's lists sorted by name of the
> oraclelinux and centos builds:
>
> oraclelinux: http://paste.fedoraproject.org/339672/96915914
> centos: http://paste.fedoraproject.org/339673/57969163
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Jeffrey Zhang
Blog: http://xcodest.me
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla] [infra] Size of images built in the gate

2016-03-14 Thread Paul Bourke 

Hi all,

I'm currently working to add new gates for the oraclelinux image base 
type (https://blueprints.launchpad.net/kolla/+spec/oraclelinux-gate) and 
had a question on size available in the gate VMs.


Right now the binary build is running out of space in the gate, as it 
exceeds the 10GB we're allocating for /var/lib/docker. For me, a current 
local build using binary+oraclelinux is clocking in at 10.01GB, where as 
the centos+binary is a little smaller at 8.89GB.


Is it written anywhere exactly what space is available in the gate VMs? 
Sam mentioned briefly that different providers used in the gate give a 
variety of disk sizes, so do we have an idea what we can reasonably bump 
the docker partition to?


The above number indicates centos will likely soon run into the same 
problem as we dockerise more of the big tent, so I think it's a good 
idea to check this now before more gates start falling over.


Thanks,
-Paul

p.s. if people are interested here's lists sorted by name of the 
oraclelinux and centos builds:


oraclelinux: http://paste.fedoraproject.org/339672/96915914
centos: http://paste.fedoraproject.org/339673/57969163

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev