Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2015-03-02 Thread Dmitry Pyzhov
Actually, we have some improvements with snapshots size and we are going to rethink our snapshots in upcoming releases. Miroslav, could you clarify the importance of this change in 6.1? On Thu, Jan 29, 2015 at 2:30 PM, Tomasz Napierala tnapier...@mirantis.com wrote: Guys, We have requests for

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2015-01-29 Thread Tomasz Napierala
Guys, We have requests for this improvement. It will help with huge environments, we are talking about 5GiB of logs. Is it on the agenda? Regards, On 22 Dec 2014, at 07:28, Bartlomiej Piotrowski bpiotrow...@mirantis.com wrote: FYI, xz with multithreading support (5.2 release) has been

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-12-22 Thread Bartlomiej Piotrowski
FYI, xz with multithreading support (5.2 release) has been marked as stable yesterday. Regards, Bartłomiej Piotrowski On Mon, Nov 24, 2014 at 12:32 PM, Bartłomiej Piotrowski bpiotrow...@mirantis.com wrote: On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote: I did this

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-12-05 Thread Dmitry Pyzhov
I've moved the bug to 6.1. And I'm going to add it to our roadmap as a separate item. On Wed, Nov 26, 2014 at 1:31 PM, Mike Scherbakov mscherba...@mirantis.com wrote: Can we put it as a work item for diagnostic snapshot improvements, so we won't forget about this in 6.1? On Tuesday,

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-26 Thread Mike Scherbakov
Can we put it as a work item for diagnostic snapshot improvements, so we won't forget about this in 6.1? On Tuesday, November 25, 2014, Dmitry Pyzhov dpyz...@mirantis.com wrote: Thank you all for your feedback. Request postponed to the next release. We will compare available solutions. On

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-25 Thread Dmitry Pyzhov
Thank you all for your feedback. Request postponed to the next release. We will compare available solutions. On Mon, Nov 24, 2014 at 2:36 PM, Vladimir Kuklin vkuk...@mirantis.com wrote: guys, there is already pxz utility in ubuntu repos. let's test it On Mon, Nov 24, 2014 at 2:32 PM,

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Vladimir Kuklin
IMO, we should get a bunch of snapshots and decide which compression to use according to the results of an experiment. XZ compression takes much longer, so you will need to use parallel xz compression utility. On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala tnapier...@mirantis.com wrote: On

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Matthew Mosesohn
I did this exercise over many iterations during Docker container packing and found that as long as the data is under 1gb, it's going to compress really well with xz. Over 1gb and lrzip looks more attractive (but only on high memory systems). In reality, we're looking at log footprints from

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Bartłomiej Piotrowski
On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote: I did this exercise over many iterations during Docker container packing and found that as long as the data is under 1gb, it's going to compress really well with xz. Over 1gb and lrzip looks more attractive (but only on

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Vladimir Kuklin
Mattherw, Dmitry I would suggest to use: 1) multi-threaded utilities 2) use xz for small snapshots (1gb) and lrzip for bigger snapshots On Mon, Nov 24, 2014 at 2:25 PM, Matthew Mosesohn mmoses...@mirantis.com wrote: I did this exercise over many iterations during Docker container packing and

Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Vladimir Kuklin
guys, there is already pxz utility in ubuntu repos. let's test it On Mon, Nov 24, 2014 at 2:32 PM, Bartłomiej Piotrowski bpiotrow...@mirantis.com wrote: On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote: I did this exercise over many iterations during Docker container