Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2015-03-02 Thread Dmitry Pyzhov
Actually, we have some improvements with snapshots size and we are going to
rethink our snapshots in upcoming releases. Miroslav, could you clarify the
importance of this change in 6.1?

On Thu, Jan 29, 2015 at 2:30 PM, Tomasz Napierala tnapier...@mirantis.com
wrote:

 Guys,

 We have requests for this improvement. It will help with huge
 environments, we are talking about 5GiB of logs.
 Is it on the agenda?

 Regards,


  On 22 Dec 2014, at 07:28, Bartlomiej Piotrowski 
 bpiotrow...@mirantis.com wrote:
 
  FYI, xz with multithreading support (5.2 release) has been marked as
 stable yesterday.
 
  Regards,
  Bartłomiej Piotrowski
 
  On Mon, Nov 24, 2014 at 12:32 PM, Bartłomiej Piotrowski 
 bpiotrow...@mirantis.com wrote:
  On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com
 wrote:
   I did this exercise over many iterations during Docker container
   packing and found that as long as the data is under 1gb, it's going to
   compress really well with xz. Over 1gb and lrzip looks more attractive
   (but only on high memory systems). In reality, we're looking at log
   footprints from OpenStack environments on the order of 500mb to 2gb.
  
   xz is very slow on single-core systems with 1.5gb of memory, but it's
   quite a bit faster if you run it on a more powerful system. I've found
   level 4 compression to be the best compromise that works well enough
   that it's still far better than gzip. If increasing compression time
   by 3-5x is too much for you guys, why not just go to bzip? You'll
   still improve compression but be able to cut back on time.
  
   Best Regards,
   Matthew Mosesohn
 
  Alpha release of xz supports multithreading via -T (or —threads)
 parameter.
  We could also use pbzip2 instead of regular bzip to cut some time on
 multi-core
  systems.
 
  Regards,
  Bartłomiej Piotrowski
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 --
 Tomasz 'Zen' Napierala
 Sr. OpenStack Engineer
 tnapier...@mirantis.com







 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2015-01-29 Thread Tomasz Napierala
Guys,

We have requests for this improvement. It will help with huge environments, we 
are talking about 5GiB of logs.
Is it on the agenda?

Regards,


 On 22 Dec 2014, at 07:28, Bartlomiej Piotrowski bpiotrow...@mirantis.com 
 wrote:
 
 FYI, xz with multithreading support (5.2 release) has been marked as stable 
 yesterday.
 
 Regards,
 Bartłomiej Piotrowski
 
 On Mon, Nov 24, 2014 at 12:32 PM, Bartłomiej Piotrowski 
 bpiotrow...@mirantis.com wrote:
 On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote:
  I did this exercise over many iterations during Docker container
  packing and found that as long as the data is under 1gb, it's going to
  compress really well with xz. Over 1gb and lrzip looks more attractive
  (but only on high memory systems). In reality, we're looking at log
  footprints from OpenStack environments on the order of 500mb to 2gb.
 
  xz is very slow on single-core systems with 1.5gb of memory, but it's
  quite a bit faster if you run it on a more powerful system. I've found
  level 4 compression to be the best compromise that works well enough
  that it's still far better than gzip. If increasing compression time
  by 3-5x is too much for you guys, why not just go to bzip? You'll
  still improve compression but be able to cut back on time.
 
  Best Regards,
  Matthew Mosesohn
 
 Alpha release of xz supports multithreading via -T (or —threads) parameter.
 We could also use pbzip2 instead of regular bzip to cut some time on 
 multi-core
 systems.
 
 Regards,
 Bartłomiej Piotrowski
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Tomasz 'Zen' Napierala
Sr. OpenStack Engineer
tnapier...@mirantis.com







__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-12-22 Thread Bartlomiej Piotrowski
FYI, xz with multithreading support (5.2 release) has been marked as stable
yesterday.

Regards,
Bartłomiej Piotrowski

On Mon, Nov 24, 2014 at 12:32 PM, Bartłomiej Piotrowski 
bpiotrow...@mirantis.com wrote:

 On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote:
  I did this exercise over many iterations during Docker container
  packing and found that as long as the data is under 1gb, it's going to
  compress really well with xz. Over 1gb and lrzip looks more attractive
  (but only on high memory systems). In reality, we're looking at log
  footprints from OpenStack environments on the order of 500mb to 2gb.
 
  xz is very slow on single-core systems with 1.5gb of memory, but it's
  quite a bit faster if you run it on a more powerful system. I've found
  level 4 compression to be the best compromise that works well enough
  that it's still far better than gzip. If increasing compression time
  by 3-5x is too much for you guys, why not just go to bzip? You'll
  still improve compression but be able to cut back on time.
 
  Best Regards,
  Matthew Mosesohn

 Alpha release of xz supports multithreading via -T (or —threads) parameter.
 We could also use pbzip2 instead of regular bzip to cut some time on
 multi-core
 systems.

 Regards,
 Bartłomiej Piotrowski
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-12-05 Thread Dmitry Pyzhov
I've moved the bug to 6.1. And I'm going to add it to our roadmap as a
separate item.

On Wed, Nov 26, 2014 at 1:31 PM, Mike Scherbakov mscherba...@mirantis.com
wrote:

 Can we put it as a work item for diagnostic snapshot improvements, so we
 won't forget about this in 6.1?


 On Tuesday, November 25, 2014, Dmitry Pyzhov dpyz...@mirantis.com wrote:

 Thank you all for your feedback. Request postponed to the next release.
 We will compare available solutions.

 On Mon, Nov 24, 2014 at 2:36 PM, Vladimir Kuklin vkuk...@mirantis.com
 wrote:

 guys, there is already pxz utility in ubuntu repos. let's test it

 On Mon, Nov 24, 2014 at 2:32 PM, Bartłomiej Piotrowski 
 bpiotrow...@mirantis.com wrote:

 On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com
 wrote:
  I did this exercise over many iterations during Docker container
  packing and found that as long as the data is under 1gb, it's going to
  compress really well with xz. Over 1gb and lrzip looks more attractive
  (but only on high memory systems). In reality, we're looking at log
  footprints from OpenStack environments on the order of 500mb to 2gb.
 
  xz is very slow on single-core systems with 1.5gb of memory, but it's
  quite a bit faster if you run it on a more powerful system. I've found
  level 4 compression to be the best compromise that works well enough
  that it's still far better than gzip. If increasing compression time
  by 3-5x is too much for you guys, why not just go to bzip? You'll
  still improve compression but be able to cut back on time.
 
  Best Regards,
  Matthew Mosesohn

 Alpha release of xz supports multithreading via -T (or —threads)
 parameter.
 We could also use pbzip2 instead of regular bzip to cut some time on
 multi-core
 systems.

 Regards,
 Bartłomiej Piotrowski
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Mike Scherbakov
 #mihgen



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-26 Thread Mike Scherbakov
Can we put it as a work item for diagnostic snapshot improvements, so we
won't forget about this in 6.1?

On Tuesday, November 25, 2014, Dmitry Pyzhov dpyz...@mirantis.com wrote:

 Thank you all for your feedback. Request postponed to the next release. We
 will compare available solutions.

 On Mon, Nov 24, 2014 at 2:36 PM, Vladimir Kuklin vkuk...@mirantis.com
 javascript:_e(%7B%7D,'cvml','vkuk...@mirantis.com'); wrote:

 guys, there is already pxz utility in ubuntu repos. let's test it

 On Mon, Nov 24, 2014 at 2:32 PM, Bartłomiej Piotrowski 
 bpiotrow...@mirantis.com
 javascript:_e(%7B%7D,'cvml','bpiotrow...@mirantis.com'); wrote:

 On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com
 javascript:_e(%7B%7D,'cvml','mmoses...@mirantis.com'); wrote:
  I did this exercise over many iterations during Docker container
  packing and found that as long as the data is under 1gb, it's going to
  compress really well with xz. Over 1gb and lrzip looks more attractive
  (but only on high memory systems). In reality, we're looking at log
  footprints from OpenStack environments on the order of 500mb to 2gb.
 
  xz is very slow on single-core systems with 1.5gb of memory, but it's
  quite a bit faster if you run it on a more powerful system. I've found
  level 4 compression to be the best compromise that works well enough
  that it's still far better than gzip. If increasing compression time
  by 3-5x is too much for you guys, why not just go to bzip? You'll
  still improve compression but be able to cut back on time.
 
  Best Regards,
  Matthew Mosesohn

 Alpha release of xz supports multithreading via -T (or —threads)
 parameter.
 We could also use pbzip2 instead of regular bzip to cut some time on
 multi-core
 systems.

 Regards,
 Bartłomiej Piotrowski
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 javascript:_e(%7B%7D,'cvml','OpenStack-dev@lists.openstack.org');
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com
 javascript:_e(%7B%7D,'cvml','vkuk...@mirantis.com');

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 javascript:_e(%7B%7D,'cvml','OpenStack-dev@lists.openstack.org');
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Mike Scherbakov
#mihgen
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-25 Thread Dmitry Pyzhov
Thank you all for your feedback. Request postponed to the next release. We
will compare available solutions.

On Mon, Nov 24, 2014 at 2:36 PM, Vladimir Kuklin vkuk...@mirantis.com
wrote:

 guys, there is already pxz utility in ubuntu repos. let's test it

 On Mon, Nov 24, 2014 at 2:32 PM, Bartłomiej Piotrowski 
 bpiotrow...@mirantis.com wrote:

 On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com
 wrote:
  I did this exercise over many iterations during Docker container
  packing and found that as long as the data is under 1gb, it's going to
  compress really well with xz. Over 1gb and lrzip looks more attractive
  (but only on high memory systems). In reality, we're looking at log
  footprints from OpenStack environments on the order of 500mb to 2gb.
 
  xz is very slow on single-core systems with 1.5gb of memory, but it's
  quite a bit faster if you run it on a more powerful system. I've found
  level 4 compression to be the best compromise that works well enough
  that it's still far better than gzip. If increasing compression time
  by 3-5x is too much for you guys, why not just go to bzip? You'll
  still improve compression but be able to cut back on time.
 
  Best Regards,
  Matthew Mosesohn

 Alpha release of xz supports multithreading via -T (or —threads)
 parameter.
 We could also use pbzip2 instead of regular bzip to cut some time on
 multi-core
 systems.

 Regards,
 Bartłomiej Piotrowski
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Vladimir Kuklin
IMO, we should get a bunch of snapshots and decide which compression to use
according to the results of an experiment. XZ compression takes much
longer, so you will need to use parallel xz compression utility.

On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala tnapier...@mirantis.com
wrote:


  On 21 Nov 2014, at 16:55, Dmitry Pyzhov dpyz...@mirantis.com wrote:
 
  We have a request for change compression from gz to xz. This simple
 change halfs our snapshots. Does anyone has any objections? Otherwise we
 will include this change in 6.0 release.

 I agree with the change, but it shouldn’t be high

 Regards,
 --
 Tomasz 'Zen' Napierala
 Sr. OpenStack Engineer
 tnapier...@mirantis.com







 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
45bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
vkuk...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Matthew Mosesohn
I did this exercise over many iterations during Docker container
packing and found that as long as the data is under 1gb, it's going to
compress really well with xz. Over 1gb and lrzip looks more attractive
(but only on high memory systems). In reality, we're looking at log
footprints from OpenStack environments on the order of 500mb to 2gb.

xz is very slow on single-core systems with 1.5gb of memory, but it's
quite a bit faster if you run it on a more powerful system. I've found
level 4 compression to be the best compromise that works well enough
that it's still far better than gzip. If increasing compression time
by 3-5x is too much for you guys, why not just go to bzip? You'll
still improve compression but be able to cut back on time.

Best Regards,
Matthew Mosesohn

On Mon, Nov 24, 2014 at 3:14 PM, Vladimir Kuklin vkuk...@mirantis.com wrote:
 IMO, we should get a bunch of snapshots and decide which compression to use
 according to the results of an experiment. XZ compression takes much longer,
 so you will need to use parallel xz compression utility.

 On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala tnapier...@mirantis.com
 wrote:


  On 21 Nov 2014, at 16:55, Dmitry Pyzhov dpyz...@mirantis.com wrote:
 
  We have a request for change compression from gz to xz. This simple
  change halfs our snapshots. Does anyone has any objections? Otherwise we
  will include this change in 6.0 release.

 I agree with the change, but it shouldn’t be high

 Regards,
 --
 Tomasz 'Zen' Napierala
 Sr. OpenStack Engineer
 tnapier...@mirantis.com







 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com
 www.mirantis.ru
 vkuk...@mirantis.com

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Bartłomiej Piotrowski
On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote:
 I did this exercise over many iterations during Docker container
 packing and found that as long as the data is under 1gb, it's going to
 compress really well with xz. Over 1gb and lrzip looks more attractive
 (but only on high memory systems). In reality, we're looking at log
 footprints from OpenStack environments on the order of 500mb to 2gb.
 
 xz is very slow on single-core systems with 1.5gb of memory, but it's
 quite a bit faster if you run it on a more powerful system. I've found
 level 4 compression to be the best compromise that works well enough
 that it's still far better than gzip. If increasing compression time
 by 3-5x is too much for you guys, why not just go to bzip? You'll
 still improve compression but be able to cut back on time.
 
 Best Regards,
 Matthew Mosesohn

Alpha release of xz supports multithreading via -T (or —threads) parameter.
We could also use pbzip2 instead of regular bzip to cut some time on multi-core
systems.

Regards,
Bartłomiej Piotrowski
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Vladimir Kuklin
Mattherw, Dmitry

I would suggest to use:
1) multi-threaded utilities
2) use xz for small snapshots (1gb) and lrzip for bigger snapshots

On Mon, Nov 24, 2014 at 2:25 PM, Matthew Mosesohn mmoses...@mirantis.com
wrote:

 I did this exercise over many iterations during Docker container
 packing and found that as long as the data is under 1gb, it's going to
 compress really well with xz. Over 1gb and lrzip looks more attractive
 (but only on high memory systems). In reality, we're looking at log
 footprints from OpenStack environments on the order of 500mb to 2gb.

 xz is very slow on single-core systems with 1.5gb of memory, but it's
 quite a bit faster if you run it on a more powerful system. I've found
 level 4 compression to be the best compromise that works well enough
 that it's still far better than gzip. If increasing compression time
 by 3-5x is too much for you guys, why not just go to bzip? You'll
 still improve compression but be able to cut back on time.

 Best Regards,
 Matthew Mosesohn

 On Mon, Nov 24, 2014 at 3:14 PM, Vladimir Kuklin vkuk...@mirantis.com
 wrote:
  IMO, we should get a bunch of snapshots and decide which compression to
 use
  according to the results of an experiment. XZ compression takes much
 longer,
  so you will need to use parallel xz compression utility.
 
  On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala 
 tnapier...@mirantis.com
  wrote:
 
 
   On 21 Nov 2014, at 16:55, Dmitry Pyzhov dpyz...@mirantis.com wrote:
  
   We have a request for change compression from gz to xz. This simple
   change halfs our snapshots. Does anyone has any objections? Otherwise
 we
   will include this change in 6.0 release.
 
  I agree with the change, but it shouldn’t be high
 
  Regards,
  --
  Tomasz 'Zen' Napierala
  Sr. OpenStack Engineer
  tnapier...@mirantis.com
 
 
 
 
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
  --
  Yours Faithfully,
  Vladimir Kuklin,
  Fuel Library Tech Lead,
  Mirantis, Inc.
  +7 (495) 640-49-04
  +7 (926) 702-39-68
  Skype kuklinvv
  45bk3, Vorontsovskaya Str.
  Moscow, Russia,
  www.mirantis.com
  www.mirantis.ru
  vkuk...@mirantis.com
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
45bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
vkuk...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Change diagnostic snapshot compression algoritm

2014-11-24 Thread Vladimir Kuklin
guys, there is already pxz utility in ubuntu repos. let's test it

On Mon, Nov 24, 2014 at 2:32 PM, Bartłomiej Piotrowski 
bpiotrow...@mirantis.com wrote:

 On 24 Nov 2014, at 12:25, Matthew Mosesohn mmoses...@mirantis.com wrote:
  I did this exercise over many iterations during Docker container
  packing and found that as long as the data is under 1gb, it's going to
  compress really well with xz. Over 1gb and lrzip looks more attractive
  (but only on high memory systems). In reality, we're looking at log
  footprints from OpenStack environments on the order of 500mb to 2gb.
 
  xz is very slow on single-core systems with 1.5gb of memory, but it's
  quite a bit faster if you run it on a more powerful system. I've found
  level 4 compression to be the best compromise that works well enough
  that it's still far better than gzip. If increasing compression time
  by 3-5x is too much for you guys, why not just go to bzip? You'll
  still improve compression but be able to cut back on time.
 
  Best Regards,
  Matthew Mosesohn

 Alpha release of xz supports multithreading via -T (or —threads) parameter.
 We could also use pbzip2 instead of regular bzip to cut some time on
 multi-core
 systems.

 Regards,
 Bartłomiej Piotrowski
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
45bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
vkuk...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev