Re: [Beowulf] Docker vs KVM paper by IBM

2015-02-05 Thread Peter Clapham
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/01/15 20:10, Joe Landman wrote: On 01/28/2015 03:03 PM, James Cuff wrote: Large dependency radii - love it! As one of those grumpy IT guys that control the spice, I for one embrace our new container overlords! Especially once all

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-29 Thread Andrew Holway
It seems that in environments where you don't care about security then docker is a great enabler so that scientists can make any kind of mess in a sandbox type environment and no one cares because your not on a public facing network. There are however difficulties in using docker with mpi so its

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-29 Thread Gavin W. Burris
On 05:09PM Wed 01/28/15 -0800, Egan Ford wrote: On Wed, Jan 28, 2015 at 11:00 AM, Gavin W. Burris b...@wharton.upenn.edu wrote: I guess I would have to ask a few questions of the developer considering docker... WHY do you need to be outside of a self-contained directory? Given that

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Jason Riedy
And Andrew Holway writes: Docker was encouraging us to do very sloppy work. Define us. My perspective is that Docker-ish things can let system staff focus on the hardware and low-level interfaces maintained exactly the way you intend. Then application consultants, if you have those on staff,

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Joe Landman
On 1/28/15, 12:30 PM, Jason Riedy wrote: And Andrew Holway writes: Docker was encouraging us to do very sloppy work. Define us. My perspective is that Docker-ish things can let system staff focus on the hardware and low-level interfaces maintained exactly the way you intend. Then application

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Andrew Holway
This is the problem that I think everyone using Docker now is looking to solve. How can you distribute an app in a reasonable manner an remove all of the silliness you don't need in the app distribution that the base OS can solve. Its seems to encourage users to do whatever they want in the

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Andrew M.A. Cater
On Wed, Jan 28, 2015 at 02:48:33PM -0500, Joe Landman wrote: You do remember this is the Beowulf list, originally comprised of researchers who decided to be their own sysadmins/hardware vendors/etc in order to get their research done, right? Even in today's mainly post-big-iron age,

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread James Cuff
Large dependency radii - love it! As one of those grumpy IT guys that control the spice, I for one embrace our new container overlords! Especially once all that messy security stuff gets sorted out. Interestingly, there is an almost identical conversation going on the XSEDE campus list as I

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Gavin W. Burris
On 08:23PM Wed 01/28/15 +, Andrew M.A. Cater wrote: So you end up with very knowledgeable users and programmers having to build their own compilers and toolchains in the teeth of sysadmins who don't want anyone to mess up their supported systems as required by the service level

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Jason Riedy
And Gavin W. Burris writes: Great, but then how do you patch for critical vulns? Beyond all the other responses (with which I agree), consider the current GHOST issue. If the possibly vulnerable bits are within container images, then for batch jobs exposure ends once the image is finished

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Kilian Cavalotti
On Wed, Jan 28, 2015 at 11:08 AM, Ellis H. Wilson III el...@cse.psu.edu wrote: No, I really don't feel like working with my IT staff every time I want to change the toolchain or recompile such-and-such magical cache replacement algorithm into this version of the kernel. Imagine that. I just

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Ellis H. Wilson III
On 01/28/2015 03:39 PM, Jason Riedy wrote: But the growing quantity of crap software from a system level that produces useful science results can be supported without the current levels of pain. This cannot be overstated. Even (or perhaps, ESPECIALLY) in CS there is an egregious amount of

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Olli-Pekka Lehto
On 28 Jan 2015, at 22:05, Ellis H. Wilson III el...@cse.psu.edu wrote: So, the obvious answer here is, provide your standard operating environments in the form of containerized/VM/whatever images quartiles 1 and 2 can use, and allow quartiles 3 and 4 to spin up their own. Multiple

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Christopher Samuel
On 29/01/15 05:32, Joe Landman wrote: Docker/VMs allow you to package your app, once, and be done with it. New app, new package. Packaging can be done programmatically. This is the appification of HPC, à la mobile phones. It brings with it the same issue that has dogged the distro

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Joe Landman
On 01/28/2015 07:02 PM, Christopher Samuel wrote: On 29/01/15 05:32, Joe Landman wrote: Docker/VMs allow you to package your app, once, and be done with it. New app, new package. Packaging can be done programmatically. This is the appification of HPC, à la mobile phones. It brings with it

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Egan Ford
On Wed, Jan 28, 2015 at 11:00 AM, Gavin W. Burris b...@wharton.upenn.edu wrote: I guess I would have to ask a few questions of the developer considering docker... WHY do you need to be outside of a self-contained directory? Given that this is mostly an HPC crowd, this answer may not be 100%

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Gavin W. Burris
You've got it all wrong, Joe. I repeat... Docker is a great enabler! Don't all of your researchers yearn to be sysadmins? Don't all sysadmins yearn to be done with OS issues in order to free up more time to chase amber lights on hardware? But seriously, more options are a good thing. Docker

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Jason Riedy
And Gavin W. Burris writes: Sure, distribute your container, but also consider writing code that can be compiled easily in a self-contained directory like a home directory. That ship sailed long ago and isn't coming back. All the world is your laptop is pretty much how it is for current

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Joe Landman
On 1/28/15, 1:19 PM, Gavin W. Burris wrote: You've got it all wrong, Joe. I repeat... Docker is a great enabler! Don't all of your researchers yearn to be sysadmins? Don't all sysadmins yearn to be done with OS issues in order to free up more time to chase amber lights on hardware? I think

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Gavin W. Burris
Docker does seem to be quite the enabler. It seems to be enabling sloppy programmers, and encouraging users that don't know how to set LD_LIBRARY_PATH to go straight for root access. Too many cooks and all. On 06:10PM Wed 01/28/15 +0100, Andrew Holway wrote: This is the problem that I think

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Gavin W. Burris
Didn't mean to upset you there, Ellis. I'm talking about every other discipline that isn't CSE. I encourage researchers to NOT be their own IT department, so that their time is freed up to do research. Obviously if your research IS the system, that is the exception. On 02:08PM Wed 01/28/15

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Gavin W. Burris
I guess I would have to ask a few questions of the developer considering docker... WHY do you need to be outside of a self-contained directory? Are you installing things that spray files all over the filesystem? Are you breaking system-level library dependencies? Great, but then how do you

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Ellis H. Wilson III
On 01/28/2015 02:00 PM, Gavin W. Burris wrote: Researchers do not want to be sysadmins, and if they do, they aren't publishing. This statement is so infuriatingly incorrect I'm embarrassed to even be responding to it. You do remember this is the Beowulf list, originally comprised of

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Joe Landman
On 01/28/2015 02:08 PM, Ellis H. Wilson III wrote: On 01/28/2015 02:00 PM, Gavin W. Burris wrote: Researchers do not want to be sysadmins, and if they do, they aren't publishing. This statement is so infuriatingly incorrect I'm embarrassed to even be responding to it. Heh ... You do

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Ellis H. Wilson III
On 01/28/2015 02:16 PM, Gavin W. Burris wrote: Didn't mean to upset you there, Ellis. I'm talking about every other discipline that isn't CSE. I encourage researchers to NOT be their own IT department, so that their time is freed up to do research. Obviously if your research IS the system,

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-28 Thread Joe Landman
On 01/28/2015 03:03 PM, James Cuff wrote: Large dependency radii - love it! As one of those grumpy IT guys that control the spice, I for one embrace our new container overlords! Especially once all that messy security stuff gets sorted out. Interestingly, there is an almost identical

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-27 Thread Joe Landman
On 01/27/2015 10:33 AM, Jason Riedy wrote: And Andrew Holway writes: The most interesting subject around docker is security and the fact that it provides pretty much null actual containerisation I know I'm more interested in it for packageization: Provide This is the problem that I think

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-27 Thread Jason Riedy
And Andrew Holway writes: The most interesting subject around docker is security and the fact that it provides pretty much null actual containerisation I know I'm more interested in it for packageization: Provide and support a very low-level, bare OS, then let different apps build an

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-26 Thread Prentice Bisbal
It's amazing what you can get published. Those results seem pretty obvious to me. On 01/21/2015 04:26 PM, Andrew Holway wrote: *yawn* On 19 August 2014 at 18:16, Kilian Cavalotti kilian.cavalotti.w...@gmail.com wrote: Hi all, On Tue, Aug 19, 2014 at 7:10 AM, Douglas Eadline

Re: [Beowulf] Docker vs KVM paper by IBM

2015-01-21 Thread Andrew Holway
*yawn* On 19 August 2014 at 18:16, Kilian Cavalotti kilian.cavalotti.w...@gmail.com wrote: Hi all, On Tue, Aug 19, 2014 at 7:10 AM, Douglas Eadline deadl...@eadline.org wrote: I ran across this interesting paper by IBM: An Updated Performance Comparison of Virtual Machines and Linux

[Beowulf] Docker vs KVM paper by IBM

2014-08-19 Thread Douglas Eadline
I ran across this interesting paper by IBM: An Updated Performance Comparison of Virtual Machines and Linux Containers Nothing like hard numbers. I created a short article with links on Cluster Monkey: http://clustermonkey.net/Select-News/docker-versus-kvm-hard-numbers-for-hpc.html --

Re: [Beowulf] Docker vs KVM paper by IBM

2014-08-19 Thread Kilian Cavalotti
Hi all, On Tue, Aug 19, 2014 at 7:10 AM, Douglas Eadline deadl...@eadline.org wrote: I ran across this interesting paper by IBM: An Updated Performance Comparison of Virtual Machines and Linux Containers It's an interesting paper, but I kind of feel it's comparing apple to oranges. They're