Re: [Beowulf] Paper describing Google's queuing system Borg

2015-04-21 Thread Scott Atchley
Is Omega the successor? The Borg paper mentions Omega : Omega [69] supports multiple parallel, specialized “verti- cals” that are each roughly equivalent to a Borgmaster minus its persistent store and link shards. Omega schedulers use optimistic concurrency control to manipulate a shared repre-

Re: [Beowulf] Paper describing Google's queuing system Borg

2015-04-21 Thread Adam DeConinck
Is Omega the successor? John Wilkes from Google gave a presentation on Omega at Lisa '13 (video @ https://www.usenix.org/cluster-management-google). If I recall correctly, he says in the talk that Omega was developed as a potential successor to their current cluster manager, but that they were

Re: [Beowulf] Paper describing Google's queuing system Borg

2015-04-16 Thread Deepak Singh
Great to see something about the Borg design out there. Google have also written about the successor to Borg, a framework called Omega. http://research.google.com/pubs/pub41684.html PDF: http://research.google.com/pubs/archive/41684.pdf On Thu, Apr 16, 2015 at 6:53 AM Chris Samuel

Re: [Beowulf] Paper describing Google's queuing system Borg

2015-04-16 Thread Alex Chekholko
There was a talk about it at LISA '13, here are the slides and video: https://www.usenix.org/cluster-management-google Cluster Management at Google by John Wilkes IIRC most of their jobs ran for just milliseconds, so as we've learned many times before, the way they do things at Google may not

[Beowulf] Paper describing Google's queuing system Borg

2015-04-16 Thread Chris Samuel
Hi all, This is a very recent (2015) paper describing the queuing system used by Google internally, called Borg. http://research.google.com/pubs/pub43438.html Full PDF available from there. Thought it might interest some folks! All the best, Chris -- Christopher SamuelSenior

Re: [Beowulf] Paper describing Google's queuing system Borg

2015-04-16 Thread Christopher Samuel
On 17/04/15 02:26, Alex Chekholko wrote: IIRC most of their jobs ran for just milliseconds, so as we've learned many times before, the way they do things at Google may not really be applicable to the rest of the world. Possibly, though a friend of mine who works there (and who pointed me at

Re: [Beowulf] Paper describing Google's queuing system Borg

2015-04-16 Thread Deepak Singh
That is correct. The architecture assumes that many of the jobs are services, which tend to run for long periods of time.   Deepak On Thu, Apr 16, 2015 at 10:19 PM, Christopher Samuel sam...@unimelb.edu.au wrote: On 17/04/15 02:26, Alex Chekholko wrote: IIRC most of their jobs ran for