Re: [gridengine users] vmem allocation

2012-12-19 Thread Reuti
Hi, Am 19.12.2012 um 18:13 schrieb Ian Kaufman: > Well, that is what you asked for - setting it to YES means applying it per > slot per the man pages. > > What kind of behavior are you really looking for? If set to JOB, then the > resource is debited from the queue resource, which in your case

Re: [gridengine users] Cluster Data Management

2012-12-19 Thread Chris Dagdigian
My own random thoughts about the storage pod and random ideas. (1) The backblaze pod we built (Rayson has my bioteam.net URL in his post below) cost roughly $12K for 100 terabytes of usable storage. Even with all the downsides and negatives to this particular hardware rig the "$12,000 for 100

Re: [gridengine users] vmem allocation

2012-12-19 Thread Brett Taylor
I wasn’t fully understanding where GE was pulling the available resources from before. It makes sense now, thanks for explaining the differences. Brett Taylor Systems Administrator Center for Systems and Computational Biology The Wistar Institute 3601 Spruce St. Room 214 Philadelphia PA 19104 T

[gridengine users] Cluster Data Management

2012-12-19 Thread Rayson Ho
Somewhat related to the email message below: I am going to play with a StoragePod 2.0 for the next project in 2013... The machine uses off-the-shelf components like motherboard, CPU, 45 disks, memory, power supply... and the real intellectual property that was released by Backblaze is the 4U custom

Re: [gridengine users] vmem allocation

2012-12-19 Thread Ian Kaufman
Well, that is what you asked for - setting it to YES means applying it per slot per the man pages. What kind of behavior are you really looking for? If set to JOB, then the resource is debited from the queue resource, which in your case, has no limit (h_vmem is set to INFINITY). So, since there is

[gridengine users] A 10, 000-node Grid Engine Cluster in Amazon EC2

2012-12-19 Thread Rayson Ho
A few weeks ago, we stress tested commlib (for those who don't know the code, commlib is the communication library in Grid Engine) to make sure that Grid Engine works in clusters larger than 10,000 nodes, and works efficiently. We blogged the experience and you can read it at: http://blogs.scalabl

Re: [gridengine users] vmem allocation

2012-12-19 Thread Brett Taylor
Ok, that seems to be working, but it changes the submission time `-l h_vmem=` to be per thread instead of per job. Not a big deal, just a little annoying to have to calculate each time. Thanks Brett Taylor Systems Administrator Center for Systems and Computational Biology The Wistar Institute

Re: [gridengine users] vmem allocation

2012-12-19 Thread Ian Kaufman
Hi Brett, Try setting consumable to YES instead of JOB. Ian On Wed, Dec 19, 2012 at 8:05 AM, Brett Taylor wrote: > Hello, > > ** ** > > I seem to be having some issues with the h_vmem setting and proper > allocation. I was under the impression that this would function much like > the CPU

Re: [gridengine users] Newbie streaming question

2012-12-19 Thread William Hay
On 19 December 2012 15:01, Vasileios Anagnostopoulos wrote: > Hi I have a program which starts with a file to be rendered. During > runtime, it takes rendering commands (plain commands actually) through web > services, executes them as render jobs (on the file) with opengl and > through web sock

[gridengine users] vmem allocation

2012-12-19 Thread Brett Taylor
Hello, I seem to be having some issues with the h_vmem setting and proper allocation. I was under the impression that this would function much like the CPU slots, and once all of the h_vmem had been allocated, that host would not accept more jobs. This however does not seem to be the case. I

Re: [gridengine users] jobs io utilization

2012-12-19 Thread Bartosz Biegun - EN
Hi, sge 2.6u2, linux 2.6.35 On 18.12.2012 19:53, Reuti wrote: Hi, Am 17.12.2012 um 10:53 schrieb Bartosz Biegun - EN: My qstat doesn't show iostats even through exec host giving io consumption in /proc/$PID/io files. Anyone knows why? which version of SGE and the kernel are you using? IIRC