Re: [galaxy-dev] Scaling tools: data available? (Peter van Heusden)

2016-03-19 Thread Peter van Heusden
Thanks Damion We've got the collectl based metric collection stuff working and that dumps data in the Galaxy DB (which you can pull out using the API). Thanks, Peter On 17 March 2016 at 10:45, Dooley, Damion wrote: > If you are looking for a simple python program to log cpu% and memory of > an

Re: [galaxy-dev] Scaling tools: data available? (Peter van Heusden)

2016-03-19 Thread Dooley, Damion
If you are looking for a simple python program to log cpu% and memory of another executable, then analyze.py might do. Its in https://github.com/Public-Health-Bioinformatics/kipper/tree/master/RDP-test -case along with a few scripts that show examples of its use. Damion >> >> On 02/21/2016 11:13

Re: [galaxy-dev] Scaling tools: data available?

2016-03-14 Thread Eric Rasche
Hi Peter, The vision I had was a cron script admins could install which would send thr past day/week's collectl/memory/parameter/dataset size logs to GRT. Again, the GRT side stalled when I hit issues with collectl locally. If you have time/energy to work on that we should chat :-) Otherwise I pl

Re: [galaxy-dev] Scaling tools: data available?

2016-03-14 Thread Peter van Heusden
Hi Eric I'm having a look at the GRT. We're collecting metrics on our local Galaxy server, I'm just trying to understand how you envision these metrics getting into the GRT? Peter On 22 February 2016 at 07:31, Eric Rasche wrote: > Hi peter, > > I've been working on this in the past but got cau

Re: [galaxy-dev] Scaling tools: data available?

2016-02-21 Thread Eric Rasche
Hi peter, I've been working on this in the past but got caught up in other projects. There's work for a server-side component here (maybe 50% done), the client side stuff could all be done through collectl but I was having issues actually collecting collectl data that had put the project on hold t

[galaxy-dev] Scaling tools: data available?

2016-02-21 Thread Peter van Heusden
Hi there We're researching the resource (CPU time and memory) requirements for RNA STAR at the moment. Specifically, we'd like to build up a database of input size to resource usage so that we can use this to feed a dynamic destination mapper (currently our STAR configuration uses a thumbsuck for