It's taking a little longer to write up posts describing each phase of what makes
http://wigner.it.mtu.edu/reports/ To not keep you waiting, I'm attaching few things with this email: 1. sge2sql.sql This one describes the structure/layout of MySQL table that will contain the information from 'accounting' file Please change 'database_name' to whatever fits your need. Import this one into MySQL before moving to the next step. 2. sge2sql.sh This does the parsing of SGE 'accounting' file and writes the output to 'sge_accounting.sql'. It's fairly well commented and should be easy enough to understand (one 'sort' and six 'sed'). Import this one into MySQL as well. This script should find 'accounting' file in the same folder. Also, the script overwrites 'sge_accounting.sql' everytime it's run - so, it may be a good idea to put 'sge_accounting.sql' under some sort of revision control system. 3. accounting.5 This is just the 'man' page - I keep it in the same folder for reference purposes, since I do not parse 'accounting' and store the data in MySQL in a Rocks front end. Once the stuff is in the database, it's your call as to what you want to do with it - I'll post/share my PHP scripts & SQL queries therein very soon. If you have questions, please let me know. Best, g -- Gowtham Information Technology Services Michigan Technological University (906) 487/3593 http://www.it.mtu.edu/ On Wed, 18 Jul 2012, Dave Love wrote: | Joe Borġ <[email protected]> writes: | | > Thanks Rayson, | > | > qacct is almost perfect, just a shame there's no XML view (like there is | > with most SGE tools) as I've written a few Python parsers. | | The accounting and reporting files are well-defined and easy to | parse/filter or, say, insert into a database. With XML you have | additional problems over parsing. The output from qstat et al is at | least undocumented and version-dependent, in some cases inconsistent | with the readable version, and possibly ill-formed. | | -- | Community Grid Engine: http://arc.liv.ac.uk/SGE/ | | _______________________________________________ | users mailing list | [email protected] | https://gridengine.org/mailman/listinfo/users |
accounting.5
Description: Unix manual page
sge2sql.sh
Description: Bourne shell script
DROP DATABASE IF EXISTS `database_name` ; CREATE DATABASE IF NOT EXISTS `database_name` DEFAULT CHARSET latin1 ; -- Use database_name database USE `database_name` ; -- -- SGE Accounting -- man accounting (on relevant Rocks cluster) for more information about fields -- OR run 'less accounting.5' DROP TABLE IF EXISTS `sge_accounting`; CREATE TABLE IF NOT EXISTS `sge_accounting` ( sge_id mediumint ( 15 ) NOT NULL AUTO_INCREMENT , sge_qname varchar ( 255 ) NOT NULL, sge_hostname varchar ( 255 ) NOT NULL, sge_group varchar ( 255 ) NOT NULL, sge_owner varchar ( 255 ) NOT NULL, sge_job_name varchar ( 255 ) NOT NULL, sge_job_number int unsigned NOT NULL, sge_account varchar ( 255 ) NOT NULL, sge_account_priority int unsigned NOT NULL, sge_submission_time int unsigned NOT NULL, sge_start_time int unsigned NOT NULL, sge_end_time int unsigned NOT NULL, sge_failed int unsigned NOT NULL, sge_exit_status int unsigned NOT NULL, sge_ru_wallclock int unsigned NOT NULL, sge_ru_utime int unsigned NOT NULL, sge_ru_stime int unsigned NOT NULL, sge_ru_maxrss int unsigned NOT NULL, sge_ru_ixrss int unsigned NOT NULL, sge_ru_ismrss int unsigned NOT NULL, sge_ru_idrss int unsigned NOT NULL, sge_ru_isrss int unsigned NOT NULL, sge_ru_minflt int unsigned NOT NULL, sge_ru_majflt int unsigned NOT NULL, sge_ru_nswap int unsigned NOT NULL, sge_ru_inblock int unsigned NOT NULL, sge_ru_oublock int unsigned NOT NULL, sge_ru_msgsnd int unsigned NOT NULL, sge_ru_msgrcv int unsigned NOT NULL, sge_ru_nsignals int unsigned NOT NULL, sge_ru_nvcsw int unsigned NOT NULL, sge_ru_nivcsw int unsigned NOT NULL, sge_project varchar ( 255 ) NOT NULL, sge_department varchar ( 255 ) NOT NULL, sge_granted_pe varchar ( 255 ), sge_slots int unsigned NOT NULL, sge_task_number int unsigned NOT NULL, sge_cpu int unsigned NOT NULL, sge_mem int unsigned NOT NULL, sge_io int unsigned NOT NULL, sge_category varchar ( 255 ), sge_iow int unsigned NOT NULL, sge_pe_taskid varchar ( 255 ), sge_maxvmem int unsigned NOT NULL, sge_arid int unsigned NOT NULL, sge_ar_submission_time int unsigned NOT NULL, UNIQUE KEY `sge_id` ( `sge_id` ), UNIQUE KEY `sge_job_number` ( `sge_job_number` ) ) ENGINE=InnoDB DEFAULT CHARSET latin1 AUTO_INCREMENT=1 ; CREATE INDEX `ssge_qname` ON `sge_accounting` ( `sge_qname` ); CREATE INDEX `ssge_hostname` ON `sge_accounting` ( `sge_hostname` ); CREATE INDEX `ssge_group` ON `sge_accounting` ( `sge_group` ); CREATE INDEX `ssge_owner` ON `sge_accounting` ( `sge_owner` ); CREATE UNIQUE INDEX `ssge_job_number` ON `sge_accounting` ( `sge_job_number` ); CREATE INDEX `ssge_submission_time` ON `sge_accounting` ( `sge_submission_time` ); CREATE INDEX `ssge_start_time` ON `sge_accounting` ( `sge_start_time` ); CREATE INDEX `ssge_end_time` ON `sge_accounting` ( `sge_end_time` ); CREATE INDEX `ssge_ru_wallclock` ON `sge_accounting` ( `sge_ru_wallclock` ); CREATE INDEX `ssge_slots` ON `sge_accounting` ( `sge_slots` ); CREATE INDEX `ssge_cpu` ON `sge_accounting` ( `sge_cpu` ); CREATE INDEX `ssge_io` ON `sge_accounting` ( `sge_io` ); CREATE INDEX `ssge_iow` ON `sge_accounting` ( `sge_iow` ); CREATE INDEX `ssge_ar_submission_time` ON `sge_accounting` ( `sge_ar_submission_time` );
_______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
