For SGE, a lot of systems let you ssh into the node and use htop. That will show you a lot of information about the node, and can help you find out which process is using up what about of memory (note, this check only works in real-time so your computational has to be long enough).
On Monday, September 12, 2016 at 1:24:20 PM UTC-7, Florian Oswald wrote: > > hi all, > > i get the following output from the SGE command `qstat -j jobnumber` of a > julia job that uses 30 workers. I am confused by the mem column. am I using > more memory than what I asked for? I asked for max 4G on each processor. > > > job-array tasks: 1-30:1 > > usage 1: cpu=00:00:08, mem=20.61903 GBs, io=0.08026, > vmem=2.684G, maxvmem=2.684G > > usage 2: cpu=00:00:13, mem=35.36547 GBs, io=0.14832, > vmem=2.754G, maxvmem=2.754G > > usage 3: cpu=00:00:16, mem=47.97179 GBs, io=0.10563, > vmem=3.084G, maxvmem=3.084G > > usage 4: cpu=00:00:17, mem=52.39960 GBs, io=0.14685, > vmem=3.146G, maxvmem=3.146G > > usage 5: cpu=00:00:13, mem=38.00948 GBs, io=0.06336, > vmem=3.208G, maxvmem=3.208G > > usage 6: cpu=00:00:14, mem=41.84277 GBs, io=0.08085, > vmem=3.208G, maxvmem=3.208G > > usage 7: cpu=00:00:16, mem=49.34722 GBs, io=0.10563, > vmem=3.208G, maxvmem=3.208G > > usage 8: cpu=00:00:18, mem=56.29933 GBs, io=0.14685, > vmem=3.208G, maxvmem=3.208G > > usage 9: cpu=00:00:21, mem=61.30837 GBs, io=0.13234, > vmem=3.145G, maxvmem=3.145G > > usage 10: cpu=00:00:18, mem=53.78262 GBs, io=0.10650, > vmem=3.209G, maxvmem=3.209G > > usage 11: cpu=00:00:19, mem=58.20804 GBs, io=0.16047, > vmem=3.208G, maxvmem=3.208G > > usage 12: cpu=00:00:19, mem=58.90526 GBs, io=0.15296, > vmem=3.209G, maxvmem=3.209G > > usage 13: cpu=00:00:13, mem=37.73257 GBs, io=0.06336, > vmem=3.208G, maxvmem=3.208G > > usage 14: cpu=00:00:15, mem=43.44044 GBs, io=0.08085, > vmem=3.208G, maxvmem=3.208G > > usage 15: cpu=00:00:19, mem=58.27114 GBs, io=0.13234, > vmem=3.208G, maxvmem=3.208G > > usage 16: cpu=00:00:17, mem=51.33971 GBs, io=0.10650, > vmem=3.208G, maxvmem=3.208G > > usage 17: cpu=00:00:19, mem=56.00911 GBs, io=0.16047, > vmem=3.208G, maxvmem=3.208G > > usage 18: cpu=00:00:19, mem=57.45101 GBs, io=0.15301, > vmem=3.209G, maxvmem=3.209G > > usage 19: cpu=00:00:19, mem=56.42524 GBs, io=0.13240, > vmem=3.208G, maxvmem=3.208G > > usage 20: cpu=00:00:18, mem=52.25189 GBs, io=0.10650, > vmem=3.209G, maxvmem=3.209G > > usage 21: cpu=00:00:20, mem=60.76601 GBs, io=0.12465, > vmem=3.208G, maxvmem=3.208G > > usage 22: cpu=00:00:22, mem=65.11690 GBs, io=0.14843, > vmem=3.207G, maxvmem=3.207G > > usage 23: cpu=00:00:18, mem=52.75353 GBs, io=0.11566, > vmem=3.146G, maxvmem=3.146G > > usage 24: cpu=00:00:15, mem=44.21442 GBs, io=0.04204, > vmem=3.207G, maxvmem=3.207G > > usage 25: cpu=00:00:20, mem=58.85802 GBs, io=0.14714, > vmem=3.209G, maxvmem=3.209G > > usage 26: cpu=00:00:18, mem=53.52543 GBs, io=0.12236, > vmem=3.208G, maxvmem=3.208G > > usage 27: cpu=00:00:20, mem=59.24938 GBs, io=0.13234, > vmem=3.208G, maxvmem=3.208G > > usage 28: cpu=00:00:20, mem=59.86234 GBs, io=0.12465, > vmem=3.208G, maxvmem=3.208G > > usage 29: cpu=00:00:18, mem=53.94314 GBs, io=0.11566, > vmem=3.209G, maxvmem=3.209G > > usage 30: cpu=00:00:16, mem=48.74432 GBs, io=0.10222, > vmem=3.208G, maxvmem=3.208G >
