Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Status: Answered = Solved Giulia Macaro confirmed that the question is solved: Thanks Anton for trying my script. I have found that if I restart the pc just before starting the test and I do not run other programs apart from Yade, it works fine for me too. I have run some more tests on the same script, recording the total memory and also the memory used by Yade only, using sysstat. Every time I was restarting the pc before starting a new test. I have found no increasing of memory (or only very slightly, from 2.38 to 2.55 % in 44 hours) if no other processes are running. But if then I start using other programs (firefox, nautilus, and geany) at the same time, the memory used by Yade increases faster (from 2.55 to 2.80 % in the following 3 hours). The same script, but with more functions to periodically save the simulation, gives a memory usage increasing from 2.21 to 4.02 in the first 23 hours (with no other programs running) and then from 4.02 to 8.23 % in the following 3 hours (with others programs running too). I understand if only the total memory would increase, but I do not see why Yade uses more memory if I open firefox, for example. Or maybe I am missing something about how the pc works. I am wondering if someone has noticed the same behaviour before. Anyway, I am also happy to not use the pc at all while running Yade and keep the memory low. -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Anton Gladky proposed the following answer: With the latest yade-daily and about 8h of simulation I do not see any memory leakage in your script. Cheers, Anton 2012/12/6 Giulia Macaro question215...@answers.launchpad.net: Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Giulia Macaro posted a new comment: At the beginning I decided to use more facets so that I could read the force acting on different layers of the vertical walls, and also the force at the centre and at the side of the bottom floor. But if it might causes problems I can get the forces from the spheres, I guess. Giulia -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Status: Answered = Open Giulia Macaro is still having a problem: Hello, I have run more tx tests with only 5,000 spheres: I've restarted the computer, launched the test, and I have not opened any other program (not even a folder, firefox, mouse, or music). After 3 hours of running with 4 out of 8 cores, the total memory usage increased from 11.13 to 11.51% (compared with the previous 22 to 32% which I had while running other programs). At the end of the test, after 18 hours, the memory arrived up to 14.29%, which is considerably less than before. It not a sceintific way, but it gives an idea of what is happening. Some observations: - As soon as I start using the pc for other stuff, the memory used by Yade starts increasing faster - Have you noticed the same behaviour? - It is good news, I assume, that the memory does not increase much if I don't run anything else. But why, while I run other programs, does the memory used by Yade increase? I am now trying to do the same (not using the pc at all) with the large simulation (which crashed with SIGKILL before). Can I decide how much memory Yade can use? Is there any command similar to -j4 given for the number of cores? Here is the script of the large simulation. It took me 3 or 4 days only to settle the packing, and other 4 days in debug mode until it crashed. Thanks for any suggestion Giulia #!/usr/bin/python # -*- coding: utf-8 -*- from yade import pack,log,params,timing,plot if utils.runningInBatch()==False: from yade import qt import os,sys,matplotlib import numpy as np from math import * from yade import * quick=False #--- # D I M E N S I O N S passing = [0, 5.72, 24.26, 35.07, 45.38, 55.17, 70.10, 79.58, 90.19, 100.0] diameters= [0.600, 0.632, 0.710, 0.770, 0.810, 0.850, 0.940, 1.000, 1.060, 1.200] radii = []; PSDup = 4 for i in range(len(diameters)): radii.append(diameters[i]*0.5e-3*PSDup) Rmin= 0.632*0.5e-3*PSDup Rmax= 1.200*0.5e-3*PSDup AvgRadius= .400e-3*PSDup Dpipe = 50e-3 # (m) pipe diameter # Initial packing dim0= 5*2*AvgRadius dim1= 9*Dpipe dim2= 4*Dpipe extra = 1*Dpipe # clearance between the packing and the periodic cell # way of roughly estimating the initial packing poroi = 0.76 porof = 0.5 nSph= int((1-porof)*(dim0*dim1*dim2)/(4/3*pi*AvgRadius**3))# approximate nSph to get the desired final packing hi = nSph*(4/3*pi*AvgRadius**3)/((1-poroi)*dim0*dim1) # initial height of the packing # Periodic cell dimensions cell0 = dim0 cell1 = dim1+2*extra cell2 = hi+2*extra # Corners of the initial packing (in sp.particleSD) minCorner = (0,extra,extra) maxCorner = (dim0,extra+dim1,extra+hi) #--- maxUnbalancedForce = 0.05 dporoMax= 0.001 dhppMax = 0.001 Damping = 0.5 #--- # M A T E R I A L S O.materials.append( FrictMat(density=2650,young=70e10,poisson=0.3,frictionAngle=radians(0), label='bottomFloorMat')) O.materials.append( FrictMat(density=2650,young=70e10,poisson=0.3,frictionAngle=radians(0), label='rightWallMat')) O.materials.append( FrictMat(density=2650,young=70e10,poisson=0.3,frictionAngle=radians(0), label='leftWallMat')) O.materials.append( FrictMat(density=2650,young=70e09,poisson=0.3,frictionAngle=radians(26), label='sandMat')) O.materials.append( FrictMat(density=7850,young=20e10,poisson=0.3,frictionAngle=radians(26*2/3),label='pipeMat')) #--- # G E O M E T R Y n0 = 3; n1 = 5; n2 = 10 # lat direction 0; lat direction 1; vert direction 2 deltah0 = dim0/n0; deltah1 = dim1/n1; deltav = hi/n2 # # ~~~ Periodic cell ~~~ O.periodic = True O.cell.setBox(Vector3(cell0,cell1,cell2)) # set size of the cell (rectangular) # ~~~ Wall ~~~ # bottom floor si = 1./n1; sj = 1./n0 for i in range(0,n1): for j in range(0,n0): x1 = (j*sj*cell0,extra*1.0+i*si*(extra*0.0+dim1),extra); x2 = (j*sj*cell0,extra*1.0+(i+1)*si*(extra*0.0+dim1),extra);x3 = ((j+1)*sj*cell0,extra*1.0+(i+1)*si*(extra*0.0+dim1),extra);x4 = ((j+1)*sj*cell0,extra*1.0+i*si*(extra*0.0+dim1),extra) O.bodies.append([utils.facet([x1,x2,x3],wire=False,fixed=True,material='bottomFloorMat'),utils.facet([x1,x3,x4],wire=False,fixed=True,material='bottomFloorMat'),]) # right wall si = 1./n2; sj = 1./n0 for
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Giulia Macaro posted a new comment: Regarding the memory leakage described in https://bugs.launchpad.net/yade/+bug/1041084, I understood it only happens when erasing a body, doesn't it? -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Status: Open = Answered Anton Gladky proposed the following answer: Hi. 2012/12/6 Giulia Macaro question215...@answers.launchpad.net: I've restarted the computer, launched the test, and I have not opened any other program (not even a folder, firefox, mouse, or music). After 3 hours of running with 4 out of 8 cores, the total memory usage increased from 11.13 to 11.51% (compared with the previous 22 to 32% which I had while running other programs). At the end of the test, after 18 hours, the memory arrived up to 14.29%, which is considerably less than before. It not a sceintific way, but it gives an idea of what is happening. Some observations: - As soon as I start using the pc for other stuff, the memory used by Yade starts increasing faster How do you measure, whether the memory, used by Yade, is increased? - Have you noticed the same behaviour? - It is good news, I assume, that the memory does not increase much if I don't run anything else. But why, while I run other programs, does the memory used by Yade increase? At least for linux-based systems it should not be so at all. Can other applications eat all the memory? I am now trying to do the same (not using the pc at all) with the large simulation (which crashed with SIGKILL before). Can I decide how much memory Yade can use? Is there any command similar to -j4 given for the number of cores? No, memory is allocating dynamically depending on the number of bodies, interactions etc. Anton -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Chareyre proposed the following answer: Could you try without this? O.trackEnergy=True O.timingEnabled=True # to enable timing services If it doesn't change the leak, then it would be also useful to remove all the postprocessing stuff, e.g. volSph=utils.getSpheresVolume(), to see if there is a leak in a very core part of the code. The ultimate tool to detect leaks is valgrind: valgrind --leak-check=full yade myscript However, I didn't try recently, and it may well give a huge amount of results due to python now, independently of yade, which does not help to find the relevant data. -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Anton Gladky proposed the following answer: What Yade version do you use? Anton 2012/12/6 Giulia Macaro question215...@answers.launchpad.net: Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Status: Answered = Open Giulia Macaro is still having a problem: Hello, I have run more tx tests with only 5,000 spheres: -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Giulia Macaro posted a new comment: I was recording the total memory with sysstat, and the memory used only by Yade with the script given in the other bug, following the process id of Yade. The other applications open were just firefox, nautilus and a music player, maybe open office sometimes. Not that much. Actually I haven't updated yade in a while, I'm using bzr 3040. Could you try without this? O.trackEnergy=True O.timingEnabled=True # to enable timing services I'll try it. I've also tried to not plot or save anything, but it didn't make any difference. I've never used Valgridnd, I'll try it. Thanks, Giulia -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Chareyre posted a new comment: Not directly related to your question, but defining the limits of your problem with so many facets is really the source of potential problems. There are known bugs with such arrangement. I would replace them by 3 boxes. -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Anton Gladky proposed the following answer: Well, some of my simulations have thousands of facets and I did not see such difficulties. Anton 2012/12/6 Chareyre question215...@answers.launchpad.net: Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Chareyre posted a new comment: Not directly related to your question, but defining the limits of your problem with so many facets is really the source of potential problems. There are known bugs with such arrangement. I would replace them by 3 boxes. -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Chareyre posted a new comment: When you need many facets, you have to live with them, and it's probably less a problem when they represent a complex geometry. But Giulia is discretizing a plane into a large number of facets, which is really useless. It could be replaced by one box, or one big facet, even. We know that coplanar facets are prone to give weird behavior when contact is on an edge. B -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Giulia Macaro posted a new comment: At the beginning I decided to use more facets so that I could read the force acting on different layers of the vertical walls, and also the force at the centre and at the side of the bottom floor. But if it might causes problems I can get the forces from the spheres, I guess. Giulia -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Anton Gladky proposed the following answer: Your large simulation was definitely broken because of huge memory consumption (probably swap was used). Why it leaks... difficult to say. It needs to be analysed on a small amount of spheres. It can be related to a bug LP:1041084 [1]. Jan is right, a test script can be useful. Anton https://bugs.launchpad.net/yade/+bug/1041084 -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
Re: [Yade-users] [Question #215877]: SIGKILL and possible memory leakage
Question #215877 on Yade changed: https://answers.launchpad.net/yade/+question/215877 Status: Open = Answered Jan Stránský proposed the following answer: Hello Giulia. could you please provide us with the script you use? I personally did some simulations with memory leaks for no obvious reason.. Thanks Jan Dne 3.12.2012 17:56 Giulia Macaro question215...@answers.launchpad.net napsal(a): New question #215877 on Yade: https://answers.launchpad.net/yade/+question/215877 Hello, I am running a very long simulation, in which a big sphere is loaded on a packing of 50,000 smaller spheres settled under gravity force. After 4 days of running, the simulation ends without any particular reason. In the terminal it is only written killed. I have rerun the same test in the debug mode, compiling Yade with debug=1, and then $ gdb /usr/bin/python GNU gdb (GDB) 7.1-ubuntu Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as x86_64-linux-gnu. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/bin/python...(no debugging symbols found)...done. (gdb) run /home/giulia/YetAnotherDynamicEngine/12.07.09_yade/YADE/bin/yade-bzr3040 -j4 --debug 2012.11.28_1_realSph.py Starting program: /usr/bin/python /home/giulia/YetAnotherDynamicEngine/12.07.09_yade/YADE/bin/yade-bzr3040 -j4 --debug 2012.11.28_1_realSph.py [Thread debugging using libthread_db enabled] Welcome to Yade bzr3040 (debug build) TCP python prompt on localhost:9000, auth cookie `cesaud' [New Thread 0x7fffd6f2c700 (LWP 7675)] [New Thread 0x7fffd672b700 (LWP 7676)] I did not get any information apart from [New Thread ... (LWP ... )] and [Thread ... (LWP ...) exited] and then at the end I got Program terminated with signal SIGKILL, Killed. The program no longer exists. I also tried (gdb) thread apply bt full But I did not get anything. And then (gdb) bt full which gave me No stack. I was recording the CPU and memory usage during the simulation. The machine has 8 cores and 12GB of memory. The CPU was constant around 40% (using 4 over 8 cores) but the total memory went from 31% to 99% in 24 hours, then constant to 99% for the following 3 days, and then the test ended. I am not erasing any body. I am creating the spheres, apply gravity, wait that the deposition ends, create the big sphere, and push the big sphere in the packing very slowly. I was expecting to use a lot of memory since I have a lot of particles, but not that much... is that possible? I am wondering if the ending of the simulation is related to the memory. At this point I am also rerunning some periodic triaxial tests, with only 5,000 particles. The memory usage has increased from 22 to 32% in 3 hours of simulation, and it is still increasing... Has anybody encountered the same increasing of memory? Is it expected in Yade? Thank you Giulia -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp -- You received this question notification because you are a member of yade-users, which is an answer contact for Yade. ___ Mailing list: https://launchpad.net/~yade-users Post to : yade-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-users More help : https://help.launchpad.net/ListHelp
