Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
The DM keeps track of the local vectors created by DMGetLocalVector(). So, if your routine gets called a million times with exactly one DMGetLocalVector() followed by one call to DMRestoreLocalVector(), then only one vector is created. Shri From: Miguel Angel Salazar de Troya salazardetr...@gmail.commailto:salazardetr...@gmail.com Date: Wed, 19 Nov 2014 14:02:10 -0600 To: barry smith bsm...@mcs.anl.govmailto:bsm...@mcs.anl.gov Cc: petsc-users@mcs.anl.govmailto:petsc-users@mcs.anl.gov petsc-users@mcs.anl.govmailto:petsc-users@mcs.anl.gov Subject: Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects You were right. I was forgetting a DMRestoreLocalVector(). Thanks. If we are creating local vectors every time we call functions such as the RHSFunction of a TS implementation, which is called many times in the TS integration, will this be a problem in terms of performance? I've noticed that it might be. This is the way it is implemented in some TS. I wanted to double check with you guys before I figure out a way to create just one local vector that is re used in functions such as RHSFunction. Miguel On Tue, Nov 18, 2014 at 12:54 PM, Barry Smith bsm...@mcs.anl.govmailto:bsm...@mcs.anl.gov wrote: On Nov 18, 2014, at 11:54 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.commailto:salazardetr...@gmail.com wrote: PetscFinalize() is inside of the class destructor (the last line of the destructor), so when the object goes out of scope, the class destructor is called and PetscFinalize() as well. Is it better to have PetscFinalize() outside of the destructor and call the destructor explicitly before? It shouldn't really matter. My guess is you must be missing calling a destroy or restore on one of the PETSc objects. Barry Miguel On Tue, Nov 18, 2014 at 11:32 AM, Barry Smith bsm...@mcs.anl.govmailto:bsm...@mcs.anl.gov wrote: On Nov 18, 2014, at 11:19 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.commailto:salazardetr...@gmail.com wrote: Hi I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example: https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called. This is normal. We cannot register the exact line the memory allocated, only the location of the PETScFunctionBegin; Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is: ierr = DMGetLocalVector(networkdm,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = VecGetArray(localX,xarr);CHKERRQ(ierr); Modify values in xarr ierr = VecRestoreArray(localX,xarr);CHKERRQ(ierr); ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMRestoreLocalVector(networkdm,localX);CHKERRQ(ierr); One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance. Do you call the class destructor yourself explicitly before the PetscFinalize()? You need to, otherwise the class may not be destroyed until after PetscFinalize() and hence the PETSc objects won't be freed when you call PetscFinalize(). You also need to make sure that you destroy ALL PETSc objects, if you miss even one PETSc object, since the objects have references
Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
On Nov 19, 2014, at 2:36 PM, Abhyankar, Shrirang G. abhy...@mcs.anl.gov wrote: The DM keeps track of the local vectors created by DMGetLocalVector(). So, if your routine gets called a million times with exactly one DMGetLocalVector() followed by one call to DMRestoreLocalVector(), then only one vector is created. In other words the DMGet/RestoreXXVector() are designed EXACTLY to be used when you need work vectors so you do not need to manage keeping the work vectors around yourself. In other words you are using it exactly right. Barry Shri From: Miguel Angel Salazar de Troya salazardetr...@gmail.com Date: Wed, 19 Nov 2014 14:02:10 -0600 To: barry smith bsm...@mcs.anl.gov Cc: petsc-users@mcs.anl.gov petsc-users@mcs.anl.gov Subject: Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects You were right. I was forgetting a DMRestoreLocalVector(). Thanks. If we are creating local vectors every time we call functions such as the RHSFunction of a TS implementation, which is called many times in the TS integration, will this be a problem in terms of performance? I've noticed that it might be. This is the way it is implemented in some TS. I wanted to double check with you guys before I figure out a way to create just one local vector that is re used in functions such as RHSFunction. Miguel On Tue, Nov 18, 2014 at 12:54 PM, Barry Smith bsm...@mcs.anl.gov wrote: On Nov 18, 2014, at 11:54 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.com wrote: PetscFinalize() is inside of the class destructor (the last line of the destructor), so when the object goes out of scope, the class destructor is called and PetscFinalize() as well. Is it better to have PetscFinalize() outside of the destructor and call the destructor explicitly before? It shouldn't really matter. My guess is you must be missing calling a destroy or restore on one of the PETSc objects. Barry Miguel On Tue, Nov 18, 2014 at 11:32 AM, Barry Smith bsm...@mcs.anl.gov wrote: On Nov 18, 2014, at 11:19 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.com wrote: Hi I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example: https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called. This is normal. We cannot register the exact line the memory allocated, only the location of the PETScFunctionBegin; Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is: ierr = DMGetLocalVector(networkdm,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = VecGetArray(localX,xarr);CHKERRQ(ierr); Modify values in xarr ierr = VecRestoreArray(localX,xarr);CHKERRQ(ierr); ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMRestoreLocalVector(networkdm,localX);CHKERRQ(ierr); One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance. Do you call the class destructor yourself explicitly before the PetscFinalize()? You need to, otherwise the class may not be destroyed until after PetscFinalize() and hence
[petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
Hi I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example: https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called. Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is: ierr = DMGetLocalVector(networkdm,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = VecGetArray(localX,xarr);CHKERRQ(ierr); Modify values in xarr ierr = VecRestoreArray(localX,xarr);CHKERRQ(ierr); ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMRestoreLocalVector(networkdm,localX);CHKERRQ(ierr); One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance. Miguel -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaz...@illinois.edu
Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
On Nov 18, 2014, at 11:19 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.com wrote: Hi I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example: https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called. This is normal. We cannot register the exact line the memory allocated, only the location of the PETScFunctionBegin; Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is: ierr = DMGetLocalVector(networkdm,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = VecGetArray(localX,xarr);CHKERRQ(ierr); Modify values in xarr ierr = VecRestoreArray(localX,xarr);CHKERRQ(ierr); ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMRestoreLocalVector(networkdm,localX);CHKERRQ(ierr); One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance. Do you call the class destructor yourself explicitly before the PetscFinalize()? You need to, otherwise the class may not be destroyed until after PetscFinalize() and hence the PETSc objects won't be freed when you call PetscFinalize(). You also need to make sure that you destroy ALL PETSc objects, if you miss even one PETSc object, since the objects have references to each other it may be that many PETSc objects do not get freed and hence -malloc_dump shows many objects still alive. Barry Miguel -- Miguel Angel Salazar de Troya Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaz...@illinois.edu
Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
PetscFinalize() is inside of the class destructor (the last line of the destructor), so when the object goes out of scope, the class destructor is called and PetscFinalize() as well. Is it better to have PetscFinalize() outside of the destructor and call the destructor explicitly before? Miguel On Tue, Nov 18, 2014 at 11:32 AM, Barry Smith bsm...@mcs.anl.gov wrote: On Nov 18, 2014, at 11:19 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.com wrote: Hi I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example: https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called. This is normal. We cannot register the exact line the memory allocated, only the location of the PETScFunctionBegin; Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is: ierr = DMGetLocalVector(networkdm,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = VecGetArray(localX,xarr);CHKERRQ(ierr); Modify values in xarr ierr = VecRestoreArray(localX,xarr);CHKERRQ(ierr); ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMRestoreLocalVector(networkdm,localX);CHKERRQ(ierr); One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance. Do you call the class destructor yourself explicitly before the PetscFinalize()? You need to, otherwise the class may not be destroyed until after PetscFinalize() and hence the PETSc objects won't be freed when you call PetscFinalize(). You also need to make sure that you destroy ALL PETSc objects, if you miss even one PETSc object, since the objects have references to each other it may be that many PETSc objects do not get freed and hence -malloc_dump shows many objects still alive. Barry Miguel -- Miguel Angel Salazar de Troya Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaz...@illinois.edu -- *Miguel Angel Salazar de Troya* Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaz...@illinois.edu
Re: [petsc-users] PetscFunctionBegin, -malloc_dump and C++ classes with PETSc objects
On Nov 18, 2014, at 11:54 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.com wrote: PetscFinalize() is inside of the class destructor (the last line of the destructor), so when the object goes out of scope, the class destructor is called and PetscFinalize() as well. Is it better to have PetscFinalize() outside of the destructor and call the destructor explicitly before? It shouldn't really matter. My guess is you must be missing calling a destroy or restore on one of the PETSc objects. Barry Miguel On Tue, Nov 18, 2014 at 11:32 AM, Barry Smith bsm...@mcs.anl.gov wrote: On Nov 18, 2014, at 11:19 AM, Miguel Angel Salazar de Troya salazardetr...@gmail.com wrote: Hi I'm implementing a problem using the TS. Most of my functions are methods inside of a class, except for the callbacks (to form the RHS and the TS monitor), which are outside of the class, although in the same .C file where the class methods are implemented. For these callbacks I followed the network example: https://bitbucket.org/petsc/petsc/src/a614f7369d93d476173b8fc6bf2463276dcbdb3a/src/snes/examples/tutorials/network/pflow/pf.c?at=master Therefore, the callbacks have the PetscFunctionBegin at the beginning and PetscFunctionReturn(0) at the end. My problems come when I run the program with -malloc_dump and I get a lot of unfreed memory. Inspecting the output I see that the line of my code where the memory is allocated corresponds with the line when PetscFunctionBegin is called. This is normal. We cannot register the exact line the memory allocated, only the location of the PETScFunctionBegin; Later in the file, I see that the function DMGetLocalVector() is called within a petsc internal routine (at the file dmget.c). I also call this routine in my callback methods few lines after PetscFunctionBegin. The procedure that I follow to use the local vectors is as the one in the network example. For vectors that I want to modify this is: ierr = DMGetLocalVector(networkdm,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalBegin(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = DMGlobalToLocalEnd(networkdm,X,INSERT_VALUES,localX);CHKERRQ(ierr); ierr = VecGetArray(localX,xarr);CHKERRQ(ierr); Modify values in xarr ierr = VecRestoreArray(localX,xarr);CHKERRQ(ierr); ierr = DMLocalToGlobalBegin(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMLocalToGlobalEnd(networkdm,localX,INSERT_VALUES,X);CHKERRQ(ierr); ierr = DMRestoreLocalVector(networkdm,localX);CHKERRQ(ierr); One last thing that I think it might be a issue here is how I destroy the petsc objects. I create the petsc objects within a class. For instance, the class has a petsc vector that later passes to the TS object to get the solution. To destroy the petsc objects, I use the class destructor, where at the end I call PetscFinalize() Inside the class I pass the callbacks to the TS routines that need them (e.g. TSSetRHSFunction() ) I can compile the code and run it, but many memory allocations are not freed. What can be the issue here? Do you know of an example using C++ classes to implement PETSc methods? Thanks in advance. Do you call the class destructor yourself explicitly before the PetscFinalize()? You need to, otherwise the class may not be destroyed until after PetscFinalize() and hence the PETSc objects won't be freed when you call PetscFinalize(). You also need to make sure that you destroy ALL PETSc objects, if you miss even one PETSc object, since the objects have references to each other it may be that many PETSc objects do not get freed and hence -malloc_dump shows many objects still alive. Barry Miguel -- Miguel Angel Salazar de Troya Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaz...@illinois.edu -- Miguel Angel Salazar de Troya Graduate Research Assistant Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign (217) 550-2360 salaz...@illinois.edu
