On 07/04/2013 10:45 PM, Derek Gaston wrote:
> David - do you call close() on a matrix before putting entries into it
> everywhere?
Looking at the code, I call zero() before commencing matrix assembly,
but not close().
> Are you relying on setting something like
> KEEP_NONZERO_PATTERN on a Petsc matrix to keep it from dropping your
> preallocation in the case where you close a matrix before setting all
> of its entries?
I'm not explicitly calling KEEP_NONZERO_PATTERN anywhere. I just use the
libMesh SparseMatrix API throughout; I don't directly use any PETSc calls.
> The reason I ask is that we've identified what I believe to be a bug
> with Petsc block matrices where they are not honoring
> KEEP_NONZERO_PATTERN. This is causing a ton of mallocs for us in
> MOOSE. So, for now, we're just disabling block matrix support in
> libmesh until we get some clarity on the issue...
OK, interesting. Disabling block matrix support (using Ben's hard-coding
suggestion) fixed the mallocs for me as well, so sounds like I might
have been running into the same issue.
Thanks,
David
>> Hi Ben,
>>
>> A bit more info re the apparent sparsity pattern issue in RBConstruction.
>>
>> The error occurs in RBConstruction::assemble_misc_matrices(),
>> specifically in the call to
>> assemble_inner_product_matrix(inner_product_matrix.get());
>>
>> inner_product_matrix is an "extra matrix" that is a member of
>> RBConstruction. It is initialized in
>> RBConstruction::allocate_data_structures, with initialization code
>> modeled on ImplicitSystem::init_matrices. To check if it's an issue
>> related to an the initialization of inner_product_matrix, I made the
>> following change:
>>
>> ------------------------------------------------------------------------------------------------------------------------
>> void RBConstruction::assemble_misc_matrices()
>> {
>> - assemble_inner_product_matrix(inner_product_matrix.get());
>> + assemble_inner_product_matrix(this->matrix);
>> + inner_product_matrix->zero();
>> + inner_product_matrix->add(1., *matrix);
>> ------------------------------------------------------------------------------------------------------------------------
>>
>> But in this case, reduced_basis_ex6 still "fails" (i.e. causes a malloc)
>> in the call to assemble_inner_product_matrix(this->matrix).
>>
>> If you have any ideas on what might be going wrong, that'd be great.
>>
>> Best,
>> David
>>
>>
>>
>>
>> On 07/04/2013 11:50 AM, David Knezevic wrote:
>>> Hi Ben,
>>>
>>> Yeah, I think the issue I'm seeing is related to blocked dof support,
>>> since it looks like you had to add "-mat_new_nonzero_allocation_err
>>> false" in c08baf8a6f41edca3b3422b897687361e73bbee3. The commit message
>>> there was "avoid preallocation error that seems to be triggered by
>>> blocked matrix storage", and this predates the recent FEMContext or
>>> DGFEMContext changes (I guess I didn't notice it back then since I
>>> wasn't on the git HEAD).
>>>
>>> Any suggestions about where I should look for the fix? I'd like to
>>> resolve this before 0.9.2 is released, if that's OK?
>>>
>>> David
>>>
>>>
>>>
>>> On 07/04/2013 02:06 AM, Kirk, Benjamin (JSC-EG311) wrote:
>>>> It could also be blocked dof support as well. I'm on my way out of town
>>>> for the holiday but will try to follow this and help the best I can.
>>>>
>>>> -Ben
>>>>
>>>> On Jul 3, 2013, at 12:49 PM, "David Knezevic" <dkneze...@seas.harvard.edu>
>>>> wrote:
>>>>
>>>>> OK, I see that it also effects reduced_basis_ex5. It might be related to
>>>>> the DFEMContext stuff that I added. Let me have a closer look...
>>>>>
>>>>>
>>>>>
>>>>> On 07/04/2013 01:28 AM, David Knezevic wrote:
>>>>>> I have an issue: It appears that reduced_basis_ex6 now requires
>>>>>> -mat_new_nonzero_allocation_err false" otherwise we get an error like
>>>>>> "New nonzero at (0,2) caused a malloc!".
>>>>>>
>>>>>> This wasn't needed previously, did something change? Note that this
>>>>>> example using a CouplingMatrix. (I use the same functionality as ex6 in
>>>>>> a production code, so it's important for me to understand this issue)
>>>>>>
>>>>>> I'll have a closer look at the git logs, but if someone knows what has
>>>>>> changed, that'd be helpful.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 07/04/2013 12:47 AM, Roy Stogner wrote:
>>>>>>> Paul's last patch is in and looks good; I think we're good to go on
>>>>>>> 0.9.2-rc2 or even 0.9.2 if everyone's ready.
>>>>>>> ---
>>>>>>> Roy
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> This SF.net email is sponsored by Windows:
>>>>>>>
>>>>>>> Build for Windows Store.
>>>>>>>
>>>>>>> http://p.sf.net/sfu/windows-dev2dev
>>>>>>> _______________________________________________
>>>>>>> Libmesh-devel mailing list
>>>>>>> Libmesh-devel@lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>>>>> ------------------------------------------------------------------------------
>>>>>> This SF.net email is sponsored by Windows:
>>>>>>
>>>>>> Build for Windows Store.
>>>>>>
>>>>>> http://p.sf.net/sfu/windows-dev2dev
>>>>>> _______________________________________________
>>>>>> Libmesh-devel mailing list
>>>>>> Libmesh-devel@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>>>> ------------------------------------------------------------------------------
>>>>> This SF.net email is sponsored by Windows:
>>>>>
>>>>> Build for Windows Store.
>>>>>
>>>>> http://p.sf.net/sfu/windows-dev2dev
>>>>> _______________________________________________
>>>>> Libmesh-devel mailing list
>>>>> Libmesh-devel@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>> ------------------------------------------------------------------------------
>>> This SF.net email is sponsored by Windows:
>>>
>>> Build for Windows Store.
>>>
>>> http://p.sf.net/sfu/windows-dev2dev
>>> _______________________________________________
>>> Libmesh-devel mailing list
>>> Libmesh-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Windows:
>>
>> Build for Windows Store.
>>
>> http://p.sf.net/sfu/windows-dev2dev
>> _______________________________________________
>> Libmesh-devel mailing list
>> Libmesh-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:
Build for Windows Store.
http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Libmesh-devel mailing list
Libmesh-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-devel
