Hallo Metthew, you were albe to reconstruct this problem on your petsc installation?
Best Klaus On 03/30/2015 01:43 PM, Matthew Knepley wrote:
On Mon, Mar 30, 2015 at 5:18 AM, Klaus Kaiser <[email protected] <mailto:[email protected]>> wrote:Hallo Metthew, at the weekend I updated petsc to the newest release version 3.5.2 and I still got this strange behavior.Okay, I will check it out. The first thing I am going to do is try with AIJ matrices instead of BAIJ, which you can probably do before me. If you get the right result, then its a bug in the BAIJ implementation,which I am confident has not been tested as thoroughly. Thanks, Matt Best Klaus On 03/27/2015 05:07 PM, Klaus Kaiser wrote:Hallo Matthew, here is a short example I used for 8 cores: int d_nnz[3]; int o_nnz[3]; d_nnz[0] = 3; d_nnz[1] = 3; d_nnz[2] = 3; o_nnz[0] = 6; o_nnz[1] = 6; o_nnz[2] = 6; ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0, d_nnz, 0, o_nnz,&A); ierr = MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE); ierr = MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE); ierr = MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE); ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0, d_nnz, 0, o_nnz,&Ah); ierr = MatCreateBAIJ(PETSC_COMM_WORLD, 3, 9, 9, 72, 72, 0, d_nnz, 0, o_nnz,&At); std::vector<double> insert(3*3*3*3, 1.0); for(int i=0;i<8;++i) { int rows[3] = {i,i+1,i+3}; int cols[3] = {i,i+1,i+3}; MatSetValuesBlocked(A, 3, rows, 3, cols, &insert[0], ADD_VALUES); } MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); MatDuplicate(A,MAT_COPY_VALUES,&Ah); MatDuplicate(A,MAT_COPY_VALUES,&At); MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY); MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY); MatAXPY(Ah,1.,At,SAME_NONZERO_PATTERN); MatAXPY(A,1.,Ah,SAME_NONZERO_PATTERN); MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); The result is, that only some values are computed correctly. Best and Thank you a lot Klaus On 03/27/2015 04:26 PM, Matthew Knepley wrote:On Fri, Mar 27, 2015 at 10:16 AM, Klaus Kaiser <[email protected] <mailto:[email protected]>> wrote: Hallo Matthew, thanks for your fast response. With "no beside the nonzero structure" I meant, that I do not create a different non-zero structure while I'm adding values to my matrices. I also tried MAT_COPY_VALUES and the result was the same. Please send the small example and we will figure out what is going wrong. Thanks, Matt I do not get a error message but the resulting matrix is wrong. Here I have a short example. The first 9 rows and first 3 colomn of the matrices Ah, At and Ah+At on the first processor (of 8): Ah: 0: 60.3553 -0.249975 2.77556e-17 1: 0 60.3553 0 2: 0 0 60.3553 3: 17.6777 0.374962 0.124987 4: 0 17.6777 0 5: 0 0 17.6777 6: -7.32233 -0.124987 0.374962 7: 0 -7.32233 0 8: 0 0 -7.32233 At: 0: 0 0 0 1: 2500 0 0 2: -4.54747e-13 0 0 3: 0 0 0 4: 1250 0 0 5: 1250 0 0 6: 0 0 0 7: 1250 0 0 8: 3750 0 0 Ah+At 0: 60.3553 -0.249975 2.77556e-17 1: 2500 60.3553 0 2: -4.54747e-13 0 60.3553 3: 17.6777 0.374962 0.124987 4: 0 17.6777 0 5: 0 0 17.6777 6: -7.32233 -0.124987 0.374962 7: 0 -7.32233 0 8: 0 0 -7.32233 you can see the first 3 rows of the resulting matrix looks exactly like what I would expect, but the last 6 rows only consists of the values of Ah. When you would also look on the matrix A and A+Ah+At you would also see, that values where both matrices have an nonzero entry are not sum correctly. Best Klaus On 03/27/2015 03:59 PM, Matthew Knepley wrote:On Fri, Mar 27, 2015 at 9:48 AM, Klaus Kaiser <[email protected] <mailto:[email protected]>> wrote: Hallo, I have a strange behavior in my code concerning the function MatAXPY. I create 3 different Matrices ierr = MatCreateBAIJ(PETSC_COMM_WORLD, block_size, local_size, local_size, system_size, system_size, 0, d_nnz, 0, o_nnz,&A); ierr = MatSetOption(A,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE); ierr = MatSetOption(A,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE); ierr = MatSetOption(A,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE); ierr = MatCreateBAIJ(PETSC_COMM_WORLD, block_size, local_size, local_size, system_size, system_size, 0, d_nnz, 0, o_nnz,&At); ierr = MatSetOption(At,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE); ierr = MatSetOption(At,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE); ierr = MatSetOption(At,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE); ierr = MatCreateBAIJ(PETSC_COMM_WORLD, block_size, local_size, local_size, system_size, system_size, 0, d_nnz, 0, o_nnz,&Ah); ierr = MatSetOption(Ah,MAT_NEW_NONZERO_ALLOCATION_ERR,PETSC_TRUE); ierr = MatSetOption(Ah,MAT_KEEP_NONZERO_PATTERN,PETSC_TRUE); ierr = MatSetOption(Ah,MAT_IGNORE_OFF_PROC_ENTRIES,PETSC_TRUE); These creations are superfluous since you use MatDuplicate() below. and want to sum these three matrixes with different factors. First I fill the Matrix A with some values, and duplicate the structure of A to the other two matrices: MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&Ah); MatDuplicate(A,MAT_DO_NOT_COPY_VALUES,&At); MatAssemblyBegin(Ah,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(Ah,MAT_FINAL_ASSEMBLY); MatAssemblyBegin(At,MAT_FINAL_ASSEMBLY); MatAssemblyEnd(At,MAT_FINAL_ASSEMBLY); After this I fill the matrices At and Ah with some other values, which are not beside the non zero structure (I also tried with just copying the Matrix A). Now after another MatAssembly I do not understand "no beside the nonzero structure". Do you mean that the nonzero structure is the same? Can you first test with MAT_COPY_VALUES?I want to add these Matrices in the form A+c*(Ah+d*At):MatAXPY(Ah,c,At,SAME_NONZERO_PATTERN); MatAXPY(A,d,Ah,SAME_NONZERO_PATTERN); When I run the method with mpi and one core everything works fine. Starting the same method with more cores, the sum of the matrices fails. It seems like some values are added Please send the full output of a failure when you use MAT_COPY_VALUES on 2 procs. Thanks, Matt correctly and many values are missed. Using DIFFERENT_NONZERO_STRUCTURE leads to the right behavior in the multi-core case, but is very slow. I checked with a viewer if all matrices have the same nonzero structure and this is the case. Does anyone know why this fails, or do I have made any wrong thoughts? I'm corrently working with a petsc version (Petsc Release Version 3.3.0, Patch 5, Sat Dec 1 15:10:41 CST 2012), I looked into the changelogs up to the current version and did not find any note about MatAXPY or MatAYPX. Best and Thanks a lot for your help Klaus-- What most experimenters take for granted before they begintheir experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener-- What most experimenters take for granted before they begin theirexperiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener--What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.-- Norbert Wiener
