The code is behaving as it should, IMHO. Here is a way to have the Mat stored the same independently of the number of processes. […] global_rows, global_cols = input_array.T.shape size = ((None, global_rows), (0 if COMM_WORLD.Get_rank() < COMM_WORLD.Get_size() - 1 else global_cols, global_cols)) […] I.e., you want to enforce that the lone column is stored by the last process, otherwise, it will be stored by the first one, and interleaved with the rest of the (0,0) block.
Thanks, Pierre > On 23 Apr 2024, at 6:13 PM, Miguel Angel Salazar de Troya > <[email protected]> wrote: > > This Message Is From an External Sender > This message came from outside your organization. > Hello, > > The following code returns a different answer depending on how many > processors I use. With one processor, the last MPIAIJ matrix is correctly > formed: > > row 0: (0, 1.) (1, 2.) (2, 3.) (3, 4.) (4, -1.) > row 1: (0, 5.) (1, 6.) (2, 7.) (3, 8.) (4, -2.) > row 2: (0, 9.) (1, 10.) (2, 11.) (3, 12.) (4, -3.) > row 3: (0, 13.) (1, 14.) (2, 15.) (3, 16.) (4, -4.) > row 4: (0, 17.) (1, 18.) (2, 19.) (3, 20.) (4, -5.) > row 5: (0, 21.) (1, 22.) (2, 23.) (3, 24.) (4, -6.) > row 6: (0, 25.) (1, 26.) (2, 27.) (3, 28.) (4, -7.) > > With two processors though, the column matrix is placed in between: > > row 0: (0, 1.) (1, 2.) (2, -1.) (3, 3.) (4, 4.) > row 1: (0, 5.) (1, 6.) (2, -2.) (3, 7.) (4, 8.) > row 2: (0, 9.) (1, 10.) (2, -3.) (3, 11.) (4, 12.) > row 3: (0, 13.) (1, 14.) (2, -4.) (3, 15.) (4, 16.) > row 4: (0, 17.) (1, 18.) (2, -5.) (3, 19.) (4, 20.) > row 5: (0, 21.) (1, 22.) (2, -6.) (3, 23.) (4, 24.) > row 6: (0, 25.) (1, 26.) (2, -7.) (3, 27.) (4, 28.) > > Am I not building the nested matrix correctly, perhaps? I am using the > Firedrake PETSc fork. Can you reproduce it? > > Thanks, > Miguel > > > ```python > import numpy as np > from petsc4py import PETSc > from petsc4py.PETSc import COMM_WORLD > > > input_array = np.array( > [ > [1, 2, 3, 4], > [5, 6, 7, 8], > [9, 10, 11, 12], > [13, 14, 15, 16], > [17, 18, 19, 20], > [21, 22, 23, 24], > [25, 26, 27, 28], > ], > dtype=np.float64, > ) > > > n_11_global_rows, n_11_global_cols = input_array.shape > size = ((None, n_11_global_rows), (None, n_11_global_cols)) > mat = PETSc.Mat().createAIJ(size=size, comm=COMM_WORLD) > mat.setUp() > mat.setValues(range(n_11_global_rows), range(n_11_global_cols), input_array) > mat.assemblyBegin() > mat.assemblyEnd() > mat.view() > > input_array = np.array([[-1, -2, -3, -4, -5, -6, -7]], dtype=np.float64) > global_rows, global_cols = input_array.T.shape > size = ((None, global_rows), (None, global_cols)) > mat_2 = PETSc.Mat().createAIJ(size=size, comm=COMM_WORLD) > mat_2.setUp() > mat_2.setValues(range(global_rows), range(global_cols), input_array) > mat_2.assemblyBegin() > mat_2.assemblyEnd() > > N = PETSc.Mat().createNest([[mat, mat_2]], comm=COMM_WORLD) > N.assemblyBegin() > N.assemblyEnd() > > PETSc.Sys.Print(f"N sizes: {N.getSize()}") > > N.convert("mpiaij").view() > ``` > > > -- > > Miguel Angel Salazar de Troya > Head of Software Engineering > > > EPFL Innovation Park Building C > 1015 Lausanne > Email: [email protected] <mailto:[email protected]> > Website: > https://urldefense.us/v3/__http://www.corintis.com__;!!G_uCfscf7eWS!YORvPf_X3sveJywRYWyVNAFZ4TMboGzB8VrtDIonGAbCuwi-L_km8zcUwHsovFRYTKz6sEij9ppa0U9uf4u2Ew$ > > <https://urldefense.us/v3/__https://www.corintis.com__;!!G_uCfscf7eWS!bC7zvBxQx0RuDXxzlOgxr_PdSp5N9ZdzjgTPmjG_ZU5WbNvHboZHFBhZksYgyDF2nO1IRXABTx5zmJLaL2NK_EYg2SnKUCzq$> > > > > >
