On Fri, Aug 14, 2015 at 2:28 PM, Junchao Zhang <junchao.zh...@gmail.com>
wrote:
> Hello,
> I have a simple AMR code using SerialMesh. I run it 70 refinements on 12
> cores.
2^70 = 1180591620717411303424, do you mean 7?
> In metis_partitioner.C, I print out the partition result as follows.
>
> 437 // Otherwise use kway
> 438 else
> 439 Metis::METIS_PartGraphKway(&n, &ncon, &csr_graph.offsets[0],
> &csr_graph.vals[0], &vwgt[0], NULL,
> 440 NULL, &nparts, NULL, NULL, NULL,
> 441 &edgecut, &part[0]);
> 442
> 443 for (int i = 0; i < mesh.n_processors(); i++) { // my code
> 444 int cnt = 0;
> 445 for (int j = 0; j < n; j++) if (part[j] == i) cnt++;
> 446 printf("METIS assigned %d elements to proc %d\n", cnt, i);
> 447 }
>
> I found in the last few iterations, the partition result is severely
> imbalanced. For example, in last iteration, proc 11 has 9M+ elements, while
> others only have hundreds or thousands.
>
> METIS assigned 2829 elements to proc 0
> METIS assigned 270 elements to proc 1
> METIS assigned 804 elements to proc 2
> METIS assigned 264 elements to proc 3
> METIS assigned 263 elements to proc 4
> METIS assigned 733 elements to proc 5
> METIS assigned 273 elements to proc 6
> METIS assigned 407 elements to proc 7
> METIS assigned 1200 elements to proc 8
> METIS assigned 4795 elements to proc 9
> METIS assigned 23404 elements to proc 10
> METIS assigned 9866034 elements to proc 11
>
>
> My mesh has only triangles, with at most one hanging node.
Your *initial* mesh has hanging nodes? We don't support this...
> I did not assign
> any weight to nodes and I found libMesh uses number of nodes as weight (in
> my case, it should be 3). I tried to changed &vwgt[0] to NULL when calling
> METIS. From the printf, I got a quite balanced partitioning. However, the
> program execution time increased a lot.
>
> Any clues?
>
I don't know what METIS does when you don't use any weights. It sounds
like it employs a more expensive algorithm, but results in a higher-quality
partitioning. Can you share the stats for this partitioning?
For what it's worth, we've noticed that METIS produces very strange
partitions on even simple geometries, provided that you pick just the right
(wrong) number of processors, so it's not surprising that it can also have
trouble when partitioning millions of elements.
https://drive.google.com/file/d/0B9BK7pg8se_iTXpJak1fZWwySm8/view?usp=sharing
https://drive.google.com/file/d/0B9BK7pg8se_ibUJCdTBzR3h1UTQ/view?usp=sharing
https://drive.google.com/file/d/0B9BK7pg8se_ia1F5ZE0zTF9pdFE/view?usp=sharing
https://drive.google.com/file/d/0B9BK7pg8se_iY2hxQkl6ZHlFTmM/view?usp=sharing
https://drive.google.com/file/d/0B9BK7pg8se_iNFI0bDNlUXJwVlE/view?usp=sharing
--
John
------------------------------------------------------------------------------
_______________________________________________
Libmesh-users mailing list
Libmesh-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libmesh-users
